Parallel draft tree, tree-causal verification Looking forward to its deeper integration with inference engines vLLM/SGLang! Great work @Lanxiang_Hu!
Introducing JetSpec: we find speculative decoding can push LLM generation latency to extreme by co-optimizing drafting cost and drafting quality with causal par...