Zhangzhe's Blog

08-09

Truncation Sampling as Language Model Desmoothing

08-08

A Contrastive Framework for Neural Text Generation

08-05

ALiBi: Train short, test long: Attention with linear biases enables input length extrapolation

07-30

KV Cache Transformer

07-18

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

07-17

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

07-12

GQA: Grouped-Query Attention

07-11

MQA: Multi-Query Attention

07-01

LLM 面试题汇总

06-17

HiPPO: Recurrent Memory with Optimal Polynomial Projections