Blog - llms

From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

A byte-level language modeling proposal where autoregressive U-Nets learn a hierarchical token representation — no tokenizer required.

llms architecture tokenization

HELMET: A Fresh Look at Context-Length Evaluation in LLMs

A benchmark system for evaluating LLMs across long contexts — summarization, long-doc QA, RAG, re-ranking, and generation with citations — beyond synthetic needle-in-a-haystack tests.

llms long-context evaluation

LoRA vs Full Fine-Tuning: A Beautiful Illusion of Equivalence

A critical look at how LoRA fine-tuning actually differs from full fine-tuning — and the 'intruder dimensions' that formalize how LoRA silently forgets part of the pre-training.

llms fine-tuning peft

Titans: Neural Long-Term Memory for Enhanced Sequence Modeling

A paper that proposes test-time learnable long-term memory for LLMs, with a 'Surprise' metric that decides what in the context is worth remembering.

llms long-context architecture