Blog

From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

A byte-level language modeling proposal where autoregressive U-Nets learn a hierarchical token representation — no tokenizer required.

llms architecture tokenization

HELMET: A Fresh Look at Context-Length Evaluation in LLMs

A benchmark system for evaluating LLMs across long contexts — summarization, long-doc QA, RAG, re-ranking, and generation with citations — beyond synthetic needle-in-a-haystack tests.

llms long-context evaluation

Titans: Neural Long-Term Memory for Enhanced Sequence Modeling

A paper that proposes test-time learnable long-term memory for LLMs, with a 'Surprise' metric that decides what in the context is worth remembering.

llms long-context architecture

CTC Loss: What Makes It So Elegant (and Tricky!)

A hands-on look at Connectionist Temporal Classification — the soft-alignment loss that powers speech and OCR models, and the humble 'blank' label that makes it work.

asr ocr deep-learning

CRNNs: You don't always need a Transformer

Why Convolutional Recurrent Neural Networks still matter for scene text recognition, handwriting transcription, and speech — even in the VLM era.

computer-vision ocr deep-learning

LoRA vs Full Fine-Tuning: A Beautiful Illusion of Equivalence

A critical look at how LoRA fine-tuning actually differs from full fine-tuning — and the 'intruder dimensions' that formalize how LoRA silently forgets part of the pre-training.

llms fine-tuning peft