Fernando Velasco
Blog - architecture
    From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

    From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

    A byte-level language modeling proposal where autoregressive U-Nets learn a hierarchical token representation — no tokenizer required.

    llms architecture tokenization
    Titans: Neural Long-Term Memory for Enhanced Sequence Modeling

    Titans: Neural Long-Term Memory for Enhanced Sequence Modeling

    A paper that proposes test-time learnable long-term memory for LLMs, with a 'Surprise' metric that decides what in the context is worth remembering.

    llms long-context architecture
© 2026 Fernando Velasco Lozano
Built with Astro · Astrofy template · source
FV
Fernando Velasco
Senior Applied Data Scientist & AI Tech Lead
  • Home
  • Projects
  • Writing
  • CV
  • Contact