https://blog.0xnullpath.cc/en/tags/deepseek/Recent content in DeepSeek on Code NowHugoenThu, 01 Oct 2026 20:30:00 +0800https://blog.0xnullpath.cc/en/posts/note-snippet-21-deepseeks-attention-evolution-dsa-csa-csa2/Thu, 01 Oct 2026 20:30:00 +0800https://blog.0xnullpath.cc/en/posts/note-snippet-21-deepseeks-attention-evolution-dsa-csa-csa2/<div class="markdown-alert markdown-alert-note"> <p class="markdown-alert-title"> <span class="markdown-alert-kind">Note</span> <span class="markdown-alert-num"></span> <span class="markdown-alert-sep">|</span> <span class="markdown-alert-text">Preface</span> </p> <div class="markdown-alert-body"> <p>This is the second post in the DeepSeek Attention evolution series. The previous one, <a href="https://blog.0xnullpath.cc/posts/note-snippet-20-deepseek-%E7%9A%84-attention%E6%BC%94%E8%BF%9B-mla/">DeepSeek&rsquo;s Attention Evolution -【MLA】</a>, covered the evolution from MHA to MLA; this post picks up with DSA, CSA and CSA2. MLA solved the storage form of the KVCache, but its compute complexity is still $O(N^2)$ and the KVCache is still $O(N)$, so it still struggles with 256K/1M long contexts. This post follows the DeepSeek V3.2-exp → V4 → V4.1-Flash line to see how it pushes the algorithmic and engineering tradeoffs to the limit, step by step, across sparse attention, KV compression and cross-layer sharing.</p>https://blog.0xnullpath.cc/en/posts/note-snippet-20-deepseeks-attention-evolution-mla/Tue, 15 Sep 2026 23:30:00 +0800https://blog.0xnullpath.cc/en/posts/note-snippet-20-deepseeks-attention-evolution-mla/<blockquote> <p>Foreword: After reading the DeepSeekV4.1 paper and seeing that DeepSeek had once again made major changes to the model architecture, I was quite excited. So I looked back at DeepSeek&rsquo;s incremental improvements to the model architecture and wrote this series of articles in the gaps while waiting for agents to finish running and slacking off at work — a summary of what I&rsquo;ve learned about model architectures over the year since I moved into AI Infra.</p>