在最困难的人工智能记忆基准测试（BEAM，1000万标记）上，使用了一个更小的模型达到了最新的技术水平。
大家好，我是Johnny，Exabase的创始人。我们刚刚在BEAM这个最具挑战性的AI记忆基准测试中，在每个规模（最高可达1000万标记）上达到了最高的报告分数。我们还使用了Gemini 3 Flash进行评估，而之前的所有领先者都依赖于一个更大的模型（Gemini 3 Pro）。
在1000万标记的规模下，这一规模远远超出了任何模型的上下文窗口，因此上下文填充并不是一个选项（除了大窗口中只有大约一半的内容可以有效利用而不会降低性能）。获得高分的唯一方法是依靠根本有效的回忆。
我们的系统（M-1）在100K时得分为76.9%，在1M时为75.0%，在10M时为68.0%。之前的领先者Hindsight的得分分别为73.4%、73.9%和64.1%，而Honcho的得分为63.0%、63.1%和40.6%，两者均使用Gemini 3 Pro，而我们使用的是Flash。
我们发现竞争差距在规模上变得更大：在100K时领先Hindsight 3.5分，在10M时领先3.9分。与Honcho的差距则从13.9分扩大到27.4分。随着语料库的增大，它过滤出了有效的回忆与强行推理/模型能力之间的差异。
M-1在每个查询中消耗的标记数量比下一个最佳系统少约20%。
关于BEAM基准测试：
BEAM测试十种记忆能力，包括一些其他基准测试未覆盖的能力：矛盾解决、事件排序和指令跟随。1000万标记的规模大致相当于与大型语言模型进行一年的长时间日常对话。
当然，我们的系统仍然远非完美，在某些类别（偏好跟随、指令跟随、摘要、回避）上表现强劲，得分在1000万标记规模下均保持在90%以上。
而我们的系统在其他方面则表现较弱：例如，多会话推理在100K时得分为44.7%，在10M时降至9.6%。尽管这个挑战似乎是这个规模下所有记忆系统的普遍问题，而非M-1特有的问题。这是我们将继续努力的方向。
方法论：
我们分支了Hindsight的开源基准测试脚本，替换了检索层，并对运行器的提示结构进行了小幅调整以便于生产使用。完整的方法论、三个规模的结果JSON以及提示生成器的链接都在论文中（论文链接见下）。
结合我们的LongMemEval结果（96.4%），M-1现在是唯一一个在所有主要记忆基准测试中，在每个规模（从115K到1000万标记）上都保持SOTA的系统。
研究论文：https://exabase.io/research/exabase-achieves-state-of-the-art-on-beam-benchmark
欢迎讨论架构、基准测试、规模挑战等话题。
查看原文
Hey HN. I'm Johnny, founder of Exabase. We just hit the highest reported score on BEAM, the hardest AI memory benchmark, at every scale up to 10 million tokens. We also ran our evaluation using Gemini 3 Flash, when all previous leaders depended on a much larger model (Gemini 3 Pro).<p>At 10M tokens, the scale is vastly larger than any model's context window, so context stuffing isn't an option (aside from the fact that only about half of a large window can be effectively utilised without degradation). The only way to score well is recall that fundamentally works.<p>Our system (M-1) scored 76.9% at 100K, 75.0% at 1M, and 68.0% at 10M. Previous leaders were Hindsight (73.4%, 73.9%, 64.1%) and Honcho (63.0%, 63.1%, 40.6%), both using Gemini 3 Pro, while we used Flash.<p>We saw the competitive gap get wider at scale: 3.5 points ahead of Hindsight at 100K, 3.9 at 10M. The gap with Honcho goes from 13.9 to 27.4 points. As the corpus gets bigger, it filters out effective recall vs. brute-forcing / model capability.<p>M-1 also consumed about 20% fewer tokens per query than the next best system.<p>About the BEAM benchmark:
BEAM tests ten memory abilities including some that other benchmarks don't cover: contradiction resolution, event ordering, and instruction following. The 10M token scale is vaguely equivalent to a year of long daily chats with an LLM.<p>Of course our system is still far from perfect, with strength in some categories (preference following, instruction following, summarization, abstention) all consistently above 90%, even at 10M token scale.<p>And our system shows weakness in others: for example, multi-session reasoning: 44.7% at 100K, collapsing to 9.6% at 10M. Although that challenge seems to be a general problem across memory systems at this scale, not M-1 specific. Something we'll continue to work on.<p>Methodology:
We forked Hindsight's open-source benchmarking script, replaced the retrieval layer, and used the runner's prompt structure with minor adjustments for production use. Full methodology, results JSON for all three scales, and the prompt generator are linked in the paper (paper linked below).<p>Combined with our LongMemEval result (96.4%), M-1 is now the only system to hold SOTA across both major memory benchmarks at every scale, from 115K to 10M tokens.<p>Research paper: https://exabase.io/research/exabase-achieves-state-of-the-art-on-beam-benchmark<p>Happy to discuss architecture, the benchmark, scale challenges etc.