展示HN：Echo – 使用开放权重模型以三分之一的成本实现与Fable相当的结果
我正在构建Echo（<a href="https://echo.tracerml.ai" rel="nofollow">https://echo.tracerml.ai</a>），这是一个尝试将多个开放权重模型整合成一个AI系统的实验，而不是选择单一模型来处理所有任务。
这个项目始于一个简单的实验。我选择了一组模型，包括GLM-5.2、Kimi K2.7等，并对它们进行了相同的评估。然后，我测量了如果对于每个问题，事先知道哪些模型会有用以及如何组合它们的输出，会发生什么。
这个假设的系统在性能上显著优于池中任何单个模型。当然，这并不是一个可以实际部署的系统，因为它依赖于在看到结果后才能知道哪些决策是好的。Echo是我试图在没有事先信息的情况下恢复这种优势的尝试。
对于每个请求，Echo会决定分配多少计算资源，哪些模型应该参与，以及如何组合它们的工作。有些提示可能只需要相对少量的推理，而其他提示则可以从多个模型在问题的不同部分上协作中获益。
在构建过程中让我感到惊讶的一点是，这些模型之间的互补性。一些整体表现明显较弱的模型，在特定问题上仍然可以非常有用，或者作为组合的一部分。
在我的第一次评估中，Echo的表现始终优于其池中最好的单个模型。它的总体结果也大致与Fable相当，而Fable是我用作比较系统之一的较强模型，且推理成本约为其三分之一。
当然，Echo仍然存在一些错误的分配或组合决策的情况。我目前花费大量时间来理解这些失败，并测试这种方法在编码和代理任务中的表现，因为在这些任务中，评估每个决策的质量变得更加困难。
我构建了一个聊天界面（echo.tracerml.ai）和一个兼容OpenAI的API（<a href="https://echo.tracerml.ai/docs/api" rel="nofollow">https://echo.tracerml.ai/docs/api</a>），以便在评估设置之外测试该系统。
这里有一个简短的高层次视频，介绍它是如何工作的：<a href="https://www.youtube.com/watch?v=lJFJSvOdXhg" rel="nofollow">https://www.youtube.com/watch?v=lJFJSvOdXhg</a>
我在这里写下了评估方法、各个模型的结果、成本和当前的局限性：<a href="https://echo.tracerml.ai/eval" rel="nofollow">https://echo.tracerml.ai/eval</a>
我非常希望你能试试它！特别是如果你遇到任何奇怪的失败案例或分配看起来不直观的地方。
查看原文
I’ve been building Echo (<a href="https://echo.tracerml.ai/" rel="nofollow">https://echo.tracerml.ai/</a>), an experiment in making one AI system out of a pool of open-weight models rather than choosing a single model and using it for every task.<p>It started with a simple experiment. I took a group of models, including GLM-5.2, Kimi K2.7 and others, and ran them on the same evaluations. Then I measured what would happen if, for each problem, you somehow knew in advance which models would be useful and how their outputs should be combined.<p>That hypothetical system performed substantially better than any individual model in the pool. Of course, it is not something you can actually deploy because it relies on knowing which decisions were good after seeing the result. Echo is my attempt to recover some of that advantage without having that information in advance.<p>For each request, Echo decides how much computation to allocate, which models should participate, and how their work should be combined. Some prompts may only need a relatively small amount of inference, while others benefit from multiple models working on different parts of the problem.<p>One thing that surprised me while building it was how complementary the models are. A model that is clearly weaker overall can still be extremely useful on particular problems or as part of a combination.<p>On my first evaluation mix, Echo consistently performed better than the best individual model in its pool. It also reached roughly the same aggregate result as Fable, which I used as one of the stronger comparison systems, at around one third of the inference cost.<p>There are still some cases where Echo makes the wrong allocation or combination decision. I’m currently spending a lot of time understanding those failures, as well as testing whether the same approach holds up on coding and agentic tasks where measuring the quality of each decision becomes much harder.<p>I built a chat interface (echo.tracerml.ai) and an OpenAI-compatible API (<a href="https://echo.tracerml.ai/docs/api" rel="nofollow">https://echo.tracerml.ai/docs/api</a>) so the system can be tested outside the evaluation setup.<p>Here is a short/high level video on how it works: <a href="https://www.youtube.com/watch?v=lJFJSvOdXhg" rel="nofollow">https://www.youtube.com/watch?v=lJFJSvOdXhg</a><p>I wrote up the evaluation methodology, individual model results, costs and current limitations here: <a href="https://echo.tracerml.ai/eval" rel="nofollow">https://echo.tracerml.ai/eval</a><p>I would love for you to try it! Especially if you hit any weird failure cases or places where the allocation looks unintuitive.