问HN：谁在使用FPGA进行机器学习推理？
随着RAM价格的上涨，我在想是否可以使用FPGA来卸载推理处理，而不需要将权重保存在RAM中？可用的RAM将用于激活、KV缓存和上下文，但不包括静态权重。权重可以从磁盘流式传输。这个方法并不是为了提高每秒处理的令牌数，而是为了以更低的成本提高吞吐量。可能会得到更好的答案/kHh？我已经开始研究这个问题，但想知道是否还有其他人考虑过或尝试过这种方法？
查看原文
With RAM price inflation, I wonder if FPGAs can be used to offload inference processing without keeping weights in RAM? The available RAM would be for activations, KV Cache, context but not static weights. Weights could be streamed from disk. This approach is not for tokens/second but throughput at a lower cost. Possibly better answers/kHh? I've started researching this, but wonder if others have considered/tried this?