问HN：你们是如何开发更具确定性的LLM管道的？
使用大型语言模型（LLMs）时，我会先设计一个提示，然后得到一些输出，但这些输出往往结构不够清晰，需要人类进行一定的监督，以验证我所获得的结果是否具有质量。而LLMs的非确定性特性是造成这种情况的主要原因。
我看到一些使用LLMs、视觉语言模型（VLMs）等的解决方案，它们能够解决一些以前已经解决过的问题（例如光学字符识别（OCR）、解析，甚至是代码生成）。它们在这些问题上表现得非常好且速度很快……90%的时间都是如此。那么，我该如何建立一个框架，以便对我的解决方案充满信心（即100%确定我知道它将会做什么）？
在标准机器学习中，我有置信度评分来判断是否接受某个输出为有价值的结果。一般来说，它是确定性的（即给定相同的输入，我总是会得到相同的输出）。而在LLMs等模型中，这一部分似乎缺失了。
查看原文
It feels like with LLMs I develop a prompt and then get some kind of output that's not very well structured and requires some kind of human oversight to verify that the result I've gotten back is of quality, and the non-deterministic nature of LLMs is the main reason for this.<p>I see solutions using LLMs, VLMs, etc that will achieve things that have been solved before (things like OCR, parsing, or even just code generation). They do these problems really well and really fast... 90% of the time. How do I get a scaffolding setup so I can be 100% confident in my solution (meaning 100% confident that I <i>know</i> what it's going to do)?<p>With standard ML, I have things like confidence scores to base whether or not I accept an output as valuable. And generally speaking, it's pure (i.e., given the same input I will always get the same output). With LLMs and the like, it feels like this piece is missing.