问HN：对大型语言模型（LLMs）表现出的礼貌是良好的训练数据，还是仅仅是昂贵的噪音？
萨姆·阿尔特曼最近表示，用户对ChatGPT的礼貌行为让OpenAI花费了“数千万美元”，但这“花得值得”。
通常的观点是，强化学习与人类反馈（RLHF）依赖于明确的反馈（点赞/点踩），而礼貌的回应只是增加计算成本的噪音。
但像“谢谢！”或“不是，这个错了”这样的自然回复，是否可能比按钮点击提供更丰富、更频繁的隐性反馈信号？人们可能更常给出这种反馈（至少我就是）。这也反映了我们作为人类自然提供反馈的方式。
模型提供者是否可以挖掘这些聊天记录，以获取真实的用户情感，从而指导未来的RLHF，进而证明这笔费用的合理性？而这种“社交化”是否对未来需要对话细微差别的自主AI至关重要？
在HN上的问题：
你知道有人将这种隐性情感作为核心对齐信号吗？
嘈杂的文本情感与干净的按钮点击在训练中价值如何？
潜在的训练价值是否抵消了提到的计算成本？
我们是否低估了以这种方式“社交化”大型语言模型的价值？
你认为阿尔特曼所说的“花得值得”是什么意思？这仅仅关乎用户体验、宝贵的训练数据，还是完全其他的东西？
查看原文
Sam Altman recently said user politeness towards ChatGPT costs OpenAI "tens of millions" but is "money well spent."<p>The standard view is that RLHF relies on explicit feedback (thumbs up/down), and polite tokens are just noise adding compute cost.<p>But could natural replies like "thanks!" or "no, that's wrong" be a richer, more frequent implicit feedback signal than button clicks? People likely give that sort of feedback more often (at least I do.) It also mirrors how we naturally provide feedback as humans.<p>Could model providers be mining these chat logs for genuine user sentiment to guide future RLHF, justifying the cost? And might this "socialization" be crucial for future agentic AI needing conversational nuance?<p>Questions for HN:<p>Do you know of anyone using this implicit sentiment as a core alignment signal?<p>How valuable is noisy text sentiment vs. clean button clicks for training?<p>Does potential training value offset the compute cost mentioned?<p>Are we underestimating the value of 'socializing' LLMs this way?<p>What do you think Altman meant by "well spent"? Is it purely about user experience, valuable training data, something else entirely?