我对人类灭绝事件炒作的反驳
“这两家公司都没有负责任地行事。他们正在直奔自我提升的超级智能，拿我们的生命在赌博。”
你可能昨天看到过一位名叫雅各布·考克森（Jacob Coxon）的Anthropic研究员发布的类似帖子，他因担心人工智能可能导致灭绝事件而辞去了公司职务。以下是我看到的一些相关引用，我想分享我对它们的看法：
1. “越是高级的人员，他们越是害怕。”
这更多地反映了硅谷的文化和激励机制，而不是其他。关于人工智能的神话在硅谷已经发展成一种自我延续的现象，金融和监管的激励措施支持着它的不断增长。
2. “我们确实真心相信人工智能可能会杀死所有人！我个人认为在下一个十年内，这种可能性超过10%。”
有什么证据吗？有没有任何暗示10%的数学依据？我没有看到任何，只是模糊地提到了ASI（超人工智能）。
3. “考克森离开了Anthropic，尽管即将上市，他仍然放弃了这么多钱，因为他对自己的信念非常真诚。”
首先，真诚并不意味着正确。而且，考虑到Anthropic目前的估值，即使在IPO前的期权也并不算牺牲，它们仍然价值非凡。我不相信这种殉道者的叙述。
4. “自我提升的人工智能正在到来，并遵循指数改善曲线。”
在我看来，这是一个隐藏在大量叙事旋转下的真相核心。是的，人工智能无疑变得更好。然而，每一个像Navier-Stokes解这样的头条背后，都有一个比头条所暗示的更加扎实、复杂且有争议的故事。
5. “我们正在接近奇点。”
……OpenAI和Anthropic的IPO。
这些极端叙事的真正不幸之处在于，我认为它们实际上是适得其反的。它们模糊了我们可以解决的具体、切实的问题，用极端的语言将复杂的立场简化为两个对立面。例如，生物武器、自动武器系统、关键基础设施中的人工智能以及大规模网络攻击都是当前值得认真关注的实际风险。
当一切都被视为潜在的末日时，没有什么会得到解决。
查看原文
"Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."<p>You may have seen a post like this yesterday from an Anthropic researcher named Jacob Coxon who quit the company in protest over fears that AI may be an extinction level event. These are a few of the quotes I saw surrounding it, and I want to share my own takes on them:<p>1. "The more senior the people, the more scared they are"<p>This says more about Silicon Valley culture and incentives than anything else. The mythology around AI has taken on a life of its own in the Valley, and the financial and regulatory incentives support it growing.<p>2. "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."<p>Any evidence? Any sort of even implied math behind the 10%? I haven't seen any, only vague illusions to ASI.<p>3. "Coxon left Anthropic, leaving so much money on the table in spite of the upcoming IPO because he's earnest in his beliefs"<p>First off, earnest doesn't mean correct. And pre-IPO options at Anthropic's current valuation even now aren't exactly a sacrifice, they're still worth an extraordinary amount. I don't believe the martyrdom framing.<p>4. "Self-improving AI is coming and following the exponential improvement curve"<p>In my opinion, this is a kernel of truth hidden under a mountain of narrative spin. Yes, AI has undoubtedly gotten much better. However, behind every headline such as the Navier-Stokes solution, there is a much more grounded, nuanced, and controversial story than the headline might suggest.<p>5. "We are approaching the singularity"<p>...of OpenAI and Anthropic's IPOs<p>The really unfortunate part of these extreme narratives is I believe they are actually counterproductive. It obscures the specific, tangible problems that we can solve in extreme language collapsing nuanced positions into two sides. For example, bioweapons, autonomous weapons systems, AI in critical infrastructure, and large-scale cyberattacks are real risks that deserve serious technical and policy attention right now.<p>When everything is a potential apocalypse, nothing gets fixed.