HackerNews中文版

《华尔街日报》 Anthropic呼吁顶尖人工智能实验室考虑放缓开发速度，指出AI系统的进步速度如此之快，以至于它们可能很快能够在没有人类干预的情况下自我改进，这可能会带来重大的社会风险。该公司在周四的一篇博客文章中表示，放慢全球AI发展速度“可能是件好事”，并披露了记录其最先进模型改进速度的内部数据。这篇文章由其内部研究院院长和政策负责人撰写，指出模型的进步似乎正朝着“递归自我改进”的方向发展，即AI系统能够在没有人类干预的情况下自行改进。一些AI业内人士将这一门槛视为潜在的危险和巨大的社会动荡的标志。文章中写道：“我们认为，世界应该有选择放慢或暂时暂停前沿AI开发的选项，以便社会结构和对齐研究能够跟上技术的进步。”它提议达成一项全球协议，以可能放慢开发速度，并建立一个机制来验证竞争对手是否遵守该协议。文章警告称，递归自我改进尚未发生，也不是不可避免的，“但可能会比大多数机构准备的要早。” Anthropic最近完成了一轮融资，公司的估值接近1万亿美元，并提交了保密文件，开始公开上市的过程。该公司最近在与ChatGPT制造商OpenAI的激烈竞争中脱颖而出，后者也预计很快会提交首次公开募股的文件。 Anthropic的年化收入预测，即初创公司常用的基于短期销售预测的年收入数字，预计到本月底将达到500亿美元，而2025年底时为90亿美元。

查看原文

WSJ Anthropic is calling for top artificial intelligence labs to weigh slowing the pace of development, suggesting that AI systems are advancing so rapidly that they may soon be able to improve themselves without human intervention in ways that could pose significant societal risks.The ability to slow global AI development would “likely be a good thing,” the company said Thursday in a blog post that disclosed internal data documenting how quickly its most advanced models are improving.The post, written by the head of its internal research institute and head of policy, noted that model advances appear to be on a path toward “recursive self-improvement,” when AI systems can improve on their own without human intervention. Some AI insiders have seen that threshold as a potential marker of danger and enormous societal upheaval.“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the post, written by Marina Favaro and Jack Clark, says. It proposes a global agreement on how to potentially slow development and a mechanism for verifying that competitors are respecting it.The post cautions that recursive self-improvement hasn’t yet happened and isn’t inevitable, “but could come sooner than most institutions are prepared for.”Anthropic recently concluded a fundraising round that valued the company at almost $1 trillion and filed confidential paperwork to begin the process of publicly listing its shares. The company has recently emerged as the front-runner in a ferocious competition for AI supremacy with ChatGPT-maker OpenAI, which is also expected to file paperwork for an initial public offering soon.Anthropic’s run-rate, a figure commonly used by startups that forecasts annual revenue based on short-term sales, is on track to reach $50 billion in annualized revenue by the end of this month, up from $9 billion at the end of 2025.

Anthropic呼吁全球暂停人工智能开发，指出“自我改进”的风险。