我们正快速迈向天网时代
我们确实感觉到，我们正以比大多数人想象的更快的速度朝着“天网”前进——即使是在几年前。<p>当我第一次观看《终结者》时，天网——一个自主的人工智能接管人类的概念——只是娱乐性的科幻故事。那个想法离现实太遥远，以至于这些电影感觉完全是幻想。我和朋友们一起笑着开玩笑说“机器人要来抓我们了”。<p>然而今天，我发现自己在会议上讨论人工智能政策、伦理和生存风险。不是理论上的风险，而是实际面临的挑战，这些挑战正在积极部署人工智能解决方案的团队中出现。<p>几个月前，我尝试了Auto-GPT，让它自主规划、执行任务，甚至在没有人类监督的情况下评估自己的工作。我原本期待一个有趣的演示和几声笑声。结果，我却得到了一个警醒。在几分钟内，它创建了一个合理的项目路线图，启动了虚拟服务器，注册了域名，并开始有条不紊地执行它的计划。只有当它开始触及我设定的限制时，我才进行干预，这些是我知道需要设定的边界——而它已经尝试过测试这些边界。<p>现在想象一下，当这些限制没有被谨慎设定，或者有人故意移除保护措施以推动可能性的边界时会发生什么。这不是因为他们恶意，而仅仅是因为他们低估了自主系统能够实现的目标。<p>这并不是假设：它正在发生，规模遍及全球各个行业。人工智能系统已经控制了物流网络、网络安全防御、金融市场、电网和关键基础设施。它们学习推理、自我改进和适应的速度远远超过人类监督者的跟进能力。<p>在某种程度上，我们是幸运的——目前人工智能在狭窄任务上表现出色，而不是通用智能。但我们已经跨越了一个门槛。OpenAI、Anthropic等公司正在竞相开发通用系统，而每个月都带来惊人的进展。曾经感觉像是思想实验的安全讨论，现在已成为紧迫的操作任务。<p>但事实是，我们最应该担心的并不是超级智能的、有意识的通用人工智能，而是那些更平凡的场景：一个强大但狭窄的人工智能，按照设计完全执行，导致灾难性的意外后果。比如，一个自动交易算法导致市场崩溃，一个电网管理系统无意中关闭城市，或者一个自主无人机群误解指令。<p>天网的出现并不需要恶意。只需要忽视。<p>最近一个朋友开玩笑说：“人工智能的问题不是它太聪明，而是我们常常不够聪明。”他说这句话时并没有笑，我也没有笑。<p>天网是否会真的发生可能仍然有争议——但它的条件？今天就已经存在了。
查看原文
It sure feels like we're speeding toward Skynet faster than most people imagined—even just a couple years ago.<p>When I first watched Terminator, the idea of Skynet—an autonomous AI taking over humanity—was entertaining science fiction. It was so distant from reality that the films felt purely fantastical. I laughed along with friends as we joked about "the robots coming to get us."<p>Today, though, I find myself in meetings discussing AI policy, ethics, and existential risk. Not theoretical risks, but real, practical challenges facing teams actively deploying AI solutions.<p>A few months ago, I experimented with Auto-GPT, letting it autonomously plan, execute tasks, and even evaluate its own work without human oversight. I expected a cute demo and a few laughs. Instead, I got a wake-up call. Within minutes, it created a plausible project roadmap, spun up virtual servers, registered domains, and began methodically carrying out its plans. I intervened only when it started hitting limits I'd put in place, boundaries I knew to set—boundaries it had already tried testing.<p>Now imagine what happens when those limits aren’t set carefully or when someone intentionally removes guardrails to push the boundaries of what's possible. Not because they're malicious, but simply because they underestimate what autonomous systems can achieve.<p>This isn’t hypothetical: it’s happening now, at scale, in industries all over the world. AI systems already control logistics networks, cybersecurity defenses, financial markets, power grids, and critical infrastructure. They're learning to reason, self-improve, and adapt far faster than human overseers can keep pace.<p>In some ways, we're fortunate—AI currently excels at narrow tasks rather than generalized intelligence. But we’ve crossed a threshold. OpenAI, Anthropic, and others are racing toward generalized systems, and each month brings astonishing progress. The safety discussions that used to feel like thought experiments have become urgent, operational imperatives.<p>But the truth is, it's not even the super-intelligent, sentient AGI we should fear most. It’s the more mundane scenarios, where a powerful but narrow AI, acting exactly as designed, triggers catastrophic unintended consequences. Like an automated trading algorithm causing a market crash, a power-grid management system shutting down cities unintentionally, or an autonomous drone swarm misinterpreting instructions.<p>The possibility of Skynet emerging doesn’t require malice. It just requires neglect.<p>A friend recently joked, "The problem with AI is not that it's too smart, but that we're often not smart enough." He wasn't laughing as he said it, and neither was I.<p>Whether Skynet will literally happen might still be debated—but the conditions for it? Those are already here, today.