教授 Rust 的 SQL 语言
当我年轻的时候，我常常想，计算机是如何“理解”一种新的编程语言的。编译器对我来说充其量只是个模糊的概念，而解释型语言则像是魔法：你输入一些东西，机器就能以某种方式知道该怎么做。
SQL并不是直接编译成机器代码，但构建一个SQL引擎迫使你面对一个类似的问题：如何让一种语言（在我的案例中是Rust）理解并执行另一种语言（SQL），而它自己的编译器对此一无所知？
数据库并不是从一堆现有代码中发展而来的，而是源于一组概念：行、列、表达式、连接、聚合、索引、事务。
我的引擎的起点是一个受到SQLite和DuckDB强烈影响的通用SQL方言，因为它们都提供了一个名为SQLLogicTest的公共格式的大量测试。我理论上认为，一个能够通过这些测试的引擎将具有坚实的兼容性基础。我的目标不是发明一种新的SQL变体，而是支持这些系统已经运行的足够多的功能，以便我可以重用它们的测试套件和查询生态系统。
我没有自己编写解析器，而是使用了`sqlparser`，它将SQL转换为抽象语法树（AST），这是查询在内存中的图形表示。从那里，我将AST映射到我自己的Rust枚举和数据结构。这是SQL停止作为文本并开始作为引擎可以推理的一组概念的地方。
我没有从优化器开始，而是从测试工具开始。我将其连接起来以运行SQLite发布的SQLLogicTest测试集，并添加了一些对DuckDB风格测试的支持。今天，我运行的所有SQLite提供的测试都通过了。这并不意味着引擎是万无一失的；这仍然是早期的绿地代码。但这确实意味着引擎的行为是基于我自己头脑之外的东西：当测试失败时，是我的实现与已知结果的对比。这个测试套件已经成为事实上的规范；对计划器或执行器的任何更改都必须保持测试通过。
起初，我以为我会为每种想要支持的数据类型编写自己的计算内核。当你在某个领域缺乏经验时，你的思维会欺骗你，让你相信重新发明轮子是“正确”的唯一方式。最终，我将引擎迁移到了Apache Arrow的内存模型上。Arrow为我提供了列式表示和一组常见操作的内核。这样，我就可以专注于查询规划和将SQL语义映射到Arrow数组上，而不是手动编写和维护所有这些内容。
这个项目是我个人的努力。为了弥补这一点，我依赖于大型语言模型（LLMs）。在任何一天，我都会在Claude Sonnet 4.5和GPT-5.x Codex之间切换，以帮助构思新特性、推理生命周期问题或探索替代设计。它们的可靠性不足以让我完全信任，但足够快速，可以作为有噪音的、上下文感知的协同程序员。
这就形成了一种平衡：我有一个相对较大的测试集，必须保持通过，我希望新特性高效，并且不希望“聪明”的LLM生成的代码背后隐藏着微妙的正确性回归。测试形成了一种保护措施。如果建议的优化破坏了行为，它就会被抛弃。如果它通过了但看起来脆弱，我会对其进行重构。
我从未编写过编译器。这个项目是我所接近的最接近的东西。它介于查询引擎、解释器和一个关于如何通过借助其他系统的肩膀走得多远的课程之间：SQLite的测试、DuckDB的查询风格、Apache Arrow的内存模型，以及有时会以错误形式建议正确想法的LLMs。
如果没有其他，这个项目回答了我童年时的疑问。计算机并不是“自动理解”新语言的。总得有人来搭建那座桥。
（如果你喜欢这个故事，请搜索“rust-llkv”并给它一个星标。）
查看原文
When I was younger, I used to wonder how a computer could "just understand" a new programming language. Compilers were a vague idea at best, and interpreted languages felt like magic: you typed something in and the machine somehow knew what to do.<p>SQL is not compiled straight to machine code, but building a SQL engine forces you to face a similar question: how do you get one language (Rust, in my case) to understand and execute another language (SQL) that its own compiler knows nothing about?<p>A database does not grow out of a pile of existing code. It grows out of a set of concepts: rows, columns, expressions, joins, aggregates, indexes, transactions.<p>The starting point for my engine is a generic SQL dialect heavily influenced by SQLite and DuckDB, because they both provide a large corpus of tests in a common format called SQLLogicTest. My theory was that an engine that could pass those tests would have a solid baseline of compatibility. My goal is not to invent a new flavor of SQL, but to
support enough of what those systems already run that I can reuse their test suites and their ecosystem of queries.<p>I did not write my own parser. I use `sqlparser`, which turns SQL into an abstract syntax tree (AST), a graph-shaped in-memory representation of the query. From there I map that AST to my own set of Rust enums and data structures. That is the point where SQL stops being text and starts being a set of concepts the engine can reason about.<p>Instead of starting with an optimizer, I started with a test harness. I wired it up to run the SQLLogicTest corpus that SQLite publishes and added some support for DuckDB-style tests. Today all of the SQLite-provided tests I run are passing. That does not mean the engine is bulletproof; this is still early greenfield code. It does mean that the behavior of the engine is anchored to something outside my own head: when a test fails, it is my implementation vs. a known result. That suite has become the de facto specification; any change to the planner
or executor has to keep it passing.<p>At the beginning I thought I would write my own compute kernels for every data type I wanted to support. When you are inexperienced in a domain, your mind plays tricks on you and convinces you that reinventing is the only way to get it "right." I eventually moved the engine onto Apache
Arrow's memory model. Arrow gives me a columnar representation and a set of kernels for common operations. Instead of hand writing and maintaining all of that, I can focus on planning queries and mapping SQL semantics onto Arrow arrays.<p>This project is a solo effort. To compensate, I lean on LLMs. On any given day I bounce between Claude Sonnet 4.5 and GPT-5.x Codex to help sketch out new features, reason about lifetime issues, or explore alternative designs. They are not reliable enough to trust blindly, but
fast enough to act as noisy, context-aware pair programmers.<p>This creates a balancing act: I have a relatively large test corpus that must keep passing, I want new features to be efficient, and I do not want subtle correctness regressions hiding behind "clever" LLM-generated
code. The tests form a guardrail. If a suggested optimization breaks behavior, it gets thrown away. If it passes but looks fragile, I refactor it.<p>I have never written a compiler. This project is the closest I have come. It sits somewhere between a query engine, an interpreter, and a lesson in how far you can get by standing on the shoulders of other systems: SQLite's tests, DuckDB's style of queries, Apache Arrow's
memory model, and LLMs that sometimes suggest the right idea in the wrong shape.<p>If nothing else, it has answered my childhood question. Computers do not just understand" new languages. Someone has to build the bridge.<p>(If you like this story, search for "rust-llkv" and give it a star.)