Tree-sitter

Tree-sitter is a parser generator and parsing library built for tools that work with code as people edit it. It parses a source file into a concrete syntax tree in milliseconds, reparses incrementally after each keystroke by reusing unchanged parts of the tree, and keeps producing useful trees even when the code is full of syntax errors. A small query language then finds patterns in those trees — for syntax highlighting, code folding, symbol outlines, structural search, and refactoring.

Tree-sitter powers syntax highlighting and code navigation in editors such as Neovim, Helix, Zed, and Emacs, in GitHub's code views, and in tools such as Difftastic and ast-grep. Hundreds of community grammars cover nearly every popular language.

TL;DR

• Grammars are written in JavaScript (grammar.js); the CLI generates a C parser with no runtime dependencies.
• Bindings exist for Rust, Python, Node.js, Go, WebAssembly, and more.
• Trees are concrete syntax trees with named nodes and fields such as name: and body:.
• After an edit, Tree-sitter reparses incrementally, reusing unchanged subtrees.
• Parsing is error-tolerant: invalid regions become ERROR or MISSING nodes instead of failing.
• Queries (S-expression patterns with captures and predicates) drive highlighting, navigation, and analysis; grammars are tested with corpus tests.

Quick Example

Find every function and its parameters in Python source with the Python bindings (py-tree-sitter 0.25 or later):

The same kind of query, saved as highlights.scm, tells an editor how to color code:

Core Concepts

Grammars

A grammar is a JavaScript module describing rules with combinators such as seq, choice, repeat, optional, field, and precedence helpers:

tree-sitter generate produces src/parser.c. Rules starting with _ are hidden from the tree, extras (such as whitespace and comments) may appear anywhere, and prec resolves ambiguities. Context-sensitive tokens — heredocs, indentation-based blocks, nested comments — use an external scanner written in C.

Concrete Syntax Trees

Every token and construct appears in the tree. Named nodes (function_definition, identifier) represent meaningful constructs; anonymous nodes ("(", "def") represent literal tokens. Fields label children by role, so queries and code can ask for name or body without depending on child positions. Each node records its byte and row/column range.

Incremental Parsing

When text changes, the editor tells Tree-sitter which byte range was edited (tree.edit(...)) and parses again with the old tree; unchanged subtrees are reused, so reparsing typically takes well under a millisecond regardless of file size. Tools can then ask which ranges changed to update only affected highlights or diagnostics.

Error Recovery

Real code in an editor is usually broken while being typed. Tree-sitter isolates syntax errors in ERROR nodes, inserts MISSING nodes for absent tokens, and keeps the rest of the tree accurate, so highlighting and navigation keep working.

Queries

Queries are patterns written as S-expressions that match tree shapes and capture nodes (@name). Predicates such as #eq? and #match? filter matches by text. Editors ship query files per language:

Testing Grammars

Corpus tests live in test/corpus/*.txt, pairing input with the expected tree:

tree-sitter test runs corpus and highlight tests; tree-sitter parse file.ext prints the tree for any file; tree-sitter playground opens an interactive web view. Parse large real-world code bases to check for ERROR nodes and performance problems.

Tree-sitter, ANTLR, and Language Servers

Editors commonly combine Tree-sitter (instant highlighting and structure) with language servers (semantic features). See ANTLR and IDEs and Editors.

Best Practices

Start From an Existing Grammar

Most languages already have maintained Tree-sitter grammars; contribute fixes upstream instead of forking.

Use Fields Generously

Fields make queries and tools robust to grammar changes and are far easier to read than positional child access.

Test Against Real Code

Run the grammar over large repositories and treat unexpected ERROR nodes as bugs; add failing snippets to the corpus.

Keep Queries Specific

Narrow patterns with fields and predicates so highlighting and navigation stay accurate and fast.

Common Mistakes

Treating the CST as an AST

Expecting Semantic Information

Reparsing From Scratch on Every Keystroke

FAQ

What is Tree-sitter?

A parser generator and incremental parsing library that produces concrete syntax trees for source code quickly and tolerantly, designed for editors and developer tools.

How is Tree-sitter used in editors?

Editors parse each open file with Tree-sitter and use queries to highlight syntax, fold code, compute indentation, show outlines, select syntax nodes, and inject parsers for embedded languages, updating trees incrementally as you type.

Is Tree-sitter a replacement for a language server?

No. Tree-sitter understands syntax only. Language servers provide semantic features such as type information, definitions across files, and compiler diagnostics. They complement each other.

How do I write a Tree-sitter grammar?

Run tree-sitter init to scaffold a project, define rules in grammar.js, generate the parser with tree-sitter generate, and iterate with corpus tests and tree-sitter parse on real files.

Which programming languages can use Tree-sitter?

The core library is C, with official or community bindings for Rust, Python, Node.js, Go, Java, Swift, WebAssembly, and others.

Related Topics

• Lexers and Parsers — Parsing fundamentals
• ANTLR — Grammar-based parsers for compilers and DSLs
• Abstract Syntax Trees — Trees for analysis and compilation
• IDEs and Editors — Where Tree-sitter is used
• Compilers & Language Implementation — The wider toolchain
• Regular Expressions — Token patterns in grammars

References

• Tree-sitter documentation
• Creating parsers (Tree-sitter documentation)
• Query syntax (Tree-sitter documentation)
• tree-sitter/tree-sitter
• py-tree-sitter documentation