How it starts

The project is built when the editor sends initialized. The entry point is load_project in server.py. Since 0.4.1 it does the fast part at once and hands the warehouse read to a background thread, so the first completion does not wait for the warehouse. dbt-ls check uses the synchronous _load_project instead.

  1. Find the project root

    find_dbt_project_root walks up from the opened folder looking for dbt_project.yml, then searches downward, skipping target, dbt_packages and virtual environments.

  2. Resolve the profile

    Profiles.locate looks for profiles.yml where dbt does: the directory given with --profiles-dir, else DBT_PROFILES_DIR, else the project root, then ~/.dbt. A directory named by the flag or the variable is the only one searched, so the server never reads a different warehouse than dbt run. ProfileTarget.from_dict picks a dataclass by type, renders env_var(), coerces types and wraps secrets. A failure here no longer stops the load: it is recorded, shown as a warning, and the server continues without live columns.

  3. Discover models, seeds and sources

    ProjectState.__post_init__ lists every .sql under the model paths as a model, every .csv under the seed paths as a seed, and every table under a sources: key in any YAML file. Both paths default to dbt's models and seeds. It also collects ref_names, every name ref() can take, packages and snapshots included, for diagnostics.

  4. Add columns from files

    Runs refresh_from_config and refresh_from_catalog in the order given by --schema-sources. Each one overwrites the columns of the models it knows about. The server now answers requests.

  5. Read the warehouse in the background

    _split_at_warehouse cuts the source list at database. A daemon thread runs that source and any after it on a copy of the state, then hands the copy over on the event loop with call_soon_threadsafe. A reload bumps load_generation, so a read that finishes late is thrown away.

  6. Pick the SQL dialect

    dialect_for_profile_type maps the profile type to a sqlglot dialect for the per-file parser. Unknown types fall back to postgres.

  7. Watch the project

    watched_files gets four watchers. run_results.json re-derives columns from the compiled SQL. Any .yml reloads the sources, matched against the last warehouse read rather than a new one. Any .csv reads the seeds again. A .sql or .py coming or going refreshes ref_names, and a .sql also runs refresh_models: known models keep what was found for them, a new one is matched against the last warehouse read, and a deleted one leaves its table among the undocumented. A change seen while the warehouse is still being read sets files_changed, so the late state is refreshed the same way before it is adopted. Each ends by checking the open files again.

Module map

All of it lives in src/dbt_ls. Five files hold nearly all the logic that is hard to remember: scope.py, alias.py, resolve.py, relations.py and model.py.

ModuleLinesWhat it owns
server.py1209LSP handlers, project loading in two stages and its error handling, the file watchers, serve_completions, serve_hover, relation_for, publishing diagnostics, columns from a saved model, the two commands.
model.py1064The Model and WarehouseTable types, model discovery and ref_names, the YAML and catalog readers, all nine warehouse readers with their registry, and the matching of warehouse tables to models and sources.
scope.py614The jinja-tolerant sqlglot dialect, the mid-edit repairs, the per-file AST cache, and turning a query into CTEs, tables and select lists.
profiles.py341Finding and reading profiles.yml, one dataclass per adapter, helpful errors for missing or misspelled keys.
state.py300ProjectState: the models, seeds, sources and undocumented tables shared by all handlers, tables (sources, then undocumented tables), the last warehouse read, the refresh methods, the merge rule, and the run_results.json reader.
diagnostics.py254diagnose: undeclared sources, unknown refs, and the column checks over sqlglot's scopes.
relations.py243What each relation in a query exposes, and whether that is every column: tables, CTEs, subqueries and *. model_columns for a saved model.
alias.py181Regex search for alias declarations, and choosing the declaration that is in scope at the cursor.
resolve.py141Giving CTE columns their data types by following them to the model or source they come from, and expanding *.
hover.py116The text of a hover: a SQL signature, where the relation lives, and its columns lined up with their types.
source.py105The SourceTable type, with the .yml that declares it, and declared(), source discovery from YAML, source columns from the catalog.
pattern.py79The regexes that decide what the cursor is completing, and which ref, source or alias.column it is on for hover and go to definition.
cli.py80Argument parsing, stdio or TCP transport, and handing dbt-ls check to check.py.
check.py169The dbt-ls check command: loads a project outside an editor and prints or returns as JSON what each schema source gave.
report.py53LoadReport: what happened during a load, per schema source, with outcome, timing and counts.
seed.py52Seed discovery and reading a CSV header.
editors/vscode231The VS Code client: one server process per dbt project, command routing.

Column completion, step by step

This is the most intricate path in the project. It mixes two techniques on purpose: regexes on raw text, which survive half-typed SQL, and a parsed tree, which knows where query blocks begin.

1. Classifypattern.py sees c. left of the cursor and returns kind column, alias c.
2. Cached treeAstCache.get returns the last version of the file that parsed. Completion itself never parses.
3. Find aliasesparse_alias_list runs three regexes over the live text: ref, source, and plain tables. Each keeps its qualifier: the source name, or the schema.
4. Pick by scopechoose_alias keeps the declaration inside the query block the cursor is in.
5. Type the CTEsresolve_ctes copies data types from models and sources onto CTE columns.
6. Look uprelation_for finds the relation among CTEs, models and state.tables, using the qualifier to tell raw.cards from crm.cards.

What wins when a name is known twice

What the project declares always wins. An undocumented warehouse table is used only when no CTE, model or documented source has the name. A bare from orders o uses one only if exactly one schema has a table called orders; with a schema in front there is no guessing.

The rule lives in one place: state.tables lists the documented sources before the undocumented tables, so any lookup that takes the first match gets the declared one. An undocumented table stays a WarehouseTable, and its type is what tells completion and hover to mark it not in sources.yml.

Parsing dbt SQL without rendering jinja

dbt_dialect in scope.py subclasses whichever sqlglot dialect the profile needs. A {{ ref('x') }} then parses as a table whose name is the jinja text, which resolve.py later matches with a regex.

src/dbt_ls/scope.py
class DbtDialect(base):
    class Tokenizer(base.tokenizer_class):
        # {{ ... }} becomes one raw-string token
        RAW_STRINGS = [*base.tokenizer_class.RAW_STRINGS, ("{{", "}}"), ("{{-", "}}")]
        # {% ... %} is skipped like a comment
        COMMENTS = [*base.tokenizer_class.COMMENTS, ("{%", "%}"), ("{%-", "%}")]

    class Parser(base.parser_class):
        # ...and a raw string is allowed wherever an identifier is
        ID_VAR_TOKENS = {*base.parser_class.ID_VAR_TOKENS, TokenType.RAW_STRING}

The weakness: a jinja expression that stands alone as a statement, rather than in place of a name, is a syntax error. That is exactly what a {{ config(...) }} header is, which is why neutralise_jinja blanks such calls out before parsing. See finding F1.

Surviving half-typed SQL

The tree is refreshed on open, change and save. When the text does not parse, repaired_variants tries rewrites that keep every line number intact: a placeholder column after a dangling a. at the end of a line, removal of a trailing comma, a placeholder in an empty SELECT, and a final SELECT appended after a lone WITH. If all fail, the previous tree is kept.

Why scope uses block starts, not block ends

find_block_range in alias.py treats a block as running from its own start to the start of the next block. sqlglot's end offsets stop at the last token it parsed, so a WHERE being typed would otherwise fall outside its own query. When no block covers the cursor, a single matching declaration is accepted as a fallback, so a brand-new CTE still completes.

Coordinates

Three conventions meet here, which is easy to trip over. LSP positions are 0-based lines. Alias and Position use 1-based lines and 0-based columns. sqlglot gives character offsets, with an inclusive end. serve_completions converts at the boundary, and offsets are always resolved against the text that was parsed, not the live buffer.

Hover

Hover reuses the lookups of completion and adds only the text. serve_hover in server.py finds what the cursor is on; hover.py turns it into Markdown and knows nothing about documents or positions.

1. What is under the cursorref_at, source_at or dotted_at in pattern.py return the regex match, so the hover's range is exactly the name.
2. Find the relationA ref by model name. A source as the first table in state.tables with that source and table name. An alias through declaration_at and relation_for, the same path as column completion.
3. Write it outrelation_hover or column_hover: a signature such as (model) stg_cards, a rule, where it lives, then the columns.

The columns are a SQL code block with the names padded so the types line up, not a Markdown table: Neovim draws a table as raw pipes unless a plugin renders it, while both editors colour a SQL block. At most 40 rows are shown, with the count underneath. Every hover that lands on a name is logged at INFO as HOVER @ line:col; misses on plain SQL only at DEBUG, because VS Code asks on every mouse rest.

Diagnostics

diagnose in diagnostics.py runs on open and on every change, and its result replaces what the file showed before. A closed file is cleared. The rule throughout: say nothing rather than something that might be wrong.

Warning or error

source() dbt cannot resolve

A warning when the warehouse has the table: it only needs declaring, and the message names the .yml to add it to. An error when nothing knows it. dbt-ls never writes the YAML.

Error

ref() to nothing

Checked against ref_names: model, seed and Python model files, snapshots by their tag or YAML entry, and dbt_packages. Two-argument and versioned refs are not judged.

Warning or error

Columns

alias.column the relation lacks; a bare column no relation in the query has, or two have; an alias nothing declares. Walks sqlglot's scopes, so CTEs, subqueries and correlated subqueries are covered.

When the column checks stay quiet

A check that needs a relation's columns runs only when it has all of them. all_columns on Model and SourceTable says so: true for columns from the warehouse, the catalog, a seed's header, a run result whose final select names every column, or a saved model's own SQL. YAML alone is never enough, because it often documents only some. A * is only as complete as what it selects from. relations.py works this out once per query, as a Known per relation whose columns are None when not all are known.

They also need a tree parsed from the text as it stands. A repaired or older tree would put the squiggle in the wrong place, so while the file does not parse only the ref and source checks run. Using s.field on a struct column, reusing a select alias in where, and a column joined with using are not flagged.

Columns that changed before the last build

The warehouse knows a model as it was last built, so a column just added upstream would be "not in" it downstream until dbt run. On save, columns_from_sql takes a model's columns from its own SQL with model_columns, when that names every column, keeps the warehouse's types for what passes through unchanged, and checks the open files again. A save during the first warehouse read is applied again when that read lands. A later save whose SQL no longer names every column, or does not parse, gives back the columns the model had before the first one, kept in sql_before.

Where column information comes from

A model starts as a name and a file path; a seed also has the columns of its header. Up to four sources then add columns, and two more events follow while you work. The rule in Model.merged_with is simple: a later source replaces the whole column list if it has any columns, takes its all_columns with it, and never touches the path.

SourceReadsGivesRuns
configEvery YAML file with a models: or seeds: keyDocumented model column names, no types; where a model or seed is builtAt load
catalogtarget/catalog.jsonNames and types as of the last dbt docs generate, for models, seeds and sourcesAt load
databaseThe warehouse in the active targetLive names and types for every table. Tables matching a model or documented source give it columns; the rest are kept as undocumented tablesIn the background, after load
run resultstarget/run_results.jsonNames from the compiled select list, types only for columns passed straight throughAfter each dbt run
saved SQLThe model's own fileEvery column, when its final select names them all; the warehouse's types where they pass throughOn save
seed headerThe seed's CSVEvery column name; types already found are keptAt load, and when a CSV changes
shell
# default: every source, warehouse last (so it wins)
dbt-ls --schema-sources config,catalog,database

# never opens a warehouse connection - works on every platform
dbt-ls --schema-sources config,catalog

Two consequences worth remembering. The order is only expressed by call order, there is no stored priority, so the run-results refresh always wins over the warehouse even though it knows less. Warehouse matching goes by schema, and by the alias, schema and database a model declares; the other sources still match by bare model name.

How the warehouse readers are built

Each reader in model.py has the same signature and is registered by profile type in _DATABASE_METHOD_REGISTRY. A matching dataclass in profiles.py is registered in _TARGET_REGISTRY. Adding a platform means adding one of each.

ibis family

DuckDB, Postgres, MySQL, SQL Server, Spark

Open an ibis connection. The first four then read every column of the database in one statement (_ALL_COLUMNS_SQL), so types read as the warehouse names them, such as numeric(18,2). If the statement fails, and always for Spark, _tables_one_by_one makes one request per table.

API family

Databricks, Athena, Glue, Redshift

Call the vendor's metadata API directly. No SQL engine needs to be running, except Redshift, which runs one query through the Data API and polls for the result.

Since 0.4.0 every reader returns a plain list of tables, each with its schema and, where known, its database. match_models and match_sources then decide which table belongs to which model or source, in one place for all nine platforms.

Adding Snowflake, BigQuery or Fabric now means writing a function that lists tables; the matching comes for free.

Since 0.4.1 the tables that match nothing are not dropped. undocumented_tables keeps them as the WarehouseTables the reader returned, on state.undocumented. A WarehouseTable answers to the same source_name (its schema), relation_schema and relation_name as a SourceTable, so completion, hover and CTE typing treat both alike. Readers still scan only the profile's own database, so sources in a second database, such as a Fabric lakehouse, are not found yet.