The project is built when the editor sends initialized. The entry point is load_project in server.py. Since 0.4.1 it does the fast part at once and hands the warehouse read to a background thread, so the first completion does not wait for the warehouse. dbt-ls check uses the synchronous _load_project instead.
find_dbt_project_root walks up from the opened folder looking for dbt_project.yml, then searches downward, skipping target, dbt_packages and virtual environments.
Profiles.locate looks for profiles.yml where dbt does: the directory given with --profiles-dir, else DBT_PROFILES_DIR, else the project root, then ~/.dbt. A directory named by the flag or the variable is the only one searched, so the server never reads a different warehouse than dbt run. ProfileTarget.from_dict picks a dataclass by type, renders env_var(), coerces types and wraps secrets. A failure here no longer stops the load: it is recorded, shown as a warning, and the server continues without live columns.
ProjectState.__post_init__ lists every .sql under the model paths as a model, every .csv under the seed paths as a seed, and every table under a sources: key in any YAML file. Both paths default to dbt's models and seeds. It also collects ref_names, every name ref() can take, packages and snapshots included, for diagnostics.
Runs refresh_from_config and refresh_from_catalog in the order given by --schema-sources. Each one overwrites the columns of the models it knows about. The server now answers requests.
_split_at_warehouse cuts the source list at database. A daemon thread runs that source and any after it on a copy of the state, then hands the copy over on the event loop with call_soon_threadsafe. A reload bumps load_generation, so a read that finishes late is thrown away.
dialect_for_profile_type maps the profile type to a sqlglot dialect for the per-file parser. Unknown types fall back to postgres.
watched_files gets four watchers. run_results.json re-derives columns from the compiled SQL. Any .yml reloads the sources, matched against the last warehouse read rather than a new one. Any .csv reads the seeds again. A .sql or .py coming or going refreshes ref_names, and a .sql also runs refresh_models: known models keep what was found for them, a new one is matched against the last warehouse read, and a deleted one leaves its table among the undocumented. A change seen while the warehouse is still being read sets files_changed, so the late state is refreshed the same way before it is adopted. Each ends by checking the open files again.
All of it lives in src/dbt_ls. Five files hold nearly all the logic that is hard to remember: scope.py, alias.py, resolve.py, relations.py and model.py.
| Module | Lines | What it owns |
|---|---|---|
server.py | 1209 | LSP handlers, project loading in two stages and its error handling, the file watchers, serve_completions, serve_hover, relation_for, publishing diagnostics, columns from a saved model, the two commands. |
model.py | 1064 | The Model and WarehouseTable types, model discovery and ref_names, the YAML and catalog readers, all nine warehouse readers with their registry, and the matching of warehouse tables to models and sources. |
scope.py | 614 | The jinja-tolerant sqlglot dialect, the mid-edit repairs, the per-file AST cache, and turning a query into CTEs, tables and select lists. |
profiles.py | 341 | Finding and reading profiles.yml, one dataclass per adapter, helpful errors for missing or misspelled keys. |
state.py | 300 | ProjectState: the models, seeds, sources and undocumented tables shared by all handlers, tables (sources, then undocumented tables), the last warehouse read, the refresh methods, the merge rule, and the run_results.json reader. |
diagnostics.py | 254 | diagnose: undeclared sources, unknown refs, and the column checks over sqlglot's scopes. |
relations.py | 243 | What each relation in a query exposes, and whether that is every column: tables, CTEs, subqueries and *. model_columns for a saved model. |
alias.py | 181 | Regex search for alias declarations, and choosing the declaration that is in scope at the cursor. |
resolve.py | 141 | Giving CTE columns their data types by following them to the model or source they come from, and expanding *. |
hover.py | 116 | The text of a hover: a SQL signature, where the relation lives, and its columns lined up with their types. |
source.py | 105 | The SourceTable type, with the .yml that declares it, and declared(), source discovery from YAML, source columns from the catalog. |
pattern.py | 79 | The regexes that decide what the cursor is completing, and which ref, source or alias.column it is on for hover and go to definition. |
cli.py | 80 | Argument parsing, stdio or TCP transport, and handing dbt-ls check to check.py. |
check.py | 169 | The dbt-ls check command: loads a project outside an editor and prints or returns as JSON what each schema source gave. |
report.py | 53 | LoadReport: what happened during a load, per schema source, with outcome, timing and counts. |
seed.py | 52 | Seed discovery and reading a CSV header. |
editors/vscode | 231 | The VS Code client: one server process per dbt project, command routing. |
This is the most intricate path in the project. It mixes two techniques on purpose: regexes on raw text, which survive half-typed SQL, and a parsed tree, which knows where query blocks begin.
pattern.py sees c. left of the cursor and returns kind column, alias c.AstCache.get returns the last version of the file that parsed. Completion itself never parses.parse_alias_list runs three regexes over the live text: ref, source, and plain tables. Each keeps its qualifier: the source name, or the schema.choose_alias keeps the declaration inside the query block the cursor is in.resolve_ctes copies data types from models and sources onto CTE columns.relation_for finds the relation among CTEs, models and state.tables, using the qualifier to tell raw.cards from crm.cards.What the project declares always wins. An undocumented warehouse table is used only when no CTE, model or documented source has the name. A bare from orders o uses one only if exactly one schema has a table called orders; with a schema in front there is no guessing.
The rule lives in one place: state.tables lists the documented sources before the undocumented tables, so any lookup that takes the first match gets the declared one. An undocumented table stays a WarehouseTable, and its type is what tells completion and hover to mark it not in sources.yml.
dbt_dialect in scope.py subclasses whichever sqlglot dialect the profile needs. A {{ ref('x') }} then parses as a table whose name is the jinja text, which resolve.py later matches with a regex.
class DbtDialect(base):
class Tokenizer(base.tokenizer_class):
# {{ ... }} becomes one raw-string token
RAW_STRINGS = [*base.tokenizer_class.RAW_STRINGS, ("{{", "}}"), ("{{-", "}}")]
# {% ... %} is skipped like a comment
COMMENTS = [*base.tokenizer_class.COMMENTS, ("{%", "%}"), ("{%-", "%}")]
class Parser(base.parser_class):
# ...and a raw string is allowed wherever an identifier is
ID_VAR_TOKENS = {*base.parser_class.ID_VAR_TOKENS, TokenType.RAW_STRING}
The weakness: a jinja expression that stands alone as a statement, rather than in place of a name, is a syntax error. That is exactly what a {{ config(...) }} header is, which is why neutralise_jinja blanks such calls out before parsing. See finding F1.
The tree is refreshed on open, change and save. When the text does not parse, repaired_variants tries rewrites that keep every line number intact: a placeholder column after a dangling a. at the end of a line, removal of a trailing comma, a placeholder in an empty SELECT, and a final SELECT appended after a lone WITH. If all fail, the previous tree is kept.
find_block_range in alias.py treats a block as running from its own start to the start of the next block. sqlglot's end offsets stop at the last token it parsed, so a WHERE being typed would otherwise fall outside its own query. When no block covers the cursor, a single matching declaration is accepted as a fallback, so a brand-new CTE still completes.
Three conventions meet here, which is easy to trip over. LSP positions are 0-based lines. Alias and Position use 1-based lines and 0-based columns. sqlglot gives character offsets, with an inclusive end. serve_completions converts at the boundary, and offsets are always resolved against the text that was parsed, not the live buffer.
Hover reuses the lookups of completion and adds only the text. serve_hover in server.py finds what the cursor is on; hover.py turns it into Markdown and knows nothing about documents or positions.
ref_at, source_at or dotted_at in pattern.py return the regex match, so the hover's range is exactly the name.ref by model name. A source as the first table in state.tables with that source and table name. An alias through declaration_at and relation_for, the same path as column completion.relation_hover or column_hover: a signature such as (model) stg_cards, a rule, where it lives, then the columns.The columns are a SQL code block with the names padded so the types line up, not a Markdown table: Neovim draws a table as raw pipes unless a plugin renders it, while both editors colour a SQL block. At most 40 rows are shown, with the count underneath. Every hover that lands on a name is logged at INFO as HOVER @ line:col; misses on plain SQL only at DEBUG, because VS Code asks on every mouse rest.
diagnose in diagnostics.py runs on open and on every change, and its result replaces what the file showed before. A closed file is cleared. The rule throughout: say nothing rather than something that might be wrong.
source() dbt cannot resolveA warning when the warehouse has the table: it only needs declaring, and the message names the .yml to add it to. An error when nothing knows it. dbt-ls never writes the YAML.
ref() to nothingChecked against ref_names: model, seed and Python model files, snapshots by their tag or YAML entry, and dbt_packages. Two-argument and versioned refs are not judged.
alias.column the relation lacks; a bare column no relation in the query has, or two have; an alias nothing declares. Walks sqlglot's scopes, so CTEs, subqueries and correlated subqueries are covered.
A check that needs a relation's columns runs only when it has all of them. all_columns on Model and SourceTable says so: true for columns from the warehouse, the catalog, a seed's header, a run result whose final select names every column, or a saved model's own SQL. YAML alone is never enough, because it often documents only some. A * is only as complete as what it selects from. relations.py works this out once per query, as a Known per relation whose columns are None when not all are known.
They also need a tree parsed from the text as it stands. A repaired or older tree would put the squiggle in the wrong place, so while the file does not parse only the ref and source checks run. Using s.field on a struct column, reusing a select alias in where, and a column joined with using are not flagged.
The warehouse knows a model as it was last built, so a column just added upstream would be "not in" it downstream until dbt run. On save, columns_from_sql takes a model's columns from its own SQL with model_columns, when that names every column, keeps the warehouse's types for what passes through unchanged, and checks the open files again. A save during the first warehouse read is applied again when that read lands. A later save whose SQL no longer names every column, or does not parse, gives back the columns the model had before the first one, kept in sql_before.
A model starts as a name and a file path; a seed also has the columns of its header. Up to four sources then add columns, and two more events follow while you work. The rule in Model.merged_with is simple: a later source replaces the whole column list if it has any columns, takes its all_columns with it, and never touches the path.
| Source | Reads | Gives | Runs |
|---|---|---|---|
config | Every YAML file with a models: or seeds: key | Documented model column names, no types; where a model or seed is built | At load |
catalog | target/catalog.json | Names and types as of the last dbt docs generate, for models, seeds and sources | At load |
database | The warehouse in the active target | Live names and types for every table. Tables matching a model or documented source give it columns; the rest are kept as undocumented tables | In the background, after load |
| run results | target/run_results.json | Names from the compiled select list, types only for columns passed straight through | After each dbt run |
| saved SQL | The model's own file | Every column, when its final select names them all; the warehouse's types where they pass through | On save |
| seed header | The seed's CSV | Every column name; types already found are kept | At load, and when a CSV changes |
# default: every source, warehouse last (so it wins)
dbt-ls --schema-sources config,catalog,database
# never opens a warehouse connection - works on every platform
dbt-ls --schema-sources config,catalog
Two consequences worth remembering. The order is only expressed by call order, there is no stored priority, so the run-results refresh always wins over the warehouse even though it knows less. Warehouse matching goes by schema, and by the alias, schema and database a model declares; the other sources still match by bare model name.
Each reader in model.py has the same signature and is registered by profile type in _DATABASE_METHOD_REGISTRY. A matching dataclass in profiles.py is registered in _TARGET_REGISTRY. Adding a platform means adding one of each.
Open an ibis connection. The first four then read every column of the database in one statement (_ALL_COLUMNS_SQL), so types read as the warehouse names them, such as numeric(18,2). If the statement fails, and always for Spark, _tables_one_by_one makes one request per table.
Call the vendor's metadata API directly. No SQL engine needs to be running, except Redshift, which runs one query through the Data API and polls for the result.
Since 0.4.0 every reader returns a plain list of tables, each with its schema and, where known, its database. match_models and match_sources then decide which table belongs to which model or source, in one place for all nine platforms.
Adding Snowflake, BigQuery or Fabric now means writing a function that lists tables; the matching comes for free.
Since 0.4.1 the tables that match nothing are not dropped. undocumented_tables keeps them as the WarehouseTables the reader returned, on state.undocumented. A WarehouseTable answers to the same source_name (its schema), relation_schema and relation_name as a SourceTable, so completion, hover and CTE typing treat both alike. Readers still scan only the profile's own database, so sources in a second database, such as a Fabric lakehouse, are not found yet.