Commands Cheatsheet¶

Taskfile.yaml only keeps targets that compose multiple commands, inject environment variables, or encode a non-obvious default. Everything else is a direct call into a third-party CLI — listed below so a forker can run the underlying command without learning a Task target that merely renamed it.

All uv run commands assume you are at the repo root.

Linting + formatting¶

uv run ruff check .                  # lint
uv run ruff check . --fix            # lint + auto-fix
uv run ruff format .                 # format in place
uv run ruff format --check .         # format check (no write)
uv run mypy . --ignore-missing-imports

task ci composes all four gates plus pytest + secret scan.

Testing¶

uv run pytest                        # full suite
uv run pytest -m unit                # unit tests only
uv run pytest -m integration         # integration tests
uv run pytest -m e2e                 # end-to-end tests
uv run pytest --cov=. --cov-report=html

Test markers are defined in pyproject.toml under [tool.pytest.ini_options].

SQLMesh¶

Run from transforms/main/ — SQLMesh picks up config.py there. Logs land in .logs/sqlmesh/ via the local transforms/main/logs symlink (gitignored; created by task install).

cd transforms/main
uv run sqlmesh plan --auto-apply     # plan + apply changes
uv run sqlmesh run                   # run scheduled models
uv run sqlmesh test                  # run SQLMesh unit tests
uv run sqlmesh ui                    # start SQLMesh UI
uv run sqlmesh plan dev              # plan into dev env

Source refresh¶

Start the local Polaris stack and configure the Polaris/AWS values from .env.example before running either command:

docker compose --env-file .env -f compose.iceberg.yml up -d
task full-refresh   # all routine sources → Iceberg → SQLMesh → Soda
task verify         # bounded DATABOX_SMOKE=1 run through the same chain

The underlying entrypoint is scripts/sources/load_dlt_iceberg.py. It validates catalog/storage configuration before launching concurrent Dagster source jobs, checks authoritative Iceberg tables and _dlt_load_status, then runs project-wide SQLMesh and every Soda contract. Source or SQLMesh failures stop the chain before quality verification.

For a real integration check without running a durable full refresh, a maintainer can manually dispatch .github/workflows/polaris-iceberg-integration.yaml through the protected polaris-iceberg-integration environment. Its six jobs use GitHub OIDC and isolated integration/ prefixes; ordinary CI never runs it. See CI routing.

Dagster (beyond `dagster:dev` / `full-refresh` / `verify`)¶

export DAGSTER_HOME="$PWD/.dagster"
export PYTHONPATH="$PWD"

uv run dagster asset materialize --select <key> \
  -f packages/databox/databox/orchestration/definitions.py

uv run dagster asset wipe --all \
  -f packages/databox/databox/orchestration/definitions.py

Pre-commit¶

uv run pre-commit run --all-files    # run every hook across every file
uv run pre-commit run ruff-check     # run one hook

Secret scan¶

python scripts/platform/check_secrets.py      # scan eligible tracked files from repo root
python scripts/platform/check_secrets.py path/to/file.py

Source layout + staging codegen¶

python scripts/sources/check_source_layout.py        # lint per-source directory layout
python scripts/analytics/generate_staging.py           # regenerate trivial-rename stg_* SQL
python scripts/analytics/generate_staging.py --check   # fail on drift (also runs in task ci)

Watching¶

Task's built-in watch mode works without a dedicated target:

task -w ci                           # re-run ci on file change