CLI Reference¶
fdl provides 8 commands with a Git-like interface.
Commands¶
| Command | Description | Git analogy |
|---|---|---|
init |
Initialize a new project | git init |
pull |
Download catalog from remote | git pull |
push |
Upload catalog to remote | git push |
metadata |
Generate metadata from dbt artifacts | — |
run |
Run a command with injected env vars | — |
gc |
Clean up orphaned data files | git gc |
config |
Get or set configuration | git config |
serve |
Start an HTTP server | — |
init¶
Initialize a new project. Creates fdl.toml, the .fdl/ directory, and a DuckLake catalog.
| Argument / Option | Description |
|---|---|
NAME |
Datasource name (required) |
--sqlite |
Use SQLite catalog (required for dlt integration) |
Generated files:
fdl.toml— Project config.fdl/ducklake.duckdb— DuckLake catalog (or.fdl/ducklake.sqlite).gitignore—.fdl/entry added
On failure, fdl.toml and .fdl/ are automatically rolled back.
pull¶
Download a catalog from remote.
| Argument | Description |
|---|---|
SOURCE |
Remote name (e.g. origin) |
Requires prior initialization with fdl init.
push¶
Upload build artifacts to remote.
| Argument | Description |
|---|---|
DEST |
Remote name (e.g. origin, local) |
Uploaded files:
ducklake.duckdb— DuckLake catalogducklake.duckdb.files/— Data files (Parquet, etc.)metadata.json— Metadatadocs/— dbt documentation
SQLite catalogs are automatically converted to DuckDB during push.
metadata¶
Generate metadata from dbt artifacts.
| Argument | Description | Default |
|---|---|---|
TARGET_DIR |
dbt target directory path | target |
Parses manifest.json and catalog.json to generate .fdl/metadata.json.
dbt documentation (docs/) is also copied to .fdl/docs/.
Requires dataset.yml in the project root.
run¶
Run a command with fdl environment variables injected.
| Argument | Description |
|---|---|
REMOTE |
Remote name (omit for local .fdl/) |
COMMAND |
Command to execute |
Injected environment variables:
| Variable | Description |
|---|---|
FDL_STORAGE |
Base storage path |
FDL_DATA_PATH |
Data files path |
FDL_ATTACH_PATH |
DuckLake attach path (for dbt) |
FDL_S3_ENDPOINT |
S3 endpoint |
FDL_S3_ACCESS_KEY_ID |
S3 access key |
FDL_S3_SECRET_ACCESS_KEY |
S3 secret key |
FDL_S3_ENDPOINT_HOST |
S3 endpoint without scheme |
Existing environment variables are never overwritten.
Examples:
# Use local .fdl/ for dbt run
fdl run -- dbt run
# Use remote storage for dbt run
fdl run origin -- dbt run
gc¶
Clean up orphaned data files on remote storage. DuckLake tracks data files by snapshot; files no longer referenced by any active snapshot are considered orphaned.
| Argument / Option | Description |
|---|---|
REMOTE |
Remote name (e.g. origin). S3 remotes only. |
--dry-run, -n |
List orphaned files and sizes without deleting |
--force, -f |
Skip confirmation prompt |
--older-than DAYS |
Only target files older than N days |
The command performs two cleanup steps:
- Runs DuckLake's
ducklake_cleanup_old_files()for files scheduled for deletion by snapshot expiration - Scans remote storage for files not in the active set (
end_snapshot IS NULL) and deletes them
Deletion is irreversible. Always use --dry-run first to review the file list.
Examples:
# Preview orphaned files and total size
fdl gc origin --dry-run
# Delete with confirmation prompt
fdl gc origin
# Delete files older than 7 days, no prompt
fdl gc origin --older-than 7 --force
config¶
Get or set configuration. Works like git config.
| Argument / Option | Description |
|---|---|
KEY |
Config key in section.name format (e.g. s3.endpoint) |
VALUE |
Value to set (omit to display current value) |
--list, -l |
List all settings |
--local |
Write to project config (fdl.toml) |
Default write target is ~/.fdl/config (user level).
With --local, writes to .fdl/config (workspace level).
Examples:
# Register a remote
fdl config --local remotes.origin s3://my-bucket
# Set S3 credentials
fdl config s3.endpoint https://r2.cloudflarestorage.com
fdl config s3.access_key_id YOUR_KEY
fdl config s3.secret_access_key YOUR_SECRET
# List all settings
fdl config --list
See Configuration for details on config resolution order.
serve¶
Start an HTTP server with CORS and Range request support.
| Argument / Option | Description | Default |
|---|---|---|
REMOTE |
Remote name (omit for project's .fdl/) |
— |
--port |
Port number | 4001 |
Examples: