feat: add DuckDB, Trino, Dremio & Spark support to CI and CLI#2135

Merged

haritamar merged 98 commits intomasterfrom

devin/ELE-5266-1772368864-code-ci

Mar 3, 2026

Commits on Mar 1, 2026

feat: add DuckDB, Trino, Dremio & Spark support to CI and CLI (part 1)

devin-ai-integration[bot]
and
haritamar
committed
feat: add Docker startup steps for Trino, Dremio, Spark in CI workflow

devin-ai-integration[bot]
and
haritamar
committed
feat: add DuckDB, Trino, Dremio, Spark profile targets

devin-ai-integration[bot]
and
haritamar
committed
feat: add Trino Iceberg catalog config for CI testing

devin-ai-integration[bot]
and
haritamar
committed
feat: add Spark Hive metastore config for CI testing

devin-ai-integration[bot]
and
haritamar
committed
feat: add Dremio setup script for CI testing

devin-ai-integration[bot]
and
haritamar
committed
feat: add Trino, Dremio, Spark Docker services to docker-compose.yml

devin-ai-integration[bot]
and
haritamar
committed
fix: address DuckDB, Spark, and Dremio CI test failures

devin-ai-integration[bot]
and
haritamar
committed
fix: use Nessie catalog source for Dremio instead of plain S3

devin-ai-integration[bot]
and
haritamar
committed
feat: add seed caching for Docker-based adapters in CI

devin-ai-integration[bot]
and
haritamar
committed
fix: move seed cache restore before Docker service startup

devin-ai-integration[bot]
and
haritamar
committed
fix: add docker-compose.yml to seed cache key and fail-fast readiness loops

devin-ai-integration[bot]
and
haritamar
committed
fix: convert ClickHouse bind mount to named Docker volume for seed caching

devin-ai-integration[bot]
and
haritamar
committed
ci: temporarily use dbt-data-reliability fix branch for Trino/Spark support

devin-ai-integration[bot]
and
haritamar
committed
fix: stop Docker containers before archiving seed cache volumes

devin-ai-integration[bot]
and
haritamar
committed
fix: add readiness wait after restarting Docker containers for seed cache

devin-ai-integration[bot]
and
haritamar
committed
fix: use Trino starting:false check for proper readiness detection

devin-ai-integration[bot]
and
haritamar
committed
fix: add Hive Metastore readiness check after container restart for Trino

devin-ai-integration[bot]
and
haritamar
committed
fix: Dremio CI - batched seed materialization, single-threaded seeding, skip seed cache for Trino

devin-ai-integration[bot]
and
haritamar
committed
fix: Dremio CI - single-threaded dbt run/test, fix cross-schema seed refs, quote reserved words

devin-ai-integration[bot]
and
haritamar
committed
fix: revert reserved word quoting, use Dremio-specific expected failures in CI validation

devin-ai-integration[bot]
and
haritamar
committed
fix: rename reserved word columns (min/max/sum/one) to avoid Dremio SQL conflicts

devin-ai-integration[bot]
and
haritamar
committed
fix: always run seed step for all adapters (cloud adapters need fresh seeds after column rename)

devin-ai-integration[bot]
and
haritamar
committed
fix: Dremio generate_schema_name - use default_schema instead of root_path to avoid double NessieSource prefix

devin-ai-integration[bot]
and
haritamar
committed
fix: Dremio - put seeds in default schema to avoid cross-schema reference issues in Nessie

devin-ai-integration[bot]
and
haritamar
committed
feat: external seed loading for Dremio and Spark via MinIO/CSV instead of slow dbt seed

devin-ai-integration[bot]
and
haritamar
committed
fix: format load_seeds_external.py with black, remove unused imports

devin-ai-integration[bot]
and
haritamar
committed
fix: use --entrypoint /bin/sh for minio/mc docker container to enable shell commands

devin-ai-integration[bot]
and
haritamar
committed

Commits on Mar 2, 2026

Commits on Mar 3, 2026