Tag: DuckDB
-
CI/CD Data Testing with dbt and DuckDB: Pre-Flight Assertions for Analytics Pipelines
Testing dbt models in continuous integration typically involves triggering a staging run on a remote cloud data warehouse such as Snowflake, BigQuery, or Databricks. While thorough, this approach introduces operational friction: warehouse spin-up delays, queueing, credit consumption, and multi-minute pipeline runs for even minor SQL modifications. By using the dbt-duckdb adapter, analytics teams can execute…
-
Serverless Analytics & Vector Search on Kubernetes: Building a Lean Data Warehouse with DuckDB, Object Storage, and PostgreSQL / pgvector
Cloud data warehouses often present teams with a difficult trade-off: either accept unpredictable SaaS compute bills or take on the operational overhead of running heavy distributed query engines like Trino or Spark. Furthermore, modern analytical platforms increasingly require bridging structured analytical files with operational relational records and vector embeddings for retrieval-augmented generation (RAG) and semantic…