Skip to content

Welcome to dagster-odp

dagster-odp (open data platform) is a Python library built on top of the Dagster orchestrator, designed to simplify the creation and management of data pipelines. With ODP, data teams can build robust pipelines using JSON/YAML configuration instead of writing extensive code.

While Dagster provides powerful orchestration capabilities, ODP takes it a step further by offering a more accessible way to define pipelines. This approach reduces the learning curve, promotes standardization, and enables faster development of data workflows.

Who is this for?

dagster-odp is ideal for data teams looking to streamline their pipeline development process, including:

  • Data Engineers seeking to standardize pipeline creation and reduce boilerplate code
  • Data Analysts and Scientists who want to create pipelines without extensive coding
  • Organizations aiming to improve collaboration between technical and non-technical team members
  • Teams using or planning to use tools like DLT, DBT, or Soda who want simplified integration

Getting Started

If you're new to dagster-odp, we recommend starting with the following resources:

  • Overview: Learn about the core concepts and benefits of dagster-odp
  • Quickstart: Set up your first ODP pipeline in minutes

Dive Deeper

Once you're familiar with the basics, explore these areas to make the most of dagster-odp:

  • Tutorials: Practical guides to help you build progressively complex pipelines and understand ODP's features in real-world scenarios.
  • Concepts: Deepen your understanding of dagster-odp's fundamental ideas including tasks, resources, and configuration.
  • Integrations: Learn about ODP's enhanced integrations with DLT, DBT, and Soda.
  • Reference: Comprehensive details on all features and configurations.