Skip to main content
Pipeline creation starts with a small form and continues in the builder.

​
Create the shell

  1. Open Data Pipelines.
  2. Click New Pipeline.
  3. Enter a pipeline name.
  4. Choose a PostgreSQL source connection.
  5. Create the pipeline and open the builder.
The shell does not ask for destination tables or SQL up front. Those decisions live in the builder.

​
Configure the Source node

  1. Click the settings icon on the Source node.
  2. Click Test connection if you want to re-check credentials.
  3. Click Discover schema or Refresh tables.
  4. Include one or more source streams.
  5. Preview the active stream.
Start with one table for the first run.

​
Configure the Destination node

  1. Click the settings icon on the Destination node.
  2. Select the PostgreSQL destination connection.
  3. Keep the first run as FULL_TABLE.
  4. Keep the write mode as Upsert.
  5. Open the SQL Layer tab.
  6. Enter the final delivery table as schema.table.
  7. Write the SQL model against the raw staged table.
  8. Validate SQL and preview output.
  9. Save the destination.

​
Run manually

Run manually from the Destination node after saving. Use run status and run history to verify row counts, failures, warnings, and duration.

​
Add automation later

Scheduling is available for Incremental destinations. Provide a replication key and validate a manual Incremental run before adding a recurring schedule.