Pipelines

Finalize a Pipeline

Finalizing turns your mappings into real tables and columns in the destination schema inside Kenseme. After that you generate the load code from the destination schema.

In the app: Data Model › Pipelines › your Pipeline › Finalize

[screenshot]
The Finalize Pipeline dialog with the table and column counts and the information notice.

Before you start

  • Run Validate in the mapping editor and fix every error. Finalize stays disabled until a validation passes. See Validate a Pipeline.
  • Your organization must not be read-only.

Steps

  1. In the mapping editor, click Finalize.
  2. Read the Finalize Pipeline dialog. It lists what will be created in the destination schema:
    • N tables with N columns
    • N materialized views (converted to tables), when you map source views
    • Lineage records linking source to destination
  3. Click Finalize.
  4. The dialog shows Creating schema objects... while the work runs in the background.
  5. When it finishes, you see Pipeline finalized — N table(s) created, N updated. and Kenseme opens the destination schema.

If it fails, the dialog shows Finalization failed. with the reason. Kenseme validates again before it builds anything, so a Pipeline with errors cannot be half-finalized: either every change lands or none does.

What gets created or updated

  • Tables and columns for every included mapping, using your destination names, data types, and key flags.
  • Surrogate keys on tables where SK is ticked. When a surrogate key is added, it becomes the key and the mapped key columns are no longer marked as the primary key.
  • Audit columns that the table’s load strategy needs, such as LoadDate, RowHash, or the SCD Type 2 history columns. See Pipeline load strategies.
  • A run-log table named zz_<destination schema>_Jobs in the etl schema. The generated loads record each run there.
  • The Pipeline status changes to Finalized.

What finalizing does not do

  • It does not generate the load procedures or functions. Do that next, from the destination schema’s Pipeline tab.
  • It does not create anything in your database, move data, or change the source schemas.

Note: The dialog says it plainly: “This does not create anything in your actual database — it creates the schema definition in Kenseme.”

Finalize again after changes

You can finalize as many times as you need. Edits in the mapping editor put the Pipeline back to Mapping In Progress; validate and finalize again to apply them.

On a repeat finalize, Kenseme updates what exists instead of starting over:

  • Existing tables and columns are updated in place. Renamed tables keep their identity.
  • New tables and columns are added.
  • Tables and columns you have since excluded are removed from the destination schema.
  • Audit columns for a strategy you moved away from are removed, and the new strategy’s columns are added.
  • Turning SK on adds the surrogate key; turning it off removes it.

Check the result

  1. In the destination schema, open the Tables tab. The new tables are there.
  2. Open a table. It has your mapped columns plus the strategy’s audit columns.
  3. Create a version of the destination schema if you want a snapshot to compare against later. See How versioning works.