Skip to content

Data migrations

Runs one named data migration that ships with a release.

How it works

Some changes can’t be expressed in SQL: rewriting event payloads to a new binary format, normalising legacy data, recomputing derived fields. Each such change ships as a coded migration — a task — in the application.

You run one task at a time, by name. There is deliberately no “run everything” mode: sweeping every migration at once is unbounded and crosses every workspace, and it used to be reachable simply by leaving a form field blank. The Task field is required, and its dropdown lists exactly the tasks the deployed version registers.

A run then:

  1. Previews by default — with Apply off, the task reports what it would change without writing anything. Read the preview first; that is what it is for
  2. Writes only when you say so — switch Apply on to save the changes. The form makes you tick I understand this writes changes that cannot be rolled back before it will submit, because applying writes across every workspace in scope and cannot be rolled back
  3. Optionally rebuilds projections — a task whose changes the projections need to pick up asks for that as a follow-up. After the run completes, Rebuild employee projections is scheduled as a new run. It appears in the job runs list as a separate entry, started only after the migrations run has completed

When it runs

The job is operator-driven. It never starts on its own — not on a schedule, and not when the application starts. Trigger it when a migration is needed, from Admin › Job Runs. It can only be run from the system workspace and is not listed in any other workspace.

It does not start automatically because every running copy of the application would then queue its own run, and several runs changing the same data at once slow the database down.

Parameters

ParameterRequiredMeaning
TaskYesThe migration to run, picked from the registered tasks. A run that names no task is rejected before any work starts, as is a name that isn’t registered
Apply (default false = dry-run for migrations that support it)NoOff (the default) previews without writing, for tasks that support a dry run; tasks that always apply ignore it. On saves the changes, which cannot be undone
Employee (optional, narrow a scoped migration to one employee)NoNarrows a task to a single employee — useful to try it on one employee before running it for everyone. Tasks that always cover everything ignore it

The programmatic API accepts the same options, plus workspace_id to narrow a scoped task to one workspace; there is no workspace picker in the trigger form.

{"jobs":["migrations"],"options":{"tasks":["<task-name>"]}}
{"jobs":["migrations"],"options":{"tasks":["<task-name>"],"apply":true}}

The options a run was given are shown in the run list’s Parameters column, which is hidden by default — switch it on from the column menu to tell two runs of the same job apart.

Job results

The result map carries dry_run (the inverse of Apply) and one key per task that ran:

KeyMeaning
dry_runtrue when nothing was written — the run only previewed
appliedWhether this task actually wrote its changes
affected_countHow many entities the task touched, or would touch
by_workspacePer-workspace counts, with workspace names
detailsThe affected entities, with whatever before/after detail the task records

A name that isn’t a registered task appears as "unknown task" against that name.

When a task marked “rebuilds projections” runs, the result also includes next_jobs: ["rebuild-employee-projections"] and that job is scheduled automatically.

Re-running

Every registered task is re-runnable on demand. Each task leaves out data that has already been migrated, so re-running after a partial success picks up where the previous run left off, and re-running a completed task changes nothing and reports nothing affected.

Troubleshooting

IssueSolution
“No task selected”Pick a task in the Task field. There is no run-everything mode — naming the task is the only way to run this job
The run reports "unknown task"The name isn’t registered in the deployed version — most often a task that has since been retired. Pick from the dropdown rather than typing a name
Run Job stays disabledApply is on and the acknowledgement below it hasn’t been ticked. Read the warning and tick the box, or switch Apply back off to preview instead
affected_count is 0The task has no pending rows — the normal state once it has completed
Errors during a taskOpen the run details. Per-row errors are surfaced; failing the whole run usually means a fatal condition. Fix the data and re-run
Projection-rebuild follow-up did not runIf the run did not complete (it failed part-way), the follow-up is not scheduled. Fix the failure, re-run the task, then re-run Rebuild employee projections manually if needed