Data migrations
Runs Go-based data migrations that ship with a release.
How it works
Some changes can’t be expressed in SQL: rewriting event payloads to a new binary format, normalising legacy data, recomputing derived fields. Each such change ships as a coded migration in the application. This job runs every registered migration, one after another:
- Asks each migration if there is work left — every migration provides a cheap “pending?” check
- Runs the pending ones — each migration scans its candidate rows and rewrites them. Reads tolerate the pre-migration state, so the deploy and the migration do not have to be in lock-step
- Optionally rebuilds projections — a migration that touches event payloads in a way projections need to re-observe declares that as a follow-up. After the migrations batch completes, the worker schedules Rebuild employee projections as a fresh run. The chained run appears in the admin UI as a separate entry started strictly after the migrations run completes
When it runs
The job is operator-driven. It is registered with the worker but never self-triggers — the API will not auto-run it on startup. Trigger it once per deploy when a migration is needed, from .
Auto-triggering was disabled because autoscaled API replicas would each queue a duplicate run, and concurrent runners against the same rows produce database contention and a flood of replay events.
Parameters
This job has no parameters. The set of migrations to run is fixed by the deployed application version.
Job results
The result map contains one key per registered migration:
| Value | Meaning |
|---|---|
"skipped" (string) | The migration’s “pending?” check returned false — no work to do |
| Nested map | The migration ran; the map contains its own counters (e.g. rows processed, errors) |
When at least one migration flagged “rebuilds projections” runs, the result also includes next_jobs: ["rebuild-employee-projections"] and the worker auto-schedules that job.
Re-running
Migrations are idempotent: each migration’s candidate query filters out rows that have already been migrated. Re-running the job after a partial success picks up where the previous run left off.
Troubleshooting
| Issue | Solution |
|---|---|
| Migration result says “skipped” | The migration has no pending rows — that’s the normal state once it has completed |
| Errors during a migration | Open the run details. Per-row errors are surfaced; failing the whole run usually means a fatal condition. Fix the data and re-run |
| Projection-rebuild follow-up did not run | If the migrations run did not enter the “completed” state (failed mid-way), the follow-up dispatcher is skipped. Fix the failure, re-run migrations, then re-run Rebuild employee projections manually if needed |