Migrations in client projects
This page is for agencies and integrators who build Exponential sites for clients and keep them running. Since 6.0.15 the changes you make to a client's database, content and settings can travel as migrations: written once, tested on your development copy, applied in the same order on the client's staging and production, and recorded in git where the client can see them.
What changes for your projects
| Before 6.0.15 | With migrations |
|---|---|
| a list of steps in a ticket, clicked through on each environment | a file per change, in the project's repository |
| "did we run that on production?" | exp:migration:status on each environment, and migrations/status/<environment>.json in git |
| an update script that someone ran twice | the record: a migration runs once per installation |
| a backup taken by hand, maybe | a caption before every run, restored after a failure |
| changes repeated for every client by hand | migrations in your own extension, applied on every client site that activates it |
Two places for a migration
| Where | Source | Namespace from --skeleton | Use for |
|---|---|---|---|
the site's migrations/ | site | Site\Migrations | changes for this one client |
extension/<name>/migrations/ of your extension | extension:<name> | Extension\<Name>\Migrations | changes every client of that extension gets |
./console exp:migration:generate --skeleton news_archive_folder --apply
./console exp:migration:generate --skeleton widget_table --source=extension:acmewidgets --applyAn extension's migrations are found only while the extension is active. A site migration that builds on one names it
in its header: requires: 20261101090000_widget_table or requires: extension:acmewidgets. When the change is one
every installation of the extension needs on upgrade, whether or not a site chose it, ship an upgrade step instead
(Upgrade step or migration?).
Environments per client
Each installation names its environment in site.ini [AdminEnvironment] Level (development, test, staging,
production, or auto with EXP_ENV). A migration header can limit a migration:
// environments: staging, production
// environments: !developmentSeed data for your development copies, test users, demo content: give those migrations environments: development.
Per-environment migrations has the details.
Imports keyed by remote id
Client sites differ in their object and node ids. Find everything by remote id or identifier, and give everything a
migration creates a remote id of your own (acme-news-folder). The same rule makes imports repeatable:
Idempotent imports keyed by remote id. A migration written this way runs
on every client's installation and can meet a half-done state after a failure.
Handover
After every run the status file and the run log are committed, tagged migrations/<environment>/<run-id> and pushed.
At handover the client gets, in the repository they own:
migrations/: every change, with its description and author in the header;migrations/status/<environment>.json: what each environment has applied;migrations/runs/: the log of every run;- the tags: one per run, never moved.
Set [RepositorySettings] Remote to the client's repository, and give the system user of each installation the push
credentials (a deploy key). Hand over the policies: who has migration/run and migration/rollback in the client's
administration (Policies, views and events).
Estimating
| Work | What to count |
|---|---|
| writing | one migration per change; schema and content apart; plan(), up(), detect(), down() |
| testing | exp:migration:check in CI; a simulation on SQLite or PostgreSQL; a test of up(), down(), up() |
| applying | a dry run and a run per environment; on MySQL/MariaDB the time of a caption (a database dump) |
| large changes | batches and resume for many objects (Large data migrations) |
Client-project checklist
- To do: The record tables exist on every environment (
6.0.15-migration-tablesis done inexp:upgrade:status). - To do: Each installation names its environment;
RunAsUseris an existing administrator login. - To do: git_manager is active and the remote takes pushes from each installation;
migrationpushis scheduled. - To do: CI runs
./console exp:migration:checkon every pull request. - To do: Every migration finds things by remote id or identifier, and says how it is taken back.
- To do: Policies: only the people who should run migrations on production have
migration/run. - To do: Disk space for captions is agreed with the host.
Go-live
- Freeze the migrations that go live;
exp:migration:checkpasses. - On staging, restored from a copy of production:
exp:migration:migrate, then--apply; check the site. - On production: deploy the code,
./console exp:migration:migrate --apply, deploy the caches (Deploying with migrations). ./console exp:migration:status --fail-on-pendingexits 0; the push went through.- Add the run's tag to the release notes for the client.
Moving a client's whole site from another system or another host is not a migration in this sense: Migrations versus migrating a site.
Where to go next
Related pages
- Migrations: the hub
- Migrations: the guide
- Migrations: the tutorial
- PHP migrations
- Migrations in the administration
- The migration commands
- The migration settings
- The migration file format
- When a migration fails
- Troubleshooting migrations
- Migrations versus migrating a site
- Integrators: start here
- Building an integration as an extension, Handover of an integration, Estimating integration work, Integration go-live checklist
- Free documentation: this page is free to copy, translate and improve under the GNU FDL 1.3.