Migrations on hosted sites #

This page is for hosting companies and database administrators who run Exponential 6.0.15 sites for customers. The sites' developers ship changes as migrations: files the site applies with ./console exp:migration:migrate --apply or from Setup > Migrations. Here is what such a run does on your servers, what it needs from them and what to watch.

What a run does #

Disk space: the captions #

Captions are the backups of runs and upgrades. With git_manager installed they are git_manager's full captions in its backup folder; without it, the dump of exp:upgrade in var/backup/upgrade/ (or migration.ini [CaptionSettings] BackupDirectory), without storage. A run takes up to three (pre, post and, after a failure, failure). Reuse (ReuseUnchanged=enabled) keeps a quiet site from adding one per run, but a site that runs often still grows the folder.

• Plan the space like a database backup of the site times the captions you keep.
• List them with ./console exp:upgrade:captions (or exp:migration:caption --list). exp:upgrade:captions --prune keeps what upgrade.ini [UpgradeSettings] CaptionKeep (10), CaptionKeepDays (90) and CaptionKeepUpgrades say; git_manager's own captions keep to its MaxBackups.
• Keep the folder on the same backup schedule as the rest of var/, or exclude it deliberately: it is a second copy.

Engines #

On MySQL/MariaDB, Oracle and MongoDB the restore of the caption is what really takes a failed run back. Make sure the dump tools of the engine (mysqldump, pg_dump, mongodump) are installed for the PHP command line user.

Background runs from the administration #

A run, a resume or a rollback started in Setup > Migrations runs as a console process of its own, as the signed-in user, and goes on when the browser is closed. It needs:

• proc_open allowed in the PHP that serves the administration (check disable_functions);
• a PHP command line binary: migration.ini [AdminSettings] PhpCliPath (empty: cronjob.ini [AdminSettings] PhpCliPath, else the usual places such as /usr/bin/php; never the php-fpm binary);
• write access to var/<site>/migration/jobs/, where each run keeps its progress file and output (JobHistory, default 50 runs).

Without them the page says so, and the run has to be started on the console. The process runs as the PHP-FPM user of the site, like every other write to var/.

Git: credentials and pushes #

The record is pushed with the credentials the system user has for the remote (an SSH key, a credential helper); nothing is stored by the migrations. A push that fails does not fail the run: the commit stays local, the status warns 1 run(s) wait for their push, and the cronjob part migrationpush pushes it later. Add that part to a cronjob group the site runs, or schedule it on its own:

php runcronjobs.php migrationpush

Logs, audit and monitoring #

For monitoring, ask each installation after a deploy:

./console exp:migration:status --fail-on-pending --format=json > var/log/migration-status.json
echo $?    # 0 nothing pending, 1 pending or failed, 3 database unreachable

The JSON (exp-migration/1) carries summary, lock (who holds it), pushes (runs waiting for their push) and orphans. Alert on exit 1 after a deploy window, on pushes that wait for more than a day, and on a lock held for hours.

What to watch #

• A run that holds the lock for a long time: exp:migration:status names the holder; a lock left by a dead process is taken over by the next run, nothing needs deleting.
• A failed run: the site was restored automatically; the developer fixes the migration and resumes it.
• Velocity or PHP-FPM not seeing a change a migration made in settings or classes: deploy as for any code change.
• Disk: the caption folder.

Migrations are not moving a site #

A migration changes one installation in place. Moving a customer's site to your servers, between hosts or to another database engine is a different job, with its own pages: Migrations versus migrating a site and Migrating customers: overview.

Where to go next #

• Migrations for administrators
• Deploying with migrations
• Engine notes and captions and rollback in practice
• Start now: the tutorial

Related pages #

• Migrations: the hub
• Migrations: the guide
• Migrations: the tutorial
• PHP migrations
• Migrations in the administration
• The migration commands
• The migration settings
• The migration file format
• When a migration fails
• Troubleshooting migrations
• Migrations versus migrating a site
• Hosting companies: start here
• Database administrators: start here
• Deploying changes, Database backups per engine, Maintenance mode, Cron jobs
• The cronjob part migrationpush
• Free documentation: this page is free to copy, translate and improve under the GNU FDL 1.3.