Maintenance

A DFS cluster does not remove files at once. When content is removed, or a cache expires, the file is only marked as expired in the metadata table, so that other servers still reading it are not disturbed. Expired files have to be purged, and the mount and the table checked against each other from time to time.

Purging expired files

# list what would be purged (the default is a dry run)
./console exp:clusterpurge --scopes=image,binaryfile,media --allow-root-user
# purge files that expired more than 30 days ago
./console exp:clusterpurge --scopes=image,binaryfile,media --expiry=30 --apply --allow-root-user

Options: --scopes (comma separated: classattridentifiers, classidentifiers, content, expirycache, statelimitations, user-info-cache, viewcache, wildcard-cache-index, image, binaryfile, media), --expiry=<days> (default 30, minimum 1), --iteration-limit (files per batch, default 100), --iteration-sleep (seconds between batches, default 1) and --memory-monitoring (logs to var/log/clusterpurge.log).

The cronjob part

The cronjob part cluster_maintenance runs the same purge for all scopes, for files expired more than 30 days ago. It is not in a default group; schedule it on one server:

php runcronjobs.php -s <siteaccess> cluster_maintenance
# or
./console cron:cluster_maintenance --allow-root-user

cronjob.ini [AdminSettings] ForbiddenParts[] keeps it off the buttons of Setup > Cronjobs, because it removes files for good. On a site that is not clustered the part says that the handler needs no purge and does nothing.

Checking the mount against the table

# files on the mount that the table does not know
./console exp:dfscleanup -S --allow-root-user
# rows in the table whose file is missing on the mount
./console exp:dfscleanup -B --allow-root-user
# then, after reading the result: delete what does not exist
./console exp:dfscleanup -S -B -D --allow-root-user

--path limits the check (default var/), --iteration-limit the batch size. Run -S and -B after an incident and before a migration.

Related pages

• Clustering
• Cronjobs