Site cache preloader: bin/php/preload.php #

Since 6 October 2026 the command runs the same crawler as Setup > Preload, with PHP's curl extension instead of wget and the curl program, one run at a time, and records every run. What follows about wget describes the earlier version. The current guide is Preloading caches.

Read this page if you clear caches on deploy and want the first visitors to get fast pages. bin/php/preload.php (also php bin/php/console exp:preload --apply, new in Exponential 6.0.15) fetches your pages before any visitor does, so the page caches are warm, and it reports every broken link with the pages that link to it.

In short #

Run it:

• after clearing all caches (exp:ezcache --clear-all);
• after a deployment or code update;
• after a scheduled cache expiry;
• from cron, to keep a busy site warm.

Prerequisites #

Run all commands from the Exponential CMS root directory:

# You should see index.php in the listing
ls index.php

You need:

Your Exponential site must be running and reachable over HTTP/HTTPS. The script reads SiteURL from settings/site.ini to know where to connect.

Quick start #

# Warm caches for the default siteaccess configured in site.ini
php bin/php/preload.php --apply

# Or run it through the console (identical result)
php bin/php/console exp:preload --apply

That is all you need for most sites.

Specifying a siteaccess #

A siteaccess is a named configuration profile in Exponential CMS (for example a front-end user site, an admin site, or a mobile site). If you have more than one you can target a specific one:

php bin/php/preload.php --apply --siteaccess=sevenx_site_user

# Same command via the console dispatcher
php bin/php/console exp:preload --apply --siteaccess=sevenx_site_user

Not sure what your siteaccess names are? Check:

grep AccessPath settings/siteaccess.ini
# or look in
ls settings/siteaccess/

What the script does — step by step #

Phase 1 — Warm section pages #

The script reads [SiteSettings] URLTranslationKeyword from site.ini to build a list of section URLs (e.g. /articles/, /products/, /blog/). Each URL is fetched once with curl. For every page it prints:

  [1/4]  https://example.com/articles/
         [ 200 OK ]  wall: 312ms  ok  size: 84.2 kB
         db        queries: 14  time: 28ms
         kernel    init: 42ms  run: 245ms  get-content: 190ms  total: 310ms
         memory    peak: 18.4 MB

• 200 OK — page fetched and cached successfully (green)
• 3xx — redirect (cyan)
• 4xx — not found or permission denied (yellow warning)
• 5xx — server error (red, first few error lines shown inline)

The wall time is how long your browser would have waited for this page. After the first fetch that number drops dramatically because the cache is warm.

Phase 2 — Spider the full site #

The script launches wget in spider mode (it visits pages but does not save them) with recursion depth 3, starting from your site root. This touches every page linked from your home page, up to three levels deep.

For every URL crawled:

  ✓  /articles/my-post                 14.2 kB  #47
  ▸  print  /articles/my-post          3.1 kB   #48
  ⬇  download  /content/download/…           #49

Final summary #

  ┌─────────────────────────────────────────┐
  │  Pages cached      204                  │
  │  Auth skipped        3 resources        │
  │  Broken links        1                  │
  └─────────────────────────────────────────┘

Reading the output #

Page speed tiers #

The wall time column tells you how fast a page responded before caching:

After the preload completes, re-fetching any of these pages should consistently show blazing or fast because the rendered HTML is now cached.

Broken links #

A broken link means one of your pages contains a hyperlink whose target does not exist or returns an error. These are worth fixing but do not prevent caching — the rest of the site is still fully warmed.

The broken link report #

Knowing that /careers is missing is only half of what an editor needs. The other half is which pages link to it, because those are the pages that have to be opened and corrected.

Every run therefore ends with a report: each missing target once, with the full address of every page that links to it.

1 broken link on 4 pages of this site.

Missing pages (404) (1)

  https://www.example.com/bold_eng/careers?nekisufiks
      linked from 4 pages:
        https://www.example.com/
        https://www.example.com/authors/tom-horvat
        https://www.example.com/authors/jane-smith
        https://www.example.com/authors/john-doe

Open each page listed under a broken link, correct the link, then run this again.

Addresses are absolute, so they can be pasted into a browser as they are. Failures are grouped by kind — Missing pages (404) separately from Server errors (500) and from No response — because they are different problems for different people.

Each line in the running log names its source as well:

/bold_eng/careers  404  linked from / and 3 other pages

At most 20 linking pages are listed per broken target, with a count of the rest. A broken link in a template or a site menu appears on every page of the site, and listing four thousand of them helps nobody — the first few name the pattern.

In Setup → Preload Sites the same report is drawn below the console as a table of clickable links, so a broken target and its pages can be opened in new tabs and fixed without transcribing anything. The plain text stays in the console too, so selecting the whole log still carries it.

Fragments are not pages #

Before this, every page of the site reported one or two broken links that did not exist:

/main        404
/debug-end   404
/fitness/main       404
/tags/view/Topics/main  404

There is no such page, and no link to fix. The crawler was cutting the fragment off an href with strtok( $href, '#' ), and strtok skips leading delimiters: given #main it returns main, not an empty string. So the skip link every page carries — <a href="#main"> — became a relative link, resolved against whatever page was being read, and was requested as a page.

Fragments are now cut with strpos/substr, which keeps the empty result, and a fragment-only href is discarded like any other empty one. This also removes a whole class of pointless requests from a bounded run: on a 250 page budget the crawler was spending a request per page on an address that could never exist.

Running on a cron job #

To keep caches continuously warm, run the preloader on a schedule. Edit your crontab:

crontab -e

Add a line like this (runs every 6 hours, logs output):

0 */6 * * * cd /path/to/exponential-root && php bin/php/preload.php --apply --siteaccess=sevenx_site_user >> /var/log/ezpreload.log 2>&1

Replace /path/to/exponential-root with the actual path to your site and sevenx_site_user with your front-end siteaccess name.

Running after a cache clear #

The most common workflow:

# 1. Clear all caches
php bin/php/console exp:ezcache --clear-all --allow-root-user

# 2. Immediately re-warm them
php bin/php/console exp:preload --apply --siteaccess=sevenx_site_user --allow-root-user

Tips #

Redirect to a log file (useful in CI/CD or when running unattended):

php bin/php/preload.php --apply --siteaccess=sevenx_site_user > var/log/preload.log 2>&1

Colors are automatically suppressed when stdout is not a terminal — the log file will contain clean plain text.

See only broken links from a previous run:

grep "BROKEN LINK" var/log/preload.log

Run just Phase 1 (section pages only, no full spider):
There is no separate flag for this yet. To skip Phase 2, interrupt the script with Ctrl+C after Phase 1 completes — Phase 1 output is flushed immediately so no work is lost.

Troubleshooting #

Cannot determine site URL: SiteSettings.SiteURL is not set
Open settings/override/site.ini.append.php (or the equivalent for your siteaccess) and confirm that [SiteSettings] contains a SiteURL line:

[SiteSettings]
SiteURL=example.com

Failed to launch wget — is it installed and on $PATH?
Install wget:

# Debian / Ubuntu
apt-get install wget

# RHEL / CentOS / Rocky
yum install wget

Running scripts as root may be dangerous
Exponential's startup system warns when you run as root. Append --allow-root-user to acknowledge:

php bin/php/preload.php --apply --allow-root-user

Phase 2 produces only a few URLs
Check that wget can actually reach your site from the server and that no firewall rule blocks outbound HTTP/HTTPS connections from the web server itself:

curl -I https://your-site.com/
wget --spider https://your-site.com/

Lots of ⊘ auth-protected lines
Pages behind HTTP Basic Auth or Exponential's login system cannot be spidered without credentials. The preloader skips them gracefully. To warm authenticated pages you would need to handle authentication separately (beyond the scope of this script).

Technical details #

• Phase 1 uses curl with -sk (silent, ignore SSL certificate errors) and captures http_code, time_total, and size_download inline. It also extracts Exponential's optional X-Debug-* HTML comments for kernel timing and database metrics if your site emits them.
• Phase 2 uses wget --spider --recursive --level=3 which visits pages without saving them. Output is read line by line with popen() so progress appears in real time rather than after the full crawl finishes.
• Binary files (images, fonts, videos, PDFs, archives) are excluded from the wget spider pass via --reject so the crawl stays focused on HTML pages.
• Auth-protected resource lines (Username/Password Authentication Failed) are collapsed into a single summary line instead of flooding the terminal.
• All PHP output is flushed immediately (flush()) so you see progress in real time even in long-running Phase 2.

Related pages #

• Preload Sites: runs in the background (28 September 2026)
• Behaviour changes, 16 to 30 September 2026
• Static cache generator
• HTTP/2 and cache warming
• Exponential Console