Storage and bandwidth #

This page is for producers and site owners who plan the hosting of a show, and for the hosts they talk to. Audio and video are the largest files a site carries, and a popular feed makes podcast apps download every new episode many times. Plan both disk and traffic before the first episode.

Contents #

1. Where files are stored
2. Versions keep files
3. How downloads are served
4. Estimating disk and traffic
5. Backups
6. What to agree with your host

Where files are stored #

Media and file fields store uploads below the site's storage directory, in original/<mime group>: audio in original/audio, video in original/video (kernel/classes/datatypes/ezmedia/ezmediatype.php). The storage directory is var/storage/ or var/<site>/storage/, depending on the installation's settings (Storage layout). On a cluster with the DFS file handler, the files live on the shared storage instead (Clustering and shared storage).

Versions keep files #

Every edit of an item makes a new version, and older versions are kept up to settings/content.ini [VersionManagement] DefaultVersionHistoryLimit (45 as shipped; VersionHistoryClass[] can set it per class). When you replace an episode's audio, the old file stays on disk as long as an old version still names it; the media datatype deletes a file only when no version or copy refers to it any more (deleteStoredObjectAttribute()).

For a show that replaces audio often, count that in, or ask the administrator for a lower version limit for the episode class. bin/php/cleanupversions.php removes versions beyond a lowered limit (comment in settings/content.ini).

How downloads are served #

Players and podcast apps fetch files from /content/download/.... The kernel checks that the visitor may read the item, then the PHP passthrough handler sends the file (kernel/classes/binaryhandlers/ezfilepassthrough/ezfilepassthroughhandler.php):

• every download occupies a PHP worker for as long as the transfer runs; a slow listener on a large file keeps it busy;
• Range requests are answered with 206 Partial Content, so players seek and apps resume instead of starting over;
• the answer carries Expires ten minutes ahead and an empty Cache-Control (eZFile::downloadHeaders() in lib/ezfile/classes/ezfile.php), so browsers and proxies may keep it only briefly.

The number of PHP workers therefore limits how many downloads run at the same moment; size the pool with that in mind (PHP-FPM tuning).

Estimating disk and traffic #

Example: 52 episodes of 57.6 MB are about 3 GB a year. With 2,000 automatic downloads each, the year's episode traffic is about 6 TB, most of it in the first days after each release. These are your own numbers to replace; measure real downloads in your web server logs (Downloads and statistics).

Backups #

Media files are part of the files half of a backup: the storage directory must be backed up with the database, at the same time (Backups: what to back up). A show's archive is often the largest part of a site's backup; agree on retention and test a restore of an episode.

What to agree with your host #

• To do: Disk space for the next year's episodes, versions and backups.
• To do: Data transfer included, and the price beyond it.
• To do: Upload limits for editors (Upload limits and large files).
• To do: The number of PHP workers, and the timeouts for long downloads.
• To do: Whether a CDN is offered, and what it may cache (CDN and external media hosting).
• To do: Backup frequency and retention of the storage directory.

Related pages #

• Sizing memory, CPU and disk
• Hosting for a website
• Disk full
• Free documentation: this page is free to copy, translate and improve under the GNU FDL 1.3.