E-mail preferences and consent: the specification #

The reference of the e-mail preference system of Exponential 6.0.15: the classes, the tables, the settings, the flows, the decisions of the mail gate and the format of the links. Read it when you need an exact answer; the walk-throughs are the administrator's guide and the developer's guide. The law requirements and the tests that prove them are on the law checklist.

Parts #

Categories #

expMailCategory is a value object: identifier, name, description, essential, defaultOn (always false for an essential category), frequencies (subset of immediate, daily, weekly), doubleOptIn, source (ini, extension, admin), handlerClass. An identifier is lower case letters, digits and underscores and starts with a letter.

expMailCategoryRegistry::instance() merges, in this order:

1. mailpreferences.ini [CategorySettings] Categories[] with a block [Category_<identifier>] each; source ini for the identifiers listed in the shipped settings/mailpreferences.ini, extension for the others.
2. Rows of expmail_category. A row with the identifier of a category of step 1 overrides its name, description, default, frequencies and double opt-in; essential and an empty handler_class stay as the settings say. Other rows are categories of source admin.
3. register(): categories added in code for this process.

The list is read again for each request ($_SERVER['REQUEST_TIME_FLOAT']), so persistent workers see admin changes. all(), optional(), essential(), get(), iniCategories(), saveAdmin(), removeAdmin(), reset().

The shipped categories:

Recipients #

expMailRecipient identifies a person. Its key is u:<user id> for an account and a:<hash> for an address without account, where <hash> is expMailSuppression::hash( $email ). fromAddress() returns the account when the address belongs to one (second argument false: the bare address). Also fromUser(), fromUserId(), fromToken(), fromPayload(), fromKey(). logKey() is what the gate log stores: the user id, or the first characters of the hash.

Tables #

All five are in the kernel schema of MySQL, PostgreSQL and SQLite, in share/db_schema.dba, and in update/database/<engine>/6.0/dbupdate-6.0.0-6.0.15.sql. Times are Unix timestamps.

expmail_preference #

One row per person and category, plus the main switch (category _master).

No row means "no choice made": the handler's reading, else the category default (off).

expmail_consent_log #

One row per change; never updated except by anonymisation.

Every row also writes an audit event: access.user.consent.erase, access.user.consent.export, or access.user.consent.change for every other action.

expmail_suppression #

expmail_pending #

A double opt-in waiting for its confirmation.

expmail_category #

The administrator's categories and overrides: identifier (unique), name, description, essential, default_on, frequencies (comma separated), double_opt_in, handler_class, priority, created, modified.

The decision for one recipient #

expMailPreferences::decision( $category ), in this order:

allows() is decision() === 'allow'.

The gate #

expMailGate::dispatch( eZMail, eZMailTransport ):

decorate( eZMail, recipient, category ) adds X-Exp-Mail-Category, and for optional mail with a known recipient the headers

List-Unsubscribe: <https://www.example.com/mailpreferences/unsubscribe/m1...>
List-Unsubscribe-Post: List-Unsubscribe=One-Click

and the footer of [FooterSettings] Template (built-in text when the template is missing): text mail at the end after -- , HTML before </body>, every text part of an ezcMailPart body. It never changes From, Reply-To, Subject or other headers.

The log ([GateSettings] LogFile, default log/mailgate.jsonl in the var directory, rotated to .1 at MaxLogSize) has one JSON object per line: t time, d decision, c category, why, r the recipient's log key, s the sender file. It holds no address. expMailGate::stats( $since ) counts it for the status page; lastResult() describes the last dispatch (category, decision: sent, partly_blocked, blocked, essential, uncategorised, unknown_category; sent, blocked, result).

Links (tokens) #

expMailToken::create( recipient, purpose, category = null, ttl = null, extra = array() ):

• purposes unsubscribe, manage, confirm;
• payload JSON { p: purpose, t: issued, x: expires (0 never), u: user id, e: address (only without account), c: category, i: pending id };
• encrypted with AES-256-GCM, 12 byte IV, 16 byte tag, additional data exp-mail-token, key HMAC-SHA-256( 'exp-mailpreferences:token', site secret );
• written as m1 + base64url( IV . ciphertext . tag ).

verify( $token, $purpose ) returns purpose, user, email, category, pending, issued, expires or null; lastError(): malformed, tampered, purpose, expired. Default lifetimes, [TokenSettings] TTL[]: unsubscribe 0 (never), manage 0, confirm 604800 (7 days); a "send me a link" link RequestLinkTTL (86400). The address is <BaseURL>/mailpreferences/<view>/<token>.

The site secret (expMailSecret): 32 random bytes, base64 in settings/override/mailpreferences.ini.append.php [SecretSettings] TokenSecret, generated on first use under a lock. derive( $purpose ) gives the key for token, hash (suppression and address keys) and anonymise.

Flows #

Turning a category on (expMailPreferences::set( $category, true, $context )): a category without double opt-in, or a context of source confirm, bridge or import, is on at once (log on, or confirm). Otherwise the state becomes pending, a row of expmail_pending is written, the log says pending, and the confirmation mail (category security, template mail/confirm.tpl) goes to the address with a confirm link, unless $context->sendConfirmation is false or the address is suppressed for a bounce, complaint or legal reason. Turning on by page, link, confirm or signup lifts the person's own unsubscribe_all suppression.

Confirming (mailpreferences/confirm/<token>): GET shows one button; the POST of Yes, confirm calls expMailPreferencesService::confirm(): confirmed, already, expired (the pending row is removed) or invalid.

Unsubscribing (mailpreferences/unsubscribe/<token>): GET shows one button; its POST, or a POST with the body List-Unsubscribe=One-Click (RFC 8058; no cookie, no form token), calls expMailPreferencesService::unsubscribe(): a link with a category turns the category off, a link without one turns the main switch off. The one-click POST answers 200 text/plain ("You are unsubscribed."), or 400 for a link that does not work. unsubscribeAll() turns the main switch off and adds the address to the suppression list (unsubscribe_all).

Send me a link (mailpreferences/request): requestLink( $email ) returns sent, invalid, rate_limited ([TokenSettings] RequestLinkLimit per hour) or blocked; the page shows the same answer for every well-formed address.

Address change (requestEmailChange( $user, $email, $context )): invalid, unchanged, in_use or pending_confirmation; the account keeps its address until the link sent to the new address is confirmed.

Sign-up: MailPreferencesPage::signupCategories() gives the boxes of the registration form, none ticked; storeSignup() stores only what was ticked, with source signup and the text of the form.

Export and erasure: export() returns the recipient, the main switch, every category with state, frequency and time, pending confirmations and the whole consent log; exportToCsv() writes it as section,key,value,detail rows. erase( $context ) writes an erase row, deletes the preferences and pending rows and anonymises every log row of the person.

Retention (expConsentLog::cleanup( $dryRun ), cronjob part mailpreferences): removes log rows older than [ConsentSettings] RetentionDays of people who are gone (anonymised, a removed account, an address without preference), removes expired pending rows and turns their categories off, and removes suppression entries older than [SuppressionSettings] RetentionDays when that is above 0.

Module views #

Pages reached by a personal link are sent with Cache-Control: private, no-store, max-age=0, Referrer-Policy: no-referrer and X-Robots-Tag: noindex, nofollow. Every page works without JavaScript.

Settings: mailpreferences.ini #

Related pages #

• The law checklist
• The commands
• The administrator's guide, the developer's guide
• The user's guide
• Upgrade notes
• Notifications