Upgrade notes: notifications
What changed in the behaviour of the notification system, what you must do when you upgrade, and how mail that the transport refuses is kept. Read it if you run Exponential 6.0.15 or later, have overridden notification templates, or call the notification classes from your own code. The pages that describe the system as it is now: the user's guide, the administrator's guide, the specification.
What you need to do
- Make sure the cronjob part
notificationruns: it is in the groupfrequent, or run it alone withphp runcronjobs.php notification. Open Setup > Notification: a red notice tells you if events wait and no run is recorded. - If you override
notification/addingresult.tplor the settings templates in your own design, read the sections below. Designs that do not override them need nothing. - Clear the caches after the upgrade (
php bin/php/ezcache.php --clear-all --allow-root-user) and reload PHP-FPM or restart Velocity, as for any code change. - Optionally remove the events and subscriptions that piled up (section "Cleaning up").
Behaviour changes
Opening /notification/addtonotification/<node> no longer subscribes
Before: a GET of the address created the subscription at once (the content action redirected there after a form post). A link
or an image on any page could subscribe a logged-in user. Now: a GET shows a confirmation page ("Notify me about updates");
the buttons are a POST that carries the form token, and only Notify me subscribes. For an item the user already follows
the page offers Stop notifications for this item. The content menu entry Notify me (the form field
ActionAddToNotification) works as before and now ends on the confirmation page, one click more.
Templates: the view renders notification/addconfirm.tpl (new), then addingresult.tpl (as before) after adding, or the
new removeresult.tpl after removing. design/standard and design/admin provide addconfirm.tpl and removeresult.tpl; a design
that falls back to them needs nothing. If your design overrides addingresult.tpl, it is still used; it now receives
redirect_uri besides redirect_url. Code that links to the address and expects it to subscribe must post the form instead:
POST /notification/addtonotification/<node> with ConfirmAddNotification (and the form token).
The settings page
admin4 and admin4l have a new overview (see the user's guide). The form fields of the handler templates keep their names
(NewRule_ezsubtree, RemoveRule_ezsubtree, SelectedRuleIDArray_ezsubtree[], Store, ReceiveDigest_ezgeneraldigest,
DigestType_, Time_, Weekday_, Monthday_, CollaborationHandlerSelection...), so templates of other designs keep working.
New: UseConfirm in the subtree form makes the view ask before removing (a form without it removes at once, as before);
FilterSubscriptions and ClearFilter; the collaboration card saves with SaveCollaboration and no longer shares the Store button
(a Store post turns the digest off when its fields are missing, so a form that posts Store must carry the digest fields).
The view accepts the new URL parameters (q) and (class) besides (offset). The view passes more template variables
(subscriptions, subscription_total, subscription_all_total, subscription_limit, subscription_classes, filter_query,
filter_class, confirm_remove, notice, can_administrate); a template of an included handler must have them passed
explicitly with include.
The settings are checked on save now: a digest type other than daily, weekly or monthly becomes daily, a time that is not one
of the offered hours becomes 0:00, a weekday outside the week becomes the first weekday, a day of the month is held to 1 to 31.
A user who never opened the page (no settings row yet) can save digest settings. The subtree form refuses nodes the user may not
read, nodes that do not exist and the same node twice, and only removes the user's own rules.
Handled events are cleaned up
Before: an event whose messages were kept for a digest stayed in the table for good after the digest was sent. Now
eZNotificationEventFilter::process() ends by removing handled events that nothing waits for
(eZNotificationEvent::cleanupHandled()). On a test installation, 7 such events were removed by a run. The status page and
exp:notification:events cleanup do the same on demand, and can remove events by age.
The filter
eZNotificationEventFilter::process( $eventIDList = null ) takes an optional list of event ids and returns an array (events,
removed, kept, failed) where it returned nothing. A handler that throws is logged and counted; the other handlers still
run and the event is not retried. Before, the error stopped the whole pass and left the rest of the events. An undefined
variable in the message for a missing handler is fixed.
Smaller fixes
| Where | Before | Now |
|---|---|---|
eZNotificationCollection::addItem( $address, $sendDate ) | the send date was ignored (always 0) | it is stored |
eZNotificationCollectionItem::fetchByDate( $date ) | the condition overwrote itself and found nothing sensible | finds the items due before $date |
eZGeneralDigestHandler::fetchHandlersForUser() | an undefined index for a handler that is listed under another name | the handler is found by id, or left out |
digest handle() | an undefined template variable when no one had a digest due | handled |
| weekly digest | an undefined index when the cronjob ran in another language than the user's | the stored weekday name, the English name and 0 to 6 are accepted |
eZCollaborationNotificationHandler::rules() | looked the rules up by e-mail address | by user id |
eZNotificationEventType::allowedTypes() | read an undefined global and a setting (AvailableEventTypes) that does not exist | reads AvailableNotificationEventTypes |
eZMailNotificationTransport::send() | a single address given as a string failed | accepted |
eZNotificationFunctionCollection::eventContent() | a fatal error for an unknown event id | returns false |
The mail transports
eZFileTransport writes to [MailSettings] FileTransportDirectory of site.ini. Not set or empty: var/log/mail, as before.
eZMailNotificationTransport::observe( $callback, $suppress ) is new (used by the status numbers and the dry run).
Cronjob
The part notification goes through expNotificationService::run(): locked (a second run is skipped with a message),
recorded, and shown on the status page. [CronjobPart-notification] in cronjob.ini is new. The output lines "Starting
notification event processing" and "Done" are kept; "Done" now says how many events and messages.
New: status page, commands, views
notification/status and notification/job (policy notification/administrate), the Setup menu entry "Notification"
(menu.ini [Leftmenu_setup] Links[notification], which was commented out since 3.5), and the commands
exp:notification:status|run|events|subscriptions with the aliases exp:notify:*. notification/runfilter keeps working and
goes through the service. Add the policy notification/administrate to roles that should see them (administrators have it).
Cleaning up after the upgrade
A site where the cronjob did not run for a long time has many pending events. One run handles them all (4 seconds for 7259 events on the test installation). Dry-run first if you like:
./console exp:notification:status
./console exp:notification:run --dry-run
./console exp:notification:events cleanup --dry-run
./console exp:notification:subscriptions list --missingRemove events that are older than you care about with exp:notification:events cleanup --older-than=90d (see
the commands). Mail that these old events would have caused is sent by the
run that handles them: if the backlog is old, remove it first, or run once with --mail-file-dir to discard the mail.
Mail the transport refuses is kept and tried again
Before: the general digest handler removed the digest items after send() without looking at the result, and the subtree
and collaboration handlers removed their items before send(). When the mail transport failed (the mail server was down,
the file transport could not write), the messages were lost and nothing showed it. Now:
- A digest whose
send()fails keeps the items of that address. They stay due, so the next run (its time event is later) sends them again. The handled event they belong to stays until its items are gone. - A message sent at once (subtree, collaboration) is sent first and its items are removed only when the transport took it. If it
did not, the items stay with no send date and the event stays handled; at the start of the next run
eZNotificationEventFilter::retryUnsent()sends them again, with the subject and text of the collection (without the threading headers). - A message that still cannot be sent after
[RuleSettings] RetryHoursofnotification.ini(default 72 hours; counted from when the digest was due, or from the time of the content for a message sent at once) is given up: its items are removed and it is counted as dropped. An item whose address cannot be mailed at all (not a valid address) is dropped at once instead of being tried for 72 hours. This keeps a permanently refused message from being tried for ever. eZNotificationEventFilter::process()returnssend_failed(messages the transport refused in this run),droppedandretried(sent now after an earlier failure) besides its earlier numbers; they are in the run record (runs.jsonl), so the status page shows not sent and given up badges on the run.- The status page and
exp:notification:statusreport a problem of level error "n messages could not be handed to the mail transport and wait for the next run; each is given up after n hours" while any wait, "The mail transport refused n messages in the last run", and a warning when messages were given up. The status command then ends with FAIL (exit code 1). exp:notification:runends withFAIL: the mail transport refused n message(s); ...and exit code 1 when a send was refused, so a cron mail or a monitoring check shows it; the work done is still kept. The cronjob part prints the same as an error line.- The run counts (
mails,recipients) count only what the transport took.eZMailNotificationTransport::observe()callbacks get the answer of the transport as a fifth argument$sent; a callback written with four parameters keeps working.
Sample (a test installation; the "transport" was the file transport pointed at a directory it cannot write, so no mail could leave):
$ ./console exp:notification:run --event=21935 --mail-file-dir=/proc/nottest-no-such-directory
PASS: 1 event(s) handled, 0 removed, 1 kept for a digest, 0 message(s) to 0 recipient(s), 150 ms.
FAIL: the mail transport refused 1 message(s); they are kept and tried again at the next run (for 72 hours). (exit code 1)
$ ./console exp:notification:run --no-time-event --event=1 --mail-file-dir=var/tmp/notification-mail/doc3
PASS: 0 event(s) handled, 0 removed, 0 kept for a digest, 1 message(s) to 1 recipient(s), 37 ms.
1 message(s) that failed earlier were sent now.What it does not cover: a mail the transport accepts and the mail server later bounces (sendmail and SMTP report only the
hand-over). Keep an eye on the mail log of the server for those. RetryHours is the only setting; there is no retry counter, so
how many times a message is tried depends on how often the cronjob runs.