Error handling and logging in integrations #

This page is for integration developers. An integration fails sooner or later: the other system is down, a record is invalid, a credential expires. Whether that is a non-event or an incident depends on how the code fails, retries and reports. The kernel's logging tools are eZDebug and logging.

Contents #

1. Three kinds of errors
2. Retries and the dead letter
3. Transactions
4. Writing to the logs
5. Reading the logs
6. What never goes into a log

Three kinds of errors #

A run that skips records must still say so: the summary line counts them, and the exit code or a status flag tells monitoring that the run was not clean.

Retries and the dead letter #

• Retry transient errors with exponential backoff and a cap (for example 2, 4, 8 ... minutes, at most an hour), and honor Retry-After from the other system.
• After a fixed number of attempts, move the item to a dead letter state (a status in your outbox table, or the dead letter of expqueue) and alert. An item retried forever hides a real problem.
• Make retries safe: the receiver must recognize a repeated delivery by an idempotency key (Writing an export).

Transactions #

eZContentFunctions::createAndPublishObject() manages its own transaction. When your code writes several rows of your own tables that belong together, wrap them:

<?php
$db = eZDB::instance();
$db->begin();
$db->query( 'UPDATE myintegration_outbox SET sent = ' . time() . ' WHERE id = ' . (int)$rowID );
$db->query( 'INSERT INTO myintegration_delivery ( outbox_id, answered ) VALUES ( ' . (int)$rowID . ', ' . time() . ' )' );
$db->commit();

Never hold a transaction open across a call to another system: a slow answer holds database locks for every editor.

Writing to the logs #

Which levels reach the log files is set in site.ini [DebugSettings] AlwaysLog[] (by default error). Use your own eZLog file for the integration's trail, so operators find it without searching the kernel logs.

A good summary message (eZLog adds the date in front):

import erp-products: rows=20412 created=12 updated=388 unchanged=19990 hidden=3 skipped=19 failed=0 seconds=274

Reading the logs #

• Setup > Log files (the log viewer) lists, follows and searches the log files and their rotations, with masking of personal data (The log viewer).

• Error references: an error page shows a reference such as ERR-0123456789, also written to error.log; the log viewer and php bin/php/console exp:logs --reference=ERR-... show every line of that request (Finding an error by its reference).

• From a shell:

  php bin/php/console exp:logs --list
  php bin/php/console exp:logs --tail=var/log/myintegration.log --lines=50
  php bin/php/console exp:logs --search=erp-product --file=var/log/error.log --rotations

• The audit trail answers "who changed what": writes through expservices, API key use and refusals, content removals (Audit). An integration that acts for users can record its own events through the audit API.

• Log files are rotated by the cronjob part logs (explogrotate.php) (Log rotation).

What never goes into a log #

• Passwords, API keys, tokens, signing secrets, session ids.
• Full payloads containing personal data; log keys and counts instead.
• Card data of any kind.

The log viewer masks e-mail and IP addresses and tokens, but masking is a safety net, not a permission to write them.

Related pages #

• eZDebug and logging
• Debugging and logs for app creators
• Monitoring integrations
• Log viewer and rotation for hosts
• Free documentation: this page is free to copy, translate and improve under the GNU FDL 1.3.