Logging in an Integration
This guide shows how to log from integration code and API modules with the Frigg logger (ADR-048). The reference for the record format, redaction, levels and deployment is Logging. This guide is the "how do I use it" part.
The rules in short
Use
this.logger. Never useconsole.*in new code.The message is fixed text. Ids, counts and names go into fields.
Give each record an
eventName:integration.<name>.<action>.Pass an error as
{ error }. Log an error one time, where you handle it. If you rethrow, do not log: the framework logs it at the boundary.Do not log tokens, bodies or personal data. Log keys, counts and lengths.
Where the logger comes from
Integration class (extends IntegrationBase)
this.logger
integration.<Definition.name>
integrationId, integrationType, userId, version
API module Api class (extends OAuth2Requester, ApiKeyRequester, …)
this.logger
module.<moduleName>
entityId, credentialId
Other files in your app (helpers, mappers, use cases)
getLogger(name)
the name you give
only the scope ids (see the next table)
const { getLogger } = require('@friggframework/core');
// One logger per file, at module scope. getLogger returns the same
// instance for the same name, and a module-scope logger keeps working
// after test resets.
const log = getLogger('integration.hubspot.mapper');You never pass the ids yourself. The integration logger reads them for each record, so a logger that you use before the integration is hydrated still gets the ids after hydration.
Ids that the framework adds
The framework opens a scope around each entry point. Every record written inside the scope gets these fields, from any logger:
HTTP route, user action, form, OAuth callback (createHandler)
requestId, handlerName, method, route, invocation
Webhook receipt (createHandler)
same as HTTP, plus integrationId from the route (POST .../webhooks/:integrationId, and a Definition.routes path with :integrationId). It goes on log records only, never into telemetry or usage attribution, because it comes from an unauthenticated URL. The plain POST .../webhooks route has no id
Queue job (Worker, QUEUE events)
requestId, plus messageId, receiveCount and integrationEvent from the SQS record, and integrationId, processId from the message body (data.* first, then the top level)
Integration event (this.send(event): USER_ACTION, CRON, QUEUE, WEBHOOK)
integrationEvent
Inside a telemetry span (this.telemetry.span(...))
trace_id, span_id, trace_flags
So a query by requestId finds every record of one invocation, and a query by processId finds every record of one sync across queue messages.
Put integrationId and processId in every queue message you send (in data or at the top level), or the worker records cannot carry them. The framework cannot resolve them later.
When the integration is hydrated, its own ids win over a logger-only id from the route or the message body.
Writing a record
On stdout this is one JSON line (shown here on several lines):
The message
Keep the message fixed, so a query can group all records of one kind.
eventName
Use
integration.<name>.<action>in integration code andmodule.<name>.<action>in an API module.<action>is snake_case and says what happened:batch_started,contact_skipped,webhook_ignored.The prefix is the logger name, so you can write
`${this.logger.name}.contact_skipped`.Give an
eventNameto everyWARNand above, and to each lifecycleINFOthat somebody will query or alert on.An
eventNameis part of your contract with the people who run the integration. Do not rename it without a reason: alerts and dashboards use it.
Fields
Fields go at the top level of the record, next to the framework fields.
Use camelCase, as the framework does.
Values can be strings, numbers, booleans, arrays and plain objects. The logger also handles
Error,Date,URL,Headers,Buffer,MapandSet.Some keys are reserved. The logger removes them from the top level and lists them in
droppedKeys:status,type,source,service,env,host,time,record,severity,tenantId,errorType,errorMessage,stackTrace. UsestatusCodefor an HTTP status andrecordTypefor a type.A field with the same name as a scope or binding field loses. For example, a call-site
integrationIdis dropped, because the logger already has it. Do not pass ids that the logger adds.body,rawBody,payloadandresponseare dropped atINFOand above. LogObject.keys(body)orbody.lengthinstead.
Choosing the level
TRACE
Step-by-step detail. Off everywhere by default.
Each page of a paginated fetch
DEBUG
Detail that helps you fix a problem. Off in production.
The field keys of a mapped record; which branch a webhook took
INFO
A lifecycle fact that somebody wants to see in production.
Sync started or finished with counts; webhook subscription renewed
WARN
Something went wrong, you handled it, and the work continues.
One contact skipped; a rate limit hit and retried; a webhook for an unknown object ignored
ERROR
The operation failed and you do not rethrow.
A batch failed and you mark the process as failed yourself
FATAL
The process cannot continue. You almost never need it in an integration.
—
INFO in production costs money for each line. Log one record per batch or per sync, not one per item. Log per item at DEBUG, or only the items that fail (WARN).
Errors
Log an error one time, at the place that decides what happens next.
You handle the error and continue: log it (
WARN, orERRORif the operation failed) with{ error }, and do not rethrow.You cannot handle it: do not log. Throw, and add context with
cause.createHandler, the express error middleware,Workerand the DLQ processor log the error one time, with all the ids.
Put the error in the
errorfield.this.logger.error('Sync failed', error)also works (the logger moves it toerror), but{ error }is the standard form.The logger writes the error as
{ type, message, code, status, stack, cause }, redacted. Thecausechain is kept to 3 levels. Other own properties of the error (for example axiosconfig,request,response) are not written, because they often hold tokens.A
FetchErrorfrom an API module already has a safe message:GET https://api.hubapi.com/crm/v3/objects/contacts?limit=REDACTED 429. Read the status witherror.statusCodeand the body witherror.body(both are still there, but not written to the log).In a queue job, an error you throw makes the message return to SQS. The
Workerlogsfrigg.worker.record_failedatWARNfor each retry. After the last retry, the DLQ processor logsfrigg.queue.dlq.message_failedatERROR. AHaltErrorstops the retries and logsfrigg.worker.record_haltedatERROR.
Ids that you learn later
The scope has the ids that exist when the entry point starts. For an id that you learn during the work, for example a new processId, use child() or a field.
child(bindings)copies the values at the time of the call.child(() => bindings)reads them again for each record. Use it when the value can change:this.logger.child(() => ({ cursor: this.cursor })).Queue messages that you send later carry the
processIdin the body, so the records of those jobs get it from the message scope.
Secrets, payloads and personal data
The logger redacts every record. It removes the values of keys such as access_token, password, apiKey, authorization, cookie, signature and your module's credentialFields. It keeps only the names of headers, replaces URL query values with REDACTED and scrubs Bearer tokens, JWTs, connection strings and long key-shaped strings in text.
Redaction is a safety net, not a permission:
Do not log a token and trust the logger. A token in an unusual shape, for example a short key in plain text, can pass.
Personal data is not redacted. Email addresses, names and phone numbers are written as they are. Log an external id, not the person.
Log the shape of a payload, not the payload.
Detail that costs time to build
The logger drops a record below the level before it reads any field. When the fields cost time to build, check the level first:
API modules
In an API module, the Api class has this.logger from Requester. When the module runs inside an integration, the records carry entityId and credentialId.
The Requester already logs:
a failed request at
DEBUG(module.<name>.request_failed, with the method, the redacted URL,statusCodeand header names), andtoken refresh steps at
DEBUGand a completed refresh atINFO.
Do not log these again. A failed request throws a FetchError; handle or rethrow it as the Errors section says.
Setting the level
The default is INFO in a deployed stage and DEBUG in a local run.
Set the level per app in the app definition:
levelsetsFRIGG_LOG_LEVELon every deployed function when it is notinfo.retentionInDayssets how long CloudWatch keeps the logs. Without it, log groups never expire, so set it in every app.To debug a deployed stage, change
FRIGG_LOG_LEVELon the function in the Lambda console (the next deploy sets it back), or deploy with a differentlogging.level.DEBUGin production can hold more data than you want to keep.
Locally, with frigg start
Nothing
DEBUG (the local default)
logging.level in the app definition
that level, info included
FRIGG_LOG_LEVEL in the shell or in .env
that level; it wins over logging.level
serverless-offline passes only AWS_* shell variables to handlers, so frigg start puts the shell FRIGG_LOG_LEVEL into the function environment for you. This happens only for the local run: a deploy never reads FRIGG_LOG_LEVEL from the shell.
Details: Deployment: the logging block.
Reading the logs
Locally, the output is JSON. Pipe it to jq:
With osls offline, handlers run in worker threads, and lines can mix. Use --useInProcess when you pipe the output.
In CloudWatch Logs Insights:
More queries: Reading logs.
Testing
Assert on log records, not on console spies.
Setup
Add the Frigg setup file to your Jest config:
Before each test, it installs a memory sink and sets the level to TRACE, so no JSON reaches the test output. It also registers the toContainNoSecretWindow matcher.
Asserting a record
createMemorySink()replaces the sinks until the next reset. The records are frozen plain objects, the same as the stdout line.Assert by
eventName, not by message text.To prove that a token does not leak, use a fake secret of 16 characters or more and check every record:
Moving existing integration code to the logger
Existing console.* calls keep working. Move them when you change the code around them:
Replace
console.log/info/warn/errorwiththis.logger.<level>. Choose the level from the table above, not from the old console method.Make the message fixed text, and move the values into fields.
Add an
eventName.Replace
console.error('...', error)followed bythrow errorwith only the throw (addcausefor context).Replace
debug(...),initDebugLog(...)andflushDebugLog(...)withthis.logger.debug(...), nothing, and a throw or one boundary log.In the tests, replace the console spy with a memory sink.
A complete example
A contact sync that fans out to queue jobs:
What each record carries without extra code:
startSyncruns in the HTTP scope:requestId,route, the integration ids,integrationEvent: 'INITIAL_SYNC', andprocessIdfrom the child logger.processBatchruns in the queue scope:requestId,messageId,receiveCount,processIdfrom the message body, the integration ids andintegrationEvent: 'PROCESS_BATCH'.The
listContactsDEBUG records carryentityIdandcredentialIdof the HubSpot module.If
processBatchthrows, theWorkerlogsfrigg.worker.record_failedwith the same ids, and the message returns to SQS.
Common mistakes
console.log(...)
A text line: no ids, no redaction, no level filter
this.logger.info(...)
Values inside the message
The message cannot group; values are not fields
Fixed message, values in fields
status: 429
Dropped (reserved key), listed in droppedKeys
statusCode: 429
Log, then rethrow
The same error appears two or three times
Throw with cause; the boundary logs
{ body: event.body } at INFO
Dropped; at DEBUG it is written
Log keys or counts
One INFO per item in a big batch
Large CloudWatch cost, noisy queries
One INFO per batch; per item at DEBUG or only failures at WARN
Passing integrationId yourself
Dropped, because the logger already has it
Leave it out
Logging an email address
Personal data in the logs, not redacted
Log the external id
A custom Lambda handler
Handlers made with createHandler, and the framework's queue and webhook handlers, open the scope for you. If you write your own raw Lambda handler, open it with runInvocationScope, so its records get requestId and the logger flushes before the function ends:
Last updated
Was this helpful?
