Repository navigation
feat!: unified structured logger from lib-node 3 - #24
Merged
Merged
Conversation
- default, development, production and custom-environment-variables are .mjs with export default - the dev config finds .env with dotenv's path list, import.meta fails once Playwright compiles it - nodemon watches mjs, a config edit restarts the api - no default keyword in the config schema, assertValid never applied it
lib-node 3 and lib-express 2 replace internalError with one pino logger (@data-fair/lib-node/log.js): JSON lines on stdout in production, pretty output in dev (pino-pretty, dev dependency). - request context: ip and host on every request, the actor through session.middleware() on the routers that read the session; not global, public routes must keep ignoring a broken cookie. A pseudo session is now refused on the non GET requests of these routers, as in the other services - what used to be internalError keeps its code; deprecated calls are warn and a badly formatted or unsafe notification url is alert: neither is an internal failure, df_internal_error stopped counting them - the "/api/v1" deprecation never ran (mounted on 'api/v1', no leading slash), it now names the url - the device registrations are no longer written in the logs (their keys let anyone push to the device), the webhook secrets no longer reach the debug output - a device registration that expired is an info line, the failing ones are warn, with codes - the 4 startup lines of webpush are one line; ui-log is a regular line with the ui-log code - eslint no-console on the server code - lib-types-builder 1.14.6 and in-range updates, the tree no longer holds lib-node 2 BREAKING CHANGE: the log output is one JSON object per line instead of plain text and the ui-log: prefix, the log collector routes must follow.
The schema said maxlength: that keyword does not exist, so this public route wrote whatever the json parser accepted in the logs.
In the query string it reached the access logs, and simple-directory warns on each call.
The loop had no error handling: a rejected query ended it as an unhandled rejection. The turns of the loop are caught and logged like in the search worker. A webhook failing for the last time was only a debug line, it is now a warn line naming the subscription and the target host.
Saving, deleting or testing a webhook subscription sends data of the account to an external url, and left no trace: info lines with the owner and the target host, never the url nor its secret header. Same for the retry and the cancel of a webhook, the subscriptions, the device registrations, the private subscriptions removed after a recipient left the sender and the erasure of a deleted identity. A device refusing its confirmation is a warn line.
An image built outside CI crashed on ENOENT when loading the admin router.
- only the root node_modules was ignored: a local build copied api and ui node_modules, ui/dist and ui/dts in the image - yarn is removed from the runtime image like npm and corepack
- .gitignore on the shared template - eslint ignores on the shared shape, tsconfig without the dead #shared path, .type excluded - test-env routes in their own router, dynamically imported: production never loads them - UiConfig derived from uiConfig - ufo removed, nothing imports it - AGENTS.md and CONTRIBUTING.md: branch length limit, df-dev down, the UI is not nuxt, broken link
…onventions - the dev pages are excluded by the vite command, not NODE_ENV that a build can inherit - server block only for vite dev, ports read with a check, strictPort - df-* components resolved by the lib-vuetify components resolver - theme and public site info loaded from simple-directory with hashed, cacheable urls, the session no longer fetches them at startup (deprecated siteInfo option)
The originator was removed from a notification, not the search texts of the event that hold its name and id: GET /notifications, the websocket and the push payload gave them to any subscriber, and they survived the deletion of the user. The notifications already stored keep them.
- webhooks: axios' timeout only watches the time between 2 chunks, a target dripping its answer held the deliveries of all the accounts. 3 s in total and 64 KB for the answer - push: 10 s for an endpoint chosen by the user, a silent one kept the request pending - mails: no redirect on the request that carries the secret
The removal preferred the root membership while the creation prefers the department one: a department admin lost a subscription at the next identity post. Both use canSubscribePrivate.
A webhook stayed in the working status forever after a restart in the middle of a delivery.
…ons by owner Their list routes and the identity webhooks scanned the collections, no index started with these fields.
cleanEvent reads the sender and the whole originator: with ?select=originator.user.name they were missing and it left the name of a super admin, or of a member of another organization. A select reaching the originator now also reads what cleanEvent needs, the others are unchanged.
The subscriptions and the webhook subscriptions stored the name of their author, nothing reads it and a deleted user's name stayed on what belongs to others. Only the id and the date are stored, and the deletion of a user removes the names written by the previous versions.
A closed port fails at once: the worker could go through waiting and working before the assertion.
Only the mails checked the url of a notification: a url template or an event could put a javascript: link in the href of the notifications list and of the notification queue. http(s) urls and the paths of the site are unchanged, the others are removed at delivery with an alert line.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Move events to lib-node 3 and lib-express 2 with a full review of its logs, and align the service on the shared data-fair conventions.
Why: events was still on lib-node 2, with plain text logs, device keys written in them and sensitive actions leaving no trace.
Logs
no-consoleon the server code.ipandhoston every request, the actor throughsession.middleware()on the routers that read the session. It is not global: public routes must keep ignoring a broken cookie.events.*codes) on webhook subscriptions and deliveries, subscriptions, devices and identity erasure. A webhook failing for the last time is a warn line.deprecatedcalls arewarnand an unsafe notification url isalert: neither is an internal failure.Fixes
javascript:…) are removed at delivery, the mails secret moves to a header.selectcan no longer skip the anonymization ofcleanEvent.created/updated, and private subscriptions follow the same rule as their creation./api/ui-logsmessage size bounded, push requests get a timeout, the API starts withoutBUILD.json, the/api/v1deprecation check runs at last.subscriptions.recipient.idandwebhook-subscriptions.owner.Conventions: config in
.mjs, vite config, hashed site theme and public info,test-envrouter dynamically imported,.dockerignore,.gitignore, eslint and tsconfig, yarn removed from the image, docs. lib-types-builder 1.14.6 and in-range dependency updates.Heads-up
df-event:lines, so the 401/403 and audit lines stay in the container logs and do not reach the per account and per host files until its routes follow (staging first, as for the other services). Nothing is lost.df_internal_errorno longer countsdeprecatednorbad-notif-url./api/v1calls now write a warn line each.created.nameuntil their author is deleted.DfRecentActivityfrom@data-fair/lib-vuetify-events(already onmain) still needs a 0.5.0 release.