changelog | [dreamwidth/dreamwidth] 034f12: Add deterministic ESN trace IDs and full pipeline ...

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: 034f12e28b75c1c3b3ce7b6ab173616484e33ee5 https://github.com/dreamwidth/dreamwidth/commit/034f12e28b75c1c3b3ce7b6ab173616484e33ee5 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-11 (Sat, 11 Apr 2026)

Changed paths: M cgi-bin/DW/Task/ESN/FilterSubs.pm M cgi-bin/DW/Task/ESN/FindSubsByCluster.pm M cgi-bin/DW/Task/ESN/FiredEvent.pm M cgi-bin/DW/Task/ESN/ProcessSub.pm M cgi-bin/LJ/ESN.pm M cgi-bin/LJ/Event/JournalNewComment.pm M cgi-bin/LJ/NotificationInbox.pm M cgi-bin/LJ/Subscription.pm M src/dwtool/main.go

Log Message:

Add deterministic ESN trace IDs and full pipeline observability

Every ESN log line now carries a deterministic trace prefix [esn ETYPEID:JOURNALID:ARG1:ARG2] derived from the event's raw_params, making end-to-end tracing possible by grepping a single string across all four ESN worker log groups. The trace ID is computable from a comment URL without needing to capture it at event-fire time.

Pipeline changes: - All four ESN task workers (FiredEvent, FindSubsByCluster, FilterSubs, ProcessSub) prefix every log line with the trace string - LJ::ESN::unique_matching_subs replaces the silent grep with a loop that logs each matches_filter rejection at DEBUG with user/sub/etypeid - JournalNewComment::matches_filter logs specific rejection reasons at each of its 9 return-0 paths via a $reject->() closure - Subscription::process logs notification construction failures and not_configured_for_user skips with trace prefix - NotificationInbox::enqueue logs successful delivery confirmation - FindSubsByCluster now checks TaskQueue->send() return values and logs users with >5000 subs that get silently skipped - FilterSubs now checks TaskQueue->send() return value - ProcessSub dead code fixed: subscription-not-found is correctly handled as a skip (COMPLETED) not routed through $failed (FAILED)

Metrics consolidation (Prometheus best practice): - All DW::Stats calls use one metric name per stage with a result tag instead of encoding the result in the metric name - dw.esn.firedevent result:started|completed|failed - dw.esn.findsubsbycluster result:started|completed|failed - dw.esn.filtersubs result:completed|dropped|failed - dw.esn.filter result:rejected - dw.esn.matches_filter result:rejected (with reason tag) - dw.esn.processsub result:processed|skipped|failed - dw.esn.process result:failed|skipped - dw.esn.inbox result:delivered|evicted

CLI tooling: - dwtool esn-trace searches all four ESN log groups in CloudWatch and pretty-prints a cross-stage timeline

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications