github: shadowy octopus with the head of a robot, emblazoned with the Dreamwidth swirl (Default)
github ([personal profile] github) wrote in [site community profile] changelog2026-04-12 12:42 am
  • Add Memory
  • Share This Entry

[dreamwidth/dreamwidth] ece0b7: Ship worker logs directly to Grafana Cloud Loki vi...

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: ece0b7970913514a534e88d20b8383a4555ebe5a https://github.com/dreamwidth/dreamwidth/commit/ece0b7970913514a534e88d20b8383a4555ebe5a Author: Mark Smith mark@dreamwidth.org Date: 2026-04-12 (Sun, 12 Apr 2026)

Changed paths: M .github/workflows/tasks/worker-birthday-notify-service.json M .github/workflows/tasks/worker-change-poster-id-service.json M .github/workflows/tasks/worker-codebuild-notifier-service.json M .github/workflows/tasks/worker-content-importer-lite-service.json M .github/workflows/tasks/worker-content-importer-service.json M .github/workflows/tasks/worker-content-importer-verify-service.json M .github/workflows/tasks/worker-directory-meta-service.json M .github/workflows/tasks/worker-distribute-invites-service.json M .github/workflows/tasks/worker-dw-change-poster-id-service.json M .github/workflows/tasks/worker-dw-distribute-invites-service.json M .github/workflows/tasks/worker-dw-embeds-service.json M .github/workflows/tasks/worker-dw-esn-cluster-subs-service.json M .github/workflows/tasks/worker-dw-esn-filter-subs-service.json M .github/workflows/tasks/worker-dw-esn-fired-event-service.json M .github/workflows/tasks/worker-dw-esn-process-sub-service.json M .github/workflows/tasks/worker-dw-import-eraser-service.json M .github/workflows/tasks/worker-dw-incoming-email-service.json M .github/workflows/tasks/worker-dw-latest-feed-service.json M .github/workflows/tasks/worker-dw-lazy-cleanup-service.json M .github/workflows/tasks/worker-dw-mass-privacy-service.json M .github/workflows/tasks/worker-dw-send-email-service.json M .github/workflows/tasks/worker-dw-sphinx-copier-service.json M .github/workflows/tasks/worker-dw-support-notify-service.json M .github/workflows/tasks/worker-dw-synsuck-service.json M .github/workflows/tasks/worker-dw-xpost-service.json M .github/workflows/tasks/worker-embeds-service.json M .github/workflows/tasks/worker-expunge-users-service.json M .github/workflows/tasks/worker-import-eraser-service.json M .github/workflows/tasks/worker-import-scheduler-service.json M .github/workflows/tasks/worker-incoming-email-service.json M .github/workflows/tasks/worker-latest-feed-service.json M .github/workflows/tasks/worker-lazy-cleanup-service.json M .github/workflows/tasks/worker-paidstatus-service.json M .github/workflows/tasks/worker-process-privacy-service.json M .github/workflows/tasks/worker-resolve-extacct-service.json M .github/workflows/tasks/worker-schedule-synsuck-service.json M .github/workflows/tasks/worker-ses-incoming-email-service.json M .github/workflows/tasks/worker-shop-creditcard-charge-service.json M .github/workflows/tasks/worker-spellcheck-gm-service.json M .github/workflows/tasks/worker-sphinx-copier-service.json M .github/workflows/tasks/worker-sphinx-search-gm-service.json M .github/workflows/tasks/worker-support-notify-service.json M config/update-workflows.py

Log Message:

Ship worker logs directly to Grafana Cloud Loki via Fluent Bit

Regenerated all 42 worker task definitions to use awsfirelens with a Fluent Bit log_router sidecar instead of the awslogs driver. Worker logs now go directly to Grafana Cloud Loki over HTTPS, bypassing CloudWatch Logs entirely.

Each task definition gains a log_router container (~15MB RSS) running the AWS-maintained Fluent Bit image from private ECR. Loki credentials are injected via SSM secrets at task startup. The worker container's logDriver changes from awslogs to awsfirelens with the grafana-loki output plugin.

Fluent Bit's own logs still go to CloudWatch (/dreamwidth/fluent-bit) so the log router itself can be debugged if needed.

Generated by config/update-workflows.py from config/workers.json.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

Commit: 0d49a7588ae0544ef51801c977e8df41c651e3b5 https://github.com/dreamwidth/dreamwidth/commit/0d49a7588ae0544ef51801c977e8df41c651e3b5 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-12 (Sun, 12 Apr 2026)

Changed paths: M cgi-bin/DW/Worker/ContentImporter.pm M cgi-bin/DW/Worker/ContentImporter/LiveJournal.pm

Log Message:

tidy

Compare: https://github.com/dreamwidth/dreamwidth/compare/d7d5b8fbd0bb...0d49a7588ae0

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications