[dreamwidth/dw-free] 692492: SQS task queue updates

[github]

Branch: refs/heads/master
Home: https://github.com/dreamwidth/dw-free
Commit: 69249237de6fc466ac57b2706982fbfc38d61675
https://github.com/dreamwidth/dw-free/commit/69249237de6fc466ac57b2706982fbfc38d61675
Author: Mark Smith <mark@dreamwidth.org>
Date: 2019-12-12 (Thu, 12 Dec 2019)

Changed paths:
M .tidyallrc
M bin/worker/dw-esn-cluster-subs
M bin/worker/dw-esn-filter-subs
M bin/worker/dw-esn-fired-event
M bin/worker/dw-esn-process-sub
M bin/worker/dw-send-email
M cgi-bin/DW/Task.pm
M cgi-bin/DW/TaskQueue.pm

Log Message:
-----------
SQS task queue updates

* Support adding a dead-letter-queue and do it for all queues
* Support exit-after-seconds to cap worker lifehood
* Support exit-after-messages to cap messages processed
* Support alarming if a message takes too long to process

[dreamwidth/dw-free] cf9aed: Worker updates.

[github]

Branch: refs/heads/master
Home: https://github.com/dreamwidth/dw-free
Commit: cf9aed697e9b49478ce33c100b5a197bb274f2b5
https://github.com/dreamwidth/dw-free/commit/cf9aed697e9b49478ce33c100b5a197bb274f2b5
Author: Mark Smith <mark@discordapp.com>
Date: 2019-12-11 (Wed, 11 Dec 2019)

Changed paths:
A etc/kubernetes/workers/generated/birthday-notify.yaml
A etc/kubernetes/workers/generated/change-poster-id.yaml
A etc/kubernetes/workers/generated/directory-meta.yaml
A etc/kubernetes/workers/generated/distribute-invites.yaml
A etc/kubernetes/workers/generated/embeds.yaml
A etc/kubernetes/workers/generated/resolve-extacct.yaml
A etc/kubernetes/workers/generated/spellcheck-gm.yaml
A etc/kubernetes/workers/generated/sphinx-copier.yaml
A etc/kubernetes/workers/generated/sphinx-search-gm.yaml
A etc/kubernetes/workers/generated/xpost.yaml
M etc/kubernetes/workers/regen-workers.pl

Log Message:
-----------
Worker updates.


Commit: c2f14083ac0c5a041006f8b1297530c04a5603dc
https://github.com/dreamwidth/dw-free/commit/c2f14083ac0c5a041006f8b1297530c04a5603dc
Author: Mark Smith <mark@discordapp.com>
Date: 2019-12-11 (Wed, 11 Dec 2019)

Changed paths:
M doc/dependencies-cpanm

Log Message:
-----------
Add missing Sphinx::Search dependency.


Commit: cd237dbe20b9f1d0782e3acfa702cb9e601bb739
https://github.com/dreamwidth/dw-free/commit/cd237dbe20b9f1d0782e3acfa702cb9e601bb739
Author: Mark Smith <mark@discordapp.com>
Date: 2019-12-11 (Wed, 11 Dec 2019)

Changed paths:
M etc/docker/worker/scripts/startup-prod.sh

Log Message:
-----------
Worker startup script.


Compare: https://github.com/dreamwidth/dw-free/compare/69249237de6f...cd237dbe20b9

[dreamwidth/dw-free] 35acf7: Better errors

[github]

Branch: refs/heads/master
Home: https://github.com/dreamwidth/dw-free
Commit: 35acf7ea13925a6820c9c225e1172d3175c3e6d6
https://github.com/dreamwidth/dw-free/commit/35acf7ea13925a6820c9c225e1172d3175c3e6d6
Author: Mark Smith <mark@dreamwidth.org>
Date: 2019-12-12 (Thu, 12 Dec 2019)

Changed paths:
M cgi-bin/DW/Task/ESN/FilterSubs.pm
M cgi-bin/DW/Task/ESN/ProcessSub.pm
M cgi-bin/DW/TaskQueue.pm

Log Message:
-----------
Better errors

Don't die out of the job when a message fails, return FAILED so we count it properly.

Also, fix counting bug in sent_messages event.