[dreamwidth/dw-free] 0196aa: Revamp importer logic to drop URLs

[github]

Branch: refs/heads/develop
Home: https://github.com/dreamwidth/dw-free
Commit: 0196aafc5ba82baaf84dcdcbf5b9ebc00bacca57
https://github.com/dreamwidth/dw-free/commit/0196aafc5ba82baaf84dcdcbf5b9ebc00bacca57
Author: mark <mark@dreamwidth.org>
Date: 2012-11-03 (Sat, 03 Nov 2012)

Changed paths:
M .gitignore
A bin/erase-imported-content.pl
M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Comments.pm
M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Entries.pm
M cgi-bin/DW/Worker/ContentImporter/Local/Comments.pm

Log Message:
-----------
Revamp importer logic to drop URLs

We've once again been bitten by the bug wherein we depend on the URLs
and end up causing duplicate posts or comment imports. While it was nice
to depend on those URLs, we don't need to.

This commit changes it so that our import_source lines are actually just
slash separated tuples that we can generate with information that we
know, and that the remote site has to provide. (Itemids, notably.)

This commit also takes care of a long-standing issue and updates the
grabbed_until columns for jobs so that long running jobs won't get
passed on to a new worker while they're still busy.


Commit: dddce898251661be710c167263dd423dbfdb5412
https://github.com/dreamwidth/dw-free/commit/dddce898251661be710c167263dd423dbfdb5412
Author: mark <mark@dreamwidth.org>
Date: 2012-11-08 (Thu, 08 Nov 2012)

Changed paths:
M .gitignore
A bin/erase-imported-content.pl
M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Comments.pm
M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Entries.pm
M cgi-bin/DW/Worker/ContentImporter/Local/Comments.pm

Log Message:
-----------
Merge remote branch 'dreamwidth/release-1.11.0' into develop


Compare: https://github.com/dreamwidth/dw-free/compare/cfecf4e3853c...dddce8982516