[dreamwidth/dw-free] 107776: Sphinx improvements, comment indexing

[github]

Branch: refs/heads/release-1.13
Home: https://github.com/dreamwidth/dw-free
Commit: 107776b66630002cb2e42a7dd78039a4a5a4dd4f
https://github.com/dreamwidth/dw-free/commit/107776b66630002cb2e42a7dd78039a4a5a4dd4f
Author: Mark Smith <mark@dreamwidth.org>
Date: 2013-02-07 (Thu, 07 Feb 2013)

Changed paths:
M bin/schedule-copier-jobs
M bin/worker/content-importer
M bin/worker/content-importer-lite
M bin/worker/content-importer-verify
M bin/worker/import-scheduler
M bin/worker/sphinx-copier
M bin/worker/sphinx-search-gm
M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Comments.pm
M cgi-bin/DW/Worker/ContentImporter/LiveJournal/Entries.pm
M cgi-bin/LJ/DB.pm
M cgi-bin/LJ/Protocol.pm
M cgi-bin/LJ/Talk.pm
M htdocs/search.bml
M htdocs/search.bml.text

Log Message:
-----------
Sphinx improvements, comment indexing

Performance improvements:

* Do not use AUTO_INCREMENT, use a better PRIMARY KEY that InnoDB that
reflects the data access patterns. (All accesses are either single-items
or groups-by-journal.)

* Use COMPRESS to save disk space.

* Don't copy entire journals every time something updates.

* Only copy once when imported -- at the end.

Functionality improvements:

* Add comment indexing and searching.

* Support delta indices for faster indexing cycles.

IMPORTANT: These changes are backwards-incompatible with search
installations. If you deploy this code, you will need to update
your database schema and delete the table and reimport (to get the
compression).

[dreamwidth/dw-free] d3351f: Make sphinx-copier status-line readable

[github]

Branch: refs/heads/release-1.13
Home: https://github.com/dreamwidth/dw-free
Commit: d3351fa940ae9cdfc37ddc2ce09599d06c2b9ab6
https://github.com/dreamwidth/dw-free/commit/d3351fa940ae9cdfc37ddc2ce09599d06c2b9ab6
Author: Mark Smith <mark@dreamwidth.org>
Date: 2013-02-07 (Thu, 07 Feb 2013)

Changed paths:
M bin/worker/sphinx-copier

Log Message:
-----------
Make sphinx-copier status-line readable

[dreamwidth/dw-free] bbfce0: Fix case where OpenIDs get underscores

[github]

Branch: refs/heads/release-1.13
Home: https://github.com/dreamwidth/dw-free
Commit: bbfce05fa911715f5dc0b55d8c826bc353c7dee9
https://github.com/dreamwidth/dw-free/commit/bbfce05fa911715f5dc0b55d8c826bc353c7dee9
Author: Mark Smith <mark@dreamwidth.org>
Date: 2013-02-07 (Thu, 07 Feb 2013)

Changed paths:
M cgi-bin/DW/Worker/ContentImporter/LiveJournal.pm

Log Message:
-----------
Fix case where OpenIDs get underscores

URLs should have dashes. This is one case where that happens.