[ckan-changes] [okfn/ckan] 56fa09: [#2788] Avoid unnecessary commit request

Thu Aug 2 11:26:23 UTC 2012

  Branch: refs/heads/2788-speed-improvements
  Home:   https://github.com/okfn/ckan
  Commit: 56fa0970f8d8658a34f9c76f511a4beef38bf559
      https://github.com/okfn/ckan/commit/56fa0970f8d8658a34f9c76f511a4beef38bf559
  Author: amercader <amercadero at gmail.com>
  Date:   2012-08-01 (Wed, 01 Aug 2012)

  Changed paths:
    M ckan/lib/search/index.py

  Log Message:
  -----------
  [#2788] Avoid unnecessary commit request

The commit can be sent on the same request with the dict to index

  Commit: 4ad7ac00af18f69a4ef27dc7fd6ba18627033ff1
      https://github.com/okfn/ckan/commit/4ad7ac00af18f69a4ef27dc7fd6ba18627033ff1
  Author: amercader <amercadero at gmail.com>
  Date:   2012-08-02 (Thu, 02 Aug 2012)

  Changed paths:
    M ckan/lib/cli.py
    M ckan/lib/search/__init__.py
    M ckan/lib/search/index.py
    M ckan/tests/lib/test_cli.py

  Log Message:
  -----------
  [#2788] Commit only once at the end of a search index rebuild

Commiting changes in the Solr search index is a heavy task that takes
significant time. We are currently commiting changes after each update,
which is probably fine for individual updated (ie users editing or
creating a dataset), but when rebuilding the whole index it is
unnecessary. A single commit at the end of the process is needed, and
that speeds the reindexing about a 85%.

A flag has been added (`-e` or `--commit-each`) to allow the old
behaviour (commiting after each edit).

Compare: https://github.com/okfn/ckan/compare/56fa0970f8d8^...4ad7ac00af18