[ckan-changes] [ckan/ckan] bab3b1: Query performance improvements.

Tue Feb 14 11:39:12 UTC 2017

  Branch: refs/heads/master
  Home:   https://github.com/ckan/ckan
  Commit: bab3b109a27713b68752187fb14bc3906909696e
      https://github.com/ckan/ckan/commit/bab3b109a27713b68752187fb14bc3906909696e
  Author: Tyler Kennedy <tk at tkte.ch>
  Date:   2017-02-07 (Tue, 07 Feb 2017)

  Changed paths:
    M ckan/model/resource.py
    M ckan/model/user.py

  Log Message:
  -----------
  Query performance improvements.

Removes a forced order_by on revisions that causes poor query
performance on many other table queries.

Replaced `count()` queries on number_of_edits and number_of_packages
with vastly improved queries. SQLAlchemy uses subqueries for `count()`,
along with an odd default unindexed order_by on the table mapper
causing even trivial counts to perform unindexed table scans.

This change reduces the average query time for a user with 500k
revisions from 15 *seconds* to under 20ms.

  Commit: 5d6949ca7602eb5039227229a0113269d213f856
      https://github.com/ckan/ckan/commit/5d6949ca7602eb5039227229a0113269d213f856
  Author: Tyler Kennedy <tk at tkte.ch>
  Date:   2017-02-13 (Mon, 13 Feb 2017)

  Changed paths:
    M ckan/templates/package/resources.html
    M ckan/templates/package/snippets/resource_item.html
    M ckan/templates/package/snippets/resources_list.html

  Log Message:
  -----------
  Move an expensive, repetitive check_access outside of the resource_item.html loop.

  Commit: cb498fc3d08ca54d1677609ad6e2caccb001e3e8
      https://github.com/ckan/ckan/commit/cb498fc3d08ca54d1677609ad6e2caccb001e3e8
  Author: Adrià Mercader <amercadero at gmail.com>
  Date:   2017-02-14 (Tue, 14 Feb 2017)

  Changed paths:
    M ckan/model/resource.py
    M ckan/model/user.py
    M ckan/templates/package/resources.html
    M ckan/templates/package/snippets/resource_item.html
    M ckan/templates/package/snippets/resources_list.html

  Log Message:
  -----------
  Merge pull request #3430 from open-data/query_performance

Query performance improvements.

Compare: https://github.com/ckan/ckan/compare/2b1ede337d96...cb498fc3d08c