[ckan-changes] [ckan/ckan] 7bd11d: Total estimation first attempt. Failing test for f...

Fri Nov 23 15:23:53 UTC 2018

  Branch: refs/heads/master
  Home:   https://github.com/ckan/ckan
  Commit: 7bd11d3f757d40023897bcdd5d5176106285dd47
      https://github.com/ckan/ckan/commit/7bd11d3f757d40023897bcdd5d5176106285dd47
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-09-21 (Fri, 21 Sep 2018)

  Changed paths:
    M CHANGELOG.rst
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/set_permissions.sql
    M ckanext/datastore/tests/helpers.py
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  Total estimation first attempt. Failing test for filters - I cant get it to accurately estimate with a filter.


  Commit: 4cad664982f7352c077fd31165fa185938a9dad1
      https://github.com/ckan/ckan/commit/4cad664982f7352c077fd31165fa185938a9dad1
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-09-21 (Fri, 21 Sep 2018)

  Changed paths:
    M CHANGELOG.rst
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/set_permissions.sql
    M ckanext/datastore/tests/helpers.py
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  Remove estimation for filters and distinct because they didnt work.


  Commit: 7760a56e94029d90462818f6b0f7fd1fb92b1c30
      https://github.com/ckan/ckan/commit/7760a56e94029d90462818f6b0f7fd1fb92b1c30
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-10-05 (Fri, 05 Oct 2018)

  Changed paths:
    M ckanext/datastore/logic/action.py
    M ckanext/reclineview/theme/public/vendor/ckan.js/ckan.js
    M ckanext/reclineview/theme/public/vendor/recline/recline.js

  Log Message:
  -----------
  Display "*about* 1000000 records" on the frontend when estimated


  Commit: af1f95c04ee3d8f0191eb517ce4aa1768ca5d3d8
      https://github.com/ckan/ckan/commit/af1f95c04ee3d8f0191eb517ce4aa1768ca5d3d8
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-10-12 (Fri, 12 Oct 2018)

  Changed paths:
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/auth.py
    M ckanext/datastore/logic/schema.py

  Log Message:
  -----------
  Add ANALYZE api for datapusher to use


  Commit: 784d1972c1dbc5de9e7ef7c09532b020162fe1e0
      https://github.com/ckan/ckan/commit/784d1972c1dbc5de9e7ef7c09532b020162fe1e0
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-10-12 (Fri, 12 Oct 2018)

  Changed paths:
    M ckanext/reclineview/theme/public/vendor/recline/recline.js

  Log Message:
  -----------
  When estimating, round to nearest 1000


  Commit: 8aa298dfcf13cc5f0b5853f6fe60d49948cb43f0
      https://github.com/ckan/ckan/commit/8aa298dfcf13cc5f0b5853f6fe60d49948cb43f0
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-10-26 (Fri, 26 Oct 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_create.py

  Log Message:
  -----------
  Adds an option to do the ANALYZE to datastore_create


  Commit: 6428c48a58d4858527f7bd12a813c670a3e89d9d
      https://github.com/ckan/ckan/commit/6428c48a58d4858527f7bd12a813c670a3e89d9d
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-10-26 (Fri, 26 Oct 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/helpers.py
    M ckanext/datastore/tests/test_create.py
    M ckanext/datastore/tests/test_delete.py
    M ckanext/datastore/tests/test_upsert.py

  Log Message:
  -----------
  Adds an option to do the ANALYZE in datastore_upsert and datastore_delete


  Commit: 23782a81bf1d8a6d64d8c5eaa253f59a845b8dda
      https://github.com/ckan/ckan/commit/23782a81bf1d8a6d64d8c5eaa253f59a845b8dda
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-10-26 (Fri, 26 Oct 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py

  Log Message:
  -----------
  pep8


  Commit: aa87446740de947933327943822dde802c05bbeb
      https://github.com/ckan/ckan/commit/aa87446740de947933327943822dde802c05bbeb
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-09 (Fri, 09 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py

  Log Message:
  -----------
  Timing tests show that asking when analyze was done is v slow - 12ms - compared to asking for the estimated row count - <1ms - so dont bother with the former. Additionally, in the case that row stats arent calculated for some reason, running ANALYZE to get this estimate is v slow - 17s for the 311 data, compared to 1.7s for SELECT COUNT(*) so lets not run ANALYZE in this case.


  Commit: c078f75c63e8756b56114309325a05cf3e773890
      https://github.com/ckan/ckan/commit/c078f75c63e8756b56114309325a05cf3e773890
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  threshold=None is now the way to turn off the estimating - makes more sense than threshold=0.


  Commit: e2f7a6d368c6c28bb5faf116bfd9e19bc5a71281
      https://github.com/ckan/ckan/commit/e2f7a6d368c6c28bb5faf116bfd9e19bc5a71281
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/logic/action.py

  Log Message:
  -----------
  Improve wording


  Commit: aa32b2379a5b2e52d3c95e4e985719f70a3bd0d5
      https://github.com/ckan/ckan/commit/aa32b2379a5b2e52d3c95e4e985719f70a3bd0d5
  Author: Ian Ward <ian at excess.org>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/auth.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/helpers.py
    M ckanext/datastore/tests/test_create.py
    M ckanext/datastore/tests/test_delete.py
    M ckanext/datastore/tests/test_search.py
    M ckanext/datastore/tests/test_upsert.py
    M ckanext/reclineview/theme/public/vendor/ckan.js/ckan.js
    M ckanext/reclineview/theme/public/vendor/recline/recline.js

  Log Message:
  -----------
  Merge pull request #4473 from ckan/datastore-estimate-total

Datastore search can estimate total, rather than expensive COUNT(*) [WIP]


Compare: https://github.com/ckan/ckan/compare/3549e3b088cf...aa32b2379a5b
      **NOTE:** This service has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.