[ckan-changes] [ckan/ckan] e6d636: datastore_search limit has a default which is NOW ...

Sun Jan 27 21:34:26 UTC 2019

  Branch: refs/heads/master
  Home:   https://github.com/ckan/ckan
  Commit: e6d6361a8bff29b9eb8dd461c8e5e6cb7c252ea6
      https://github.com/ckan/ckan/commit/e6d6361a8bff29b9eb8dd461c8e5e6cb7c252ea6
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_search.py
    M doc/maintaining/configuration.rst

  Log Message:
  -----------
  datastore_search limit has a default which is NOW configurable with ckan.datastore.search.rows_max


  Commit: ab19c888fcdf1401bf2c1848e6b24a6278778a17
      https://github.com/ckan/ckan/commit/ab19c888fcdf1401bf2c1848e6b24a6278778a17
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  Modernize some search tests


  Commit: b381dc23da5b8d1af2ea5a063dd4538e6426efb2
      https://github.com/ckan/ckan/commit/b381dc23da5b8d1af2ea5a063dd4538e6426efb2
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py

  Log Message:
  -----------
  Initial work on limit added to datastore_search_sql


  Commit: e647bbd665bc5e79cf92762ccc79ad856ef0deb0
      https://github.com/ckan/ckan/commit/e647bbd665bc5e79cf92762ccc79ad856ef0deb0
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-23 (Fri, 23 Nov 2018)

  Changed paths:
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  Try to get tests passing on circleci. Both version ok on 2.7.6 but circleci is 2.7.15 I think


  Commit: 33aefe9ea6d9f7f034a80af03a6dde5c0ba71014
      https://github.com/ckan/ckan/commit/33aefe9ea6d9f7f034a80af03a6dde5c0ba71014
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-30 (Fri, 30 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/tests/test_search.py
    M doc/maintaining/configuration.rst

  Log Message:
  -----------
  Add limit to datastore_search_sql


  Commit: 1f0cde2ac933cbbabab3edc52a0c307394b4049d
      https://github.com/ckan/ckan/commit/1f0cde2ac933cbbabab3edc52a0c307394b4049d
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-11-30 (Fri, 30 Nov 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py

  Log Message:
  -----------
  Simplify


  Commit: d31f688cbc59fee6ede6b70a0a8120ea01fecdaa
      https://github.com/ckan/ckan/commit/d31f688cbc59fee6ede6b70a0a8120ea01fecdaa
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/tests/test_dump.py

  Log Message:
  -----------
  Modernize test_dump.py


  Commit: a65294d4b1513ff14ed89fd8d02099992c48a8cd
      https://github.com/ckan/ckan/commit/a65294d4b1513ff14ed89fd8d02099992c48a8cd
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py

  Log Message:
  -----------
  records_truncated key is only returned if the result is truncated. Simpler and more backwards compatible for clients.


  Commit: 9ec856b02f5c39047dadd21b07d2666c16896a1a
      https://github.com/ckan/ckan/commit/9ec856b02f5c39047dadd21b07d2666c16896a1a
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/controller.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_dump.py

  Log Message:
  -----------
  Compatibility between PAGINATE_BY and rows_max. Fixes a bug where if the rows_max is less than PAGINATE_BY then dump_as misses records.


  Commit: 1d4d27de8327814e61bffbc8fa7cfdbc9af2d54e
      https://github.com/ckan/ckan/commit/1d4d27de8327814e61bffbc8fa7cfdbc9af2d54e
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py
    M doc/maintaining/configuration.rst

  Log Message:
  -----------
  Default rows_max is now 32000


  Commit: 5ed1b2d18da13e979485de30a833756007f8897e
      https://github.com/ckan/ckan/commit/5ed1b2d18da13e979485de30a833756007f8897e
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  datastore_search now also has returns "records_truncated"


  Commit: 613ba51443155cb83523e8bdb1f2ca5395f28618
      https://github.com/ckan/ckan/commit/613ba51443155cb83523e8bdb1f2ca5395f28618
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/controller.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/tests/test_dump.py

  Log Message:
  -----------
  Different way to attack the dump bug - WIP


  Commit: 54b56ba6c16ceac6718e45b6cd97f6093a53e529
      https://github.com/ckan/ckan/commit/54b56ba6c16ceac6718e45b6cd97f6093a53e529
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    A ckan/cli/__init__.py
    A ckan/cli/cli.py
    A ckan/cli/db.py
    A ckan/cli/search_index.py
    A ckan/cli/server.py
    M ckan/config/middleware/flask_app.py
    M ckan/config/solr/schema.xml
    M ckan/controllers/api.py
    M ckan/lib/dictization/model_dictize.py
    M ckan/lib/dictization/model_save.py
    M ckan/lib/helpers.py
    M ckan/lib/search/__init__.py
    M ckan/lib/search/query.py
    M ckan/logic/action/get.py
    M ckan/logic/action/patch.py
    M ckan/tests/legacy/lib/test_resource_search.py
    M ckan/tests/legacy/logic/test_action.py
    M ckan/tests/lib/dictization/test_model_dictize.py
    M ckan/tests/logic/action/test_get.py
    M ckan/views/__init__.py
    M ckan/views/user.py
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/auth.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/helpers.py
    M ckanext/datastore/tests/test_create.py
    M ckanext/datastore/tests/test_delete.py
    M ckanext/datastore/tests/test_dump.py
    M ckanext/datastore/tests/test_search.py
    M ckanext/datastore/tests/test_upsert.py
    M ckanext/reclineview/theme/public/vendor/ckan.js/ckan.js
    M ckanext/reclineview/theme/public/vendor/recline/recline.js
    M doc/theming/index.rst
    M setup.py

  Log Message:
  -----------
  Merge branch 'dump' into 4561-limit-datastore_search


  Commit: fc0b4650d649569df93029db0dad6b8be8f0c396
      https://github.com/ckan/ckan/commit/fc0b4650d649569df93029db0dad6b8be8f0c396
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/controller.py
    M ckanext/datastore/tests/test_dump.py
    M doc/maintaining/configuration.rst
    M doc/maintaining/datastore.rst

  Log Message:
  -----------
  X-Records-Up-To-Rows-Max implemented for Datastore dump call. Pretty complicated - not sure if it is worth it.


  Commit: 497795cc3770a6ffac5501a90039341033e6a4b0
      https://github.com/ckan/ckan/commit/497795cc3770a6ffac5501a90039341033e6a4b0
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/controller.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_dump.py
    M ckanext/datastore/tests/test_search.py
    M doc/maintaining/configuration.rst
    M doc/maintaining/datastore.rst

  Log Message:
  -----------
  datastore dump is NOT limited by rows_max.


  Commit: 74d8fb2ff4f77d8c20d4eef2508aaaf166769df8
      https://github.com/ckan/ckan/commit/74d8fb2ff4f77d8c20d4eef2508aaaf166769df8
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/controller.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/tests/test_dump.py

  Log Message:
  -----------
  Reworded comments and made tests easier to understand.


  Commit: 0e0c9573a6f5cd7af44fd78c42c9e633c9acb1d7
      https://github.com/ckan/ckan/commit/0e0c9573a6f5cd7af44fd78c42c9e633c9acb1d7
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-07 (Fri, 07 Dec 2018)

  Changed paths:
    M ckanext/datastore/tests/test_dump.py
    M doc/maintaining/configuration.rst

  Log Message:
  -----------
  Pep8


  Commit: f91f818fc9b5adae16420c239b8cbd67b6546867
      https://github.com/ckan/ckan/commit/f91f818fc9b5adae16420c239b8cbd67b6546867
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2018-12-14 (Fri, 14 Dec 2018)

  Changed paths:
    M ckanext/datastore/backend/postgres.py

  Log Message:
  -----------
  Use pop() for efficiency


  Commit: 51337ff02aaf1ac2f216563a4c0c5b1a27c5700a
      https://github.com/ckan/ckan/commit/51337ff02aaf1ac2f216563a4c0c5b1a27c5700a
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2019-01-25 (Fri, 25 Jan 2019)

  Changed paths:
    M CHANGELOG.rst
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_search.py
    M doc/maintaining/configuration.rst

  Log Message:
  -----------
  Make configurable the default limit of datastore_search. Note it in the docs & changelog.


  Commit: 8bc80d7a612ed644ed137709427da5817da2a7e2
      https://github.com/ckan/ckan/commit/8bc80d7a612ed644ed137709427da5817da2a7e2
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2019-01-25 (Fri, 25 Jan 2019)

  Changed paths:
    M CHANGELOG.rst

  Log Message:
  -----------
  limit has defaulted to 100 since 2012, so no need to mention here


  Commit: 4171ce9abb8a0b2e77979022ef285f8ef95b7e5b
      https://github.com/ckan/ckan/commit/4171ce9abb8a0b2e77979022ef285f8ef95b7e5b
  Author: David Read <david.read at hackneyworkshop.com>
  Date:   2019-01-27 (Sun, 27 Jan 2019)

  Changed paths:
    M CHANGELOG.rst
    M ckanext/datastore/backend/postgres.py
    M ckanext/datastore/controller.py
    M ckanext/datastore/logic/action.py
    M ckanext/datastore/logic/schema.py
    M ckanext/datastore/tests/test_dump.py
    M ckanext/datastore/tests/test_search.py
    M doc/maintaining/configuration.rst
    M doc/maintaining/datastore.rst

  Log Message:
  -----------
  Merge pull request #4562 from ckan/4561-limit-datastore_search

Add a hard upper limit to datastore_search(_sql) rows returned


Compare: https://github.com/ckan/ckan/compare/b8568db82440...4171ce9abb8a
      **NOTE:** GitHub Services has been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      We will provide an alternative path for the email notifications by January 31st, 2019.