[ckan-changes] [ckan/ckan] 8e4c5c: [#1725] Move validations to IDataStore's validate_...

Wed May 28 12:34:53 UTC 2014

  Branch: refs/heads/1725-extend-datastore
  Home:   https://github.com/ckan/ckan
  Commit: 8e4c5c78072b9728f7c742664f807995a0a834ab
      https://github.com/ckan/ckan/commit/8e4c5c78072b9728f7c742664f807995a0a834ab
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-27 (Tue, 27 May 2014)

  Changed paths:
    M ckanext/datastore/db.py
    M ckanext/datastore/interfaces.py
    M ckanext/datastore/plugin.py
    M ckanext/datastore/tests/test_interface.py

  Log Message:
  -----------
  [#1725] Move validations to IDataStore's validate_query() method

This commit is the first step towards moving all validations of the received
`data_dict` to a single place. This was the easiest way I could think of to
allow extensions to modify our validation rules.

Every IDataStore extension should implement a `validate_query()` method that
receives the context, the data_dict, and the list of fields the current
resource has. The data_dict is a copy, so the extension is able to mess with it
without problems.

Inside that method, the extension will check everything it understands (e.g.
filters, fields, sorts). When finding something, it should remove it from the
data_dict. In the end, return the data_dict with everything it understands
removed. This new data_dict will be passed to the next extension.

After we do this with every extension available, we expect to have a data_dict
with empty values. For example:

```
    # Before calling
    data_dict = {
  'fields': ['age', 'distance'],
  'filters': {
      'age': 25,
      'distance': 5
  }
    }

    # After calling
    data_dict = {
  'fields': [],
  'filters': {}
    }
```

In that case, the validation succeeds and we can keep going. If there's still
any element left in the data_dict, that means that no extension understands it,
and we'll throw a ValidationError.

  Commit: 6d24cca287e3f789be65b3282606f7c3d7ef8baf
      https://github.com/ckan/ckan/commit/6d24cca287e3f789be65b3282606f7c3d7ef8baf
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-27 (Tue, 27 May 2014)

  Changed paths:
    A ckanext/datastore/tests/sample_datastore_plugin.py
    M ckanext/datastore/tests/test_interface.py
    M setup.py

  Log Message:
  -----------
  [#1725] Create proper plugin for testing

Instead of loading the plugin in a hackish way, we create a proper plugin and
load it the same way any other plugins is loaded.

  Commit: 675126f9d4eef025b155f1600d6f6aa34ea25b68
      https://github.com/ckan/ckan/commit/675126f9d4eef025b155f1600d6f6aa34ea25b68
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-27 (Tue, 27 May 2014)

  Changed paths:
    M ckanext/datastore/db.py
    A ckanext/datastore/helpers.py
    M ckanext/datastore/logic/action.py
    A ckanext/datastore/tests/test_helpers.py
    M ckanext/datastore/tests/test_unit.py

  Log Message:
  -----------
  [#1725] Move db._get_list to datastore_helpers.get_list

This method was already being used by action.py, and we'll need it soon enough
in other parts of the code, so we'd better make it public.

  Commit: f84e4c7239d00f263ec1fa89a9686c186511bf2c
      https://github.com/ckan/ckan/commit/f84e4c7239d00f263ec1fa89a9686c186511bf2c
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-28 (Wed, 28 May 2014)

  Changed paths:
    M ckanext/datastore/db.py
    M ckanext/datastore/interfaces.py
    M ckanext/datastore/plugin.py
    M ckanext/datastore/tests/sample_datastore_plugin.py

  Log Message:
  -----------
  [#1725] Refactor DataStore's WHERE code to the DataStore plugin

The intent is for us to use the same extension points as third-party plugins,
so we can test that they work and that they're extensible enough.

  Commit: d2124503dfef1697965eca439decdd9902f43529
      https://github.com/ckan/ckan/commit/d2124503dfef1697965eca439decdd9902f43529
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-28 (Wed, 28 May 2014)

  Changed paths:
    M ckanext/datastore/db.py
    M ckanext/datastore/interfaces.py
    M ckanext/datastore/plugin.py

  Log Message:
  -----------
  [#1725] Adds search_data() extension point and change DataStore to use it

In a high level, extensions that implement this method are expected to add what
they want into a query_dict dictionary, defined as:

```
{
    'select': [],    # list
    'ts_query': '',  # string
    'sort': [],      # list
    'limit': 100,    # int
    'offset': 0      # int
}
```

This dictionary will be passed to each extension, and they can add or remove
attributes to it. Because of this, the order the extensions are loaded in the
.ini file matter.

At this point, the validations defined in validate_query() have already run.

Compare: https://github.com/ckan/ckan/compare/aa82afc577e9...d2124503dfef