[ckan-changes] [ckan/ckan] 1a1400: [#1725] Add tests for when searching/deleting elem...

Thu May 22 20:49:28 UTC 2014

  Branch: refs/heads/1725-extend-datastore
  Home:   https://github.com/ckan/ckan
  Commit: 1a1400607fc871c221a29ebb19e1fa84bf08ed2d
      https://github.com/ckan/ckan/commit/1a1400607fc871c221a29ebb19e1fa84bf08ed2d
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-22 (Thu, 22 May 2014)

  Changed paths:
    M ckanext/datastore/tests/test_delete.py
    M ckanext/datastore/tests/test_search.py

  Log Message:
  -----------
  [#1725] Add tests for when searching/deleting elements passing invalid filters

  Commit: f3aa38783c6b70ff77e00737c1dd5a2aef31c628
      https://github.com/ckan/ckan/commit/f3aa38783c6b70ff77e00737c1dd5a2aef31c628
  Author: Vitor Baptista <vitor at vitorbaptista.com>
  Date:   2014-05-22 (Thu, 22 May 2014)

  Changed paths:
    M ckanext/datastore/db.py
    A ckanext/datastore/interfaces.py
    A ckanext/datastore/tests/test_interface.py

  Log Message:
  -----------
  [#1725] Adds IDataStore interface and extension point to change WHERE clauses

I prefered to extend on `where` instead of `search_data` and `delete_data`
separately to keep the extension code DRY.

The IDataStore extensions' `where` methods are called with the `filters` dict
received from the user. They must check if there's any filter they want to
process and create a `clause` list with elements as tuples following the
pattern:

    (SQL_CLAUSE, PARAM1, PARAM2, ...)

The SQL_CLAUSE can be anything valid on the WHERE clause, for example
`AVG(score) > %s AND AVG(score) < %s`. You have to substitute any user-defined
parameters for `%s`, doesn't matter what their type is (int, string, ...).
Then, for each `%s` inside SQL_CLAUSE, you need to add one PARAM. For example:

    ('AVG(score) > %s AND AVG(score) < %s', 10, 20)

This pattern is needed to be able to sanitize user input and avoid SQL
Injections.

After that, you'll need to delete every key from the `filters` dict that you've
processed. Following our example, you'll have to:

    del filters['score_between']

We need this because we validate the filters' column names against the
DataStore resource's column names. Any filter on a column that doesn't exist on
the resource is invalid and raises an error. If you didn't remove
`filters['score_between']`, you'll receive a "Field 'score_between' doesn't
exist" error.

This is done to give the user a friendlier error message, and avoid another
possible SQL Injection vector.

Compare: https://github.com/ckan/ckan/compare/1a1400607fc8^...f3aa38783c6b