[ckan-dev] Datasets without current revision

Thu Jan 5 18:14:58 UTC 2012

Hi all,

I'm investigating some exceptions that occur in the test.ckan.org
instance when rebuilding the search index. The exception returned is a
NotFound logic one:

  File "/home/okfn/var/srvc/test.ckan.net/src/ckan/ckan/lib/search/__init__.py",
line 134, in rebuild
    {'id': pkg.id}
  File "/home/okfn/var/srvc/test.ckan.net/src/ckan/ckan/logic/action/get.py",
line 517, in package_show_rest
    package_show(context, data_dict)
  File "/home/okfn/var/srvc/test.ckan.net/src/ckan/ckan/logic/action/get.py",
line 357, in package_show
    package_dict = package_dictize(pkg, context)
  File "/home/okfn/var/srvc/test.ckan.net/src/ckan/ckan/lib/dictization/model_dictize.py",
line 119, in package_dictize
    raise NotFound

This is where it's fired:
https://github.com/okfn/ckan/blob/master/ckan/lib/dictization/model_dictize.py#L145

After some digging, I found that they are caused by some of the
packages not having a "current" revision.

See for instance, the package "eu-ebr" in the test.ckan.org database:

test.ckan.org=# select name,state,revision_id,current from
package_revision where name = 'eu-ebr';
  name  | state  |             revision_id              | current
--------+--------+--------------------------------------+---------
 eu-ebr | active | 1d28dd1d-c3aa-483d-81b3-9be73080c089 | f
(1 row)

And on the thedatahub.org database:

ckan.net=> select name,state,revision_id,current from package_revision
where name = 'eu-ebr';
  name  | state  |             revision_id              | current
--------+--------+--------------------------------------+---------
 eu-ebr | active | 1d28dd1d-c3aa-483d-81b3-9be73080c089 | t
(1 row)

I'm not sure if this was caused by a bad DB import when reinstalling
test.ckan.org on s023 or something else is going on.
Any clues on that?

On a related note, shouldn't we capture these exceptions when
rebuilding the search index so the process does not stop and most of
the datasets are indeed indexed? (Right now, test.ckan.org only shows
607 datasets, because the 608th threw the exception)

Adrià