[ckan-dev] datastore_active flag overwritten during harvesting

Wed Jan 23 10:51:46 UTC 2019

Hi,

I'm working on a CKAN setup that makes use of ckanext-harvest, ckanext-dcat
(with a custom profile), DataStore and DataPusher. I'm facing a bug where
during the harvesting process for a dataset with lots of resources, many of
them end the process with the datastore_active flag set to false (thus
views are not created).

The resources that suffer from this vary between each execution of the
harvesting process. I can confirm the data is correctly indexed in the
DataStore, and that at some point of the process the flag was correctly set
to True (so I think something is overwriting it later).

For the resources that fail, I've checked that when datapusher_hook is
called, the flag has already been set to False.

I've seen a couple of issues related to this, but one was already solved
and the solution proposed in the other did not work for me:

- https://github.com/ckan/ckan/issues/3245
- https://github.com/ckan/datapusher/issues/100

¿Has anyone experienced this before? ¿Any clues about what can be happening?

Thanks!

Alberto Miedes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20190123/709ab76a/attachment-0001.html>