[ckan-dev] connecting Drupal and CKAN

Fri Sep 30 13:58:45 UTC 2011

Hi David

Thanks for this.

Mart and Paul

How are things now?

Best wishes

Jillian Mathews

On 27 Sep 2011, at 21:58, David Raznick wrote:

> Hello
> 
>> In the README.txt of the ckanpackage module it says:
>> 
>>         “If you now run the drush command
>> 
>>         drush  ckan_checkallandnew
>> 
>>         All the data packages on the CKAN site will be created as Drupal
>> nodes."
>> 
>> 
>> 
>> This works fine if there are no ckanpacakge nodes in the Drupal database
>> already.
>> 
>> 
>> 
>> Then it says:
>> 
>>         "Periodically run the following command to query the CKAN API for
>> any updates and ensure the Drupal nodes are updated accordingly.
>> 
>>         drush ckan_update"
>> 
>> 
>> 
>> This is where our problem starts. The ckan_update command seems to work
>> erratically. Sometimes the updates on packages in the CKAN repository
>> (changed values, new packages, deleted packages) get through, but most of
>> the time they do not. Sometimes duplicate or empty nodes (even without a
>> title) are created in the Drupal database. Most of the duplicates get the
>> “unpublished” state by the way (I don’t know if this is by design for
>> updated nodes).
>> 
>> 
>> 
>> After concluding that the ckan_update command was not working reliably we
>> tried to use the ckan_checkllandnew command instead (knowing that it would
>> go through all packages again). This also did not work reliably if there
>> were already packages in the Drupal database.
>> 
>> 
>> 
>> We finally resorted to automatically deleting all ckan packages from the
>> Drupal database every night at 01:00 and running the ckan_checkallandnew
>> command at 01:15 to update the Drupal database on a daily basis.
>> 
>> 
>> 
>> My first question is if you have a newer version of the module available
>> that possibly addresses issues with updating the ckanpackage nodes in
>> Drupal?
> 
> The latest version is located in the ckan_dgu branch of the repository
> on drupal.org.
> 
> i.e
> http://drupalcode.org/project/ckan.git/shortlog/refs/heads/ckan_dgu
> 
> We have not had any particular problems with the dgu import process
> for quite a while, however there setup may be different from yours.
> 
> I have copied in ckan-dev, and can any follow ups also be sent there
> because there are more people looking at the list that may be able to
> help.
> 
> Thanks
> 
> David