[ckan-dev] Should the CKAN Harvester copy groups?

Thu Feb 7 11:49:09 UTC 2013

Hi Konrad,

Dealing with remote groups is tricky, that's why we decided to not
implement it for the time being. What happens if the remote group does
not exist locally? Do you try to recreate it with the remote group
details? For instances on where  you know beforehand what groups will
you find that could make sense, but on others like Publicdata.eu that
harvest very different instances you probably don't want random groups
being created.
One possible implementation would be to have a config option to manage
group creation. Default should be to ignore groups (as it is now), but
you could also set it to "only_local" or something like that to only
assign datasets to a group if it exists locally, or "create" to try
and recreate remote groups on your local instance. We probably would
need to think about authorization implications for this.
It is unlikely that we will get time on the core team to do this soon,
so if you really need this for your project perhaps you could try and
do a patch + pull request? :)

Hope this helps,

Adrià

On 6 February 2013 14:40, Konrad Reiche <konrad.reiche at gmail.com> wrote:
> Hello everyone,
>
> I was using the CKAN Harvester extension to harvest packages from a remote
> CKAN instance.
> Unfortunately, the harvest process does not copy the CKAN's remote groups,
> which is also
> documented here:
>
> https://github.com/okfn/ckanext-harvest/blob/master/ckanext/harvest/harvesters/ckanharvester.py#L248
>
> "Ignore remote groups for the time being"
>
> Would it make sense to add a group registration here, so that the groups can
> be copied?
>
> Best,
> Konrad
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>