[ckan-dev] Should the CKAN Harvester copy groups?

[Konrad Reiche]

Hi Adrià,

that is a great proposal and I have started a fork with a feature branch 
tackling that problem.

Do you have any idea how one would handle API 1 vs. 2 or later? API 1 
uses the names to identify groups and 2 or later uses the unique identifier.

If I want to import the groups, should I use in API 1 the name and in 
API 2 the unique identifier for that task? What if you have on your 
local CKAN the groups "Population", "Water" and "Taxes" and on a remote 
CKAN as well, but with different unique identifiers? Tough look and one 
should configure it then to API 1 or should it look on the name as well 
with API 2?

For API reasons I guess the former case, but what do you think about that?

Best,
Konrad

On 02/07/2013 12:49 PM, Adrià Mercader wrote:
> Hi Konrad,
>
> Dealing with remote groups is tricky, that's why we decided to not
> implement it for the time being. What happens if the remote group does
> not exist locally? Do you try to recreate it with the remote group
> details? For instances on where  you know beforehand what groups will
> you find that could make sense, but on others like Publicdata.eu that
> harvest very different instances you probably don't want random groups
> being created.
> One possible implementation would be to have a config option to manage
> group creation. Default should be to ignore groups (as it is now), but
> you could also set it to "only_local" or something like that to only
> assign datasets to a group if it exists locally, or "create" to try
> and recreate remote groups on your local instance. We probably would
> need to think about authorization implications for this.
> It is unlikely that we will get time on the core team to do this soon,
> so if you really need this for your project perhaps you could try and
> do a patch + pull request? :)
>
> Hope this helps,
>
> Adrià
>
>
>
> On 6 February 2013 14:40, Konrad Reiche <konrad.reiche at gmail.com> wrote:
>> Hello everyone,
>>
>> I was using the CKAN Harvester extension to harvest packages from a remote
>> CKAN instance.
>> Unfortunately, the harvest process does not copy the CKAN's remote groups,
>> which is also
>> documented here:
>>
>> https://github.com/okfn/ckanext-harvest/blob/master/ckanext/harvest/harvesters/ckanharvester.py#L248
>>
>> "Ignore remote groups for the time being"
>>
>> Would it make sense to add a group registration here, so that the groups can
>> be copied?
>>
>> Best,
>> Konrad
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev