[ckan-dev] how may I add a new harvesting source?

Mon Oct 14 07:49:02 UTC 2013

Dear Adrià,

Thanks for answering. Indeed this was the first command that I tried to run:

paster --plugin=ckanext-harvest harvester source source-name-test
url-to-source-csw csw 

(from the development.ini folder).
Then I tried with CSW, and other obvious combinations.

The output that I get now (with stable version 2.1, before was version 2.2)
is:

[1] 16840
[2] 16841
-bash: csw: command not found
(pyenv)[ckan at mtester ckan]$
Please provide a source type

[1]-  Exit 1                  paster --plugin=ckanext-harvest harvester
source source source-name-test url-to-source-csw 
[2]+  Done                    service=CSW

So, it seems that the "csw" parameter is not get as a parameter but as a
bash command. 
Anyway, it recognizes the service as a CSW by itself, but it asks for a
service type. 

If I have a look at the list of registered sources, this is not included, so
it has not been added (I'm able to add it from the web interface, going
directly to page ../harvest/new).

Kind regards,
-Elena 

> -----Original Message-----
> From: ckan-dev-bounces at lists.okfn.org [mailto:ckan-dev-
> bounces at lists.okfn.org] On Behalf Of Adrià Mercader
> Sent: Friday, October 11, 2013 7:06 PM
> To: CKAN Development Discussions
> Subject: Re: [ckan-dev] how may I add a new harvesting source?
> 
> Hi Elena,
> 
> Don't worry about the "sources" command not showing the type, it's just a
> bug.
> 
> Creating a source via the CLI works for me, for example using:
> 
> paster harvester source test-csw-source http://url.for.the.csw csw -c
> ../ckan/development.ini
> 
> Note the order of the params (You can get help for the command running
> "paster harvester -h -c ../ckan/development.ini")
> 
> harvester source {name} {url} {type} [{title}] [{active}] [{owner_org}]
> [{frequency}] [{config}]
>     - create new harvest source
> 
> Does this work for you?
> 
> The CLI is not very maintained, so there may be some bug somewhere.
> 
> Adrià
> 
> 
> 
> On 11 October 2013 13:45, Elena Camossi
> <elena.camossi at ext.jrc.ec.europa.eu> wrote:
> > Hi everyone,
> >
> >> There should be a page for inserting new harvesting sources at
> >>   ... /harvest/new
> >
> > I inserted a new CSW harvesting source using the form from this page.
> > I can see it from http://mysite/harvest under the csw type, and going
> > to the details of the source, I check that the source type is correctly
set to
> CSW.
> > However, if I run the following command from the python virtual
> > environment pyenv to list all the sources
> >
> > paster --plugin=ckanext-harvest harvester sources
> >
> > I see that every CSW source has source type set to None.
> >
> > So, If I want to use the CLI to add a new source instead, running
> > harvester source , I still don't know how to specify this source type
> > and getting either error
> >
> > 'Please provide a source type'
> > or
> > 'ckan.logic.ValidationError: {'name': ['Url must be purely lowercase
> > alphanumeric (ascii) characters and these symbols: -_'], 'source_type':
> > [u'Unknown harvester type: None. Have you registered a harvester for
> > this type?']}'
> >
> > Can someone add something on this? It's not a problem anymore, I have
> > added the source already, but I would like to be able to do it from the
CLI
> too.
> >
> > Kind regards,
> > .e
> >
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev