[ckan-dev] pycsw

Fri Oct 11 12:03:03 UTC 2013

Dear Adria',

I answer on behalf of Hilde for what I can, just to speed up the thread. I
hope this will not mess up.  

> -----Original Message-----
> From: ckan-dev-bounces at lists.okfn.org [mailto:ckan-dev-
> bounces at lists.okfn.org] On Behalf Of Adrià Mercader
> Sent: Friday, October 11, 2013 1:06 PM
> To: CKAN Development Discussions
> Subject: Re: [ckan-dev] pycsw
> 
> Hi Hilde,
> 
> I'm sorry to hear that you are finding problems. The ckan-pycsw is
relatively
> recent addition and may still have some rough edges.
> 
> Let's take it one step at a time and see if we can make progress, see
below
> for comments.
> 
> On 10 October 2013 15:19, Hildegard Gerlach
> <hildegard.gerlach at jrc.ec.europa.eu> wrote:
> > Dear Angelos and Adrià,
> > I am using Redhat 6, ckan version 2.2a PostgreSQL 8.4.13, Postgis
> > 1.5.3
> >  SQLAlchemy-0.7.8   (installed by CKAN requirements.txt)
> 
> I strongly recommend always using stable versions unless you are
developing
> CKAN or the extensions. Use ckan 2.1, and the "stable"
> branches of ckanext-spatial and ckanext-harvest The rest of the versions
> seem fine
> 

See below for the comment on version downgrade

> > I am using a Postgis database for CKAN and have done harvesting from 2
> > remote sources.
> Great.
> 
> > However I don't manage to load these sources into the newly created
> > database for pycsw.
> > In the pycsw.cfg file I have only changed the home and database
settings.
> > The database table has been created successfully with this command
> >
> > paster ckan-pycsw setup -p /etc/ckan/default/pycsw.cfg
> That sounds fine
> 
> 
> > In the meantime I went back to version 2.1, but I am not sure if this
> > made any change.
> > I did
> >
> > git checkout release-v2.1
> > git pull
> >
> > pip install --upgrade -r requirements.txt
> >
> > for CKAN, the harvester and spatial extension. Is this the correct way ?
> I wouldn't use the --upgrade flag unless you find some version number
> conflicts. This will upgrade all dependencies which might have unexpected
> results
> 
>

What do you suggest then? Is there any way to go back to version 2.1
avoiding to install everything from scratch?

> > When harvesting before I had not specified anything in the
> > development.ini file for Now I have set it to
> > ckan.spatial.validator.profiles = iso19193eden
> That is good, this is probably the best profile for generic iso documents
> (iso19193ngdc is even a bit more permissive)
> 
> > and tried to reharvest, but then I get another error:
> > Error gathering the identifiers from the CSW server [Please use the
> > updated 'getrecords2' method instead of 'getrecords'. The 'getrecords'
> > method will be upgraded to use the 'getrecords2' parameters in a
> > future version of OWSLib
> That might be an issue with an incompatible version of OWSLib that got
> installed when you upgraded the requirements, we've had some issues with
> incompatibilities with owslib.
> 
> What version of OWSLib do you have installed? Run the following with your
> virtualenv activated and post the result:
> 
> pip freeze | grep -i owslib

this is the output:

-e
git+https://github.com/geopython/OWSLib.git@4b0a62cd37a5a03aafc5c0cd9606e065
8d5ac664#egg=OWSLib-dev

> 
> > A workaround, but not solution, is here:
> > https://www.seegrid.csiro.au/wiki/Infosrvices/CKANHarvestingGuide
> > The workaround (not a fix) for the above error is to comment out the
> > lines
> > (186-188) that raised the DeprecationWarning exception in the
> > following Python source file
> (/usr/lib/ckan/default/src/owslib/owslib/csw.py).
> > #raise DeprecationWarning("""Please use the updated 'getrecords2'
> > method instead of 'getrecords'. #The 'getrecords' method will be
> > upgraded to use the 'getrecords2' parameters #in a future version of
> > OWSLib.""")

> Sounds like we should update our harvesters to use getrecords2. We have
> never found this, so there definitely seems to be a conflict with a newer
> version of OWSlib.
> We'll investigate, but we'll need to OWSLib version you are using that you
can
> find with the command I mentioned before.
> 
> 
> > Anyway, if  I look at the harvest object like you described I don't
> > know what is wrong with it. It is too big to post on the list.
>
> Can you post it online somewhere and link to it? I'm curious to see what
> might be causing the ckan-pycsw command to fail.
> 
> 
> > I have also another problem, I cannot see the "Add harvest source"
> > button on the
> >
> > http://{my-ckan-instance}/harvest
> >
> > Website. It has disapperared.
> Make sure that you are using the latest "stable" branch on
ckanext-harvest,
> we fixed some problems with this a while ago:
>
> git checkout stable
> git pull
> 
> (and of course that you are logged in)
> 

Running the 1st command I got this error: error: You have local changes to
'ckan/templates/package/read.html'; cannot switch branches.

Anyway, with this "temporary" 2.1 version the button is still missing. I
have posted the same question on the list in a separate mail and the
temporary solution is to go directly to the page http://mysite/harvest/new.
This worked.

> 
> > Sorry, but I am getting so many issues at the same time.
> Sorry about the troubles, hopefully we'll get there.
> 
> Adrià
> 

Thanks a lot for your help.
-Elena 