[ckan-dev] ckanext-spatial and pycsw synchronization workflow

Tue Nov 19 18:05:29 UTC 2013

Hi Tom,

On 15 November 2013 00:34, Tom Kralidis <tomkralidis at gmail.com> wrote:

> ckanext-spatial docs mention that only harvested documents are made
> available to pycsw.

Feeding pycsw an ISO document (harvested from eg a CSW server on this
case) and let her deal with the parsing and storing in the repo seemed
like the easiest and fastest solution for a first step.

> What is blocking us from modifying
> https://github.com/okfn/ckanext-spatial/blob/master/bin/ckan_pycsw.py
> to sync all CKAN metadata, not just harvested metadata.

If I understood correctly (please correct me if wrong) pycsw always
needs an xml document to add a record to its repository.

If that is the case, if there was a reliable way of generating these
metadata files from a CKAN domain object (eg this [1] in JSON) as you
suggest later on we could import those as we do with the harvested
ones already.
The challenge will be to do this on a generic enough way that doesn't
involve users writing a custom template for their case. Ryan Clark
from the Arizona Geological Survey did a great job in this direction
using a template (adapted to their own needs) that was filled with
values from the CKAN dataset.

https://gist.github.com/rclark/5886908

It all basically depends on how easy is to generate meaningful ISO
documents from CKAN dataset that can be parsed and imported from
pycsw.

As an aside, is there a lower level API on pycsw that allows to create
records for instance from dict or list of values? This way we will
avoid having to generate the metadata file and will just need to adapt
the CKAN dict. (although I'm not sure if you still an actual metadata
file stored, as I see a "xml" field in the pycsw model)

Hope this makes sense,

Adrià

>
> Is this because we are unable to yield formal metadata (ISO, FGDC) XML
> from native CKAN local datasets?  If yes, if we had a JSON to ISO XML
> metadata converter for local datasets only (whereas harvested datasets
> would already be in a formal metadata standard), would this work?
>

> ..Tom

[1] http://demo.ckan.org/api/3/action/package_show?id=test-preview-geojson

> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev