[ckan-dev] Datastore & filestore via API

Tue Aug 7 13:45:44 UTC 2012

Hi Michele,

i've just writen a code to do exactly the same you are talking about. I'm
cleaning it up (removing client stuff) to publish it, but i can point you
some directions in the meantime.

Let's address to you problems individually:

2012/8/7 Michele Barbera <barbera at spaziodati.eu>

> 1) Create a new dataset via the API
>

Ok, if you did it already, we can just ignore it.

> 2) Create a new resource for this dataset and upload a zip file in the
> ckan-filestore (local)
>

I would break your code in two parts: create a new resource and upload a
zip file in filestore.

*Create a new resource*

You could use ckanclient extension to do this:
https://github.com/okfn/ckanclient

        # Get API user and website domain data
        user = get_action('get_site_user')({'model': model, 'ignore_auth':
True}, {})
        ckan_url = config.get('app:main','ckan.site_url').rstrip('/')

        # Ckan client data and api URL
        api_url = urlparse.urljoin(config.get('app:main','ckan.site_url'),
'api')
        ckan = ckanclient.CkanClient(
                                     base_location=api_url,
                                     api_key=user.get('apikey'),
                                     is_verbose=True,
                                    )

        # This dict will contain resource data
        metadata = dict()
        metadata['url'] = 'http://something'
        identifier = 'some_name'
        format = 'xml'

        # Describe
        resource_entity = {
                           'name' : identifier,
                           'url' : metadata['url'],
                           'format' : format,
                           'extras' : metadata
        }

        package_entity = ckan.package_entity_get(package_name)

        # Add resource as an element for package. It's the only way to save
        package_entity['resources'].append(resource_entity)

        # This will update the package in package_entity
        ckan.package_entity_put(package_entity)

*Store file*
*
*
To store file you will could use the ofs library. Take a look at this code:
https://github.com/eduardosan/ckanclient/blob/master/ckanclient/loaders/util.py

> 3) Create another resource for the same dataset and upload a CSV file into
> the ckan datastore
>

This should do the trick:

        # Authenticate and remove resource from system
        user = get_action('get_site_user')({'model': model, 'ignore_auth':
True}, {})
        api_url = urlparse.urljoin(config.get('app:main','ckan.site_url'),
'api')

        # Elastic search URL
        webstore_request_url = '%s/api/data/%s/%s' % (ckan_url,
self.package_name, identifier)
        client = DataStoreClient(webstore_request_url)

        # Convert data to Json format
        metadata['id'] = identifier
        post_data = json.dumps(metadata, ensure_ascii=False)
        headers = dict()
        headers['Authorization'] = user.get('apikey')
        client._headers = headers
        list_data = list()
        list_data.append(post_data)

        # Insert data on Elastic Search
        self.response = client.upsert(list_data)

You will need DatastoreClient to do so.

I hope it helps.

-- 
Eduardo Santos
Analista de Sistemas

http://www.eduardosan.com
http://twitter.com/eduardosan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20120807/2debfd1e/attachment-0001.html>