[ckan-dev] Unable to upload file to S3 using ckanclient

Tue Feb 19 19:17:25 UTC 2013

Hi there,

I'm struggling the the file upload using ckanclient. I setup my S3 bucket
and everything works fine when using the web frontend (file is uploaded
correctly, the path works etc.)

But when I try to do the same with ckanclient using the upload_file method
(using the example from the README:
https://github.com/okfn/ckanclient/blob/master/README.rst).

My code:

ckan = ckanclient.CkanClient(api_key="<api key>", base_location="
http://ckan.dev:5000/api")

# Register the dataset.
dataset_entity = {
    'name': 'my_upload_dataset',
    'tags': 'test',
    'notes': 'Notes about the test',
    }
ckan.package_register_post(dataset_entity)

# Download the file
test_url = '
https://commondatastorage.googleapis.com/ckannet-storage/2011-11-24T112025/AfTerFibre_21nov2011.csv
'
file_name = test_url.split('/')[-1]
urllib.urlretrieve (test_url, file_name)

# Upload the file to S3
dataset_entity = ckan.add_package_resource(dataset_name, file_name,
name=file_name, resource_type='data', description='Some CSV', format='csv')

After that I have new dataset in CKAN, referencing the following URL:
http://ckan.liip.ch.s3.amazonaws.com/storage/f/2013-02-19T184215/AfTerFibre_21nov2011.csv

This URL is wrong and the file does not get uploaded at all. But ckanclient
does not indicate an error or anything.

Am I doing something completely wrong? Or is S3 not supported by
ckanclient? I saw that there is an open pull requests concerning S3 (
https://github.com/okfn/ckanclient/pull/14). Is this going to be merged
soon? Is it necessary?

Sorry for all my newbish questions, but I'm really stuck here.

Best Regards
Stefan

-- 
Liip AG  //  Feldstrasse 133 //  CH-8004 Zurich
Tel +41 43 500 39 81 // GnuPG 0x7B588C67 // www.liip.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130219/f7d21835/attachment.html>