[ckan-dev] File upload via API and ckanclient

[Hendrik Bunke]

Hi,

I'm trying to upload a file via API and with ckanclient (Python).
It doesn't work and as far as can see there is an authorization
error somewhere in CKAN. I'm aware that there have been others on
this list with similar problems, but so far I haven't found a
solution. So I did some debugging, and here's what I found.

For this test and debugging purposes upload is done with a very
simple script that submits the API key, defines the filename and
so on. Upload to a local CKAN instance is then done via
ckanclient.upload_file(filename). 

I've set a pdb at ckan/controllers/storage.py in function
authorize (I've tried many others, but this seems to be the most
helpful). CKAN log is set to debug, now let's see what happens:

    2013-03-28 11:43:19,750 DEBUG [ckan.lib.base] Received API Key: <some-long-here-hidden-key>
    2013-03-28 11:43:19,758 DEBUG [ckan.logic] check access - user u'bunke', action dashboard_new_activities_count
    2013-03-28 11:43:19,758 DEBUG [ckan.logic] Access OK.
    2013-03-28 11:43:19,759 DEBUG [ckan.logic] check access - user u'bunke', action dashboard_activity_list
    2013-03-28 11:43:19,759 DEBUG [ckan.logic] Access OK.
    > /home/bunke/dev/ckan/src/ckan/ckan/controllers/storage.py(106)authorize()
    -> if not is_authorized:

Everything's fine here. Asking for is_authorized:
    
    (Pdb) is_authorized
    True

So, API key is correct and authorization (called by storage.py
upload_handle) for file upload seems to be correct, right?

Going some steps further: 

    (Pdb) n
    > /home/bunke/dev/ckan/src/ckan/ckan/controllers/storage.py(418)auth_form()
    -> data = self._get_form_data(label)
    (Pdb) data
    {'action': '/storage/upload_handle', 'fields': [{'name': 'key', 'value': u'2013-03-28T114319/2011-33.html'}]}

even the necessary userobject is still available and correct:    

    (Pdb) print c
    <pylons.util.AttribSafeContextObj at 0xb29bdec action=u'auth_form',
    author=u'bunke', controller=u'ckan.controllers.storage:StorageAPIController',
    environ={'HTTP_AUTHORIZATION': '<API-key>'},
    label=u'2013-03-28T114319/2011-33.html', language='en', new_activities=0,
    pylons=<pylons.util.PylonsContext object at 0xb175fac>,
    remote_addr='127.0.0.1', start_response=<function repl_start_response at
    0xb28dfb4>, user=u'bunke', userobj=<User
    id=a3bd42dd-3583-40d9-97c7-76cade9f8ac0 name=bunke ope...True>>

After that some wsgi and middleware actions are performed, and
then suddenly we are back to storage.py.authorize:

    (Pdb) cont
    2013-03-28 11:49:58,777 DEBUG [ckan.new_authz] Using auth profile at ckan.logic.auth
    > /home/bunke/dev/ckan/src/ckan/ckan/controllers/storage.py(106)authorize()
    -> if not is_authorized:

And --surprise, surprise-- there's no authorization anymore:
    (Pdb) is_authorized
    False

Somewhere in the WSGI process authorization got lost, or did I
get something completely wrong here? The last call before going
back to authorize came from here:

    /home/bunke/dev/ckan/local/lib/python2.7/site-packages/paste/httpserver.py(296)wsgi_execute()->None
    -> result = None

Why am I back to authorize here? I could dig my way through the
wsgi code but that's hard. There's the private method in
ckanclient _post_multipart(). Does it really use the credentials
(e.g. the API key?)

thanks in advance
hendrik


-- 
Hendrik Bunke
http://gplus.to/hbunke
http://twitter.com/hbunke
http://www.hbxt.org