[ckan-dev] File upload via API and ckanclient

Thu Mar 28 13:24:54 UTC 2013

Well, I was right with my assumption that _post_multipart in
ckanclient is responsible (and I should have looked there in the
first place). It turned out that it's missing the
authentification headers. Putting them there like so:

    h.putheader('Authorization', self.api_key)
    h.putheader('X-CKAN-API-Key', self.api_key)

resolves the problem and uploads the file.

Unfortunately now there are some other problems (esp. with the
files url) but I think that could be dealt with somehow.

BTW: anyone here who contributes to ckanclient? 

best regards, and sorry for the noise
hendrik

--On 2013-03-28 12:53, Hendrik Bunke wrote:
> Hi,
> 
> I'm trying to upload a file via API and with ckanclient (Python).
> It doesn't work and as far as can see there is an authorization
> error somewhere in CKAN. I'm aware that there have been others on
> this list with similar problems, but so far I haven't found a
> solution. So I did some debugging, and here's what I found.
> 
> For this test and debugging purposes upload is done with a very
> simple script that submits the API key, defines the filename and
> so on. Upload to a local CKAN instance is then done via
> ckanclient.upload_file(filename). 
> 
> I've set a pdb at ckan/controllers/storage.py in function
> authorize (I've tried many others, but this seems to be the most
> helpful). CKAN log is set to debug, now let's see what happens:
> 
>     2013-03-28 11:43:19,750 DEBUG [ckan.lib.base] Received API Key: <some-long-here-hidden-key>
>     2013-03-28 11:43:19,758 DEBUG [ckan.logic] check access - user u'bunke', action dashboard_new_activities_count
>     2013-03-28 11:43:19,758 DEBUG [ckan.logic] Access OK.
>     2013-03-28 11:43:19,759 DEBUG [ckan.logic] check access - user u'bunke', action dashboard_activity_list
>     2013-03-28 11:43:19,759 DEBUG [ckan.logic] Access OK.
>     > /home/bunke/dev/ckan/src/ckan/ckan/controllers/storage.py(106)authorize()
>     -> if not is_authorized:
> 
> Everything's fine here. Asking for is_authorized:
>     
>     (Pdb) is_authorized
>     True
> 
> So, API key is correct and authorization (called by storage.py
> upload_handle) for file upload seems to be correct, right?
> 
> Going some steps further: 
> 
>     (Pdb) n
>     > /home/bunke/dev/ckan/src/ckan/ckan/controllers/storage.py(418)auth_form()
>     -> data = self._get_form_data(label)
>     (Pdb) data
>     {'action': '/storage/upload_handle', 'fields': [{'name': 'key', 'value': u'2013-03-28T114319/2011-33.html'}]}
> 
> even the necessary userobject is still available and correct:    
> 
>     (Pdb) print c
>     <pylons.util.AttribSafeContextObj at 0xb29bdec action=u'auth_form',
>     author=u'bunke', controller=u'ckan.controllers.storage:StorageAPIController',
>     environ={'HTTP_AUTHORIZATION': '<API-key>'},
>     label=u'2013-03-28T114319/2011-33.html', language='en', new_activities=0,
>     pylons=<pylons.util.PylonsContext object at 0xb175fac>,
>     remote_addr='127.0.0.1', start_response=<function repl_start_response at
>     0xb28dfb4>, user=u'bunke', userobj=<User
>     id=a3bd42dd-3583-40d9-97c7-76cade9f8ac0 name=bunke ope...True>>
> 
> After that some wsgi and middleware actions are performed, and
> then suddenly we are back to storage.py.authorize:
> 
>     (Pdb) cont
>     2013-03-28 11:49:58,777 DEBUG [ckan.new_authz] Using auth profile at ckan.logic.auth
>     > /home/bunke/dev/ckan/src/ckan/ckan/controllers/storage.py(106)authorize()
>     -> if not is_authorized:
> 
> And --surprise, surprise-- there's no authorization anymore:
>     (Pdb) is_authorized
>     False
> 
> Somewhere in the WSGI process authorization got lost, or did I
> get something completely wrong here? The last call before going
> back to authorize came from here:
> 
>     /home/bunke/dev/ckan/local/lib/python2.7/site-packages/paste/httpserver.py(296)wsgi_execute()->None
>     -> result = None
> 
> Why am I back to authorize here? I could dig my way through the
> wsgi code but that's hard. There's the private method in
> ckanclient _post_multipart(). Does it really use the credentials
> (e.g. the API key?)
> 
> thanks in advance
> hendrik
> 
> 

-- 
Dr. Hendrik Bunke
http://gplus.to/hbunke
http://twitter.com/hbunke
http://www.hbxt.org