[ckan-dev] file store api problems

[Nigel Babu]

Hi!

I'm not an R expert, but from basic reading, I think there's a few mistakes
you're making.

When you upload a file, you don't need to upload a JSON, in fact, you
shouldn't. You should send them as key value pairs, like in a form. This is
a bit confusing because you do need to send in JSON *if and only if* you're not
uploading a file. Because of how file uploads work, the API is a bit
inconsistent. Does that help? It might end up something like this (I'm *not* an
R person and thoroughly guessing):

POST(uploadResource,  config=add_headers(Authorization =
    ckanapi,"Content-Type"="application/json;charset=utf-8"),
    body=list(package_id="cmtresults2",format"="JSON",
    upload=upload_file("CleanedData/test.json")))


On Wed, Jun 11, 2014 at 09:30:18AM -0400, Sasha Cuerda wrote:
> Hello all,
>
> I am trying to set up some R scripts to upload files our ckan file store.
>
> Using the httr package
>
>  POST(uploadResource,  config=add_headers(Authorization =
> ckanapi,"Content-Type"="application/json;charset=utf-8"),
>           body=list("{\"package_id\":\"cmtresults2\",\"format\":\"JSON\"}",
> upload=upload_file("CleanedData/test.json")))
>
> Throws a 400 code with the following error;
>
>  Status: 400
>   Content-type: application/json;charset=utf-8
> "Bad request - JSON Error: Error decoding JSON data. Error:
> JSONDecodeError('Expecting value: line 1 column 2 (char 1)',) JSON data
> extracted from the request:
> '--------------------------775bdb2706256deb\\r\\nContent-Disposition:
> form-data;
> name=\"\"\\r\\n\\r\\n{\"package_id\":\"cmtresults2\",\"format\":\"JSON\"}\\r\\n--------------------------775bdb2706256deb\\r\\nContent-Disposition:
> form-data; name=\"upload\"; filename=\"test.jsonC\"\\r\\nContent-Type:
> application/octet-stream\\r\\n\\r\\n[\\n{  ...
> }\\n]\\r\\n--------------------------775bdb2706256deb--\\r\\n'"
>
> I suspect that the problem hear has to do with how httr is double escaping,
> but I'm not sure how to fix that.
>
> When I switch my code to
>
> POST(uploadResource,  config=add_headers(Authorization =
> ckanapi,"Content-Type"="application/json;charset=utf-8"),
>           body =
> "{\"package_id\":\"cmtresults2\",\"format\":\"JSON\",\"upload\":\"CleanedData/test.json\"}")
>
> I get a different error
>
> Status: 409
>   Content-type: application/json;charset=utf-8
> {"help": "Appends a new resource to a datasets list of resources.\n\n
>  :param package_id: id of package that the resource needs should be added
> to.\n    :type package_id: string\n    :param url: url of resource\n
>  :type url: string\n    :param revision_id: (optional)\n    :type
> revisiion_id: string\n    :param description: (optional)\n    :type
> description: string\n    :param format: (optional)\n    :type format:
> string\n    :param hash: (optional)\n    :type hash: string\n    :param
> name: (optional)\n    :type name: string\n    :param resource_type:
> (optional)\n    :type resource_type: string\n    :param mimetype:
> (optional)\n    :type mimetype: string\n    :param mimetype_inner:
> (optional)\n    :type mimetype_inner: string\n    :param webstore_url:
> (optional)\n    :type webstore_url: string\n    :param cache_url:
> (optional)\n    :type cache_url: string\n    :param size: (optional)\n
>  :type size: int\n    :param created: (optional)\n    :type created: iso
> date string\n    :param last_modified: (optional)\n    :type last_modified:
> iso date string\n    :param cache_last_updated: (optional)\n    :type
> cache_last_updated: iso date string\n    :param webstore_last_updated:
> (optional)\n    :type webstore_last_updated: iso date string\n    :param
> upload: (optional)\n    :type upload: FieldStorage (optional) needs
> multipart/form-data\n\n    :returns: the newly created resource\n
>  :rtype: dictionary\n\n    ", "success": false, "error": {"__type":
> "Validation Error", "url": ["Missing value"]}}
>
> If I change to
>
> POST(uploadResource,  config=add_headers(Authorization =
> ckanapi,"Content-Type"="application/json;charset=utf-8"),
>           body =
> "{\"package_id\":\"cmtresults2\",\"format\":\"JSON\",\"url\":\"CleanedData/test.json\"}")
>
> I get a success response, but the attached resource is a pointer to the
> dummy url that I included. In the ckan interface it looks like a file was
> uploaded, but if I click on it, I get a sad face.
>
> Any thoughts?
>
> I am able to use the python requests library and post the code easily, but
> I am hoping to find a native R solution as large chunks of my workflow are
> already set up in R.
>
> Thanks,
>
> Sasha

> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev


--
Nigel Babu
Developer, Open Knowledge