[ckan-dev] file store api problems

[Sasha Cuerda]

Hello Nigel,

Thanks for your response. I'll check it out and report back. I believe I
tried that approach as well, but I'll go back over my notes and code.

Thanks,

Sasha


On Fri, Jun 13, 2014 at 12:10 AM, Nigel Babu <nigel.babu at okfn.org> wrote:

> Hi!
>
> I'm not an R expert, but from basic reading, I think there's a few mistakes
> you're making.
>
> When you upload a file, you don't need to upload a JSON, in fact, you
> shouldn't. You should send them as key value pairs, like in a form. This is
> a bit confusing because you do need to send in JSON *if and only if*
> you're not
> uploading a file. Because of how file uploads work, the API is a bit
> inconsistent. Does that help? It might end up something like this (I'm
> *not* an
> R person and thoroughly guessing):
>
> POST(uploadResource,  config=add_headers(Authorization =
>     ckanapi,"Content-Type"="application/json;charset=utf-8"),
>     body=list(package_id="cmtresults2",format"="JSON",
>     upload=upload_file("CleanedData/test.json")))
>
>
> On Wed, Jun 11, 2014 at 09:30:18AM -0400, Sasha Cuerda wrote:
> > Hello all,
> >
> > I am trying to set up some R scripts to upload files our ckan file store.
> >
> > Using the httr package
> >
> >  POST(uploadResource,  config=add_headers(Authorization =
> > ckanapi,"Content-Type"="application/json;charset=utf-8"),
> >
> body=list("{\"package_id\":\"cmtresults2\",\"format\":\"JSON\"}",
> > upload=upload_file("CleanedData/test.json")))
> >
> > Throws a 400 code with the following error;
> >
> >  Status: 400
> >   Content-type: application/json;charset=utf-8
> > "Bad request - JSON Error: Error decoding JSON data. Error:
> > JSONDecodeError('Expecting value: line 1 column 2 (char 1)',) JSON data
> > extracted from the request:
> > '--------------------------775bdb2706256deb\\r\\nContent-Disposition:
> > form-data;
> >
> name=\"\"\\r\\n\\r\\n{\"package_id\":\"cmtresults2\",\"format\":\"JSON\"}\\r\\n--------------------------775bdb2706256deb\\r\\nContent-Disposition:
> > form-data; name=\"upload\"; filename=\"test.jsonC\"\\r\\nContent-Type:
> > application/octet-stream\\r\\n\\r\\n[\\n{  ...
> > }\\n]\\r\\n--------------------------775bdb2706256deb--\\r\\n'"
> >
> > I suspect that the problem hear has to do with how httr is double
> escaping,
> > but I'm not sure how to fix that.
> >
> > When I switch my code to
> >
> > POST(uploadResource,  config=add_headers(Authorization =
> > ckanapi,"Content-Type"="application/json;charset=utf-8"),
> >           body =
> >
> "{\"package_id\":\"cmtresults2\",\"format\":\"JSON\",\"upload\":\"CleanedData/test.json\"}")
> >
> > I get a different error
> >
> > Status: 409
> >   Content-type: application/json;charset=utf-8
> > {"help": "Appends a new resource to a datasets list of resources.\n\n
> >  :param package_id: id of package that the resource needs should be added
> > to.\n    :type package_id: string\n    :param url: url of resource\n
> >  :type url: string\n    :param revision_id: (optional)\n    :type
> > revisiion_id: string\n    :param description: (optional)\n    :type
> > description: string\n    :param format: (optional)\n    :type format:
> > string\n    :param hash: (optional)\n    :type hash: string\n    :param
> > name: (optional)\n    :type name: string\n    :param resource_type:
> > (optional)\n    :type resource_type: string\n    :param mimetype:
> > (optional)\n    :type mimetype: string\n    :param mimetype_inner:
> > (optional)\n    :type mimetype_inner: string\n    :param webstore_url:
> > (optional)\n    :type webstore_url: string\n    :param cache_url:
> > (optional)\n    :type cache_url: string\n    :param size: (optional)\n
> >  :type size: int\n    :param created: (optional)\n    :type created: iso
> > date string\n    :param last_modified: (optional)\n    :type
> last_modified:
> > iso date string\n    :param cache_last_updated: (optional)\n    :type
> > cache_last_updated: iso date string\n    :param webstore_last_updated:
> > (optional)\n    :type webstore_last_updated: iso date string\n    :param
> > upload: (optional)\n    :type upload: FieldStorage (optional) needs
> > multipart/form-data\n\n    :returns: the newly created resource\n
> >  :rtype: dictionary\n\n    ", "success": false, "error": {"__type":
> > "Validation Error", "url": ["Missing value"]}}
> >
> > If I change to
> >
> > POST(uploadResource,  config=add_headers(Authorization =
> > ckanapi,"Content-Type"="application/json;charset=utf-8"),
> >           body =
> >
> "{\"package_id\":\"cmtresults2\",\"format\":\"JSON\",\"url\":\"CleanedData/test.json\"}")
> >
> > I get a success response, but the attached resource is a pointer to the
> > dummy url that I included. In the ckan interface it looks like a file was
> > uploaded, but if I click on it, I get a sad face.
> >
> > Any thoughts?
> >
> > I am able to use the python requests library and post the code easily,
> but
> > I am hoping to find a native R solution as large chunks of my workflow
> are
> > already set up in R.
> >
> > Thanks,
> >
> > Sasha
>
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
> --
> Nigel Babu
> Developer, Open Knowledge
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140613/70ebc439/attachment-0003.html>