[ckan-dev] Creating resource with upload from an extension

Fri Apr 29 11:30:58 UTC 2016

Dear Pedro,

if you want to call the "resource_update" function directly then I think
you need to construct the file upload object yourself:

   import cgi
   from ckan.plugins.toolkit import get_action

   def update_resource_file(resource_id, f):
       context = {
           'ignore_auth': True,
           'user': '',
       }
       upload = cgi.FieldStorage()
       upload.filename = getattr(f, 'name', 'data')
       upload.file = f
       data = {
           'id': resource_id,
           'url': 'will-be-overwritten-automatically',
           'upload': upload,
       }
       return get_action('resource_update')(context, data)

   with open('ckanext/extractor/tests/test.pdf', 'rb') as f:
       print update_resource_file(resource_id, f)

You have to provide the URL setting, but it will be overwritten
automatically with the download URL of the uploaded file.

Regards,
Florian

"ckan-dev" <ckan-dev-bounces at lists.okfn.org> schrieb am 29.04.2016
11:01:36:

> Von: Pedro G Silva <pedro.gracio at deimos.com.pt>
> An: lucia.espona at wsl.ch,
> Kopie: CKAN Development Discussions <ckan-dev at lists.okfn.org>
> Datum: 29.04.2016 11:02
> Betreff: Re: [ckan-dev] Creating resource with upload from an extension
> Gesendet von: "ckan-dev" <ckan-dev-bounces at lists.okfn.org>
>
> Hi Lucia
> Thank you for your code. You are using the API from ckan and what
> I'm looking for is using CKAN internal methods, like
> this plugins.toolkit.get_action('resource_create')(context,resource_dict)
>
> On Fri, Apr 29, 2016 at 9:15 AM, <lucia.espona at wsl.ch> wrote:
> Dear Pedro
>
> I am a just a newbie but in case this is helpful, for creating
> resources to the FileStore using the API I put an empty string as a
> value for URL and it worked, maybe you can try :) See below my code.
>
> Good luck
> Lucia
>
> ################## CREATE RESOURCE (UPLOAD FILE) #################
>
> ckan_api_url = ckan_url + 'resource_create'
> request_header = {'X-CKAN-API-Key':user_token}
>
> resource_dict = {}
> resource_dict['package_id'] = package_id
> resource_dict['name'] =  'Tree Report Table'
> resource_dict['description'] = 'Report with measurements of trees'
> resource_dict['format'] = 'csv'
> resource_dict['id'] = 'tree-data-99999-2015'
> resource_dict['url'] = ''
>
> requests.post(ckan_api_url, data=resource_dict,
> headers=request_header, files=[('upload', file('H:\\Envidat_project\
> \dev\\test\\API-test\\data\\treedata_99999_2015.txt'))])
>
> data_string = urllib.quote(json.dumps(resource_dict))
>
> # Make the HTTP request.
> response = urllib2.urlopen(request, data_string)
> assert response.code == 200
>
> # Use the json module to load CKAN's response into a dictionary.
> response_dict = json.loads(response.read())
>
> # Check the contents of the response.
> assert response_dict['success'] is True
> result = response_dict['result']
> pprint.pprint(result)
>
> _________________________________________________________
> Dr. Lucia Espona Pernas
>
> Swiss Federal Institute for Forest, Snow and Landscape Research WSL
> Hauptgebäaude Labortrakt (HL D34)
> Zürcherstrasse 111
> 8903 Birmensdorf
> Switzerland
>
> +41 44 739 28 71 phone direct
> +41 44 739 21 11 reception
>
> www.wsl.ch
>
> -----"ckan-dev" <ckan-dev-bounces at lists.okfn.org> wrote: -----
> To: Jonas Oppenlaender <jonas.oppenlaender at seme4.com>
> From: Pedro G Silva
> Sent by: "ckan-dev"
> Date: 28.04.2016 14:11
> Cc: CKAN Development Discussions <ckan-dev at lists.okfn.org>
> Subject: Re: [ckan-dev] Creating resource with upload from an extension
>

> Hi
> Your solution does not produce any error, although if I
> have resource_dict['url'] = None,
> [ckanext.harvest.model] Validation Error: {'URL': 'Missing value'}
>
> If I had a value I get that value on the resource link, not a
> resource pointing to the filestore (or ckan internal storage).
>
> Any ideas?
> I'm thinking on using the API inside the extension, but this is a
> very hacky solution..
>
> On Wed, Apr 27, 2016 at 7:29 PM, Jonas Oppenlaender <
> jonas.oppenlaender at seme4.com> wrote:
> Hi Pedro,
>
> You can just pass it an open file pointer, like this:
> resource_dict['upload'] = open(‘/path/to/file’, ‘rb’)
>
> If you use an already open file, you might need to reset the current
> position before calling the API:
> existingfilepointer.seek(0)
>
> Regards,
> Jonas Oppenlaender
>
>
>
> From: Pedro G Silva
> Sent: 27 April 2016 17:51
> To: CKAN Development Discussions
> Subject: [ckan-dev] Creating resource with upload from an extension
>
> Hi
> I have a custom extension which does some parsing on files and then
> adds them to the dataset created. How should I use the field upload
> to pass a file? Any ideas?
>
> resource_dict={}
> resource_dict['package_id'] = package_id
> resource_dict['upload'] = ????
> resource_dict['name'] = 'Product Download'
> oi = plugins.toolkit.get_action('resource_create')(context,resource_dict)
>
> --
>
>
>
>
> [Bild entfernt]
>
> Pedro Gracio Silva
> Aerospace Systems / Production Center
>
> Av. D. Joao II, Lt 1.17.01, 10 • 1998-023 Lisboa, Portugal
> Ph: +351 21 893 3020 • Fax: +351 21 896 9099
>
>
>
>
>
>

>
> --
>
> [Bild entfernt]
>
> Pedro Gracio Silva
> Aerospace Systems / Production Center
>
> Av. D. Joao II, Lt 1.17.01, 10 • 1998-023 Lisboa, Portugal
> Ph: +351 21 893 3020 • Fax: +351 21 896 9099
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>

>
> --
>
> [Bild entfernt]
>
> Pedro Gracio Silva
> Aerospace Systems / Production Center
>
> Av. D. Joao II, Lt 1.17.01, 10 • 1998-023 Lisboa, Portugal
> Ph: +351 21 893 3020 • Fax: +351 21 896 9099
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20160429/5832ba41/attachment-0003.html>