[ckan-discuss] Management of numerous resources

[Fawcett, David (MNIT)]

For the automated upload, I suggest structuring your images and their metadata in a standard way on a local filesystem and then writing code that crawls your images, parses/queries your metadata, builds the appropriate JSON object for that dataset/resource, and then pushes it to CKAN via that API.

We are doing this with Python and the ckanapi module.

David.

From: ckan-discuss-bounces at lists.okfn.org [mailto:ckan-discuss-bounces at lists.okfn.org] On Behalf Of Stéphane Guidoin
Sent: Thursday, November 07, 2013 2:48 PM
To: ckan-discuss at lists.okfn.org
Subject: [ckan-discuss] Management of numerous resources

Dear CKAN users and developers,

For the implementation of CKAN Montréal, we are facing a case where I would like to have some external opinions: how to manage a dataset with potentially a very high number of resources.

I have a precise case at hand: the photographical archive of the city; for the moment hundreds, but potentially thousands of pictures. Ideally we're search a solution where upload could be automated and where browsing would remain... human (listing thousands of resource does not sound human to me)

For the moment (solution inherited from the previous hand tailored portal), they upload and excel speadsheet with the list of individuals picture and corresponding urls (which are managed outside of the data portal). Not optimal, needs manual management and not everything is in the data portal.

Does anybody has some ideas / existing solution for such a case?

Best

Stéphane

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-discuss/attachments/20131108/56cfba4a/attachment.htm>