[ckan-dev] Hierarchical folder structure for a dataset

Mon Aug 14 13:44:26 UTC 2017

Hi Damian

Thanks for the explanation! It is an interesting concept, we have looked
at doing something similar. In the end we didn't because we also want
data from researchers who don't use the shared storage infrastructure.

Cheers,
Harald

On 2017-08-11 11:24, Damian Steer wrote:
> On 11/08/17 09:04, Harald von Waldow wrote:
>> Hi Damian
>>
>> That looks pretty interesting. I would also not mind seeing how you
>> designed the user-interface to let people upload hierarchically
>> structured datasets. Is the code for that publicly available?
> 
> Ah, I was too brief originally:
> 
>> We only use CKAN as a catalogue - the data is held externally
> 
> Our data publications come from our institutional research data store: a
> fancy shared drive essentially. Our publications are stored on the same
> infrastructure.
> 
> We have a publication tool that lets users enter metadata and pick the
> folder they want to deposit. We then copy the data to our space, and
> save the metadata with it in a hidden file. The folder is shared over
> the web (example [1] and the corresponding ckan page [2]).
> 
> We have deposits up to 100GB, which would be painful to do in the
> repositories we looked at. There are some custom uploaders now, and
> things like rsync of course. But hacking in that simple pick-a-folder
> thing was hard.
> 
> We also think we can maintain a static folder shared over apache for up
> to 20 years (which is our guarantee). [3]
> 
> So CKAN comes in right at the end, and is an nice, searchable shop-front
> for our datasets. We periodically look for new or changed datasets (via
> http), grab the metadata and push it into CKAN using the web api.
> 
> Working well so far, I'm happy to say.
> 
> Damian
> 
> [1] <https://data.bris.ac.uk/datasets/upjtf9os1dzr154phmgvrupib/>
> [2] <https://data.bris.ac.uk/data/dataset/upjtf9os1dzr154phmgvrupib>
> [3] Obviously we hope CKAN is around forever!

-- 
Harald von Waldow
Eawag
ICT Services
Ueberlandstrasse 133
8600 Duebendorf
http://www.eawag.ch

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170814/b96e90a1/attachment-0003.sig>