[ckan-dev] Hierarchical folder structure for a dataset

[Damian Steer]

On 11/08/17 09:04, Harald von Waldow wrote:
> Hi Damian
> 
> That looks pretty interesting. I would also not mind seeing how you
> designed the user-interface to let people upload hierarchically
> structured datasets. Is the code for that publicly available?

Ah, I was too brief originally:

 > We only use CKAN as a catalogue - the data is held externally

Our data publications come from our institutional research data store: a 
fancy shared drive essentially. Our publications are stored on the same 
infrastructure.

We have a publication tool that lets users enter metadata and pick the 
folder they want to deposit. We then copy the data to our space, and 
save the metadata with it in a hidden file. The folder is shared over 
the web (example [1] and the corresponding ckan page [2]).

We have deposits up to 100GB, which would be painful to do in the 
repositories we looked at. There are some custom uploaders now, and 
things like rsync of course. But hacking in that simple pick-a-folder 
thing was hard.

We also think we can maintain a static folder shared over apache for up 
to 20 years (which is our guarantee). [3]

So CKAN comes in right at the end, and is an nice, searchable shop-front 
for our datasets. We periodically look for new or changed datasets (via 
http), grab the metadata and push it into CKAN using the web api.

Working well so far, I'm happy to say.

Damian

[1] <https://data.bris.ac.uk/datasets/upjtf9os1dzr154phmgvrupib/>
[2] <https://data.bris.ac.uk/data/dataset/upjtf9os1dzr154phmgvrupib>
[3] Obviously we hope CKAN is around forever!
-- 
Damian Steer
Senior Technical Researcher
Research IT
+44 (0) 117 39 41724