[ckan4rdm] Contents of ckan4rdm digest...

Thu Feb 13 12:10:52 UTC 2014

Hi All,

We (dartportal.leeds.ac.uk) created a structured automated metadata 
creation system to handle our data ingest of thousands of stuff 
(different formats, datasets and sensors). There is definitely a need 
for this, especialy for those who may want to bulk upload archives.

Our code is not generic - I'm happy to share it if you would like to 
generify it, or if it helps in the general thrust. I'm in the process of 
writing a blog post on our experiences for OKF archaeology subsection

Best

Ant
On 13/02/14 12:00, ckan4rdm-request at lists.okfn.org wrote:
>
> Message: 1
> Date: Wed, 12 Feb 2014 13:46:36 +0100
> From: Hannes Thiemann <thiemann at dkrz.de>
> To: ckan4rdm at lists.okfn.org
> Subject: Re: [ckan4rdm] automatic metadata extraction
> Message-ID: <52FB6D2C.7020300 at dkrz.de>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Dear Joe,
>
> I believe such a thing could be pretty useful for any domain specific
> repository where the number of used formats is limited. Automatic
> metadata generation is an important added value as the ease by which the
> data can be found is greatly improved.
>
> Best, Hannes
>
>
>
> Am 08.02.2014 04:22, schrieb Joe Tsoi:
>> Hi,
>>
>> I'm one of the CKAN core devs and I've created a ckan extension that
>> is a bit of a toy example. I hope that it might interest some people
>> on this list. The ckan extension is for FITS images generally used in
>> astronomy. When a fits file is uploaded to ckan, it automatically
>> parses the file and extracts the metadata and saves it against the
>> ckan resource, it also generates a greyscale jpeg image of the fits
>> file which serves as the image preview when the resource is previewed
>> in ckan.
>>
>> I've setup a demo of it at http://astro-joet.rhcloud.com/dataset and
>> I've uploaded a couple of sample images taken from the Hubble fits
>> sample images. The source code for the extension is available at
>> https://github.com/joetsoi/ckanext-astro . I've actually added a
>> custom extension point to ckan which this extensions uses that isn't
>> currently available in vanilla ckan, but I hope to get some form of it
>> into ckan core. It's just a demo, so I'm pretty sure if you upload a
>> non fits image it'll break, and it's pretty hacky and is probably
>> buggy.
>>
>> Anyway I'd like to hear if this sort of automatic metadata extraction
>> is of any use to anyone, not just for astronomy, but for any type of
>> file. I'm not an astronomer or anything,  I just thought it might
>> serve as a good example.
>>
>> Regards,
>> Joe
>> _______________________________________________
>> ckan4rdm mailing list
>> ckan4rdm at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan4rdm
>>