[ckan4rdm] CKAN as research data repository

Fri Oct 24 01:57:49 UTC 2014

Hi Claire,

great to hear you're heading in the same direction!

Unfortunately we had to turn our repository private as we had to include
sensitive information (no time to factor them out). Our lxc-container
architecture is changing, so we're moving away form the docker setup for
now.
Having said that, CKAN offer an official docker image now:
http://docs.ckan.org/en/latest/maintaining/installing/install-using-docker.html
Our only trick was to map out the dynamic folders (file store, database) to
a persistant folder on the host machine.

ATM I'm building our own AWS Ubuntu 14.10 VM running CKAN 2.2.1, solR,
postgis, and a few extensions following the default "installation from
source" instructions. We serve two separate instances (with two .ini config
files and separate databases) from one installation with one datapusher.
We're currently running the VM as a t2.medium machine with 100GB storage
and nightly snapshots. Our IT guys did the networking / AWS security
settings which requires an advanced degree in black magic.

The tricky bits were to migrate the database from our live instance to the
new images, the datapusher setup (instructions are pretty broken), and
setting correct folder permissions for the log files, and SQLalchemy
version problems - not sure when/whether my write-up will be publishable,
but happy to assist if you get stuck at specific points!

Cheers,
Florian

*Florian Mayer*
*Marine Science Information Management | **Ecoinformatics*
Department of Parks and Wildlife | Science and Conservation Division
Keiran McNamara Conservation Science Centre
17 Dick Perry Av, KENSINGTON WA 6151
*Stop*
<https://maps.google.com/maps?q=17+Dick+Perry+Av,+KENSINGTON+WA+6151&hl=en&sll=-31.994724,115.882598&sspn=0.004337,0.005681&t=h&hnear=17+Dick+Perry+Ave,+Kensington+Western+Australia+6151,+Australia&z=16>*
Collaborate
(**LI* <http://www.linkedin.com/pub/florian-mayer/5b/308/963> *GS*
<http://scholar.google.com.au/citations?hl=en&user=UK_dR0wAAAAJ> *RG*
<http://www.researchgate.net/profile/Florian_Mayer4/>*) **Listen*
<http://soundcloud.com/florian-w-mayer/>

On Thu, Oct 23, 2014 at 10:04 PM, Claire Reis <Claire.Reis at umanitoba.ca>
wrote:

>  Hi Florian,
>
> I realize this is a year old message, but we are trying also to use CKAN
> as a data repository.  I am a biologist at the University of Manitoba in
> Manitoba, Canada and am managing a new data network designed to store and
> serve out open access scientific data about the Lake Winnipeg Basin, which
> covers 4 Canadian provinces and 4 US States.  We are currently exploring
> options to migrate the data server portion from the MS visual basic
> programming it was originally designed in (due to Federal government
> restrictions of the time) to an architecture that will fit at the
> University.  CKAN is looking like a good option except I don’t have much
> programming experience, nor does our computer services with this kind of
> software and no budget to hire outside help at this time. I have tried
> installing CKAN from the instructions on the CKAN website, including using
> virtual servers for Ubuntu 12.04, as well as trying from source code using
> Ubuntu 14, AND trying the data.gov.uk package, but none of these have
> been successful.  I am wondering if you have an updated link for your code
> and instructions as I see the current links no longer exist?
>
>
>
> I currently have a test version of CKAN working in the cloud on Acquia but
> this is not a viable long term solution for us.
>
>
>
> Thanks,
>
> Claire
>
>
>
> Hi everyone,
>
>
>
> we're setting up CKAN for RDM for a Marine Research and Monitoring group
>
> within a state government department in Western Australia. We've got
>
> additional constraints as we deal with sensitive data about threatened
>
> species and communities. We'll be looking at maintaining proper ISO 13139 /
>
> ANZLIC MCP metadata on a CKAN-harvested GeoNetwork catalog for spatially
>
> referenced datasets, but first need to get our scientists' heads around the
>
> paradigm change from spreadsheets to CKAN.
>
>
>
> The selling points of CKAN were the user community, the intuitive GUI, the
>
> search performance, and the filestore/datastore plus API. With a more
>
> technical end user audience we could have used a GeoNetwork / GeoServer /
>
> Thredds stack, but our main data manipulation paradigm is still the trusty
>
> spreadsheet, an internal Wiki for knowledge management and R / Python /
>
> Latex for report automation.
>
>
>
> Our servers run Ubuntu 12.04 LTS VMs with a few home-made application
>
> hosting conventions beyond CKAN's default settings / ports / directories.
>
> To take some of the headache out of the manual installation and
>
> customization steps, we've created a fabric script [0] to automate the
>
> setup (incl file store, data store and spatial preview). I've also added
>
> the setup of initial orgs, groups and users (datasets and resources to
>
> follow) using the API [1].
>
> Feel free to take our installer for a spin, and I'd highly value your
>
> opinion and feedback!
>
>
>
> [0] Code repository https://bitbucket.org/dpaw/yes_you_ckan/overview
>
> [1] Installation instructions
>
> https://bitbucket.org/dpaw/yes_you_ckan/wiki/Home
>
>
>
> Cheers,
>
> Florian
>
>
>
>
>
> Claire Herbert
>
> Coordinator
>
> Lake Winnipeg Basin Information Network
>
>
>
> Centre For Earth Observation Science
>
> Department of Environment and Geography
>
> 522 Wallace Building
>
> University of Manitoba
>
> Winnipeg, Canada, R3T 3N2
>
> Phone: (204) 474-8657
>
>
>
> Follow us on twitter - @LWBIN_UM
>
> Web: http://lwbi.cc.umanitoba.ca/
>
>
>
> _______________________________________________
> ckan4rdm mailing list
> ckan4rdm at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan4rdm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan4rdm/attachments/20141024/69dfb38f/attachment-0003.html>