[ckan-dev] CKAN Enterprise Stack on AWS

[Adrià Mercader]

Hi Shane,

Thanks for sharing this. Really interesting stuff.


Adrià

On 21 July 2016 at 10:05, Shane Davis <shane.davis at linkdigital.com.au> wrote:
> Hi All,
>
> I'm starting to immerse myself more deeply into the CKAN dev world.
>
> Having read https://lists.okfn.org/pipermail/ckan-dev/2016-July/010130.html
> by Luiz Felipe I thought it worthwhile to share where Link is at with
> Enterprise CKAN builds given we want to open source our work once we are
> happy with it's stability.
>
> What we've been working on for quite some time is a robust, scalable and
> easily maintained HA CKAN stack. Our technology of choice has been AWS, and
> more specifically, AWS CloudFormation and OpsWorks.
>
> Where we are at the moment is a fairly stable build that has the following
> features:
> - Gluster File System for HA shared file store across two service nodes
> - SolrCloud with Apache Zookeeper across two service nodes
> - CKAN 2.5.2 across load balanced EC2-Instances
> - AWS Postgres RDS with Multi-AZ and Postgis extensions installed and
> enabled.
>
> The CloudFormation templates we have allow highly configurable DEV,UAT and
> Production VPCs to be provisioned.
> Standalone OpsWorks stacks with the features described above can then be
> provisioned into any one of the VPCs.
>
> OpsWorks provides the following key benefits:
> Layers are used to define service roles. Hence our build has seperate layers
> for all the key components such as GFS, Solr, Zookeeper etc.
> An instance can be assigned to one or more layers. Thus you might have two
> service nodes that provide GFS, Solr and Redis. Later as your environment's
> requirements expand, you can create instances that take those roles on
> separately.
> CKAN extensions can be deployed as apps within the stack. These apps (CKAN
> Extensions) can then be deployed to individual instances/servers, or to a
> whole layer.
>
> The existing ruby based Chef recipes should provide pretty good insights to
> anyone interested in adapting them to other environments or cloud platforms.
> They are as close to bare basics as possible which should lend them towards
> adaptation easily.
>
> Longer term our ambition is to have a straight out CloudFormation template
> for those not interested in running an OpsWorks stack. We're also fairly
> confident that the build can be turned into Docker containers so rather than
> using OpsWorks, or indeed AWS, it would be possible to deploy a fully
> scalable HA stack using Docker on one or more systems.
>
> As things keep progressing I'll keep the list updated. We plan to provide
> the repo with a blog post and possibly video taped presentation of some
> sort.
>
> Cheers.
> Shane.
>
>
>
> ------------------------------------------------------------------
> SHANE DAVIS  |  SERVICE DELIVERY MANAGER
> Link Digital
> shane.davis at linkdigital.com.au
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>