[okfn-help] The Future of Our Tech Infrastructure: RFC

Fri Apr 16 11:22:48 UTC 2010

Hi Rufus,

Thanks for picking up the service oriented initiative I started!

My comments:

1. The well-trodden concept of "service orientation" (and the included 
subject of SOA) is set out in this (and other) books, which, having seen 
TFOOTI I would strongly recommend you buy and read:
[1] http://www.cambridge.org/catalogue/catalogue.asp?isbn=0521843367

2. TFOOTI conflicts in purpose, but does not reference, two already 
existing Wiki pages:

(a) The Asset Inventory is the keystone of any service orientated 
effort. It is the "well known" repository for all service-related docs:
[2] http://knowledgeforge.net/okfn/tasks/wiki/AssetInventory

(b) The Service Oriented Architecture policy document is the overarching 
policy document for the design of service-oriented services. It already 
contains most of the questions that remain to be answered (but TFOOTI 
hardly contains any of the answers):
[3] http://knowledgeforge.net/okfn/tasks/wiki/ServicesPolicy

3. "Unix philosophy" does not pertain to service orientation. That's a 
level error. To be specific, the central distinction of service 
orientation is not singularisation. Service orientation is also not 
primarily the object-oriented doctrine of high cohesion and low coupling 
(but that isn't excluded). Instead, service orientation is about 
satisfaction: of a service level agreement which has "agreed 
expectations for the measurable levels of service a provider must 
achieve and terms of use that a customer must comply with". In other 
words, services and users must adequate to a set of expectations, but 
neither services nor users are restricted to having only one purpose. In 
respect of service design, it's an inappropriate and unhelpful doctrine, 
and I recommend removing the statement.

4. The purpose of service oriented architecture is neither reaping 
economies of scope and scale, nor simplifying internal architecture, nor 
following the "Unix philosophy". Rather, the purpose of service oriented 
architecture (SOA) consists in both conditioning individual service 
designs and bridging the two main areas of service orientated activity: 
  service level management (SLM) and service execution management (SEM). 
For brevity, descriptions of SLM and SEM are beyond the scope of this email.

5. The diagram of "Service Architecture" doesn't make sense to me on two 
levels:

Firstly, it appears to be an arrangement of services, so it would be 
better to denote it as a Service Design. As indicated, service oritented 
architecture is about conditioning service designs and bridging SLM and 
SEM. It is constituted of the concerns I have set out in [3] above. 
Service design determines a specification of service (SoS), which may 
involve an arrangement of services. Hence...

Secondly, if it is a service design, then I don't understand why it 
resembles an n-tier software design (with interface, middleware, 
persistence). A service dependency diagram wouldn't look like that. 
Perhaps it's some other kind of diagram, one I'm not familiar with?

6a. RFC what? There is neither an RFC number nor an OKF RFC process. Why 
use "RFC" when it's just a draft and you just want some comments?

6b. Infrastructure what? If your infrastructure is physical and 
organisational structures which are critical to the operation of the 
OKF, then not all OKF infrastructure is provided by the OKF (e.g. you 
use train and telephone systems) and not all services provided by the 
OKF are infrastructural (e.g. you provide services to others). So, 
presuming TFOOTI is "doing one thing and one thing well" either it's 
concerned with securing OKF infrastructure or it's concerned with how to 
sustain quality services. I recommend cutting it like this: firstly 
establish how to sustain quality services; then use this approach to 
secure quality infrastructure.

That's probably enough for now. :-)

Best wishes,

John.

Rufus Pollock wrote:
> Useful associated diagram:
> <http://knowledgeforge.net/okfn/tasks/wiki/ServiceArchitecture>
> (Source version of this text at [1])
> 
> The Future
> ==========
> 
> Having developed, deployed and maintained apps and services at the
> OKFN for the last 5 1/2 years we are not at a point of relative
> maturity and I think it is time to take stock of where we are and look
> to the future.
> 
> We currently support more than 10 production or semi-production
> application services (apps or app services from hereon) as well as
> lots of bleeding edge development. The majority of these apps are OKFN
> projects but we also do some service provision to others.
> 
> It appears likely that both internal and external demand is only
> likely to grow. Furthermore, there is some important re-architecting
> we can do that both ease our ability to scale and significantly
> improve the app development process by providing key base services
> (see below) in a standardized way.
> 
> Our aim: solid, scalable infrastructure on which to develop the open
> knowledge application "stack"
> 
> Some Observations on the World at Present
> =========================================
> 
>   * We are in a world in which hardware/bw are **cheap** and rented
>   * Larger machines are cheaper per compute unit than smaller machines.
>   * Clear economies of scope across projects and services
>   * Many application services require a common a set of 'base'
> services/infrastructure ranging from file storage to the full-text
> search.  While each of these can be provided as part of the individual
> app (i.e. by integrating search software into the app) it is
> increasingly attractive to provide these at the **service** level.
>   * In general, we (and others) are moving towards a more
> "service-oriented" architecture in which different services are
> integrated together
> 
> Service-oriented architecture
> =============================
> 
> **Aim:** Reap benefits of economies of scope and scale, simplify our
> internal architecture, and follow the UNIX philosophy (each tool
> (service) does one thing and one thing well).
> 
> Key base services provided:
> 
>   * Bare-metal machines
>   * Relational databases: mysql and postgresql
>   * RDF: 4store
>   * Keyvalue: redis + mongo (couchdb?)
>   * Filestorage (?): more than bare-metal machines
>   * Messaging and job queue: celery + redis + rabbitmq
>   * Indexing: solr + xapian (?)
>   * Mailing lists: lists.okfn.org
>   * MX: mail exchange but no mail accounts
> 
> For each of these we deliver:
> 
>   * Provisioning and availability
>   * Backup
>   * Redeployment from backup
> 
> General architecture for a application service detailing its use of
> key base services is shown in diagram here:
> 
> <http://knowledgeforge.net/okfn/tasks/wiki/ServiceArchitecture>
> 
> Appendix: The Unix Philosophy
> =============================
> 
> See unix-philosophy.txt
> 
> [1]: <http://knowledgeforge.net/okfn/sysadmin/file/b9c53a5e1765/doc/future-of-infrastructure.txt>
> 
> _______________________________________________
> okfn-help mailing list
> okfn-help at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/okfn-help