[ckan-discuss] interface versions

[John Bywater]

I've been thinking about how supporting different versions of CKAN's 
interfaces could work....

MOTIVATION
The need to support multiple versions of CKAN's interfaces arises 
because of the need to decouple the development cycles of CKAN from the 
development cycles of CKAN clients.

In other words, without support for multiple interface versions, either 
the development of CKAN's interfaces must be restricted to those changes 
which preserve backwards compatibility, or all CKAN clients must be 
required to synchronise upgrades with upgrades of CKAN.

With support for multiple interface versions, incompatible changes can 
be introduced by introducing a new version of the interface. Current 
clients can continue to use the old interface. New clients can be 
written against the new interface. Current clients can be upgraded to 
work with the new interface.

OVERVIEW
A "service oriented" approach is being taken. This approach involves 
authoring a series of Specifications of Service (SoS). Each SoS defines 
its services as a set of interfaces. Each interface is defined by a set 
of operations.

A migration path through a non-backwards compatible change to an 
interface would involve authoring three SoSs: for the service with the 
old interface version; for the service with both the old and the new 
interface version; and for the service with only the new interface version.

Then, over time, agreed customer expectations will follow these SoSs 
stepwise: a customer will expect to be able to use the old interface; 
then later, the same customer will expect to be able to use both 
interfaces; later still, a customer will expect to be able to use only 
the new interface (hopefully because the customer has determined that 
the old interface is redundant for her purposes).

A Service Level Agreement (SLA) can be used to record the association of 
a particular customer with a particular SoS and a particular duration of 
time.

TERMS OF REFERENCE
Service Level Agreement (SLA) references a Specifications of Service 
(SoS), a customer, a provider, and time period.

A Specification of Service (SoS) has a Service Specification and may
have many Software Unit Specifications

A Service Specification has Services, Software Services, Web
Services, Service Interfaces, and Interface Operations.

   - A Service may require other Services.

   - A Software Service is a type of Service.

   - A Software Service is defined by its Interfaces.

   - An Interface may require many Interfaces.

   - An Interface may offer many Operations.

   - An Interface may require many Software Units.

   - Web Service is a type of Software Service.

Software Unit Specification has Software Units such that:

   - A Software Unit may offer Interfaces.

EXAMPLE SoS EVOLUTION
For illustrative purposes, there are three example Specification of
Service documents below showing the REST API developing through a series
of major versions.

The first SoS (vA) shows the current state of CKAN.

The second SoS (vB) adds a new operation to the REST interface.
Therefore clients of vA are compatible with the new REST interface, but
new clients are not compatible with the old interface. [NB An interface
without versions would be perfectly adequate for client migration in
this case.]

The third SoS changes a data format, therefore clients of vB are not
compatible with the new REST interface. Again, new clients are not
compatible with the old interface. [NB An interface without versions
would fail to be adequate for client migration in this case.]

The fourth SoS drops an old interface. [NB The interface can only be
removed from the Software Unit when all users have agreed to stop using
the old interface.]

== CKAN Specification of Service (vA) ==

Software Services:
   - CKAN

Interfaces:
   - CKAN:
     - CWUI vW
     - CRST vX

Operations:
   - CRST vX:
     - PackageRegisterGet, PackageEntityGet

Software Units:
   - CKAN vL (provides: CWUI vW, REST vX)


== CKAN Specification of Service (vB) [added operation] ==

Software Services:
   - CKAN

Interfaces:
   - CKAN:
     - CWUI vW
     - CRST vX
     - CRST vY

Operations:
   - CRST vX:
     - PackageRegisterGet(PackageFormatP)
     - PackageEntityGet(PackageFormatP)
   - CRST vY
     - PackageRegisterGet(PackageFormatP)
     - PackageEntityGet(PackageFormatP)
     - PackageEntityPut(PackageFormatP)

Software Units:
   - CKAN vM (provides: CWUI vW, REST vX, REST vY)


== CKAN Specification of Service (vC) [changed message format] ==

Software Services:
   - CKAN

Interfaces:
   - CKAN:
     - CWUI vW
     - CRST vX
     - CRST vY

Operations:
   - CRST vX:
     - PackageRegisterGet(PackageFormatP)
     - PackageEntityGet(PackageFormatP)
   - CRST vY
     - PackageRegisterGet(PackageFormatP)
     - PackageEntityGet(PackageFormatP)
     - PackageEntityPut(PackageFormatP)
   - CRST vZ
     - PackageRegisterGet(PackageFormatQ)
     - PackageEntityGet(PackageFormatQ)
     - PackageEntityPut(PackageFormatQ)

Software Units:
   - CKAN vN (provides: CWUI vW, REST vX, REST vY, REST vZ)



== CKAN Specification of Service (vD) [original interface removed] ==

Software Services:
   - CKAN

Interfaces:
   - CKAN:
     - CRST vX
     - CRST vY

Operations:
   - CRST vY
     - PackageRegisterGet(PackageFormatP)
     - PackageEntityGet(PackageFormatP)
     - PackageEntityPut(PackageFormatP)
   - CRST vZ
     - PackageRegisterGet(PackageFormatQ)
     - PackageEntityGet(PackageFormatQ)
     - PackageEntityPut(PackageFormatQ)

Software Units:
   - CKAN vN (provides: CWUI vW, REST vX, REST vY, REST vY)


== CKAN Specification of Service (vE) [software unit updated] ==

Software Services:
   - CKAN

Interfaces:
   - CKAN:
     - CRST vX
     - CRST vY

Operations:
   - CRST vY
     - PackageRegisterGet(PackageFormatP)
     - PackageEntityGet(PackageFormatP)
     - PackageEntityPut(PackageFormatP)
   - CRST vZ
     - PackageRegisterGet(PackageFormatQ)
     - PackageEntityGet(PackageFormatQ)
     - PackageEntityPut(PackageFormatQ)

Software Units:
   - CKAN vO (provides: CWUI vW, REST vY, REST vY)


SERVICE ORIENTED MANAGEMENT IMPLICATIONS
It follows that:

1. CKAN is required only to support interfaces that are included in an 
SoS that is referenced by at least one SLA.

2. SLAs referencing SoSs with old interfaces may not be issued after the 
new interface has been deployed.

3. After all SLAs referencing SoSs with an old interface have been 
terminated or have expired, the old interface may be removed from the 
software and would therefore no longer be available.

4. Care must be taken when issuing SLAs to balance the wishes of 
customers to have their services available indefinitely with the wishes 
of the service providers to continue to enhance their offering.

5. An effective service desk process is required to acknowledge and 
resolve in a transparent and responsive manner all issues raised by all 
customers.

SOFTWARE ARCHITECTURAL IMPLICATIONS
It also follows that:

1. Having one domain model support many interfaces requires good tests
so that adapters can be added safely to the existing implementations of
old interfaces, to permit advancements of the domain model.

2. Unwanted aspects of the model must not be removed until there are
no longer any interfaces depending on those aspects.

3. Interfaces must be supported for a duration of time that is
sufficient for service clients to be upgraded.

4. There must be close alignment between the agreed expectations of
service capabilities and the implementation tests. This can be reliably
achieved by driving both the agreed expectations and the tests from a
version controlled SoS document that includes definitions of the 
interfaces (and the operations of the interfaces).

5. An interface can be removed from the Software Unit once no SLA exists
that references a Service specified to include that interface.

6. Interface resource locations must include designators for both the 
name and the version of the interface.