[ckan-discuss] [Fwd: Re: API For Package Name]

Wed Nov 24 09:19:11 GMT 2010

[ caution: design discussion below :-) ]

Hi David,

Thanks for all the information.

As the architect of the CKAN API, and probably the person most closely 
associated with the CKAN API*, I'd like to say a couple of things about 
the CKAN API.

My mild mannered objection to that "slug" design is that a distinctly 
*RPC* style is being introduced into the CKAN API, which is architected, 
developed, and widely understood to be a *RESTful* API.

Since the style of an API is all about providing for developer intuition 
  by having a consistent design, the effect of mixing in RPC-styled 
aspects to a RESTful-styled API is to make an API that is less 
consistent, less approachable, and therefore less lovely.

Ed Summers has described the CKAN API as lovely. He wrote, "the fact 
that it only took an hour or so was all due to the way you have 
leveraged CKAN, and CKAN's lovely REST API."
http://lists.okfn.org/pipermail/ckan-discuss/2010-September/000542.html

I'd like to know how James and you understand this loveliness to have 
been constructed? Did you realise it is a purposefully designed affect? 
Can't you see that mixing styles isn't really a style at all?

David Read wrote:
> John,
> 
> This doesn't seem to be RESTful

Um, why? The RESTful approach is an architectural style. "Creating a 
slug from a string" is a function, not an architectural style. So that's 
a category error.

Just about any collection of services can be presented as a RESTful API.

Therefore "creating a slug from a string" *can* also be presented as a 
RESTful API, even if one doesn't know how.

> so I wouldn't suggest putting it in
> the Model API.

Since slugs are used to constrain attribute values in the model, it does 
seem to be fully and exclusively an aspect of the domain model. It isn't 
just presentation, and it isn't just persistence.

> Rather, a new 'util' API. James has already documented
> this in ckan/doc/api/version2.rst.
> 

With just a little bit of thought, both tag completion and slug creation 
can be done in a RESTful style.

Not doing so is what Larry Wall would call "false laziness". :-)

I propose we rework the tag completion and slug creation to eliminate 
the unnecessary and incompatible API style that has been introduced.

> e.g. /api/2/util/package/create_slug?title=Package+1+Title+Typed+So+Far
>

To recap, the reasons for constructing CKAN's API as a RESTful API (and 
not any other kind of API) are:

1.  The basic architecture of the web does not force any type of service
architecture - a given service could be implemented in a wide variety of
styles.

2.  It is important that services work consistently within the API - so
we want to have one (and only one) style.

3.  Representational State Transfer (REST) is an architectural style -
or "pattern" - guiding on the architecture of web services.

4.  Representational State Transfer (REST) is commonly understood to
mean that the URL represents a noun and the HTTP method represents one
of several standard verbs. There's more to it, but that's the start.

5.  Just about any collection of services can be presented as a RESTful
API, and by doing so, it gains the advantage of a familiar, intuitive,
style.

6.  A few years ago, Rufus asked me to design an API for CKAN that would
present a familiar, intuitive style, and we identified, investigated,
and adopted the RESTful style for that purpose.

So the design is being complicated unnecessarily, since:

- We can separate slugs as a separate and independent concern.

- We can say that a slug is a resource.

- We can design a locator template for a slug resource.

- We can avoid putting a verb in the locator template.

- We can code the whole set of slug resources with a single function.

I should add that there are still several aspects of the RESTful API
which need to be fixed. In order to "release early", CKAN's API was 
never properly finished.

We should be looking to reduce that technical debt, and not to increase 
it like this.

I'm also worried that not much of this seems to be understood....

> We have one current use case:
> * munge a title to a package name
> And two similar things done in the data importers, but I see no reason
> to have these in the api right now. But I'll mention them in case it
> guides the API:
> * munge a badly-formed name to a well-formed package name
> * munge a word/phrase into a tag name
> 

Very useful.

> I'm going to detach all importers from any centralised munger or api,
> to ensure they never change.
> 

That's probably for the best. :-)

> I'm not convinced adding the word slug in here helps, since only one
> purpose of a package name or tag name is to be a slug, although that
> is the reason it is constrained.
> 

I hear that. However, given that for date-times we would expect:

     created_date = datetime.new()

and not:

     created_date = datetime.create_created_date()

therefore, for "slugs" we might expect:

     name = slug(title)

and not:

     name = create_package_slug_from_title(title)

Hence:

     /api/model/slug/{string}

and not:

     /api/util/package/create_slug?title={string}

Why can't it be that simple?

Since these considerations (and others I have omitted for brevity) are 
at the core of the longevity which the RESTful style is an attempt to 
obtain, I feel compelled to speak up.

Best wishes,

John.

PS Puzzlingly enough, I just noticed that, for whatever reasons, the 
blog James wrote on data.gov.uk doesn't mention that the data.gov.uk 
catalogue API (a CKAN API) is a RESTful API. It seems that James is not 
investing in the existing architecture of the CKAN API.
http://data.gov.uk/blog/announcing-datagovuk-catalogue-apis

PPS The blog post about the Legislation API mentions that it is a 
RESTful API. Seems like a more approachable introduction to an API.
http://data.gov.uk/blog/legislationgovuk-api

*The last time I was Richard Stirling, he actually referred to the CKAN 
API as 'your API'. He had been enjoying writing code against the 
data.gov.uk Catalogue API, and wanted to let me know.

> David
> 