[okfn-help] CKAN/Microfacts questions

[John Bywater]

Rufus Pollock wrote:
> Sorry for the delay in reply but it took me longer to get things 
> working than I imagined (on the new machine etc).

No worries.

> Part of this was my own fault in that I installed Genshi 0.4 and then 
> had to work out why it wasn't working. The good side of this is that 
> a) I've removed the dependency on markdown (markdown is in webhelpers) 
> b) ckan now works with genshi 0.4 c) I have found one genuine bug.
>
> Hope I haven't broken anything and all the gory details in the log 
> messages:
>
> http://knowledgeforge.net/ckan/trac/changeset/245
> http://knowledgeforge.net/ckan/trac/changeset/244

Looks good!

>
> As detailed in this 2nd message there is one genuine bug: package 
> create action returns 201 but also returns content which is apparently 
> illegal for http spec (using my version of paste fixture i get an 
> assertion error).

Yes, this was hedged poorly, but is now resolved in favour of 200. We
don't need to return 201. What else could possibly be implied by a HTTP
200 from a POST request to the create action? REST doesn't say you need
to return 201. Nobody's asking CKAN to return 201. We don't need to
return 201!

I've changed the code, the test, and the wiki page. I also added this
quickly, please do review:

http://project.knowledgeforge.net/ckan/trac/wiki/RestfulAPI#POSTMethodParams

You might want to write a client against this, to test the interaction.
One thing I wondered about is how the licenses are to be retrieved and
selected by a machine client:

http://project.knowledgeforge.net/ckan/trac/wiki/RestfulAPI#CurrentDevelopmentIssues


>> Firstly, have you seen/tried the package tag autocompletion function? 
>> It's very like the full-screen version of delicious's edit page (ie 
>> you get a horizontal list that starts hard-left). Only thing is that 
>> editing mid-list doesn't autocomplete, but I haven't tried to address 
>> that.
>
> Looks really nice. Great stuff :)

Thanks a lot.

>> Secondly, I fixed up the package controller error handling and HTTP 
>> response status codes. The REST thing also just clicked, so I wrote 
>> about it here, here, and here:
>> http://desire.appropriatesoftware.net/requirements/60/
>> http://knowledgeforge.net/ckan/trac/ticket/27#comment:3
>> http://knowledgeforge.net/ckan/trac/ticket/27#comment:4
>>
>> Following that general conception and CKAN-specific analysis, I wrote 
>> this piece of documentation:
>> http://knowledgeforge.net/ckan/trac/wiki/RestfulAPI
>
> Again this all looks really good.

Thanks. Did you show me that X-Tech paper? (Did you see it being
presented??)

>
>> Basically, I wanted to ask you whether you understand 
>> Routes/Pylons/CKAN enough to match the 'api/rest' path and set a 
>> request param (or context attribute?) called 'format' to 'json'. I 
>> didn't look into this cos I thought you might just know. If we can 
>> just map the URLs I wrote in that document to the same controllers 
>> but with the request param set that would be very nice.
>
> There are two obvious ways to do this:
>
> a) Mess around with routes so that it does this. This would mean 
> messing around with the setup of config/routing.py

Yes, I don't know what. Perhaps adding something like this would do it?

map.connect('api/:api/:controller/:action/:id')

Perhaps that would make Pylons pass api='rest' with the controller
method params?

>
> b) create a new controller controllers/api.py with a method called 
> rest whose job is to strip off the rest of the url and then redispatch 
> based on that adding format=json to the request

What does redispatching look like in Pylons?

>
> However personally I think we should not bother with this and just 
> have an extra query parameter named 'format' (which perhaps we can 
> just extract out in the BaseController). This then determines what is 
> returned out  (html/json at this point defaulting to html). If this 
> seems a bit repetitive in that each controller has to extract the 
> parameter and then act on it we could move to the decorator model 
> where every action returns a dict which is either converted to json or 
> to template ...

The decorator approach is certainly an option....

But, the first decision should be between these alternatives:

EITHER

http://ckan.net/api/rest/package/list/:page
http://ckan.net/api/rest/package/create/:name
http://ckan.net/api/rest/package/update/:name
http://ckan.net/api/rest/package/read/:name

where the create and update POST params are purely object properties.

OR

http://ckan.net/package/list/:page?format=json
http://ckan.net/package/create/:name
http://ckan.net/package/update/:name
http://ckan.net/package/read/:name?format=json

where the create and update methods send a 'format' POST param which has
nothing to do with the objects being accessed.

My opinion is that this in part confuses the format of the method
responses with the location of the resources, and in part confuses the
format of the method responses with the properties of the resources. So
it isn't a very good design, and certainly not what REST is about, which
requires a clear separation of these concerns. It both looks cheaper,
and will turn out to be more expensive. So I think it's the worse
alternative.

There's going to be a series of minor differences between machine and
human client interaction, so it's probably worth anticipating this
design load, and establishing a clear separation between the two modes.


>> BTW I think it makes a lot of sense to distinguish the locations of 
>> your web browser interface and your machine client interfaces. This 
>> is how many of the more famous web services (esp. flickr.com) have 
>> been done.
>
> I see that but is it worth the added complexity at this point in time? 
> I'm undecided.

I reckon it's a less complex design, just more precisely conceived....


>> Thirdly, what's the state of microfacts? Should I be able to do 
>> pretty much the same as I can with CKAN? I notice there is no 
>> microfacts-admin, so am I correct to infer there no special test data?
>
> No, not at present. Everything is setup in the (one current) test 
> itself. There is some json data in demo/ but that is not yet used by 
> the code.

OK, great.


>> Also, might I ask you to check that you have no uncommitted changes 
>> (that I need) on your machines?
>
> No. Everything committed.

Great.

>> Other than that, what did you think about AJAX-style aspects to 
>> microfacts thread view? Shall we start without any for now? (Is there 
>> already a thread view?? I didn't get it running yet....)
>
> There is no WUI at all at present (i.e. no controllers, no templates 
> etc etc). Just a bare-bones model. First step is probably to copy and 
> paste ckan css (and js if necessary) and start on thread/factlet views.

Fantastic, microfacts coming up! ;-)

Best wishes,

John.

>
> ~rufus
>
>
>
>


-- 
Appropriate Software Foundation
Registered in England and Wales
17 Chapel Street, Hyde Cheshire
Company number: 04977110
W: appropriatesoftware.net
T: 0870 720 2944
M: 0781 139 2292