[wdmmg-dev] [ckan-dev] Started work on the 'webstore' (a datastore with web API)

Thu Jul 7 19:08:47 UTC 2011

Hi James,

On Thu, Jul 7, 2011 at 8:03 PM, James Casbon <casbon at gmail.com> wrote:
> On 7 July 2011 18:26, Friedrich Lindenberg
> <friedrich.lindenberg at okfn.org> wrote:
>
>> Would appreciate any feedback!
>
> Looks very promising.  I checked the create table code and the type
> guessing is a little basic (always text) and done on a row basis.
> Since you are using JSON and will therefore load the entire set of
> posted data before you process it, you might want to guess the type
> based on all the data posted.

Yes, the type-guessing could use some extension, it was basically
meant as a stub. The question is whether we want to try and actually
guess non-JSON columns as well and what happens when an update is
submitted that doesn't conform to the given type?

> I would also like to be able to nest tables (maybe I've spent too long
> with mongo ;) ), so that I could encapsulate projects in a single URL
> scheme.  FWIW the difficulty in doing this with flask decorators is
> what made me use something other than flask when I made something
> similar.

I'm not sure if this is one or two things. What I want ASAP are SQL
views, which don't require much flask decoration but only some
RESTification (ideas?)

The other thing is actual nested data, i.e. having a dict in a cell.
This is not hard (we have it in CKAN) but essentially you end up with
a blob. My thought was to maybe rather use the kind of unrolling that
formencode does:
http://formencode.org/Validator.html#http-html-form-input - thats
maybe a bit weird but interesting!

> Obviously security is going to be a concern when you are executing sql
> input.  Have you given any thought to this?  Esp when you would want
> to be embedding this in other javascript apps.

Its ugly. At the moment, the model is: you can run SQL on *your own*
databases and since these are currently SQLite, isolation should not
be a problem. SQL write is intentionally modelled as a PUT operation
which puts a high cost on using it and might help to reduce the danger
of simple CSRF. I'm really unsure we want SQL-executing JS apps at all
- and if so, the SQL statements must be parsed or run on a read-only
DB connection.

- Friedrich

> But all in all, looking impressive.
>
> James
>

-- 
Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/

http://twitter.com/pudo
http://pudo.org