[ckan-dev] Forms in CKAN

[Friedrich Lindenberg]

Hi all,

its a fantastic idea to tackle forms now, I think they are ripe for a
nice refactoring :-)

On Fri, Jan 14, 2011 at 7:38 PM, David Read <david.read at okfn.org> wrote:
> All the excitement in the form is around the 'extra' fields, with
> every customer wanting a different set of them. So I'd say we need to
> be able to have the different variants of the extra fields - text,
> select box, date, date range, etc. as common 'widgets' which can be
> reused for every form that we do.

I'd like to be more radical: we should be handling extras as
first-class keys. This is a bit of a pet-fight of mine, but I think
that we want pkg.as_dict() to return extras.update(pkg_data) and, in
the reverse, the forms to really accept random fields by default. In
effect, Package should have an update_from_dict method which sorts out
the question of what to put where, not before!

> The one that is missing is converting from the field to the API format
> (although it could be added).

Important point: Getting rid of any differences in form handling for
input coming from the Web UI and the REST API.

> One idea Rufus is making the package dictionary the 'central' format
> to build html from, and bypass the field value. I'm not convinced this
> is saving us much for all the effort it would be.

Choosing a dictionary as a main representation seems like a clean idea
(and very helpful in my secret and sinister plan to slap some NoSQL
thing onto the back of CKAN ;)

> The Fields in common.py could do with a bit of a tidy up, but do they
> encapsulate most things about a form widget in one place in the code
> and is pretty logical.
>
> Rufus says it is a mistake to "merge too much model/validation/form
> generation into one thing" but I quite like encapsulation.

I don't think having HTML-yielding Python Widgets is as evil as I
first assumed it would be, they should just each go into their own
source file with the HTML inlined (last I checked the group form uses
stuff from templates/package/*.html, so I'd rather have that in python
than there).

> The configuration using the form builder needs a bit of tightening up
> to avoid some gotchas, but basically putting together a form is pretty
> simple.

There I disagree. I think we want two kinds of validation: system
validation and tenant validation. The first kind would be
squeezing-it-into-the-db-and-getting-it-out-again things like "name"
constraints, valid extra key names, and authz questions like who can
change "state" or assign which "groups". Tenant validation, on the
other hand, holds specific knowledge so as not to allow "department"
to be "Department of Space Invaders" etc. IMHO it would be extemely
valuable to extract this into a seperate spec having roughly the
following structure:

- define a domain (e.g ckan.net)
  - define a form (e.g linguistic resource)
    - define a group of fields (e.g. "language information")
      - define a field, e.g. "language" with a name, label,
description, type, value range and widget hint

As you will be quick to point out, this is not too far from what we
have at the moment. My point is that we should have another go at
separating system and semantic validation and that the latter does not
need to be written in python. This could very well be a
"metadata_schema.json" or we could even call in the RDF Ghostbusters -
its a case for them if I ever saw one (a group of fields being an
ontology etc.). I imagine CKAN having a WUI metadata editor plugin in
the future, where people can describe different forms for their
specific package classes as needed. If this ends up being Protégé, so
be it.

Best regards,

 Friedrich