[ckan-dev] Forms in CKAN

Mon Jan 17 16:32:58 UTC 2011

Hello

I am still not sure of the exact requirements but having tried forms before
I will have my say anyway.

I have looked at the packages Rufus suggested.

>1. Flatland: http://discorporate.us/projects/flatland/docs/tip/
>* Only does 2+3 which is a good thing IMO

This is an interesting take on the issue.  I really like the concept and the
ideas but I hate the implementation.  It uses a declarative style like

class form(base):
    field = Integer
    field2 = Text(validation = moo)

This looks innocuous but it involves complicated metaclass trickery to make
work.  This would be ok if you knew you never had to dig deeper that that
but you inevitably will.  I personally don't like things that take your code
and turn it into something different.   It does not deal with ordering
properly either from what I can tell.

I also do not like the way it flattens things. (more below)

>2. WTForms: http://wtforms.simplecodes.com/
> * Used in standard flask docs: <
http://flask.pocoo.org/docs/patterns/wtforms/>

Nothing new here and the same declarative style.

My take on it is this, based on very short talk with Seb and Pudo and what I
have done before.

I agree with Rufus there should be a general dict for each entity (an entity
being important things like a package or resource) and that dict can have
the the ability to be extended.  It should have two forms a flattened and
unflattend one.

unflattened.

{name: "fred"
 groups: ["a", "b"]
 extras:  [{"thing1" : "value1",
            "thing2" : "value2"},
                  ..
           ]
 }

The flattened one should use the great python feature of having tuples as
keys so.   Tuples are better as they can be sorted properly i.e by depth, so
are easy to unflatten/flatten.

{
(name,):  "fred",
(groups, 0):  "a",
(groups, 1):  "b",
(extras, 0, thing1): "value1",
(extras, 0, thing2): "value2"
...
}

The flattend one is much more useful.  To send to the front end and back for
starters.  You can convert the keys to strings.  I recommend the delimiter
being a double underscore.  Its readable and all you have to do is make sure
no field has a double underscore in.  So a key sent back and forth to the
front end will look like  extras__0__thing1.

The disadvantage of the flattened form are that doing some lookups i.e
looking for children of a particular node   are o(n) and not o(1).  However
the size of the forms are never going to big enough for that to make much of
a difference.

The other advantage to the flattend form is you can assign validation easily
by having a mirrored dict like.

{
(name,):  [vaidator1, validator2],
(groups, 0):  [group_validator],
(groups, 1):  [group_validator],
(extras, 0, thing1): [extra_thing1_validator],
(extras, 0, thing2): [extra_validator]
...
}

You can have other mirrored ones for things like defaults, field_types if
needed.

The forms themselves should basically be in html/templates with the ids
using the to the key convention (plus a pefix).  They are much easier to
update this way.

I think the validation should probably just be formencode ones, just because
they have the greatest set of predefined ones and making your own is pretty
simple.
Do not use the formencode.schemas though for the same metaclass reasons
above.

Hope that helps

David

On Sun, Jan 16, 2011 at 11:15 AM, Friedrich Lindenberg
<friedrich at pudo.org>wrote:

> Hi all,
>
> its a fantastic idea to tackle forms now, I think they are ripe for a
> nice refactoring :-)
>
> On Fri, Jan 14, 2011 at 7:38 PM, David Read <david.read at okfn.org> wrote:
> > All the excitement in the form is around the 'extra' fields, with
> > every customer wanting a different set of them. So I'd say we need to
> > be able to have the different variants of the extra fields - text,
> > select box, date, date range, etc. as common 'widgets' which can be
> > reused for every form that we do.
>
> I'd like to be more radical: we should be handling extras as
> first-class keys. This is a bit of a pet-fight of mine, but I think
> that we want pkg.as_dict() to return extras.update(pkg_data) and, in
> the reverse, the forms to really accept random fields by default. In
> effect, Package should have an update_from_dict method which sorts out
> the question of what to put where, not before!
>
> > The one that is missing is converting from the field to the API format
> > (although it could be added).
>
> Important point: Getting rid of any differences in form handling for
> input coming from the Web UI and the REST API.
>
> > One idea Rufus is making the package dictionary the 'central' format
> > to build html from, and bypass the field value. I'm not convinced this
> > is saving us much for all the effort it would be.
>
> Choosing a dictionary as a main representation seems like a clean idea
> (and very helpful in my secret and sinister plan to slap some NoSQL
> thing onto the back of CKAN ;)
>
> > The Fields in common.py could do with a bit of a tidy up, but do they
> > encapsulate most things about a form widget in one place in the code
> > and is pretty logical.
> >
> > Rufus says it is a mistake to "merge too much model/validation/form
> > generation into one thing" but I quite like encapsulation.
>
> I don't think having HTML-yielding Python Widgets is as evil as I
> first assumed it would be, they should just each go into their own
> source file with the HTML inlined (last I checked the group form uses
> stuff from templates/package/*.html, so I'd rather have that in python
> than there).
>
> > The configuration using the form builder needs a bit of tightening up
> > to avoid some gotchas, but basically putting together a form is pretty
> > simple.
>
> There I disagree. I think we want two kinds of validation: system
> validation and tenant validation. The first kind would be
> squeezing-it-into-the-db-and-getting-it-out-again things like "name"
> constraints, valid extra key names, and authz questions like who can
> change "state" or assign which "groups". Tenant validation, on the
> other hand, holds specific knowledge so as not to allow "department"
> to be "Department of Space Invaders" etc. IMHO it would be extemely
> valuable to extract this into a seperate spec having roughly the
> following structure:
>
> - define a domain (e.g ckan.net)
>  - define a form (e.g linguistic resource)
>    - define a group of fields (e.g. "language information")
>      - define a field, e.g. "language" with a name, label,
> description, type, value range and widget hint
>
> As you will be quick to point out, this is not too far from what we
> have at the moment. My point is that we should have another go at
> separating system and semantic validation and that the latter does not
> need to be written in python. This could very well be a
> "metadata_schema.json" or we could even call in the RDF Ghostbusters -
> its a case for them if I ever saw one (a group of fields being an
> ontology etc.). I imagine CKAN having a WUI metadata editor plugin in
> the future, where people can describe different forms for their
> specific package classes as needed. If this ends up being Protégé, so
> be it.
>
> Best regards,
>
>  Friedrich
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20110117/ec62930e/attachment-0001.html>