[ckan-dev] Scheming validators not updating dataset metadata issue

[Grant Paton-Simpson]

Hi,

Apologies if my question is really basic. I'm new to the mailing list as
I've just begun working with CKAN for the New Zealand open government
data initiative.

It seems I have a problem with the scheming extension. Even though my
dataset and resource metadata fields are displaying correctly in the
display and edit pages my custom validators don't seem to alter the
metadata (whether harvesting or updating). The custom validators are
being called and the log shows that the key, data, errors, and context
values are as they should be. But the data remains unchanged. Anything
obvious I'm missing?

Here is an extract from the schema.json file I've set scheming to use:

    {
      "field_name": "organization",
      "label": "Organisation",
      "validators": "ignore_missing add_eggs_and_spam"
    }

Here is the test validator I've added to
/usr/lib/ckan/default/src/ckan/ckan/logic/converters.py. I put it there
temporarily to ensure it is called:

def add_eggs_and_spam(key, data, errors, context):
    warn = logging.warning
    warn("Calling add_eggs_and_spam")
    warn("key is {}".format(key))
    warn("data is {}".format(data))
    warn("value is {}".format(data.get(key)))
    #warn("errors is {}".format(errors))
    #warn("context is {}".format(context))
    warn("keys BEFORE {}".format(sorted(data.keys())))
    data[key] = "eggs and spam"
    warn("keys AFTER {}".format(sorted(data.keys())))

Here are example log results proving it is being called:

Calling add_eggs_and_spam
key is (u'organization',)
data is {('extras', 4, 'key'): ...
...

Related note - I am able to alter data via the Manager > Update Dataset
button but not all fields update. Some do, a couple don't (e.g.
organization).

Any ideas? Hopefully the problem is something simple and obvious :-)

All the best,
Grant

-- 
Grant Paton-Simpson

e: grantps at catalyst.net.nz
m: 64 (0) 29 771-3462
d: 64 (0) 9  972-1325

Catalyst IT (Auckland)
Level 5, Rabobank Tower
2 Commerce Street
Auckland 1010
New Zealand