[ckan-dev] What is the current state and future of the extras field?

Fri Nov 8 13:03:04 UTC 2013

Hi Ian,

thanks for responding. So would you recommend staying with nested fields
on extras?

The problem arises for metadata consumer. If they use the API to
retrieve the packages as JSONs and parse them into a language specific
data structure, then they need to deal with the recursive JSON parsing
on the extra field values themselves.

I always found this to be an obstacle which could be removed by avoiding
nested fields altogether and keeping it flat.

Regarding the validation, I am looking more into validation outside of CKAN.

I am querying metadata records and want to validate, for instance are
certain fields set, are values of a certain field part of a list of
valid members (enums) etc.

For this I chose JSON Schema, simply because the API serializes the
packages as JSON.

This works fine for me, but I am still trying to figure out how to
design the schema. Currently, I am validating against the output of
/rest/dataset, because /action/package_show delivers more information
than necessary.

The identifier or revision identifier of an extras field is not so
important to the validation, because it is not really somethings the
data provider, the package creator influences.

Best,
Konrad

On 11/07/2013 09:38 PM, Ian Ward wrote:
> On Thu, Nov 7, 2013 at 3:11 PM, Konrad Reiche <konrad.reiche at gmail.com> wrote:
>> Hey there,
>>
>> what is the current state of the extras field in CKAN packages? As far
>> as I have learned subfields should be kept flat, without additional
>> nesting. If nested, then the nested fields need to be serialized to strings.
>>
>> "extras": { "my_field": { "my_subfield": 122 } }
>>
>> becomes
>>
>> "extras": { "my_field": "{\"my_subfield\": 122}" }
> 
> Yes, extras are stored as string values in an "extras" database table,
> so if you pass other types of objects they get converted into strings
> before they are stored.
> 
> Extras on resources are different, though. They get stored together in
> a single JSON string in the resource, so I'm not sure if they maintain
> nested values or not.
> 
>> Also asking, because I am looking for further methods to validate CKAN
>> packages against a JSON schema and I wonder if it would be better to
>> define extras as an array of objects or as an object as it was before in
>> the Model API. Since, Model and Search API are deprecated I guess the
>> former case.
> 
> Validation of dataset and resource fields is done with an IDatasetForm
> plugin using python code. What sort of validation are you looking for?
> 
> Ian
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
> 