[ckan-dev] corupted JSON-string with API search

[Scheel, Thomas]

Hi Sean,

the search URL for the call is <IP>/api/search/dataset?sort=metadata_modified+desc&all_fields=1&limit=10
the result is like this:
...
extras": {
    "geographical_granularity": "",
    ...
    "comments": "{u'date': u'Tue Aug 21 13:01:48 GMT 2012', u'comment': u'testing ckan', u'userId': u'11702'}",
    ...
}

but when calling the dataset directly  I get the correct resonse:
...
"extras": {
    "geographical_granularity": "",
    ...
    "comments": [
        {
            "date": "Tue Aug 21 13:01:48 GMT 2012",
            "comment": "testing ckan",
            "userId": "11702"
        }
    ],
   ...
...

Best / Viele Grüße,

Thomas Scheel


-----Ursprüngliche Nachricht-----
Von: ckan-dev-bounces at lists.okfn.org [mailto:ckan-dev-bounces at lists.okfn.org] Im Auftrag von Sean Hammond
Gesendet: Freitag, 17. August 2012 13:14
An: CKAN Development Discussions
Betreff: Re: [ckan-dev] corupted JSON-string with API search

Hi Thomas, can you provide examples of exactly what URLs you're calling when you call a dataset with the search API?

Thanks

On Wed, Aug 15, 2012 at 06:11:42PM +0200, Scheel, Thomas wrote:
> Hello devs,
> 
>  
> 
> after testing the new CKAN 1.7.1 we found some strange handling of the 
> values in some extra fields. We store some JSON arrays or objects in 
> extra fields like this:
> 
> "extras": {
> 
>                 "comments":  [
> 
>                                {
> 
>                                                "date": "Tue Jan 31
>                                                12:01:53 GMT 2012",
> 
>                                                "comment": "comment
>                                                123",
> 
>                                                "userId": "10875"
> 
>                                },
> 
>                                {
> 
>                                                "comment": "lorem ipsum
>                                                ertzui",
> 
>                                                "date": "Wed Aug 15
>                                                12:26:12 CEST 2012",
> 
>                                                "userId": "niko3"
> 
>                                }
> 
>                 ],
> 
>                 ...
> 
> }
> 
> which is no problem so far.
> 
> When we call the dataset via api/rest/dataset/... CKAN returns the 
> correct result, but when we call the dataset with the search API ith
> all_fields=1 parameter CKAN returns this corrupted string: 
> 
>  
> 
> "comments": "{u'comment': u'comment 123', u'date': u'Tue Jan 31
> 12:01:53 GMT 2012', u'userId': u'10875'} {u'date': u'Wed Aug 15
> 12:26:12 CEST 2012', u'comment': u'lorem ipsum ertzui', u'userId':
> u'niko3'}",
> 
>  
> 
> As you can see the array is removed with all comma separators between 
> the array fields. Also all keys and values were suffixed with a u so 
> it is actually no valid JSON anymore.
> 
>  
> 
> I'm not sure what causes this behavior but it is very unsatisfying 
> because it destroys the functionality to add your own data to extra 
> fields like it was in older releases.  Is there any possibility that 
> CKAN returns the correct result in a valid JSON syntax like it does in 
> get dataset (api/rest/dataset/...) and why CKAN manipulates the value 
> of the data field?
>  
> When I searched the documentation what may cause this behavior I found 
> this sample result in http://docs.ckan.org/en/latest/apiv3.html
>  
>  {'count': 2,
> 
> 'results': [ { <snip> }, { <snip> }],
> 
> 'search_facets': {u'tags': {'items': [{'count': 1,
> 
>                                         'display_name': u'tolstoy',
> 
>                                         'name': u'tolstoy'},
> 
>                                        {'count': 2,
> 
>                                         'display_name': u'russian',
> 
>                                         'name': u'russian'}
> 
>                                       ]
> 
>                             }
> 
>                   }
> 
> }
> 
>  
> 
> Which is actually no valid JSON too and may cause parsing errors of 
> JSON frameworks or parsers.
> 
>  
> 
> Thanks in advance.
> 
>  
> 
> Best / Viele Grüße,
> 
>  
> 
> Thomas Scheel
> 
> ---
> 
> Fraunhofer-Institut für Offene
> 
> Kommunikationssysteme FOKUS
> 
> Kompetenz-Zentrum eGovernment
> 
>  
> 
> Kaiserin-Augusta-Allee 31
> 
> 10589 Berlin, Deutschland
> 
> Tel.  + 49 30 3463 9225
> 
> E-Mail: Thomas.Scheel at fokus.fraunhofer.de 
> <mailto:Thomas.Scheel at fokus.fraunhofer.de>
> 
> Internet: www.fokus.fraunhofer.de <http://www.fokus.fraunhofer.de>
> 
>  
> 

> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev


_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/ckan-dev