[data-protocols] json-stat

Thu Aug 9 18:47:54 BST 2012

Dear Friedrich,

I'm Xavier Badosa, promoter of JSON-stat.org. I'm also involved in
SDMX-JSON, a similar initiative (making the impenetrable SDMX
apps-friendly).

In none of the those two standards, the "split model" is followed. A split
model seems efficient in an exchange context but I'm not so sure it is a
good fit for dissemination in the mobile apps space. A REST API that
follows the split models requires a package that first must be uncompressed
and then its components must be processed.

I agree that probably nothing is as concise for data as a CSV file. That's
why the JSON-stat data part is very similar to a CSV file (it can be
considered as a JSON serialization of a CSV file).

If you look at the JSON-stat documentation

http://json-stat.org/doc/

you'll notice that the standard allows for data-only responses that look
like this:

{
   "dataset" : {
      "value" : [4729, 4832, 9561]
   }}

(As you can see, this isn't very far away from a CSV file and it's as
concise as a CSV file.)

JSON-stat allows also for metadata-only responses. In practical terms, this
means that the developer can choose to have data and metadata together
(mixed response) or separated (like in the split model).

What do you think?

X.

PS: By the way, everyone is welcome to join our discussion group

https://groups.google.com/forum/#!forum/json-stat

CSV has been discussed in relation with streaming support in this thread

https://groups.google.com/d/topic/json-stat/W1--ezvUUOQ/discussion

On Thu, Aug 9, 2012 at 2:18 PM, Friedrich Lindenberg <
friedrich.lindenberg at okfn.org> wrote:

> Dear Vincenzo,
>
> this is really fascinating, thanks for the link! I've always found SDMX a
> bit impenetrable (not only because its XML), and I fully agree that JSON is
> a much easier way to represent this information. What I find interesting is
> that you have decided to describe both the metadata and data model in JSON,
> but to also package it with the actual data.
>
> As an alternative approach, Google's DSPL (
> https://developers.google.com/public-data/) splits these out - using XML
> for the metadata and normal CSV for the actual observations. Nick and I
> have done a bit of additional work to transfer this into JSON, you can find
> a simple converter here: https://github.com/nickstenning/dspljson ...
>
> I slightly prefer the split approach, as nothing can be quite as concise
> for data as a CSV file (and it's now no longer an issue to load that via
> JavaScript). What do you think?
>
> - Friedrich
>
> On Wed, Aug 8, 2012 at 9:56 AM, Vincenzo Patruno <patrunomeister at gmail.com
> > wrote:
>
>> Hi all!
>>
>> I know,... it's summertime and most of you are probably on vacation.....
>> Anyway, my name is Vincenzo Patruno and I'm IT specialist at ISTAT, the
>> italian statistical office.
>>
>> As probably you already know, some national statistical offices and seven
>> international organizations (like for instance IMF, OECD, Eurostat, ...)
>> are supporting the SDMX initiative (http://sdmx.org/)
>>
>> In short, me and some colleagues around the world, we believe we should
>>  need a different architectural model and a different protocol for data
>> transmission. The reason is to permit developers to use data easily inside
>> web apps and mobile apps.
>>
>> This is the sense of Json-Stat initiative (http://json-stat.org/)
>> lanched "from the bottom" last january.
>> Json-stat is a "json based" schema that permits to use easily datasets
>> inside applications. Just to be clear, for dataset we intend something
>> modeled as follow:
>>
>> Suppose we have the following table:
>>
>> Demographic Balance for the year 2010
>> Italy
>>                                                         males       females
>>    total
>> Resident population on 1st January 2928740331052925 60340328Live births
>> 289185 272759561944Deaths 286094301394587488 Natural increase3091 -28635
>> -25544
>>
>>
>>
>> A dataset "Demographic Balance" can be:
>>
>> geo        time         sex       demographic balance               value
>>           italy       2010         m        Resident population on 1st
>> jan 29287403 italy               2010 f           Resident population on
>> 1st jan 31052925[....]
>>
>> or, better
>>
>> geo       time          sex      demographic balance                value
>>           37         2010         m        001
>>                        29287403 37                 2010 f           001
>>                                                  31052925
>>
>>
>> where "37" is the code for "italy" and 001 the code for "Resident
>> population on 1st jan" etc
>>
>> This is what we call usually "multidimensional dataset". (But every
>> dataset is "multidimensional" because has almost the "geo" and "time"
>> dimension ....) What we need is probably to promote the culture for
>> modeling dataset. What we should promote is modeling both data and
>> structured metadata.
>>
>> To obtain something like this (open it on your web browser)
>>
>> http://apistat.istat.it/?q=getdatajson&dataset=DCIS_POPSTRBIL&dim=1,0,0,0&lang=1&tr=&te=
>>
>> that is a structured dataset that can be easily used inside applications
>> for an "interactive" use of data.
>>
>> Here some example.
>>
>> http://www.vincenzopatruno.org/json-stat/
>> Enjoy!
>>
>> VP
>>
>>
>>
>> --
>> Vincenzo Patruno
>> http://www.segnalazionit.org
>> http://www.vincenzopatruno.org
>>
>> “ If you want a track team to win the high jump you find one person who
>> can jump seven feet, not seven people who can jump one foot. ”.
>>
>> _______________________________________________
>> data-protocols mailing list
>> data-protocols at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/data-protocols
>>
>>
>
> _______________________________________________
> data-protocols mailing list
> data-protocols at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/data-protocols
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/data-protocols/attachments/20120809/a94680ad/attachment-0001.htm>