[okfn-labs] Calendar fields (date/time)

[Rufus Pollock]

More great feedback questions :-) - its really appreciated so please keep
it coming.

Also thanks to Michael's and Alioune's comments too!

On 19 February 2014 22:58, Stefan Urbanek <stefan.urbanek at gmail.com> wrote:
[snip]

PROPOSAL
>
> Here is a proposal for specification and for the data package creators.
>
> Specification should say that all `date`, `time` and `datetime` fields
> should conform to an ISO standard or some other well defined standard.
> Current spec says:
>
> * *date*: a date. This MUST be in ISO6801 format YYYY-MM-DD or, if not, a
> format field must be provided describing the structure.
> * *time*: a time without a date
> * *datetime*: a date-time. This MUST be in ISO 8601 format of
> YYYY-MM-DDThh:mm:ssZ in UTC time or, if not, a format field must be
> provided.
>
> I would highly recommend to drop the *"... if not, a format field must be
> provided describing the structure."* as it will cause many automated
> processing complications and will result in serious data quality issues.
>

I like this proposal a lot but there are certain questions I have:

- What do data packagers do with time series data they get that is just a
year (e.g. as in your example of bond yields). Personally i've found myself
more and more just converting these to actual dates e.g. 2004 =>
2004-01-01, 2004 Q3 => 2004-10-01 etc. This means you get a real date but
at the cost of adding some precision that wasn't really there.

- the date type did have some use in, for example, affecting how the graph
and grid views work. For example, setting a field to date means that the
graphing library would interpret that field as a date. This is is somewhat
minor and this isn't a problem if you have already converted "2004" to
"2004-01-01"

So, to summarize: I agree with making the change you propose re types - are
you happy to open an appropriate issue re JSON Table Schema on
https://github.com/dataprotocols/dataprotocols/issues


> Information that a field contains a year or any other combination of
> calendar units should be in some kind of analytical metadata. We don't have
> to define them yet, as they might be very case-specific.
>

I think string plus pattern/format attribute should be ok for this. I also
note we have a units spec http://dataprotocols.org/units/ (not much used
yet and not formally integrated into JSON Table Schema but it could be)

Rufus

[snip]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20140221/1a498c70/attachment-0004.html>