[datacatalogs] Few quick issues with the DCIP spec

Thu Nov 22 13:09:55 UTC 2012

On 20 November 2012 23:35, <therzog1 at worldbank.org> wrote:
> Hi all,
>
> I'm coming a bit late to this, but I wanted to ask a couple of quick questions about http://spec.datacatalogs.org. I'll probably have more later after a deeper review. Please let me know if you prefer me to post these on the GitHub issue tracker (I looked, but there didn't seem to be much action there compared to this listserv).

Good to send here as well. We're intending to use the issue tracker
more as a way to log and track issues but the listserv is the best
place to discuss.

> 1. Has there been any thought to future proofing the API? I noticed there are several forward-looking comments about things like pagers which, if/when implemented, might cause some compatibility issues. Let's say, for instance, that a future version of the spec implements a paging option: it doesn't look like there's any provision for a client to know what version(s) the API server supports.  This could be addressed in several ways, for instance requiring "/v1/" in the endpoint, or via content negotiation, each with its set of advantages and drawbacks. Just wondering if this was discussed at all.

This is an excellent question Tim. As you say the possibility of
upgrades is envisioned so considering versioning is important. I've
created an issue for this here:

https://github.com/dataprotocols/data-catalog-spec/issues/3

I'd be interested in what others thought. My feeling is that we should
at least put version info somewhere so clients know what version they
are dealing with. Perhaps the meta field could be used to list the
various version available.

> 2. Under "Help Dataset API Call:" the first two example URLs are confusing. It seems one of these should return a list of Datasets, as described above under "List Dataset API Call"

Agreed and ticketed: https://github.com/dataprotocols/data-catalog-spec/issues/4

I also note there is some discussion about the best way to do bulk
download of an entire catalog (this could in fact be an ultra-simple
Level 0 before the Level 1 of a REST API - e.g. the first supported by
e.g. CKAN was just a bulk dump of the entire catalog). This is
separate but could affect the meaning of the /dataset/ url.

> 3. The change_type: "delete" case is interesting. Is this used for datasets that no longer exist in the catalog? What is the appropriate server response for the corresponding URL? If the URL should return a record, how should the response record indicate the dataset is deleted? If it shouldn't return a record, should it instead return a response code 410 (Gone)? Should the URL still be included as a field in these cases?

Issue: https://github.com/dataprotocols/data-catalog-spec/issues/5

Yes, my understanding of "delete" case was that it was for the case
you describe. The 410 response code would then be appropriate and I
think we should note that.

> 4. Should Dataset API requests support 404 responses? That would distinguish between malformed requests, and requests for unrecognized datasets.

Agreed and ticketed: https://github.com/dataprotocols/data-catalog-spec/issues/6

Rufus