[ckan-dev] Package modification date

Tue May 17 14:19:43 UTC 2011

On 17 May 2011 12:30, William Waites <ww at styx.org> wrote:
> The atom feed and revision API are both complicated pains to deal with when all you want is the modified date or more importantly packages modified since X. This has been pointed out before.

I guess we have a 2 step process to get the revision list, then query
each revision. This follows our REST model. Do suggest an alternative!

> Looking at revisions only makes sense from a ckan centric view offbeat world and the atom feed is not machine readable. It is a shame to have to resortto scraping to get the information we need out of ckan...

As I have reiterated today (having already said to you a couple of
weeks ago), the atom feed is not for machines - it is designed to be
human readable. Instead, look at revision search which is. No scraping
required - JSON format.

David

>
>
> David Read <david.read at okfn.org> a écrit :
>
>>On 17 May 2011 11:08, David Raznick <kindly at gmail.com> wrote:
>>> 2011/5/17 Adrià Mercader <amercadero at gmail.com>
>>>>
>>>> Hi all,
>>>>
>>>> I'm working on the CKAN harvester, and I need to check when a package
>>>> was last modified. I'm using the json API to get the remote packages.
>>>> Two different questions here:
>>>>
>>>> * For an individual package, "metadata_modified" looks like the one to
>>>> look at:
>>>> http://it.ckan.net/api/2/rest/package/2eb938e1-0aa0-4a33-8e2b-d7d8f6b41101
>>>>
>>>> The thing is that it is not always present, like in this case:
>>>> http://hu.ckan.net/api/2/rest/package/5f343aba-bef5-4f3f-abfd-672f2de1dbe0
>>>
>>> This must be an old/buggy version.   It would be sorted if we upgraded it,
>>> as the data is still there.
>>>>
>>>> Should I fall back to the revision
>>>>
>>>> (http://hu.ckan.net/api/2/rest/revision/6fe1c323-2094-4955-887b-abee19ab4f28)
>>>> to look at the last modified date in that case?
>>>> For the packages that have "metadata_modified", is this equivalent to
>>>> the revision timestamp?
>>>>
>>>> * If I wanted to query only the packages modified from a certain date,
>>>> I guess I should use the revision search api:
>>>> http://it.ckan.net/api/search/revision?since_time=2011-04-17
>>>>
>>>> My question is, are the revisions returned only referring to changes
>>>> in packages, or there could be also changes in groups, users, etc on
>>>> this list?
>>>
>>>
>>> Changes in metadata_modified include changes to extras, resources and tags
>>> as well.
>>>
>>> The revision_id against the package (above) only shows changes to the
>>> package object itself *not* the extras, resources and tags.
>>>
>>> No method shows changes to users and groups.
>>
>>(only because we've not seen a use case for it yet...)
>>
>>> Sadly the easiest way is to get changes since a certain date from the atom
>>> feed.  However that needs to be parsed.  This feed gives changes to extras,
>>> resources and tags.
>>
>>Actually the revision search does this very nicely. The Atom feed has
>>a similar functionality, but is formatted more for humans and
>>syndication software.
>>
>>David
>>
>>>
>>>>
>>>>
>>>> Thanks!
>>>>
>>>> Adrià
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>
>>>
>>
>>_______________________________________________
>>ckan-dev mailing list
>>ckan-dev at lists.okfn.org
>>http://lists.okfn.org/mailman/listinfo/ckan-dev
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>