[ckan-dev] Dpm and DataDeck rewriting. What about ckanclient?

Rufus Pollock rufus.pollock at okfn.org
Fri Feb 15 11:37:22 UTC 2013


On 14 February 2013 20:17, Daniel Graziotin <dgraziotin at task3.cc> wrote:
> Hello everyone,
>
> A couple of years ago, I worked on DataDeck <https://github.com/dgraziotin/datadeck> to make it the standard Python GUI for dpm <https://github.com/okfn/dpm>.
> Due to issues with dpm development, I then switched to dpm and helped there.
> Dpm was becoming a mess. Many changes were happening to CKAN, further limiting the development while rising dpm complexity.
> There were nice ideas floating around dpm; one of them was to turn it to the git of OpenData (sort of). Unfortunately, I could not be of more help and development stopped.

The development hasn't really stopped but taken another form. The
simple data package route has in some sense split out from CKAN and
exists as specs such as:

http://www.dataprotocols.org/en/latest/data-packages.html

And as a new mini-project: http://datasets.okfnlabs.org/

This latter is focused on: curation (selected datasets of high
quality), and a specific approach - see
http://datasets.okfnlabs.org/about. It may well use CKAN or parts
thereof as needed but atm it's mainly focused on following the data
packages spec, simple tools, and CSV + git for data.

This approach is completely complementary to CKAN - the data packages
spec is directly based on the CKAN JSON structure and there can be
reuse from both directions (e.g. use CKAN as the catalog or datastore,
while CKAN may be able to utilize some of the pure data packages
tooling and approach ...)

For an intro to the idea, see this slide deck: http://bit.ly/datasets-slides

BTW: primary discussion on datasets.okfnlabs.org is taking place on
okfn-labs: http://lists.okfn.org/mailman/listinfo/okfn-labs (see e.g.
http://lists.okfn.org/pipermail/okfn-labs/2013-February/000621.html).
(We want to avoid too much noise on the ckan-dev list -- there's
plenty of CKAN specific stuff to be discussed)

> Nowadays, dpm is old and bloated while DataDeck is completely useless. I would like to dedicate some more time to the OpenData movement.
> My free time is still scarce (I am a PhD student) but I am planning to rewrite dpm / fork it to sdpm, or simple data package manager.
> I would like to have it working a la (Debian) APT, plus the creation and upload of packages. Nothing more than that.
> When sdpm reaches some maturity, I would like to rewrite my DataDeck to support it.

Personally, I think the best way at the moment for the pure packages
route laid out above.

> Luckily, @CKANproject on Twitter has just pointed me to the recent discussion "API docs update" <http://lists.okfn.org/pipermail/ckan-dev/2013-February/003891.html>.
> Dpm relies on ckanclient. Now I see that there is uncertainty with ckanclient as well.
> I am with those who want ckanclient to exist: other people rely on it, e.g., me :-)

I'm pretty sure ckanclient will go on being maintained - it's pretty
heavily used and we're contributing patches to it (it's fully
functional atm I think!). If you were happy to contribute, I don't
think there would be any problems here.

> What do you think about the sdpm + DataDeck idea? Think of the couple like apt and Synaptic package manager

I note that much of the DataDeck stuff (at least as I approached it)
is now operational in: http://explorer.okfnlabs.org/ ...

> Regarding the API version / ckanclient: should I use the actual ckanclient? Should I wait for a better ckanclient instead?

I think you should use the actual ckanclient if you need a python API
to ckan - it's fully functional right now and would not need much work
if improvements are needed.

> I could spend some effort on ckanclient. However, I am more confident on developing at a higher level.
> Another reason to work on a simple dpm would be to focus only on a part of the APIv2/v3, to keep complexity down and maximize my effort in my limited free time.
>
> Any feedback / counterproposals?

See above :-)

Rufus




More information about the ckan-dev mailing list