[ckan-dev] R: Dpm and DataDeck rewriting. What about ckanclient?

Fri Feb 15 13:36:11 UTC 2013

Hello Sean and Rufus, 
> Date: Fri, 15 Feb 2013 11:02:46 +0100
> From: Sean Hammond <sean.hammond at okfn.org>
> 
> > What do you think about the sdpm + DataDeck idea? Think of the couple
> > like apt and Synaptic package manager
> 
> This sounds to me kind of like a command-line interface for CKAN's action API?
> e.g. to print out the details of the gold prices package:
> 
>     $ dpm package show gold_prices
> 
> etc. It seems like a nice idea to me and should be easy enough to do as a thin
> wrapper around the action api.	

It would work exactly like that. This is somehow how dpm (called datapkg at that time) started years ago.
While I think dpm is going to be something terrific and beyond exploiting CKAN alone, it is my perception that some people just need a simple command-line for CKAN.
Dpm is going to be far beyond that, as Rufus reports below.

> 
> > Regarding the API version / ckanclient: should I use the actual
> > ckanclient? Should I wait for a better ckanclient instead?
> 
> Honestly, I think developing a new ckanclient as a thin wrapper around the
> action api _would_ be a fairly high-level development job.
> 
> I and others would be happy to discuss the design with you on this list.
> 

If nobody has taken care of the action API, I might focus on that instead. 

> Date: Fri, 15 Feb 2013 11:37:22 +0000
> From: Rufus Pollock <rufus.pollock at okfn.org>
> 
> The development hasn't really stopped but taken another form. The simple data
> package route has in some sense split out from CKAN and exists as specs such
> as:
> 
> http://www.dataprotocols.org/en/latest/data-packages.html
> 
> And as a new mini-project: http://datasets.okfnlabs.org/
> 
> This approach is completely complementary to CKAN - the data packages spec is
> directly based on the CKAN JSON structure and there can be reuse from both
> directions (e.g. use CKAN as the catalog or datastore, while CKAN may be able
> to utilize some of the pure data packages tooling and approach ...)
> 
> For an intro to the idea, see this slide deck: http://bit.ly/datasets-slides
> 

Whoa, Rufus (nice to hear back from you).
You are always involved in crazy stuff. Cheers for all the material. 
I am more familiar with the datapackage.json format than the dataset format employed by CKAN (e.g. http://datahub.io/api/rest/dataset/osm) to be honest.

Why is there a difference between a CKAN package/dataset and a dpm dapata package? 
Why aren't both project using the same data format/structure?
I think this is something you have all discussed for very long so please just point me to the discussion in case.

> > I would like to have it working a la (Debian) APT, plus the creation and upload
> of packages. Nothing more than that.
>
> Personally, I think the best way at the moment for the pure packages route laid
> out above.
> 
> I'm pretty sure ckanclient will go on being maintained 
> > Regarding the API version / ckanclient: should I use the actual ckanclient?
> Should I wait for a better ckanclient instead?
> 
> I think you should use the actual ckanclient if you need a python API to ckan - it's
> fully functional right now and would not need much work if improvements are
> needed.
> 

I see a ckanclient rewriting / enhancement as the creation of "libckanclient" for Python. 
On top of that well-made library there might come sdpm (or whatever name it would have) and then sdpm gui.
I still think there is value in obtaining datasets through a command line or GUI. 

On the other hand, I am interested in this difference between CKAN datasets/packages and dpm data packages.

What if this libckanclient returned Python objects following datapackage.json and metadata specs?
Could this be of any help for both projects?  This is just an idea.

Sorry if I am giving inputs which you might all have discussed many times in the past. 
My studies did not let me have a continuous involvement here but I am trying to fix that. 