[ckan-dev] Proposals for simplifying dpm

Fri Nov 25 23:41:44 UTC 2011

Hi Rufus and all dpm developers,
I was thinking about the ease of use (and development effort) of dpm.I
see contributing to datasets like a Wikipedia page contribution - plus
the resources - not a continuous bouncing of patches between
developers.I think that making dpm acting like git is a very high
target. What about a package manager with some subversion/rsync
capabilities?
Example:-dpm add resource.extAdds a local resource to datapackage.json
- i.e., package.resources). To add external resources, something
beautiful would be
dpm add http://blah.gov.ext/study/resource.csv (just links theresource
URL and name to datapackage.json).-dpm delete resource.extDeletes the
resource file and the relative entry in datapackage.json).-dpm
commitUploads the new resources, the modified resources (by looking at
the hash value) & the updated dataset to CKAN.-dpm updateFetches the
dataset files from CKAN & the resources if they changed. In case of
conflict, overwrite them or at least abort the operation and warn the
user. External resources - that are missing a hash value - will be
(re)downloaded if the user confirms it.-dpm statusExplains which
resources would be uploaded in case of a dpm commit.-dpm reset (with
confirmation)Destroys the local package; downloads the dataset and the
resources again.
It does not seem efficient and requires that users are aware of the
behaviors, but a well written documentation would help. Most
importantly, it ensures that everybody has the latest version of
everything.
I am not belittling the current plans. By lowering the target, I think
that a beautiful working tool may appear in a shorter time. I am quite
sure that a typical dpm user will be satisfied with these
functionalities.

This is just a stream of consciousness, there may be many things left
out and is just a start point for launching further ideas.
Let me know what you think about it.
Best Regards
-- 
Daniel Graziotin
- http://task3.cc