[ckan-changes] [ckan/ckan] 762a51: Add IResourceUpload plugin interface

Tue Sep 16 10:06:50 UTC 2014

  Branch: refs/heads/master
  Home:   https://github.com/ckan/ckan
  Commit: 762a51979df1d62c2eb8cad2418440f2c8348c58
      https://github.com/ckan/ckan/commit/762a51979df1d62c2eb8cad2418440f2c8348c58
  Author: Sean Hammond <seanhammond at lavabit.com>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Add IResourceUpload plugin interface

  Commit: 4755cdd7e123bb53f30e76c804c7697a1fcff98b
      https://github.com/ckan/ckan/commit/4755cdd7e123bb53f30e76c804c7697a1fcff98b
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Generalise IResourceUpload into IResourceModification

Instead of proliferating the interfaces with special resource
cases, by creating yet another resource interface for any resource
creation (not only uploads), IResourceModification incorporates
both uploads and links, but after_create (which replaces
after_upload) provides an additional parameter indicating whether
the resource was uploaded or not.

This entail renaming but also a slightly different workflow in
calling the interface since we must catch deletions before
creations.

  Commit: cdf7765ceed938558cdf9cd3e720e19cbe33c589
      https://github.com/ckan/ckan/commit/cdf7765ceed938558cdf9cd3e720e19cbe33c589
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/lib/uploader.py

  Log Message:
  -----------
  Add comments to better explain the upload process

The upload process is slightly hard to follow and just by looking
at the code it looks like if one accidentally sets a filename and
then also sets clear on the ResourceUpload, it would remove the
just created file.

This is handled by javascript in the resource form which sets or
resets (to an empty string) the clear_upload input field which is
slightly disconnected to the upload function of ResourceUpload.

So, in order to make the code better stand on its own and to
further explain the assumption it makes about self.filename and
self.clear and how those work together, the upload function now
has some comments to guide developers and divert us from breaking
things.

  Commit: 96b3b6388f683c080844d9a8e4d9c241dd107a3f
      https://github.com/ckan/ckan/commit/96b3b6388f683c080844d9a8e4d9c241dd107a3f
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Remove 'file uploaded' and 'file deleted' return values from uploader

These were added to the previous IResourceUpload to know if a file
was being uploaded or if an uploaded file was deleted.

Looking into the ckan extension "ckanext-datapackager", for which
the original ckan branch, "datapackager", was developed (which is
where IResourceUpload comes from), it turns out that the file
deletion trigger isn't used. So it is save to assume that no-one
uses it (since it's in a special branch) and it can therefore be
removed.

The 'file uploaded' trigger can be inferred from the resource
dictionary's key 'url_type' which has the value 'upload' when a
file has been uploaded, but is set to an empty string when the
resources just links to an external file. Therefore we do not need
to track file uploads via a special parameter. This also makes the
IResourceModification after_create method use in extensions easier
to in case someone adds a different url_type and wants to process
'upload', '', and their special type of url_type.

Also note that after uploading a file the upload method returns.
That's still a good idea from the IResourceUpload implementation
and is left in there because that makes it slightly clearer that
after uploading a file the method should never even get to the
self.clear check (since we do not remove files if we upload them
as explained in the comments).

  Commit: 10be57ad39e8cdde0af0369cfe6294bc65222cf6
      https://github.com/ckan/ckan/commit/10be57ad39e8cdde0af0369cfe6294bc65222cf6
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Add after_update to IResourceModification for resource updates

In case extensions want to do different things based on creations
and updates, it is better to split them up into two different
methods.

  Commit: dc7d224586d4a708976e39001317d191f2f935e8
      https://github.com/ckan/ckan/commit/dc7d224586d4a708976e39001317d191f2f935e8
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/logic/action/create.py
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Add pre-processor methods to IResourceModifications

Add before_(create,update,delete) to IResourceModification to
complement the existing after_(create,update,delete) methods.

This allows extensions to parse data before actually creating,
updating or deleting it. Since 'file deleted' was removed in a
previous commit as it wasn't used, it could still be useful. By
adding the pre-processors to resource updates, one can now infer
file deletion from the resource being passed into the before_create
and before_update since those include the magic 'clear_upload'
input which indicates whether a file will be deleted.

  Commit: 15e28894f48f94a057d303f6b96c700024fcd72a
      https://github.com/ckan/ckan/commit/15e28894f48f94a057d303f6b96c700024fcd72a
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Update IResourceModification documentation

Minor edits to the documentation of IResourceModification methods
to make the documentation clearer.

  Commit: 37e8bd974d99216d1efbef88b58fc2f2ba147d13
      https://github.com/ckan/ckan/commit/37e8bd974d99216d1efbef88b58fc2f2ba147d13
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/logic/action/delete.py
    M ckan/plugins/interfaces.py

  Log Message:
  -----------
  Add call to before_delete and after_delete in IResourceModification

Until now before_delete and after_delete weren't being called when
a resource was deleted. They are now called in the appropriate
places but that required a change in the parameters passed in (for
the sake of performance).

The parameters sent to before_delete are context, resource and
resources where context is just the context dictionary, resource
is the data_dict being passed in (it should only contain the id
of the resource that should be deleted) and resources which is the
list of the package's resources.

The parameters sent to after_delete are the context dictionary
and the resources list (so not the resource dictionary) and the
resources list now does not contain the resource that was deleted
(after all, it was deleted).

This way we can still just use the list comprehension which is
faster than a for loop as we should not take a performance hit
for an edge case like this one (e.g. by switching to a for loop
or something).

  Commit: 79346a68d0eaa3f8e89cfe34cc863e38c6f3fea0
      https://github.com/ckan/ckan/commit/79346a68d0eaa3f8e89cfe34cc863e38c6f3fea0
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/tests/ckantestplugins.py
    M ckan/tests/logic/test_action.py
    M setup.py

  Log Message:
  -----------
  Add test cases for the IResourceModification plugin interface

This adds one test case for each of the three sets of hooks:

* One test case for before_create and after_create
* One test case for before_update and after_update
* One test case for before_delete and after_delete

  Commit: bc4988a382cc697834f755c14195e01eee174695
      https://github.com/ckan/ckan/commit/bc4988a382cc697834f755c14195e01eee174695
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/tests/ckantestplugins.py
    M ckan/tests/logic/test_action.py
    M setup.py

  Log Message:
  -----------
  Revert "Add test cases for the IResourceModification plugin interface"

This reverts commit 9f034dcff42ffeccca63a8ec835897a184e1f4ea.

The commit added tests to the "old" version of tests which has now
been deprecated and new_tests should be used instead of it.

  Commit: 072a693ae7a88e7318208aa39e0daebc7dd6ffd4
      https://github.com/ckan/ckan/commit/072a693ae7a88e7318208aa39e0daebc7dd6ffd4
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    A ckanext/example_iresourcemodification/__init__.py
    A ckanext/example_iresourcemodification/plugin.py
    A ckanext/example_iresourcemodification/tests/test_example_iresourcemodification.py
    M setup.py

  Log Message:
  -----------
  Add example plugin implementation and tests for IResourceModification

This adds an example ckan extension that uses the IResourceModification
plugin interface and then adds three tests for that example
implementation:

* One test case for before_create and after_create
* One test case for before_update and after_update
* One test case for before_delete and after_delete

  Commit: 85fb7b07af98f90316642858a5c0e001e4d30cfc
      https://github.com/ckan/ckan/commit/85fb7b07af98f90316642858a5c0e001e4d30cfc
  Author: Tryggvi Bjorgvinsson <tryggvi.bjorgvinsson at okfn.org>
  Date:   2014-09-12 (Fri, 12 Sep 2014)

  Changed paths:
    M ckan/logic/action/create.py
    M ckan/logic/action/delete.py
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py
    A ckanext/example_iresourcecontroller/__init__.py
    A ckanext/example_iresourcecontroller/plugin.py
    A ckanext/example_iresourcecontroller/tests/test_example_iresourcecontroller.py
    R ckanext/example_iresourcemodification/__init__.py
    R ckanext/example_iresourcemodification/plugin.py
    R ckanext/example_iresourcemodification/tests/test_example_iresourcemodification.py
    M setup.py

  Log Message:
  -----------
  Merge IResourceModification into IResourceController

This is done for consistency in the code (this makes it similar
to IPackageController).

It is worth noting that IResourceController already had a
before_show method which is called in the other methods as well
and some parameters could therefore be redundant but they are left
intact because it becomes difficult to know what is what if we ask
devs to use before_show in order to retrieve a variable they want
to use (and before_show can actually be called multiple times in
some cases).

  Commit: d74c4cb91695196a707248b1f1c5b2d0b0f482fe
      https://github.com/ckan/ckan/commit/d74c4cb91695196a707248b1f1c5b2d0b0f482fe
  Author: amercader <amercadero at gmail.com>
  Date:   2014-09-16 (Tue, 16 Sep 2014)

  Changed paths:
    M ckan/lib/uploader.py
    M ckan/logic/action/create.py
    M ckan/logic/action/delete.py
    M ckan/logic/action/update.py
    M ckan/plugins/interfaces.py
    A ckanext/example_iresourcecontroller/__init__.py
    A ckanext/example_iresourcecontroller/plugin.py
    A ckanext/example_iresourcecontroller/tests/test_example_iresourcecontroller.py
    M setup.py

  Log Message:
  -----------
  Merge branch 'tryggvib-resource-modification-interface'

Compare: https://github.com/ckan/ckan/compare/f83332a1ab61...d74c4cb91695