[ckan-dev] potential lost resource in case of 2 simultaneous requests ?

Fri Jun 30 22:59:07 UTC 2017

Hello,

I'm wondering if the fact that resource_create() ( and similarly
resource_update() ) does a package_show()
<https://github.com/OCHA-DAP/hdx-ckan/blob/dev/ckan/logic/action/create.py#L285>
followed later by a package_update()
<https://github.com/OCHA-DAP/hdx-ckan/blob/dev/ckan/logic/action/create.py#L302>
can potentially lead to a lost resource in some special cases.

Since postgres uses by default "read committed" transaction isolation I
think the following could happen:
2 almost simultaneous requests (R1, R2) come to the API and are dealt with
by different processes/threads. Both are modifying dataset *D which already
has one resource (resource1)*

TIMELINE

   1.  (R1) starts *resource_create*(resource2) on dataset D
   2.  (R2) starts *resource_update*(resource1) on dataset D
   3.  (R2) does package_show(D) => *gets D with just resource1*
   4.  (R2) changes resource1 in D
   5.  (R1) does package_show(D) => gets D with resource1
   6.  (R1) adds resource2 to D => D.resources = [resource1, resource2]
   7.  (R1) does package_update(D)
   8.  (R1) resource_create() finishes, everything is committed
   successfully to the db => D has 2 resources in the db
   9.  (R2) does package_update(D) - please note that here D only has one
   resource as read in step 3
   10.  (R2) resource_update() finishes, everything is committed
   successfully to the db => D has just resource1 (resource2 disappears)

Question: is this something that seems possible ? I reproduced this locally
on a slightly modified CKAN running paster but that could also mean that I
have something misconfigured or changed. Before starting to think about
strategies for avoiding this scenario (like a different transaction
isolation) is there some mechanism in CKAN that would prevent this ? Did
anyone stumble onto such an issue ?

Thank you,
Alex Gartner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170701/85900222/attachment-0002.html>