[ckan-dev] Alternative upsert datastore API function

Tue Oct 30 12:15:04 UTC 2018

Hi all,

My CKAN (ckan 2.8) loads CSV files into datastore database tables. I update
these tables with upsert datastore API function.

Datastore follows this sequence:

Step 1:
file.csv is generated at 8:00 AM and it contains:
id, address, extra
1, address1, data1
2, address2, data2
3, address3, data3

Step 2:
upsert is executed and table has:
id, address, extra
1, address1, data1
2, address2, data2
3, address3, data3

Step 3:
A new file.csv is generated at 8:05 AM and it contains:
id, address, extra
1, new-address1, data1
5, address5, data5

Step 4:
upsert is executed and datastore table contains this:
id, address, extra
1, new-address1, data1
2, address2, data2
3, address3, data3
5, address5, data5

So file.csv is different from datastore table. I need rows id=2 and id=3
were deleted, I mean, table should be:
id, address, extra
1, new-address1, data1
5, address5, data5

The only solution I can find is calling datastore delete API and after
create API each time.

Is there any other option? I mean, is there another API function or upsert
parameter?

Some files has more than 10.000 rows, and I think it could be and expensive
solution if I have to delete and create a table each time.

Thanks.

-- 
Isabel M. Ruiz Mellado
http://about.me/isabelruiz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20181030/e75561aa/attachment.html>