[ckan-dev] Help! - data resources gone missing

Mon May 19 03:40:38 UTC 2014

We are also on CKAN 2.2.

Like Alex the files that disappeared have had their state set to deleted, however setting it back to active and then switching the dataset from private-public-private did not make them appear again.  The files that disappeared on me we some large zip files, a couple of jpgs, tsv, and pdf - not limieted to csv files as Alex experienced.

I noticed that one of the files position integer setting was duplicated, so I changed it (the xlsx file) to 12 to avoid a conflict, but the resources still did not show up (apart from 1).  Have decided to delete the original dataset and reload, being careful to put in the order I want so I don't need to try reordering which seems to trigger the problem.

[cid:image002.png at 01CF7368.09A36550]

------------------------------

Message: 3

Date: Sat, 17 May 2014 17:52:25 +1000

From: "Alex (Maxious) Sadleir" <maxious at gmail.com<mailto:maxious at gmail.com>>

To: CKAN Development Discussions <ckan-dev at lists.okfn.org<mailto:ckan-dev at lists.okfn.org>>

Subject: Re: [ckan-dev] Help! - data resources gone missing

Message-ID:

                <CAL3EF=9P=JbdfZJ4DL3aVKnYWC_uZqvY=Qsy3Kqi+oJ84ROA-g at mail.gmail.com<mailto:CAL3EF=9P=JbdfZJ4DL3aVKnYWC_uZqvY=Qsy3Kqi+oJ84ROA-g at mail.gmail.com>>

Content-Type: text/plain; charset=UTF-8

Haven't upgraded datapusher since the 2.2 release so fbc6dc3 on stable branch.

I see there are some fixes so will deploy and see if it is resolved, maybe if this is a known severe issue let ckan-announce know?

On Thu, May 15, 2014 at 3:38 PM, Nigel Babu <nigel.babu at okfn.org<mailto:nigel.babu at okfn.org>> wrote:

> This sounds terrible. Can you tell me what's the latest commit in the

> datapusher version you're using? I remember hearing about this bug and

> us fixing it.

>

> Nigel Babu

> Developer, Open Knowledge

>

>

> On 15 May 2014 08:23, Alex (Maxious) Sadleir <maxious at gmail.com<mailto:maxious at gmail.com>> wrote:

>>

>> I have experienced this too - my best theory is it happens to

>> datasets with very small CSV files and there is some kind of race

>> condition in the DataPusher updating the resources.

>>

>> On datasets that it occurs (7 or so small CSVs), it occurs

>> consistently after it occurs for the first time whenever another file

>> is added or existing file edited. Interestingly the resources are all

>> marked as deleted within ms of each other and if I manually set the

>> state back to Active in postgres, I have to set the dataset Private

>> and Public to trigger them to reappear.

>>

>> On Thu, May 15, 2014 at 12:43 PM, Aaron McGlinchy

>> <McGlinchyA at landcareresearch.co.nz<mailto:McGlinchyA at landcareresearch.co.nz>> wrote:

>> > Hi,

>> >     I created a dataset and had uploaded a number of resources

>> > yesterday (about 10, of various file formats).

>> >

>> > Today I had some more resources to add, and I edited the dataset

>> > and began uploading the additional resources.  Whilst doing so I

>> > used the reorder resources button to arrange them in the order I

>> > wanted.  When I did this with the first additional resource it

>> > worked fine, but when I added another resource and went to reorder,

>> > the list of resources to rearrange that came up only had a single

>> > resource (the one just loaded) and all the ones I had loaded earlier disappeared.

>> >

>> > When I clicked back to the Dataset it only showed that single resource.

>> > However I knew the link for some of the resources I had previously

>> > loaded so I opened that and the resource I check is still there,

>> > just not showing up under the dataset.

>> >

>> > What can I do to get them back listed under the dataset?

>> >

>> > Thanks

>> > Aaron

>> >

>> > ________________________________

>> >

>> > Please consider the environment before printing this email

>> > Warning: This electronic message together with any attachments is

>> > confidential. If you receive it in error: (i) you must not read,

>> > use, disclose, copy or retain it; (ii) please contact the sender

>> > immediately by reply email and then delete the emails.

>> > The views expressed in this email may not be those of Landcare

>> > Research New Zealand Limited. http://www.landcareresearch.co.nz

>> > _______________________________________________

>> > ckan-dev mailing list

>> > ckan-dev at lists.okfn.org<mailto:ckan-dev at lists.okfn.org>

>> > https://lists.okfn.org/mailman/listinfo/ckan-dev

>> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev

>> _______________________________________________

>> ckan-dev mailing list

>> ckan-dev at lists.okfn.org<mailto:ckan-dev at lists.okfn.org>

>> https://lists.okfn.org/mailman/listinfo/ckan-dev

>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev

>

>

>

> _______________________________________________

> ckan-dev mailing list

> ckan-dev at lists.okfn.org<mailto:ckan-dev at lists.okfn.org>

> https://lists.okfn.org/mailman/listinfo/ckan-dev

> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev

>

------------------------------

Subject: Digest Footer

_______________________________________________

ckan-dev mailing list

ckan-dev at lists.okfn.org<mailto:ckan-dev at lists.okfn.org>

https://lists.okfn.org/mailman/listinfo/ckan-dev

Unsubscribe: https://lists.okfn.org/mailman/optionss/ckan-dev

------------------------------

End of ckan-dev Digest, Vol 43, Issue 62

****************************************

________________________________

Please consider the environment before printing this email
Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails.
The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140519/05b498e4/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 111689 bytes
Desc: image001.png
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140519/05b498e4/attachment-0003.png>