[ckan-dev] Help! - data resources gone missing

Aaron McGlinchy McGlinchyA at landcareresearch.co.nz
Tue May 20 23:39:58 UTC 2014


Aaargh!  It has happened again, this time I did not attempt to reorder resources (I did edit some resource descriptions, and spontaneousy all but one of the approx 5 resources I had uploaded disappeared).

Nigel - can you please point me to the link for the branch you think fixes this issue, I will ask my Tech guy to implement it (is it difficult to do/any instructions I can point him too)?

Thanks
Aaron

------------------------------

Message: 2
Date: Mon, 19 May 2014 09:44:42 +0530
From: Nigel Babu <nigel.babu at okfn.org>
To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
Subject: Re: [ckan-dev] Help! - data resources gone missing
Message-ID:
        <CALGFUD7NG8UDo1+cBeBbTGv=chmYXTuMsottzdD370pdYoxGBg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Alex and Aaron,

I believe the latest commit in the stable branch fixes this issue. I'll arrange for an email out to ckan-announce today.

Nigel Babu
Developer, Open Knowledge


On 19 May 2014 09:10, Aaron McGlinchy <McGlinchyA at landcareresearch.co.nz>wrote:

>  We are also on CKAN 2.2.
>
>
>
> Like Alex the files that disappeared have had their state set to
> deleted, however setting it back to active and then switching the
> dataset from private-public-private did not make them appear again.
> The files that disappeared on me we some large zip files, a couple of jpgs, tsv, and pdf ?
> not limieted to csv files as Alex experienced.
>
>
>
> I noticed that one of the files position integer setting was
> duplicated, so I changed it (the xlsx file) to 12 to avoid a conflict,
> but the resources still did not show up (apart from 1).  Have decided
> to delete the original dataset and reload, being careful to put in the
> order I want so I don?t need to try reordering which seems to trigger the problem.
>
>
>
> [image: cid:image002.png at 01CF7368.09A36550]
>
>
>
> ------------------------------
>
>
>
> Message: 3
>
> Date: Sat, 17 May 2014 17:52:25 +1000
>
> From: "Alex (Maxious) Sadleir" <maxious at gmail.com>
>
> To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
>
> Subject: Re: [ckan-dev] Help! - data resources gone missing
>
> Message-ID:
>
>                 <
> CAL3EF=9P=JbdfZJ4DL3aVKnYWC_uZqvY=Qsy3Kqi+oJ84ROA-g at mail.gmail.com>
>
> Content-Type: text/plain; charset=UTF-8
>
>
>
> Haven't upgraded datapusher since the 2.2 release so fbc6dc3 on stable
> branch.
>
>
>
> I see there are some fixes so will deploy and see if it is resolved,
> maybe if this is a known severe issue let ckan-announce know?
>
>
>
> On Thu, May 15, 2014 at 3:38 PM, Nigel Babu <nigel.babu at okfn.org> wrote:
>
> > This sounds terrible. Can you tell me what's the latest commit in
> > the
>
> > datapusher version you're using? I remember hearing about this bug
> > and
>
> > us fixing it.
>
> >
>
> > Nigel Babu
>
> > Developer, Open Knowledge
>
> >
>
> >
>
> > On 15 May 2014 08:23, Alex (Maxious) Sadleir <maxious at gmail.com> wrote:
>
> >>
>
> >> I have experienced this too - my best theory is it happens to
>
> >> datasets with very small CSV files and there is some kind of race
>
> >> condition in the DataPusher updating the resources.
>
> >>
>
> >> On datasets that it occurs (7 or so small CSVs), it occurs
>
> >> consistently after it occurs for the first time whenever another
> >> file
>
> >> is added or existing file edited. Interestingly the resources are
> >> all
>
> >> marked as deleted within ms of each other and if I manually set the
>
> >> state back to Active in postgres, I have to set the dataset Private
>
> >> and Public to trigger them to reappear.
>
> >>
>
> >> On Thu, May 15, 2014 at 12:43 PM, Aaron McGlinchy
>
> >> <McGlinchyA at landcareresearch.co.nz> wrote:
>
> >> > Hi,
>
> >> >     I created a dataset and had uploaded a number of resources
>
> >> > yesterday (about 10, of various file formats).
>
> >> >
>
> >> > Today I had some more resources to add, and I edited the dataset
>
> >> > and began uploading the additional resources.  Whilst doing so I
>
> >> > used the reorder resources button to arrange them in the order I
>
> >> > wanted.  When I did this with the first additional resource it
>
> >> > worked fine, but when I added another resource and went to
> >> > reorder,
>
> >> > the list of resources to rearrange that came up only had a single
>
> >> > resource (the one just loaded) and all the ones I had loaded
> >> > earlier
> disappeared.
>
> >> >
>
> >> > When I clicked back to the Dataset it only showed that single
> resource.
>
> >> > However I knew the link for some of the resources I had
> >> > previously
>
> >> > loaded so I opened that and the resource I check is still there,
>
> >> > just not showing up under the dataset.
>
> >> >
>
> >> > What can I do to get them back listed under the dataset?
>
> >> >
>
> >> > Thanks
>
> >> > Aaron
>
> >> >
>
> >> > ________________________________
>
> >> >
>
> >> > Please consider the environment before printing this email
>
> >> > Warning: This electronic message together with any attachments is
>
> >> > confidential. If you receive it in error: (i) you must not read,
>
> >> > use, disclose, copy or retain it; (ii) please contact the sender
>
> >> > immediately by reply email and then delete the emails.
>
> >> > The views expressed in this email may not be those of Landcare
>
> >> > Research New Zealand Limited. http://www.landcareresearch.co.nz
>
> >> > _______________________________________________
>
> >> > ckan-dev mailing list
>
> >> > ckan-dev at lists.okfn.org
>
> >> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>
> >> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
> >> _______________________________________________
>
> >> ckan-dev mailing list
>
> >> ckan-dev at lists.okfn.org
>
> >> https://lists.okfn.org/mailman/listinfo/ckan-dev
>
> >> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
> >
>
> >
>
> >
>
> > _______________________________________________
>
> > ckan-dev mailing list
>
> > ckan-dev at lists.okfn.org
>
> > https://lists.okfn.org/mailman/listinfo/ckan-dev
>
> > Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
> >
>
>
>
>
>
> ------------------------------
>
>
>
> Subject: Digest Footer
>
>
>
> _______________________________________________
>
> ckan-dev mailing list
>
> ckan-dev at lists.okfn.org
>
> https://lists.okfn.org/mailman/listinfo/ckan-dev
>
> Unsubscribe: https://lists.okfn.org/mailman/optionss/ckan-dev
>
>
>
>
>
> ------------------------------
>
>
>
> End of ckan-dev Digest, Vol 43, Issue 62
>
> ****************************************
>
> ------------------------------
>
> Please consider the environment before printing this email
> Warning: This electronic message together with any attachments is
> confidential. If you receive it in error: (i) you must not read, use,
> disclose, copy or retain it; (ii) please contact the sender
> immediately by reply email and then delete the emails.
> The views expressed in this email may not be those of Landcare
> Research New Zealand Limited. http://www.landcareresearch.co.nz
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/pipermail/ckan-dev/attachments/20140519/a7430471/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 111689 bytes
Desc: not available
URL: <https://lists.okfn.org/pipermail/ckan-dev/attachments/20140519/a7430471/attachment.png>

------------------------------

Subject: Digest Footer

_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
https://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: https://lists.okfn.org/mailman/optionss/ckan-dev


------------------------------

End of ckan-dev Digest, Vol 43, Issue 65
****************************************

________________________________

Please consider the environment before printing this email
Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails.
The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz



More information about the ckan-dev mailing list