[ECODP-dev] Dataset issue resolved

Fri Jan 25 09:12:45 UTC 2013

On Thu, Jan 24, 2013 at 9:58 PM, Bert Van Nuffelen <
bert.van.nuffelen at tenforce.com> wrote:

> Hi Darwin,
>
> what was the fix? as it was from a backup created with a script
> following the rules specified by Ian
> on a system release 00.07.00. So this has to work.
>
> If the name is disappeared then there is an indication of an important
> issue.
>
> * is the backup containing all the data?
>

yes

> * can a backup of rel 00.07.00 be inserted in rel 00.07.50?
>

yes, no backup is corrupted

> * is there a change in ckan rel 00.07.00 - rel 00.07.50 which requires
> additional fields? Different constraints?
>

There was a change in the last release that mean we ordered resources by
name and there was a bug that meant if the name was not present then an
error would be raised.  The data *never* had a name but in our testing we
only tried it on datasets with one so we did not catch the bug.  Name in
resource was never mandatory.

So the mistake is us not testing against real imported data. The data
itself is fine.

>
> Also, I am really concerned about the fact that RDF2CKAN execution,
> hence REST calls to CKAN, can lead to a corrupted DB.
>

The db is not corrupted.

> This should be avoided at any time. The outside semantics is that
> either the REST calls are atomic: either they completely succeed
> either they fail and then all changes to the DB regarding this call
> are retracted. Can you confirm this is the case?
>

Per call they are atomic, completely wrapped in a transaction with a single
commit.

> It happened with this release. That is all I know at the moment.
>
> Also, What is the procedure for the PO if they encounter this
> situation? What do they have to do? Suppose only 1 dataset is in this
> situation, can CKAN detect that and provide a mean to clean up the
> database?
>

The updates are all atomic we have never had a corrupt database before.

>
> btw, this is "life data". The data is injested by rdf2ckan from
> publishers contributed rdf2ckan-packages.
> This is the procedure we have provided to PO.
> So any issue we see here is a real topic to solve.
>
> kind regards,
>
> Bert
>
>
> 2013/1/24 Darwin Peltan <darwin.peltan at okfn.org>:
> > Hi Bert,
> >
> > There was an issue with the release where viewing a dataset with imported
> > resources that didn't have a name would cause a server error when you
> tried
> > to view the dataset page (as imported resources don't have names).
> >
> > David has fixed that issue now on the test server. This is a good
> example of
> > where having a copy of the live database would be very useful for our
> > testing.
> >
> > Best,
> >
> > Darwin
> >
> > Darwin Peltan
> > Project Manager
> >
> > The Open Knowledge Foundation
> > http://www.okfn.org
> >
> > Skype: darwinp
> > Twitter: @darwin
> >
> > _______________________________________________
> > Ecodp-dev mailing list
> > Ecodp-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ecodp-dev
> >
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen
>
> _______________________________________________
> Ecodp-dev mailing list
> Ecodp-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ecodp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20130125/a46be6e8/attachment.html>