[ECODP-dev] migration of the namespace in production

Wed Dec 18 10:32:00 UTC 2013

Hi David,

but for some reason EU ODP CKAN  does not allow web interface edits for api
inserted datasets.

So I am wondering if I will stumble into this issue.

Bert

2013/12/18 David Raznick <david.raznick at okfn.org>

> Hello,
>
> I am not sure why it made the data take up that much space, we only do
> updates in place.   We also tested the update against the most recent
> dump of data we had from them without much space.  You should be able
> to run it while the site is live, without downtime.
>
> As to your other question. Yes you should be able to do normal api
> updates, if you do *not* put an "rdf" field, then the metadata can be
> edited afterwards.
>
> Thanks for the warning here, I tackle that.

> In theory you can use RDF2CKAN, but missing out posting the rdf json
> field, and that should work.
>
> David
>
> On 18 December 2013 10:12, Bert Van Nuffelen
> <bert.van.nuffelen at tenforce.com> wrote:
> > Hi David,
> >
> > Well they encountered a problem during the migration: postgreSQL died
> > because of diskspace problems (the script seems to create so much
> additional
> > space, that it overflow the available space).
> > But then they added more resources, and restarted the process. It took
> > ultimately 8 hours to finish. (including all reindexations)
> >
> > To my other question: it is possible to use the api calls to do an update
> > and have the web publishers being able to edit the data afterwards?
> >
> > kind regards,
> >
> > Bert
> >
> >
> >
> > 2013/12/18 David Raznick <david.raznick at okfn.org>
> >>
> >> Hello
> >>
> >> It should be safe to rerun the paster command again, it was designed
> >> to be re-runable.
> >>
> >> The resource type was definately part of the migration (on the test
> >> system this worked), so the only reasons that this would fail would be
> >> that.
> >>
> >> 1. The migration stopped because of an error.  In this case we should
> >> know what the error is, so we can fix it.
> >> 2. It was stopped in the middle and not given time to finish.
> >>
> >> In both cases it needs to be run again.
> >>
> >> Thanks
> >>
> >> David
> >>
> >>
> >>
> >> On 17 December 2013 21:57, Bert Van Nuffelen
> >> <bert.van.nuffelen at tenforce.com> wrote:
> >> > Hi David & Darwin,
> >> >
> >> > we face in production the next issue: the migration script (a paster
> >> > command) that would update all occurrences of the old namespace
> >> > "ec.europa.eu/open-data" to the new namespace "open-data.europa.eu"
> has
> >> > not
> >> > resulted in the desired end result.
> >> > It seems that the change for the datasets has not been in effect.
> >> >
> >> > I wonder why it happend, but more I am working towards some
> alternative
> >> > to
> >> > get it fixed. I thought I could update the database via the api, but
> >> > probably this will lead to not editable datasets for the datasets that
> >> > are
> >> > entered via the web interface. Is that right?
> >> >
> >> > An example of interest can be:
> >> >
> >> >
> >> >
> http://open-data.europa.eu/data/dataset/pan-european-opinion-poll-on-occupational-safety-and-health.rdf
> >> >
> >> > In that we see that the publisher url is now the correct value
> >> > <dct:publisher
> >> >
> >> > rdf:resource="
> http://publications.europa.eu/resource/authority/corporate-body/OSHA"/>
> >> >
> >> > but that the references to the document types point to the old
> >> > namespace.
> >> >
> >> > Any idea how we can resolve this further?
> >> >
> >> > kind regards,
> >> >
> >> > Bert
> >> >
> >> > --
> >> > Bert Van Nuffelen
> >> >
> >> > Semantic Technologies Software Architect at TenForce
> >> > www.tenforce.be
> >> >
> >> > Bert.Van.Nuffelen at tenforce.com
> >> > Office: +32 (0)16 31 48 60
> >> > Mobile:+32 479 06 24 26
> >> > skype: bert.van.nuffelen
> >> >
> >> > _______________________________________________
> >> > ecodp-dev mailing list
> >> > ecodp-dev at lists.okfn.org
> >> > https://lists.okfn.org/mailman/listinfo/ecodp-dev
> >> >
> >> _______________________________________________
> >> ecodp-dev mailing list
> >> ecodp-dev at lists.okfn.org
> >> https://lists.okfn.org/mailman/listinfo/ecodp-dev
> >
> >
> >
> >
> > --
> > Bert Van Nuffelen
> >
> > Semantic Technologies Software Architect at TenForce
> > www.tenforce.be
> >
> > Bert.Van.Nuffelen at tenforce.com
> > Office: +32 (0)16 31 48 60
> > Mobile:+32 479 06 24 26
> > skype: bert.van.nuffelen
> >
> > _______________________________________________
> > ecodp-dev mailing list
> > ecodp-dev at lists.okfn.org
> > https://lists.okfn.org/mailman/listinfo/ecodp-dev
> >
> _______________________________________________
> ecodp-dev mailing list
> ecodp-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ecodp-dev
>

-- 
Bert Van Nuffelen

Semantic Technologies Software Architect at TenForce
www.tenforce.be

Bert.Van.Nuffelen at tenforce.com
Office: +32 (0)16 31 48 60
Mobile:+32 479 06 24 26
skype: bert.van.nuffelen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/ecodp-dev/attachments/20131218/8e2f7b92/attachment-0001.html>