[ckan-dev] problem with import of latin character

Tue May 31 13:27:04 UTC 2011

On 31 May 2011 13:14, <p.romain at cg33.fr> wrote:

>
> Hi David,
>
> Thank you very much for taking some time for looking into our problems.
> If I believe the CHANKGELOg.txt file we use now the v1.3.2 version.
>

The changelog should tell you about previous releases, and is not
necessarily the current version.

> I am a bit confused about the version because the web page displays a
> v1.4.a mention

In 1.4a the 'a' stands for alpha - i.e. you are using a pre-release version
of the source (probably between 1.3.2 and 1.3.3), rather than an official
release. This is because you followed the instructions in README for a
developer install, so you would have got the latest code at the time. This
is probably fine, although you may want to match it to your deploy version,
which should be a release version.

> Maybe I messed up the ckan distribution but I am not sure if I =should do
> an upgrade or not.
> Actually our problem of latin character disapeared although we did not
> change anything to the code.

Great!

David

As you will see in a different post regarding production deployment, I am a
> bit lost in the process of setting up a production environnement with
> CKAN...

>
> Cordialement,*
> Pascal Romain**
> responsable domaine applicatif gestion documentaire*
> SPEC*
> Direction des Systèmes d'Information*
> 05 56 99 33 33 poste 6643*
> * <http://www.gironde.fr/>
>
>
> *David Read <david.read at okfn.org>*
>
> Envoyé par : ckan-dev-bounces at lists.okfn.org
>
> 30/05/2011 16:42
> Veuillez répondre à
> CKAN Development Discussions <ckan-dev at lists.okfn.org>
>
> A
> CKAN Development Discussions <ckan-dev at lists.okfn.org>
> cc
> ckan-dev-bounces at lists.okfn.org, y.ngarboui at cg33.fr
> Objet
> Re: [ckan-dev] problem with import of latin character
>
>
>
>
>
>
> On 30 May 2011 14:49, <*p.romain at cg33.fr* <p.romain at cg33.fr>> wrote:
>
> Hi,
>
> Here is the output from python import_data_file.py test_importer_full.xls
>
> 2011-05-30 15:10:20,880 DEBUG [ckan.lib.base] Access OK.
> 2011-05-30 15:10:20,916 DEBUG [ckan.lib.base] Retrieving request params:
> UnicodeMultiDict([('{"name": "annakarenina", "title": "A Novel By Tolstoy",
> "version": "0.7a", "url": "*http://www.annakarenina.com*<http://www.annakarenina.com/>",
> "n
> otes": "Some test notes\\n\\n### A 3rd level heading\\n\\n**Some bolded
> text.**\\n\\n*Some italicized text.*\\n\\nForeign characters:\\nu with
> umlaut \\u00fc\\n66-style quote \\u201c\\nforeign word: th\\u00fc
> mb\\n \\nNeeds escaping:\\nleft arrow <\\n\\n<*http://ckan.net/*<http://ckan.net/>>\\n\\n",
> "state": "active", "license_id": "other-open", "tags": "russian tolstoy",
> "groups": "david roger", "extras": {"genre": "romantic novel",
>  "original media": "book"}, "resources": [{"url": "*
> http://www.annakarenina.com/download/x=1&y=2*<http://www.annakarenina.com/download/x=1&y=2>",
> "format": "plain text", "description": "Full text. Needs escaping: \\"
> Umlaut: \\u00fc", "hash": "abc123", "al
> t_url": "alt123", "size": "123"}, {"url": "*
> http://www.annakarenina.com/index.json*<http://www.annakarenina.com/index.json>",
> "format": "json", "description": "Index of the novel", "hash": "def456",
> "alt_url": "alt345", "size": "345"}]}', u'1')])
> 2011-05-30 15:10:20,916 DEBUG [ckan.lib.base] Retrieving request POST:
> UnicodeMultiDict([('{"name": "annakarenina", "title": "A Novel By Tolstoy",
> "version": "0.7a", "url": "*http://www.annakarenina.com*<http://www.annakarenina.com/>",
> "not
> es": "Some test notes\\n\\n### A 3rd level heading\\n\\n**Some bolded
> text.**\\n\\n*Some italicized text.*\\n\\nForeign characters:\\nu with
> umlaut \\u00fc\\n66-style quote \\u201c\\nforeign word: th\\u00fcmb
> \\n \\nNeeds escaping:\\nleft arrow <\\n\\n<*http://ckan.net/*<http://ckan.net/>>\\n\\n",
> "state": "active", "license_id": "other-open", "tags": "russian tolstoy",
> "groups": "david roger", "extras": {"genre": "romantic novel", "
> original media": "book"}, "resources": [{"url": "*
> http://www.annakarenina.com/download/x=1&y=2*<http://www.annakarenina.com/download/x=1&y=2>",
> "format": "plain text", "description": "Full text. Needs escaping: \\"
> Umlaut: \\u00fc", "hash": "abc123", "alt_
> url": "alt123", "size": "123"}, {"url": "*
> http://www.annakarenina.com/index.json*<http://www.annakarenina.com/index.json>",
> "format": "json", "description": "Index of the novel", "hash": "def456",
> "alt_url": "alt345", "size": "345"}]}', u'1')])
> 2011-05-30 15:10:20,917 DEBUG [ckan.lib.base] Request data extracted:
> {u'name': u'annakarenina', u'title': u'A Novel By Tolstoy', u'url': u'*
> http://www.annakarenina.com* <http://www.annakarenina.com/>', u'notes':
> u'Some test notes\n\n### A 3
> rd level heading\n\n**Some bolded text.**\n\n*Some italicized
> text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote
> \u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow <\n\n<*
> http://ckan* <http://ckan/>
> .net/>\n\n', u'tags': u'russian tolstoy', u'state': u'active', u'version':
> u'0.7a', u'groups': u'david roger', u'license_id': u'other-open',
> u'resources': [{u'hash': u'abc123', u'description': u'Full text. Ne
> eds escaping: " Umlaut: \xfc', u'format': u'plain text', u'url': u'*
> http://www.annakarenina.com/download/x=1&y=2*<http://www.annakarenina.com/download/x=1&y=2>',
> u'alt_url': u'alt123', u'size': u'123'}, {u'hash': u'def456',
> u'description': u'Index of the n
> ovel', u'format': u'json', u'url': u'*
> http://www.annakarenina.com/index.json*<http://www.annakarenina.com/index.json>',
> u'alt_url': u'alt345', u'size': u'345'}], u'extras': {u'genre': u'romantic
> novel', u'original media': u'book'}}
> 2011-05-30 15:10:20,925 DEBUG [ckan.controllers.apiv1.package] Created
> object annakarenina
> 2011-05-30 15:10:20,940 ERROR [ckan.controllers.apiv1.package] unexpected
> keywords {'alt_url': u'alt123', 'size': u'123'}
>
> These columns the spreadsheet are 'resource extras', which were introduced
> in CKAN release 1.3.2. Which version of CKAN are you loading into? The debug
> looks suspiciously like an old release. With the latest code I get this and
> no error:
>
> 2011-05-30 15:37:42,528 DEBUG [ckan.logic.action.create] Created object
> annakarenina
>
> We noticed that the ckanclient is calling the apiv1 : is this correct ?
>
> API version is specified through the base location in your script. e.g. you
> append '/2' to your base_location (e.g. '*http://localhost:5000/api/2*<http://localhost:5000/api/2>')
> and use API version 2. However, v1 should do the same as v2 for loading
> packages - so this is not a concern here.
>
> David
>
>
>
>
>
> Cordialement,*
> Pascal Romain**
> responsable domaine applicatif gestion documentaire*
> SPEC*
> Direction des Systèmes d'Information*
> 05 56 99 33 33 poste 6643*
> * <http://www.gironde.fr/>
>
> *Rufus Pollock <**rufus.pollock at okfn.org* <rufus.pollock at okfn.org>*>*
> Envoyé par : *ckan-dev-bounces at lists.okfn.org*<ckan-dev-bounces at lists.okfn.org>
>
> 26/05/2011 14:07
>
>
> Veuillez répondre à*
> **rufus.pollock at okfn.org* <rufus.pollock at okfn.org>; Veuillez répondre à
> CKAN Development Discussions <*ckan-dev at lists.okfn.org*<ckan-dev at lists.okfn.org>
> >
>
> A
> CKAN Development Discussions <*ckan-dev at lists.okfn.org*<ckan-dev at lists.okfn.org>
> >
> cc
> *y.ngarboui at cg33.fr* <y.ngarboui at cg33.fr>
> Objet
> Re: [ckan-dev] problem with import of latin character
>
>
>
>
>
>
> 2011/5/26  <*p.romain at cg33.fr* <p.romain at cg33.fr>>:
> >
> > Hi,
> >
> > We wrote a script for the bulk import of our catalogues of data packages
> but
> > we can't import data with latin characters. Is there something to set in
> the
> > python script that we are missing (encoding or something..)
>
> To add to David's comment:
>
> It would be useful to see the output from script and also log output
> from the server -- if you could run it with paster serve {config-file}
> you'll see out put from server in terminal.
>
> Rufus
>
> _______________________________________________
> ckan-dev mailing list*
> **ckan-dev at lists.okfn.org* <ckan-dev at lists.okfn.org>*
> **http://lists.okfn.org/mailman/listinfo/ckan-dev*<http://lists.okfn.org/mailman/listinfo/ckan-dev>
>
>
> __________________________________________________________________
>
> Ce message et toutes les pièces jointes sont confidentiels et établis à
> l'intention exclusive de ses destinataires. Ce message ne constitue pas un
> document officiel. Seuls les documents revêtus de la signature du Président
> du Conseil Général ou d'un de ses délégataires sont de nature à engager le
> Département.
> Toute utilisation ou diffusion non autorisée est interdite. Tout message
> électronique est susceptible d'altération et le Département de la Gironde
> décline toute responsabilité au titre de ce message s'il a été altéré,
> déformé, falsifié.
>
> _______________________________________________
> ckan-dev mailing list*
> **ckan-dev at lists.okfn.org* <ckan-dev at lists.okfn.org>*
> **http://lists.okfn.org/mailman/listinfo/ckan-dev*<http://lists.okfn.org/mailman/listinfo/ckan-dev>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>
> __________________________________________________________________
>
> Ce message et toutes les pièces jointes sont confidentiels et établis à
> l'intention exclusive de ses destinataires. Ce message ne constitue pas un
> document officiel. Seuls les documents revêtus de la signature du Président
> du Conseil Général ou d'un de ses délégataires sont de nature à engager le
> Département.
> Toute utilisation ou diffusion non autorisée est interdite. Tout message
> électronique est susceptible d'altération et le Département de la Gironde
> décline toute responsabilité au titre de ce message s'il a été altéré,
> déformé, falsifié.
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20110531/84095259/attachment-0001.html>