[ckan-dev] problem with import of latin character

Mon May 30 13:49:02 UTC 2011

Hi,

Here is the output from python import_data_file.py test_importer_full.xls

2011-05-30 15:10:20,880 DEBUG [ckan.lib.base] Access OK.
2011-05-30 15:10:20,916 DEBUG [ckan.lib.base] Retrieving request params: 
UnicodeMultiDict([('{"name": "annakarenina", "title": "A Novel By 
Tolstoy", "version": "0.7a", "url": "http://www.annakarenina.com", "n
otes": "Some test notes\\n\\n### A 3rd level heading\\n\\n**Some bolded 
text.**\\n\\n*Some italicized text.*\\n\\nForeign characters:\\nu with 
umlaut \\u00fc\\n66-style quote \\u201c\\nforeign word: th\\u00fc
mb\\n \\nNeeds escaping:\\nleft arrow <\\n\\n<http://ckan.net/>\\n\\n", 
"state": "active", "license_id": "other-open", "tags": "russian tolstoy", 
"groups": "david roger", "extras": {"genre": "romantic novel",
 "original media": "book"}, "resources": [{"url": 
"http://www.annakarenina.com/download/x=1&y=2", "format": "plain text", 
"description": "Full text. Needs escaping: \\" Umlaut: \\u00fc", "hash": 
"abc123", "al
t_url": "alt123", "size": "123"}, {"url": 
"http://www.annakarenina.com/index.json", "format": "json", "description": 
"Index of the novel", "hash": "def456", "alt_url": "alt345", "size": 
"345"}]}', u'1')])
2011-05-30 15:10:20,916 DEBUG [ckan.lib.base] Retrieving request POST: 
UnicodeMultiDict([('{"name": "annakarenina", "title": "A Novel By 
Tolstoy", "version": "0.7a", "url": "http://www.annakarenina.com", "not
es": "Some test notes\\n\\n### A 3rd level heading\\n\\n**Some bolded 
text.**\\n\\n*Some italicized text.*\\n\\nForeign characters:\\nu with 
umlaut \\u00fc\\n66-style quote \\u201c\\nforeign word: th\\u00fcmb
\\n \\nNeeds escaping:\\nleft arrow <\\n\\n<http://ckan.net/>\\n\\n", 
"state": "active", "license_id": "other-open", "tags": "russian tolstoy", 
"groups": "david roger", "extras": {"genre": "romantic novel", "
original media": "book"}, "resources": [{"url": 
"http://www.annakarenina.com/download/x=1&y=2", "format": "plain text", 
"description": "Full text. Needs escaping: \\" Umlaut: \\u00fc", "hash": 
"abc123", "alt_
url": "alt123", "size": "123"}, {"url": 
"http://www.annakarenina.com/index.json", "format": "json", "description": 
"Index of the novel", "hash": "def456", "alt_url": "alt345", "size": 
"345"}]}', u'1')])
2011-05-30 15:10:20,917 DEBUG [ckan.lib.base] Request data extracted: 
{u'name': u'annakarenina', u'title': u'A Novel By Tolstoy', u'url': 
u'http://www.annakarenina.com', u'notes': u'Some test notes\n\n### A 3
rd level heading\n\n**Some bolded text.**\n\n*Some italicized 
text.*\n\nForeign characters:\nu with umlaut \xfc\n66-style quote 
\u201c\nforeign word: th\xfcmb\n \nNeeds escaping:\nleft arrow 
<\n\n<http://ckan
.net/>\n\n', u'tags': u'russian tolstoy', u'state': u'active', u'version': 
u'0.7a', u'groups': u'david roger', u'license_id': u'other-open', 
u'resources': [{u'hash': u'abc123', u'description': u'Full text. Ne
eds escaping: " Umlaut: \xfc', u'format': u'plain text', u'url': 
u'http://www.annakarenina.com/download/x=1&y=2', u'alt_url': u'alt123', 
u'size': u'123'}, {u'hash': u'def456', u'description': u'Index of the n
ovel', u'format': u'json', u'url': 
u'http://www.annakarenina.com/index.json', u'alt_url': u'alt345', u'size': 
u'345'}], u'extras': {u'genre': u'romantic novel', u'original media': 
u'book'}}
2011-05-30 15:10:20,925 DEBUG [ckan.controllers.apiv1.package] Created 
object annakarenina
2011-05-30 15:10:20,940 ERROR [ckan.controllers.apiv1.package] unexpected 
keywords {'alt_url': u'alt123', 'size': u'123'}

We noticed that the ckanclient is calling the apiv1 : is this correct ?

Cordialement,
Pascal Romain
responsable domaine applicatif gestion documentaire
SPEC
Direction des Systèmes d'Information
05 56 99 33 33 poste 6643

Rufus Pollock <rufus.pollock at okfn.org> 
Envoyé par : ckan-dev-bounces at lists.okfn.org
26/05/2011 14:07
Veuillez répondre à
rufus.pollock at okfn.org; Veuillez répondre à
CKAN Development Discussions <ckan-dev at lists.okfn.org>

A
CKAN Development Discussions <ckan-dev at lists.okfn.org>
cc
y.ngarboui at cg33.fr
Objet
Re: [ckan-dev] problem with import of latin character

2011/5/26  <p.romain at cg33.fr>:
>
> Hi,
>
> We wrote a script for the bulk import of our catalogues of data packages 
but
> we can't import data with latin characters. Is there something to set in 
the
> python script that we are missing (encoding or something..)

To add to David's comment:

It would be useful to see the output from script and also log output
from the server -- if you could run it with paster serve {config-file}
you'll see out put from server in terminal.

Rufus

_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/ckan-dev

__________________________________________________________________

Ce message et toutes les pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Ce message ne constitue pas un document officiel. Seuls les documents revêtus de la signature du Président du Conseil Général ou d'un de ses délégataires sont de nature à engager le Département. 
Toute utilisation ou diffusion non autorisée est interdite. Tout message électronique est susceptible d'altération et le Département de la Gironde décline toute responsabilité au titre de ce message s'il a été altéré, déformé, falsifié.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20110530/c946aa33/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 7254 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20110530/c946aa33/attachment-0001.jpe>