[ckan-dev] Encoding problems with DataStore

Wed Jun 14 16:48:44 UTC 2017

Hi everybody,

I'm having encoding problems with a non-ASCII CSV file that I manually 
upload to CKAN and which is then stored in the DataStore by the 
DataPusher.

If I store the file on disk encoded as UTF-8 then the non-ASCII characters 
in it get mangled in the DataStore (e.g. "ß" is mangled to "\u0102\xf9" 
instead of "\xdf"). If, on the other hand, I store the file locally in 
Latin1 encoding then it is represented correctly in the DataPusher. 
However, in that case, downloading the file will obviously also return it 
as Latin1, whereas I want the download to be in UTF-8. This happens both 
for Firefox and Chrome.

The strange thing is that I do have previously uploaded other UTF-8 
encoded CSV files without any problems.

I'm currently out of ideas, so any pointers would be much appreciated.

Best regards,
Florian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20170614/cd650350/attachment-0002.html>