[ckan-dev] basic question on configuring SSL with CKAN

Tue Nov 7 08:13:22 UTC 2017

Why do you suspect the python version?
If there is a problem with the SSL_VERIFY I strongly suspect datapusher.
And as there have been some fixes for this functionality just recently, I'd
recommend to use the latest version of datapusher.
The SSL_VERIFY feature is available since datapusher 0.0.11 and has been
improved in both 0.0.12 and 0.0.13.

To check which version you have, open the following file:
/usr/lib/ckan/datapusher/src/datapusher/datapusher/__init__.py

I quickly checked the latest 2.6 deb package on http://packaging.ckan.org/
and they include datapusher 0.0.12, so I guess this is what you have
installed.

You wrote
> But the changes looks like are not being reflected

Are any values from the datapusher_settings.py reflected? Or is just the
SSL_VERIFY not working? Note that only with datapusher 0.0.13 the
SSL_VERIFY is applied to download the remote file. Before SSL_VERIFY is
only used to disable the SSL check for the CKAN API calls.

- Stefan

On Mon, Nov 6, 2017 at 12:05 PM, gustavo w <gustavo.waku at gmail.com> wrote:

> thanks for the tip.
>
> But I suspect it's the python version. I am using python 2.7.6
> Does it work good on python 2.7.9 ?
> Which python version do you recommend?
>
> thanks a lot,
> gustavo
>
> On Tue, Oct 31, 2017 at 1:15 AM, Stefan Oderbolz <stefan.oderbolz at liip.ch>
> wrote:
>
>> I'm not sure what version of datapusher is being installed when using the
>> package installation of CKAN, as usually you need to install this
>> seperately. But I don't have any experience with the deb package.
>>
>> Can you try to manually install the latest version of datapusher from
>> https://github.com/ckan/datapusher ?
>>
>> On Oct 26, 2017 19:25, "gustavo w" <gustavo.waku at gmail.com> wrote:
>>
>>> Hello folks,
>>>
>>> Is there any detailed guide to disable the SSL_VERIFY in the
>>> datapusher_settings.py?
>>> I modified it, rebooted the server, but it does not reflect my change.
>>> Any idea whats going on? I tried in 2.6.2 and 2.7.2
>>>
>>> thanks,
>>> gustavo
>>>
>>>
>>> On Wed, Oct 25, 2017 at 2:17 PM, gustavo w <gustavo.waku at gmail.com>
>>> wrote:
>>>
>>>> I did install the ckan as a package install (.deb), version 2.6.2
>>>> I do not know how to check the datapusher version.
>>>> thx
>>>>
>>>> On Wed, Oct 25, 2017 at 11:44 AM, Stefan Oderbolz <
>>>> stefan.oderbolz at liip.ch> wrote:
>>>>
>>>>> What version of datapusher do you use? How did you install it?
>>>>>
>>>>> On Wed, Oct 25, 2017 at 3:15 PM, gustavo w <gustavo.waku at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Stefan,
>>>>>>
>>>>>> I just modified the files at my machine, adding the SSL_VERIFY =
>>>>>> 'False'
>>>>>> *./etc/ckan/datapusher_settings.py*
>>>>>>
>>>>>> *./usr/lib/ckan/datapusher/src/datapusher/deployment/datapusher_settings.py*
>>>>>>
>>>>>> *the file looks like this:*
>>>>>> *-------*
>>>>>> import uuid
>>>>>>
>>>>>> DEBUG = False
>>>>>> TESTING = False
>>>>>> SECRET_KEY = str(uuid.uuid4())
>>>>>> USERNAME = str(uuid.uuid4())
>>>>>> PASSWORD = str(uuid.uuid4())
>>>>>>
>>>>>> NAME = 'datapusher'
>>>>>>
>>>>>> # database
>>>>>>
>>>>>> SQLALCHEMY_DATABASE_URI = 'sqlite:////tmp/job_store.db'
>>>>>>
>>>>>> # webserver host and port
>>>>>>
>>>>>> HOST = '0.0.0.0'
>>>>>> PORT = 8800
>>>>>>
>>>>>> # logging
>>>>>>
>>>>>> #FROM_EMAIL = 'server-error at example.com'
>>>>>> #ADMINS = ['yourname at example.com']  # where to send emails
>>>>>>
>>>>>> #LOG_FILE = '/tmp/ckan_service.log'
>>>>>> STDERR = True
>>>>>>
>>>>>> SSL_VERIFY = 'False'
>>>>>> *-------*
>>>>>>
>>>>>> *But the changes looks like are not being reflected. I have rebooted
>>>>>> the machine, but still nothing changes...*
>>>>>> *Do I have to do anything else?*
>>>>>>
>>>>>> thanks a lot,
>>>>>> Gustavo
>>>>>>
>>>>>> On Wed, Oct 25, 2017 at 9:48 AM, Stefan Oderbolz <
>>>>>> stefan.oderbolz at liip.ch> wrote:
>>>>>>
>>>>>>> Hi Gustavo,
>>>>>>>
>>>>>>> there was just recently a bugfix for this issue:
>>>>>>> https://github.com/ckan/datapusher/issues/149
>>>>>>> So you could either update to the latest version of datapusher or
>>>>>>> try to set SSL_VERIFY to 'False' (including the quotes).
>>>>>>>
>>>>>>> - Stefan
>>>>>>>
>>>>>>> On Tue, Oct 24, 2017 at 6:57 PM, gustavo w <gustavo.waku at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I installed ckan 2.6.2, and its working fine (via deb package),
>>>>>>>> with data explorer ok.
>>>>>>>> after I set the SSL protocol using the tutorial available at: (
>>>>>>>> https://github.com/ckan/ckan/wiki/SSL), whenever I upload a new
>>>>>>>> CSV file, the data is not displayed anymore into the dataexplorer.
>>>>>>>>
>>>>>>>> When I force the datastore to process the CSV file, I get the
>>>>>>>> following error:
>>>>>>>>
>>>>>>>> "Erro: File "/usr/lib/ckan/datapusher/lib/
>>>>>>>> python2.7/site-packages/apscheduler/scheduler.py", line 512, in
>>>>>>>> _run_job retval = job.func(*job.args, **job.kwargs)
>>>>>>>> File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py",
>>>>>>>> line 300, in push_to_datastore resource = get_resource(resource_id,
>>>>>>>> ckan_url, api_key)
>>>>>>>> File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py",
>>>>>>>> line 250, in get_resource 'Authorization': api_key}
>>>>>>>> File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/api.py",
>>>>>>>> line 110, in post return request('post', url, data=data, json=json,
>>>>>>>> **kwargs)
>>>>>>>> File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/api.py",
>>>>>>>> line 56, in request return session.request(method=method, url=url,
>>>>>>>> **kwargs)
>>>>>>>> File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/sessions.py",
>>>>>>>> line 488, in request resp = self.send(prep, **send_kwargs)
>>>>>>>> File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/sessions.py",
>>>>>>>> line 609, in send r = adapter.send(request, **kwargs)
>>>>>>>> File "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/adapters.py",
>>>>>>>> line 497, in send raise SSLError(e, request=request)
>>>>>>>> SSLError(SSLError(SSLError(1, '_ssl.c:510: error:14090086:SSL
>>>>>>>> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify
>>>>>>>> failed'),),)"
>>>>>>>>
>>>>>>>> Looks like it's an error with python lib SSL protocol.
>>>>>>>> I was instructed to follow the tutorial at:
>>>>>>>> http://docs.ckan.org/projects/datapusher/en/latest/using
>>>>>>>> .html#configuring-ssl-verification
>>>>>>>>
>>>>>>>> the tutorial says:
>>>>>>>> *"If you still have problems verifying certificates, or maybe for
>>>>>>>> test purposes, you can switch the verification off in
>>>>>>>> datapusher_settings.py:*
>>>>>>>> *SSL_VERIFY = False"*
>>>>>>>>
>>>>>>>> *I set SSL_VERIFY = False in datapusher_settings.py*
>>>>>>>> *but the server does not reflect this change.*
>>>>>>>>
>>>>>>>> *I found two files like this in my machine.*
>>>>>>>> *./etc/ckan/datapusher_settings.py*
>>>>>>>>
>>>>>>>> *./usr/lib/ckan/datapusher/src/datapusher/deployment/datapusher_settings.py*
>>>>>>>>
>>>>>>>> *I changed both and rebooted the machine.*
>>>>>>>> *somehow it does not work.*
>>>>>>>> *Do I have to recompile somehow the file?*
>>>>>>>>
>>>>>>>> *Can anyone give me directions on how to set the SSL_VERIFY flag
>>>>>>>> into the datapusher_settings? do I have to recompile anything else?*
>>>>>>>>
>>>>>>>> *Thanks a lot,*
>>>>>>>> *Gustavo*
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ckan-dev mailing list
>>>>>>>> ckan-dev at lists.okfn.org
>>>>>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
>>>>>>> Tel +41 43 500 39 80 <+41%2043%20500%2039%2080> // GnuPG 0x7B588C67
>>>>>>> // www.liip.ch
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ckan-dev mailing list
>>>>>>> ckan-dev at lists.okfn.org
>>>>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> ckan-dev mailing list
>>>>>> ckan-dev at lists.okfn.org
>>>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
>>>>> Tel +41 43 500 39 80 <+41%2043%20500%2039%2080> // GnuPG 0x7B588C67
>>>>> // www.liip.ch
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> ckan-dev mailing list
>>>>> ckan-dev at lists.okfn.org
>>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>

-- 
Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20171107/9de12fef/attachment-0003.html>