[ckan-dev] Self Signed Certificate and DataPusher

Thu Jan 29 06:35:32 UTC 2015

Hi Michael,

I think this error probably occurs in a few places in CKAN, because by 
default the library we use for HTTP (python-requests) wants to verify 
certificates.

To temporarily work around this (and because datapusher typically only 
pushes locally anyway, it's not too heinous) we need to specify 
verify=False in any calls to requests.get/post.  This specific call is 
at https://github.com/ckan/datapusher/blob/master/datapusher/jobs.py#L224

Until this is fixed, you might try changing that line to something like

r = requests.post(url,
                       data=json.dumps({'id': resource_id}),
                       headers={'Content-Type': 'application/json',
                                'Authorization': api_key},
		      verify=False
                       )

You might also want to try setting

      SetEnv REQUESTS_CA_BUNDLE /path/to/my.crt

in your apache config for datapusher. This'll be the lowest impact (as 
you're not changing code) but I've never used it myself for this purpose.

There needs to be a better solution. Longer term we probably need to 
document the requests option for specifying the list of trusted CAs, 
and/or provide configuration - I've opened a ticket here 
https://github.com/ckan/ckan/issues/2249

Hope this helps a little.

Ross

Michael Speth wrote:
> Greetings,
> We have a test instance of CKAN running with https enabled using a
> self-signed cert. We get the following error from the DataPusher:
>
> Error: [u' File
> "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/apscheduler/scheduler.py",
> line 512, in _run_job\n retval = job.func(*job.args, **job.kwargs)\n',
> u' File "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py",
> line 222, in push_to_datastore\n resource = get_resource(resource_id,
> ckan_url, api_key)\n', u' File
> "/usr/lib/ckan/datapusher/src/datapusher/datapusher/jobs.py", line 178,
> in get_resource\n \'Authorization\': api_key}\n', u' File
> "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/api.py",
> line 94, in post\n return request(\'post\', url, data=data, json=json,
> **kwargs)\n', u' File
> "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/api.py",
> line 49, in request\n return session.request(method=method, url=url,
> **kwargs)\n', u' File
> "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/sessions.py",
> line 457, in request\n resp = self.send(prep, **send_kwargs)\n', u' File
> "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/sessions.py",
> line 569, in send\n r = adapter.send(request, **kwargs)\n', u' File
> "/usr/lib/ckan/datapusher/lib/python2.7/site-packages/requests/adapters.py",
> line 420, in send\n raise SSLError(e, request=request)\n',
> u"SSLError(SSLError(SSLError(1, '_ssl.c:504: error:14090086:SSL
> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed'),),)"]
>
> Is this due to not having a certificate authority signed cert? If so, is
> there any way around this? Or do we need to disable https (not desirable)?
>
> Thanks
>
> --
> Michael Speth
> DevOps Computer Engineer
> Landcare Research Consultant
>
> ------------------------------------------------------------------------
>
> Please consider the environment before printing this email
> Warning: This electronic message together with any attachments is
> confidential. If you receive it in error: (i) you must not read, use,
> disclose, copy or retain it; (ii) please contact the sender immediately
> by reply email and then delete the emails.
> The views expressed in this email may not be those of Landcare Research
> New Zealand Limited. http://www.landcareresearch.co.nz
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev