[CKAN-Security] X-Forwarded-Host HTTP header is always trusted and is used in url_for

Mon Aug 13 14:38:04 UTC 2018

Thanks Phil for the heads up and Adria for including us in the thread.

Fortunately, none of our direct government partners are using CKAN below
2.5, but we have relationships with some who do and will relay the info.

As we're currently talking about the CKAN 3.0 roadmap, taking this
opportunity to add "proactive opt-in monitoring" this to the discussion as
well

https://github.com/ckan/ideas-and-roadmap/issues/218

Best,
Joel

On Mon, Aug 13, 2018 at 6:21 AM Adrià Mercader <adria.mercader at okfn.org>
wrote:

> Thanks for the heads up Philip,
>
> It's  a tricky one. We knew that all <2.5.x sites would be vulnerable,
> but there are really important sites in there so perhaps is worth
> trying to reach out (adding Joel and Stephen to the thread in case
> they can help out with the American/Australian ones). Of course we are
> missing other sites, but discussing the issue in a blog post or
> mailing list also gives further exposure to the exploit.
>
> We'll discuss the best approach in tomorrow's dev meeting.
>
> Just to remind, the patch that should be applied is this one:
>
> https://github.com/ckan/ckan/pull/2599
>
> On 11 August 2018 at 01:06, Philip Ashlock - QXA <philip.ashlock at gsa.gov>
> wrote:
> > Thanks Adria, we implemented the patch and it looks like it solved the
> issue
> > (we also needed to update production.ini to use the full domain with
> https).
> >
> > The researcher who flagged this issue presented at Black Hat yesterday
> and
> > he also published a blog post which cited our website and has been
> getting a
> > lot of attention (it's been one of the top links on Hacker News). For
> those
> > following ckan issues on github obviously this has been known for a
> while,
> > but it's now fair to say that the vulnerability has been disclosed in a
> way
> > that makes it much easier for others to discover and exploit.
> >
> > I've done some casual scanning of CKAN sites and the majority of those I
> > checked appear to be running a version of 2.5 or older that's vulnerable
> to
> > this. However, most did not appear to be using caching on the URLs I
> tested
> > so they haven't yet fully activated the potential for the cache poisoning
> > exploit. The only one I've found so far that does use caching and is
> > exploitable is https://www.data.vic.gov.au. That said, if any of these
> other
> > sites were to implement a CDN or caching in the future they would
> > immediately expose themselves to the exploit.
> >
> > I'm probably not going to spend too much time looking for impacted sites
> or
> > trying to notify folks, but I wanted to raise this in case that might be
> of
> > interest to somebody or if you think this should be discussed more
> publicly
> > on a mailing list or something.
> >
> > Some of the sites that look like they're vulnerable but which I couldn't
> > determine if they were using caching in a way that could be exploited
> > include:
> >
> > https://www.europeandataportal.eu/data/
> > http://data.europa.eu/euodp/en/data/
> > http://datos.gob.cl/dataset
> > http://www.data.go.jp/data/dataset
> > https://data.overheid.nl/data/dataset
> > https://catalogue.data.wa.gov.au/dataset
> > http://data.ottawa.ca/
> > https://data.kk.dk/
> > https://data.boston.gov
> > http://www.wprdc.org
> > https://www.opendataphilly.org/dataset
> >
> >
> > I just tested for this by running commands like:
> >
> > curl -H 'x-forwarded-host: foo.com' http://data.europa.eu/euodp/en/data/
> |
> > grep data-site-root
> >
> > When there's caching in place I poisoned it like this:
> >
> > curl -i -s -k -X $'GET' \
> >     -H $'Host: www.data.vic.gov.au' -H $'Accept-Encoding: gzip,
> deflate' -H
> > $'Accept: /' -H $'Accept-Language: en' -H $'User-Agent: Mozilla/5.0
> > (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)' -H
> > $'x-forwarded-host: foo.com' -H $'Connection: close' \
> >     $'https://www.data.vic.gov.au/data/dataset?q=test1234456abcdef' >
> > /dev/null
> >
> > And then I verified it had been poisoned by running:
> >
> > curl https://www.data.vic.gov.au/data/dataset?q=test1234456abcdef | grep
> > data-site-root
> >
> > In any case, I just wanted to spread the news that this will have a bit
> more
> > public visibility now.
> >
> > Best,
> > Phil
> >
> > On Thu, Jun 7, 2018 at 10:07 AM, Adrià Mercader <adria.mercader at okfn.org
> >
> > wrote:
> >>
> >> Thanks for the details.
> >>
> >> As of CKAN 2.5 we included code to sanitize the host used in fully
> >> qualified URLs generated by `url_for`, making sure it is using the one
> >> defined in the `ckan.site_url` configuration setting:
> >>
> >> https://github.com/ckan/ckan/pull/2599
> >>
> >> Would you mind testing if that patch solves the issue? My
> >> understanding is that it will but I might have missed something along
> >> the way.
> >>
> >> I guess now it's a good time to mention that a lot has moved forward
> >> on recent CKAN releases :)
> >>
> >> Let me know if the patch works and if you need any more details.
> >>
> >> Best,
> >>
> >> Adrià
> >>
> >>
> >>
> >> On 6 June 2018 at 16:27, John Jediny - QXA <john.jediny at gsa.gov> wrote:
> >> > Here is the full report
> >> >
> >> > On Tue, Jun 5, 2018 at 6:06 PM, Philip Ashlock - QXA
> >> > <philip.ashlock at gsa.gov> wrote:
> >> >>
> >> >> Hello,
> >> >>
> >> >> Data.gov uses CKAN and is participating in a bug bounty program
> through
> >> >> hackerone.com. We are currently responding to a vulnerability that
> >> >> comes
> >> >> from libraries used by CKAN (specifically the url_for function
> provided
> >> >> by
> >> >> routes/flask). I haven't seen this reported with routes or flask, but
> >> >> my
> >> >> understanding is that routes is a port of the rails routing
> >> >> functionality
> >> >> and it does look like this vulnerability was reported under rails
> (also
> >> >> via
> >> >> HackerOne). You can see that vulnerability report here
> >> >> https://github.com/rails/rails/issues/29893
> >> >>
> >> >> If I understand correctly that this is a vulnerability impacting
> other
> >> >> CKAN instances and it is not something currently being addressed, we
> >> >> wanted
> >> >> to make sure it was brought to your attention. While I am aware that
> >> >> this
> >> >> appears to be caused by upstream libraries, I wanted to start by
> >> >> addressing
> >> >> it with CKAN, because the researcher who reported it to us on
> HackerOne
> >> >> has
> >> >> indicated they expect to publish their findings soon and include it
> in
> >> >> a
> >> >> talk at the Black Hat conference and they will likely use CKAN as an
> >> >> example.
> >> >>
> >> >> If you can confirm my understanding of this problem is correct and
> that
> >> >> it
> >> >> hasn't already been addressed or reported to the upstream libraries
> >> >> (routes/flask), we'd also appreciate assistance in bringing the
> >> >> disclosure
> >> >> to the attention of those developers.
> >> >>
> >> >> If you need more information or a proof of concept beyond the
> >> >> documentation reported with the issue on rails, we'd be happy to
> >> >> provide
> >> >> that.
> >> >>
> >> >> Best,
> >> >> Phil
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Title: IT Specialist
> >> > Program: Data.gov
> >> > Office: Technology Transformation Service (TTS)
> >> > Agency: General Services Administration
> >> > Github: JJediny
> >> >
> >> > _______________________________________________
> >> > CKAN security
> >> > https://lists.okfn.org/mailman/listinfo/security
> >> >
> >> >
> https://lists.okfn.org/mailman/options/security/adria.mercader%40okfn.org
> >> >
> >> > Repo: https://github.com/ckan/ckan-security
> >>
> >
>

-- 

Joel Natividad

OpenGov

+1 (347) 565-5635

jnatividad at opengov.com | @jqnatividad <http://twitter.com/jqnatividad>

Silicon Valley
<https://www.google.com/maps/place/OpenGov+Inc/@37.4859652,-122.2121292,15z/data=!4m2!3m1!1s0x0:0xb84d4c3f06ecd893>
| Washington DC
<https://www.google.com/maps/place/OpenGov,+Inc./@38.915617,-77.047496,17z/data=!4m6!1m3!3m2!1s0x89b7b7cf85e25661:0xf3c11f6e3f635963!2sOpenGov,+Inc.!3m1!1s0x89b7b7cf85e25661:0xf3c11f6e3f635963>
| Portland
<https://www.google.com/maps/place/220+NW+8th+Ave,+Portland,+OR+97209/@45.5248202,-122.6802196,17z/data=!3m1!4b1!4m2!3m1!1s0x54950a01a6fdee1f:0x7f2eb4b084a75646>
| New York
<https://www.google.com/maps/place/33+Irving+Pl,+New+York,+NY+10003/@40.7274072,-73.9853282,1067a,20y,338.96h,44.7t/data=!3m1!1e3!4m2!3m1!1s0x89c2599f450ace29:0xcc9ef4cf79798e7c>
| London <https://goo.gl/maps/NFBzYXcrDcz>
<https://goo.gl/maps/NFBzYXcrDcz>
<https://goo.gl/maps/NFBzYXcrDcz>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.okfn.org/mailman/private/security/attachments/20180813/0076ab85/attachment-0003.html>