[CKAN-Security] X-Forwarded-Host HTTP header is always trusted and is used in url_for

Mon Aug 13 10:21:49 UTC 2018

Thanks for the heads up Philip,

It's  a tricky one. We knew that all <2.5.x sites would be vulnerable,
but there are really important sites in there so perhaps is worth
trying to reach out (adding Joel and Stephen to the thread in case
they can help out with the American/Australian ones). Of course we are
missing other sites, but discussing the issue in a blog post or
mailing list also gives further exposure to the exploit.

We'll discuss the best approach in tomorrow's dev meeting.

Just to remind, the patch that should be applied is this one:

https://github.com/ckan/ckan/pull/2599

On 11 August 2018 at 01:06, Philip Ashlock - QXA <philip.ashlock at gsa.gov> wrote:
> Thanks Adria, we implemented the patch and it looks like it solved the issue
> (we also needed to update production.ini to use the full domain with https).
>
> The researcher who flagged this issue presented at Black Hat yesterday and
> he also published a blog post which cited our website and has been getting a
> lot of attention (it's been one of the top links on Hacker News). For those
> following ckan issues on github obviously this has been known for a while,
> but it's now fair to say that the vulnerability has been disclosed in a way
> that makes it much easier for others to discover and exploit.
>
> I've done some casual scanning of CKAN sites and the majority of those I
> checked appear to be running a version of 2.5 or older that's vulnerable to
> this. However, most did not appear to be using caching on the URLs I tested
> so they haven't yet fully activated the potential for the cache poisoning
> exploit. The only one I've found so far that does use caching and is
> exploitable is https://www.data.vic.gov.au. That said, if any of these other
> sites were to implement a CDN or caching in the future they would
> immediately expose themselves to the exploit.
>
> I'm probably not going to spend too much time looking for impacted sites or
> trying to notify folks, but I wanted to raise this in case that might be of
> interest to somebody or if you think this should be discussed more publicly
> on a mailing list or something.
>
> Some of the sites that look like they're vulnerable but which I couldn't
> determine if they were using caching in a way that could be exploited
> include:
>
> https://www.europeandataportal.eu/data/
> http://data.europa.eu/euodp/en/data/
> http://datos.gob.cl/dataset
> http://www.data.go.jp/data/dataset
> https://data.overheid.nl/data/dataset
> https://catalogue.data.wa.gov.au/dataset
> http://data.ottawa.ca/
> https://data.kk.dk/
> https://data.boston.gov
> http://www.wprdc.org
> https://www.opendataphilly.org/dataset
>
>
> I just tested for this by running commands like:
>
> curl -H 'x-forwarded-host: foo.com' http://data.europa.eu/euodp/en/data/ |
> grep data-site-root
>
> When there's caching in place I poisoned it like this:
>
> curl -i -s -k -X $'GET' \
>     -H $'Host: www.data.vic.gov.au' -H $'Accept-Encoding: gzip, deflate' -H
> $'Accept: /' -H $'Accept-Language: en' -H $'User-Agent: Mozilla/5.0
> (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)' -H
> $'x-forwarded-host: foo.com' -H $'Connection: close' \
>     $'https://www.data.vic.gov.au/data/dataset?q=test1234456abcdef' >
> /dev/null
>
> And then I verified it had been poisoned by running:
>
> curl https://www.data.vic.gov.au/data/dataset?q=test1234456abcdef | grep
> data-site-root
>
> In any case, I just wanted to spread the news that this will have a bit more
> public visibility now.
>
> Best,
> Phil
>
> On Thu, Jun 7, 2018 at 10:07 AM, Adrià Mercader <adria.mercader at okfn.org>
> wrote:
>>
>> Thanks for the details.
>>
>> As of CKAN 2.5 we included code to sanitize the host used in fully
>> qualified URLs generated by `url_for`, making sure it is using the one
>> defined in the `ckan.site_url` configuration setting:
>>
>> https://github.com/ckan/ckan/pull/2599
>>
>> Would you mind testing if that patch solves the issue? My
>> understanding is that it will but I might have missed something along
>> the way.
>>
>> I guess now it's a good time to mention that a lot has moved forward
>> on recent CKAN releases :)
>>
>> Let me know if the patch works and if you need any more details.
>>
>> Best,
>>
>> Adrià
>>
>>
>>
>> On 6 June 2018 at 16:27, John Jediny - QXA <john.jediny at gsa.gov> wrote:
>> > Here is the full report
>> >
>> > On Tue, Jun 5, 2018 at 6:06 PM, Philip Ashlock - QXA
>> > <philip.ashlock at gsa.gov> wrote:
>> >>
>> >> Hello,
>> >>
>> >> Data.gov uses CKAN and is participating in a bug bounty program through
>> >> hackerone.com. We are currently responding to a vulnerability that
>> >> comes
>> >> from libraries used by CKAN (specifically the url_for function provided
>> >> by
>> >> routes/flask). I haven't seen this reported with routes or flask, but
>> >> my
>> >> understanding is that routes is a port of the rails routing
>> >> functionality
>> >> and it does look like this vulnerability was reported under rails (also
>> >> via
>> >> HackerOne). You can see that vulnerability report here
>> >> https://github.com/rails/rails/issues/29893
>> >>
>> >> If I understand correctly that this is a vulnerability impacting other
>> >> CKAN instances and it is not something currently being addressed, we
>> >> wanted
>> >> to make sure it was brought to your attention. While I am aware that
>> >> this
>> >> appears to be caused by upstream libraries, I wanted to start by
>> >> addressing
>> >> it with CKAN, because the researcher who reported it to us on HackerOne
>> >> has
>> >> indicated they expect to publish their findings soon and include it in
>> >> a
>> >> talk at the Black Hat conference and they will likely use CKAN as an
>> >> example.
>> >>
>> >> If you can confirm my understanding of this problem is correct and that
>> >> it
>> >> hasn't already been addressed or reported to the upstream libraries
>> >> (routes/flask), we'd also appreciate assistance in bringing the
>> >> disclosure
>> >> to the attention of those developers.
>> >>
>> >> If you need more information or a proof of concept beyond the
>> >> documentation reported with the issue on rails, we'd be happy to
>> >> provide
>> >> that.
>> >>
>> >> Best,
>> >> Phil
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Title: IT Specialist
>> > Program: Data.gov
>> > Office: Technology Transformation Service (TTS)
>> > Agency: General Services Administration
>> > Github: JJediny
>> >
>> > _______________________________________________
>> > CKAN security
>> > https://lists.okfn.org/mailman/listinfo/security
>> >
>> > https://lists.okfn.org/mailman/options/security/adria.mercader%40okfn.org
>> >
>> > Repo: https://github.com/ckan/ckan-security
>>
>