[ckan-dev] API abuse

Thu Apr 18 14:02:22 UTC 2013

Interesting point about DoS attacks. However I'm not convinced open
data sites are attractive targets at all. So I think throttling by IP
should be sufficient in today's world. If that changes then you're
right that requiring an API key might be necessary.

Dave

On 18 April 2013 14:20, Peder Jakobsen | gmail <pjakobsen at gmail.com> wrote:
> This is not an easy problem to deal with and most solutions will have
> drawbacks.  For planned Denial of Service attacks, it possible to forge the
> IP in the packets being sent, so banning IP addresses is not a permanent
> solution even if it stops one attacker.   If you feel the attacks are coming
> from a particular part of the world, you could block an entire IP range.  :(
>
> How about just insisting on a key for all API calls?  That way you could
> limit the number of requests allow per day per key, that's how a lot of web
> services are designed AFAIK.
>
> Peder
>
>
>
>
> On Thu, Apr 18, 2013 at 6:13 AM, David Read <david.read at hackneyworkshop.com>
> wrote:
>>
>> We had an incident yesterday caused by a java web bot making
>> simultaneous connections to our CKAN API. Averaging 10 requests per
>> second, it caused serious server problems - postgres filling the CPU
>> use, Apache spawning lots of processes. Normally big loads are not a
>> problem for us because of using a cache in front of CKAN, but because
>> the API v3 is not easily cached, it caused the problems.
>>
>> The user was POSTing requests to package_show, without api key. Nagios
>> alerted us to the slowing server and we banned their IP manually
>> within a few minutes to take it back to normal. But it has become a
>> concern.
>>
>> Does anyone have any thoughts on how the CKAN community might deal
>> with this sort of behaviour better, either in the design of CKAN or
>> with server software?
>>
>> David
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>