[ckan-dev] API abuse

[Peder Jakobsen | gmail]

This is not an easy problem to deal with and most solutions will have
drawbacks.  For planned Denial of Service attacks, it possible to forge the
IP in the packets being sent, so banning IP addresses is not a permanent
solution even if it stops one attacker.   If you feel the attacks are
coming from a particular part of the world, you could block an entire IP
range.  :(

How about just insisting on a key for all API calls?  That way you could
limit the number of requests allow per day per key, that's how a lot of web
services are designed AFAIK.

Peder




On Thu, Apr 18, 2013 at 6:13 AM, David Read
<david.read at hackneyworkshop.com>wrote:

> We had an incident yesterday caused by a java web bot making
> simultaneous connections to our CKAN API. Averaging 10 requests per
> second, it caused serious server problems - postgres filling the CPU
> use, Apache spawning lots of processes. Normally big loads are not a
> problem for us because of using a cache in front of CKAN, but because
> the API v3 is not easily cached, it caused the problems.
>
> The user was POSTing requests to package_show, without api key. Nagios
> alerted us to the slowing server and we banned their IP manually
> within a few minutes to take it back to normal. But it has become a
> concern.
>
> Does anyone have any thoughts on how the CKAN community might deal
> with this sort of behaviour better, either in the design of CKAN or
> with server software?
>
> David
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130418/ece763bb/attachment-0001.html>