[ckan-dev] Solr out of memory

[Exversion]

First thing I would check is how your heap size compares to your actual memory size. The JVM is greedy. If you set your heap size to 5GB on a 4GB machine it will routinely run itself into the ground 

Sent from my iPhone

> On Dec 6, 2015, at 4:51 AM, Carl Lange <carl at derilinx.com> wrote:
> 
> Hi all,
> 
> Recently we moved from an m3.large aws instance to a 4GB linode, having discovered that we didn't require nearly as much power as we thought. However, since then, our Solr core has been causing some major issues. To begin with, it runs out of memory very quickly (with a JavaOutOfMemory exception). Other times, I simply get a 500 from solr, with no other info. I have only about 1200 datasets - surely I don't need more than 4GB of RAM to search this?
> 
> Could anyone point me in the direction of the best way to debug these issues? I find myself restarting jetty every ten minutes in order to get my search back, which is a little unsustainable ;)
> 
> The problem seems to have manifested itself after google decided to crawl all our search pages yesterday, 1 per second. I've brought this rate down to 0.3 a second, which has helped a little. Prior to this, we had reasonably stable search for about a month.
> 
> I notice that there are quite a large number of indexed fields in schema.xml - are these all necessary? Same goes for stored fields. (I'm using a slightly modified version of the data.gov.uk schema).
> 
> Really though, I'm just shooting in the dark - I don't know if it's my schema, or if it's anything else, and so some info on how to debug this would be great.
> 
> Cheers,
> Carl
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20151206/5208b236/attachment-0003.html>