[ckan-dev] Solr out of memory

[Carl Lange]

I hadn't set -Xmx previously, so I've set it to 2048M. However, now solr
simply dies when it hits that limit, which seems like odd behaviour…

On 6 December 2015 at 12:12, Exversion <marianne at exversion.com> wrote:

> First thing I would check is how your heap size compares to your actual
> memory size. The JVM is greedy. If you set your heap size to 5GB on a 4GB
> machine it will routinely run itself into the ground
>
> Sent from my iPhone
>
> On Dec 6, 2015, at 4:51 AM, Carl Lange <carl at derilinx.com> wrote:
>
> Hi all,
>
> Recently we moved from an m3.large aws instance to a 4GB linode, having
> discovered that we didn't require nearly as much power as we thought.
> However, since then, our Solr core has been causing some major issues. To
> begin with, it runs out of memory very quickly (with a JavaOutOfMemory
> exception). Other times, I simply get a 500 from solr, with no other info.
> I have only about 1200 datasets - surely I don't need more than 4GB of RAM
> to search this?
>
> Could anyone point me in the direction of the best way to debug these
> issues? I find myself restarting jetty every ten minutes in order to get my
> search back, which is a little unsustainable ;)
>
> The problem seems to have manifested itself after google decided to crawl
> all our search pages yesterday, 1 per second. I've brought this rate down
> to 0.3 a second, which has helped a little. Prior to this, we had
> reasonably stable search for about a month.
>
> I notice that there are quite a large number of indexed fields in
> schema.xml - are these all necessary? Same goes for stored fields. (I'm
> using a slightly modified version of the data.gov.uk schema).
>
> Really though, I'm just shooting in the dark - I don't know if it's my
> schema, or if it's anything else, and so some info on how to debug this
> would be great.
>
> Cheers,
> Carl
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20151206/c57a209d/attachment-0003.html>