[okfn-help] Adding a new Index to the existing OKFN solr server for bibliographica - queries.

Tue Apr 26 11:13:00 BST 2011

Hi,

I understand that there is already a Solr instance somewhere with some
capacity, and from various conversations I believe it has indexes for
CKAN and some 5 or 6 other services.

Rufus (cc'd) has asked me to add an index to this instance to support
bibliographica, to hold its index of records - I have a few questions
about this instance in this case!

Index to add:

 - 25 fields, some copies for faceting and combined search
 - approx 3 million 'docs', with 200+ chars per doc in total
 - needs to be accessible and updatable from the bibliographica.org
pylons frontend IP address.

Questions: (VM -> *Java* VM from this point on)

 - Is there capacity (RAM/Heap/etc) for this sort of index on the
existing solr VM?
 - Single VM (multicore), multiple VM (tomcat-adminned), or combination?
 - If multicore:
	- How is the instance set up for multicore? using SolrCore admin
servlet (ie HTTP API for admin)? or is it using a file directory layout
system?
 - Which public facing services are supported by this instance and more
importantly, who would be the people to let know that a large index will
be added to this?
	 - ie who should be awake and aware while the index is being made, in
case it hits Heap/Stack/other OOM errors and an index corruption happens
elsewhere in the solr instance VM?
 - which server is running this solr instance and what standard backup
and maintenance routines are in place for it? eg critical files, /etc
and indexes rsync'd to remote machine, etc.

Thanks

Ben