[ckan-dev] CKAN 2.5.2 test suite errors

Mon Aug 8 23:48:30 UTC 2016

Hi Jim,

If you download
https://github.com/DataShades/opswx-ckan-cookbook/blob/master/files/default/ckan_solr_conf.zip
and extract it into a /conf where your solr core in the solr data directory
is, this schema and solr config works.

The setup we execute which will work in a simple set up as well is;
Download and extract Solr 5.2.2
Run the setup script that comes with Solr.
We move the /var/solr directory to /data/solr so it's picked up by our
daily backup process.
We create a core in /data/solr/data. Typical naming would be dev_ckan.
We create a core.properties file in /data/solr/data/dev_ckan with a single
name=dev_ckan
We extract the archive I linked to above to  /data/solr/data/dev_ckan/conf
and create a /data/solr/data/dev_ckan/data directory.

It's so long since we created this config for Solr 5 that I can't remember
the specific changes we made. From memory it was changing the data types
Solr uses. When Solr changed from 4 to 5, a couple of the data types
changed. That at least is my foggy recollection.

I'm fairly certain the process outlined above should get you up and
running. No doubt adding this core to the Solr install you have should
suffice after you restart Solr and point CKAN at it.

Shane.

------------------------------------------------------------------
*SHANE DAVIS*  |  *SERVICE DELIVERY MANAGER*
Link Digital

On Tue, Aug 9, 2016 at 8:31 AM, Espeland, James <james.espeland at montana.edu>
wrote:

> Hi Shane,
>
>
>
> I tried Solr 5.5.2 and while Solr seems to run just fine, I run into the
> same types of Solr errors (as Solr 6.0.x) when I run the CKAN test suite
> (undefined field, field can’t be found, unknown field).  I also tried Solr
> 4.10.4, but ran into numerous problems configuring and running that version
> of Solr.
>
>
>
> I saw your message regarding the work you’ve done on an enterprise stack
> for AWS, but at this time, we’re mostly interested in setting up a local
> instance of CKAN.
>
>
>
> I remember that you mentioned that you had seen similar errors to the ones
> I am seeing.  Can you recall what you did to correct those errors?  I
> suspect that it will require a configuration change of some sort.
>
>
>
> Thank you again for any assistance you can offer to me!
>
>
>
> Best regards,
>
>
>
> --- Jim Espeland
>
> james.espeland at montana.edu
>
>
>
>
>
> *From:* Shane Davis [mailto:shane.davis at linkdigital.com.au]
> *Sent:* Wednesday, August 3, 2016 3:42 PM
> *To:* CKAN Development Discussions <ckan-dev at lists.okfn.org>
> *Cc:* Espeland, James <james.espeland at montana.edu>
> *Subject:* Re: [ckan-dev] CKAN 2.5.2 test suite errors
>
>
>
> Hi Jim,
>
>
>
> Solr 5.5.2 definitely works. We're using that as a standard right now. If
> I get the opportunity over the next week, I'll try a Solr 6.1 install. When
> we moved from Solr 4 to Solr 5 I remember running into issues not
> dissimilar to the one's you're reporting. We're in final stages of UAT with
> Datashades which is a CKAN 2.5.2 build using Solr 5.5.2. At an absolute
> minimum that will provide an insight into the Solr config we're using.
>
>
>
> I'm extremely keen to get a release up on GitHub even if it's not a
> version one. There's a couple of known issues I need to fix beforehand, but
> a stable release is very close. I'll notify the list when it's actually
> published.
>
>
>
>
> Shane.
>
>
>
> ------------------------------------------------------------------
> *SHANE DAVIS*  |  *SERVICE DELIVERY MANAGER*
> Link Digital
>
> www.linkdigital.com.au
> p: *02 6111 2907 <02%206111%202907>* | f: 02 6248 5582
> m: 0400 611 428
>
> t: @link_sdm
>
> s: link_sdm
> PO Box 199 Canberra ACT 2601
> 32 Lonsdale Street Braddon ACT 2612
>
> [image: Image removed by sender.] [image: Image removed by sender.] [image:
> Image removed by sender.]
>
>
>
> [image: Image removed by sender.] [image: Image removed by sender.] [image:
> Image removed by sender.]
>
>
>
>
>
> On Thu, Aug 4, 2016 at 4:51 AM, Espeland, James <
> james.espeland at montana.edu> wrote:
>
> Hello Adri?,
>
> Thank you very much for your reply to my request for assistance.
>
> Yes, it does appear that my Solr instance is not picking up the schema
> file.
>
> I'm not committed to using Solr 6.x, but I would like to use the most
> recent version that is known to work with the schema.xml file provided by
> CKAN.  I'm trying to set up CKAN 2.5.2.
>
> Can you recommend to me the latest known version of Solr that will work
> with the schema.xml file provided by CKAN?
>
> Thank you again,
>
> --- James Espeland
>
>
> > Date: Wed, 3 Aug 2016 14:26:09 +0100
> > From: Adri? Mercader <adria.mercader at okfn.org>
> > To: CKAN Development Discussions <ckan-dev at lists.okfn.org>
> > Subject: Re: [ckan-dev] CKAN 2.5.2 test suite errors
> > Message-ID:
> >       <CAGJR8iJje3jFBG-
> > bcRJBJgLqkZxRSwGP1ieueGUw18f3L_4XYg at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > Hi James,
> >
> > These errors seem to suggest that Solr is not using the right schema
> file.
> >
> > I'm not familiar with Solr 6 but I seem to recall from other messages on
> the list
> > that it required a special setup to use file based schemas.
> >
> > Maybe someone with more experience can provide more details.
> >
> > Adri?
> >
> >
> >
> > On 21 July 2016 at 20:34, Espeland, James <james.espeland at montana.edu>
> > wrote:
> > > Hello Everyone,
> > >
> > >
> > >
> > > I have been setting up a pilot instance of CKAN 2.5.2 following the
> > > CKAN documentation for Release 2.5.2.  I can run the Paste development
> > > server from the command line and see the CKAN front page.  The
> > > documentation then says to ?run CKAN?s tests to make sure that
> everything?s
> > working?.
> > >
> > >
> > >
> > > I'm still getting a number of errors when I run the tests.
> > >
> > >
> > >
> > > My pilot environment includes:
> > >
> > >
> > >
> > > centos 6.7
> > >
> > > ckan 2.5.2
> > >
> > > solr 6.0.0
> > >
> > > java 1.8.0
> > >
> > > python 2.7.11
> > >
> > > postgres 9.4.8
> > >
> > > jetty 9.3.8
> > >
> > >
> > >
> > > I have created a solr core named ckan1.  I set the solr_url entry in
> > > test-core.ini to:
> > >
> > >
> > >
> > > http://127.0.0.1:8393/solr/ckan1
> > >
> > >
> > >
> > > I soft-linked the ckan src schema.xml into my core config directory as
> > > follows:
> > >
> > >
> > >
> > > ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml
> > > /var/solr/data/ckan1/conf/schema.xml
> > >
> > >
> > >
> > > When I run the tests, I get the following types of errors in the
> > > solr.log
> > > file:
> > >
> > >
> > >
> > > **********
> > >
> > >
> > >
> > > 2016-07-21 19:05:02.647 ERROR (qtp1634198-20) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
> > > undefined field site_id
> > >
> > >
> > >
> > > 2016-07-21 19:05:09.544 ERROR (qtp1634198-15) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: s
> > >
> > > ort param field can't be found: metadata_modified
> > >
> > >
> > >
> > > 2016-07-21 19:05:21.759 ERROR (qtp1634198-13) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: E
> > >
> > > RROR: [doc=812b3b0a-30e4-4aa6-bcd6-9e1fe2072d22] unknown field
> > 'data_dict'
> > >
> > >
> > >
> > > 2016-07-21 19:05:40.053 ERROR (qtp1634198-13) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: E
> > >
> > > RROR: [doc=9315d7ca-7c3e-4e59-92f6-8d494e1d374d] unknown field
> > > 'res_description'
> > >
> > >
> > >
> > > 2016-07-21 19:05:56.431 ERROR (qtp1634198-15) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: u
> > >
> > > ndefined field capacity
> > >
> > >
> > >
> > > 016-07-21 19:06:56.184 ERROR (qtp1634198-15) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: E
> > >
> > > RROR: [doc=abed426b-cfa4-45b1-8403-cbf23c3fa899] unknown field
> > 'maintainer'
> > >
> > >
> > >
> > > 2016-07-21 19:07:38.205 ERROR (qtp1634198-15) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: E
> > >
> > > RROR: [doc=0cdcd6d7-8cf6-4767-bb3a-f9ed82141a6f] unknown field
> > 'owner_org'
> > >
> > >
> > >
> > > 2016-07-21 19:08:51.554 ERROR (qtp1634198-17) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: E
> > >
> > > RROR: [doc=d9567b82-d3f0-4c17-b222-d9a7499f7940] unknown field
> > 'capacity'
> > >
> > >
> > >
> > > 2016-07-21 19:08:51.638 ERROR (qtp1634198-20) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: E
> > >
> > > RROR: [doc=test-index] unknown field 'views_recent'
> > >
> > >
> > >
> > > 2016-07-21 19:08:51.727 ERROR (qtp1634198-16) [   x:ckan1]
> > > o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
> > > undefined field entity_type
> > >
> > >
> > >
> > > **********
> > >
> > >
> > >
> > > I suspect that something isn?t configured correctly.
> > >
> > >
> > >
> > > Any help or advice regarding how to correct this would be greatly
> > > appreciated.
> > >
> > >
> > >
> > > Best regards,
> > >
> > >
> > >
> > > --- Jim Espeland
> > >
> > > james.espeland at montana.edu
> > >
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20160809/3f124f73/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 524 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20160809/3f124f73/attachment-0003.jpg>