[ckan-dev] solr

Tue May 24 15:13:44 UTC 2011

On 24 May 2011 13:52, David Read <david.read at okfn.org> wrote:
[...]
> > I don't see it as a feature but as refactoring to reduce technical debt.
>
> I'll try and summarize what I see as the pluses & minuses of dropping postgres:

I think dropping postgres is a really major step. I'd personally argue
in the other direction: doing a bit of work to finish off the
abstraction of the search interface that we started. At the moment the
only thing we are really using that isn't in postgres is facets and
that shouldn't be that hard to do. Requiring SOLR by default is a big
ask -- it's a significant extra bit of kit to maintain.

Rufus

> benefits:
> * we don't have to explain any more that faceting is only provided if
> you use SOLR
> * future search features don't have to be implemented in both SOLR and postgres.
> * we could refactor to remove the code to allow different search
> backends (but maybe this is useful to keep as it is) which could be
> seen as reducing technical debt
>
> drawbacks:
> * we have to write tests for solr search (we should do this anyway)
> * all our team have to learn solr admin & schema writing
> * there are some question marks in my mind about solr's schema
> flexibility to cope with extras. I'd like to see the schema to be
> automatically derived from the CKAN package schema, yet isn't it right
> that the SOLR schema is designed to live in /etc/ and changes mean
> restarting SOLR? I'm sure we could get round this, but starts to feel
> inflexible. Especially for writing tests. Friedrich can you enlighten
> me on this?
>
> David
>
> > As I say, I would be happy with officially deprecating non-solr
> > installs rather than migrating or similar.  But we owe our new users
> > clarity over what is recommended, I think.
> >
> > Seb
> >
> >
> >>>> We're in a funny half-way place right now as the Solr stuff needs more
> >>>> hand-holding and documentation to allow people to easily run with it,
> >>>> but with postgres you don't get our full functionality.
> >>>
> >>> Thats very true and my opinion on it would be to drop Postgres FTS
> >>> support. Solr search is excellent and since search is a core feature
> >>> of CKAN (until we've all figured out what "data collaboration" really
> >>> means its not inaccurate to say it is *the* core) I think the
> >>> additional overhead of deploying a Java-based service might be
> >>> justified. Removing Psql-FTS would drive down the complexity of the
> >>> search code significantly.
> >>>
> >>> This certainly needs some more discussion but I'd like this thread to
> >>> end in voting or a BDFL judgement.
> >>> - Friedrich
> >>>
> >>> _______________________________________________
> >>> ckan-dev mailing list
> >>> ckan-dev at lists.okfn.org
> >>> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >>>
> >>
> >> _______________________________________________
> >> ckan-dev mailing list
> >> ckan-dev at lists.okfn.org
> >> http://lists.okfn.org/mailman/listinfo/ckan-dev
> >>
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
> >
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev

--
Co-Founder, Open Knowledge Foundation
Promoting Open Knowledge in a Digital Age
http://www.okfn.org/ - http://blog.okfn.org/