[ckan-discuss] Search

Tue Aug 17 10:21:15 BST 2010

Friedrich,

Many thanks for this - top tip! I'll update the deployment doc with
this info too.

David

On 17 August 2010 00:13, Friedrich Lindenberg <friedrich at pudo.org> wrote:
> Hi David,
>
> the same thing has been haunting me with regards to another Pylons app, but I finally figured it out: pastedeploy won't by default read the config file to configure the logger when run from mod_wsgi. In order for this to happen, we'll have to amend the launcher script for CKAN:
>
> [...]
> from paste.deploy import loadapp
> config_filepath = os.path.join(instance_dir, config_file)
> from paste.script.util.logging_config import fileConfig
> fileConfig(config_filepath)
> application = loadapp('config:%s' % config_filepath)
>
> That will allow us to configure a logger like this:
>
> # Logging configuration
> [loggers]
> keys = root, ckan
>
> [handlers]
> keys = file
>
> [formatters]
> keys = generic
>
> [logger_root]
> level = INFO
> handlers = file
>
> [logger_ckan]
> level = DEBUG
> handlers = file
> qualname = ckan
>
> [handler_file]
> class = handlers.RotatingFileHandler
> formatter = generic
> level = NOTSET
> args = ('/var/log/ckan/de.log', 'a', 2048, 3)
>
> [formatter_generic]
> format = %(asctime)s %(levelname)-5.5s [%(name)s] %(message)s
>
> I'll take a few minutes tomorrow to adapt the other ckan instances. That should give us a clearer picture, especially with regards to common API errors - and maybe wrt out current performance problems.
>
> Friedrich
>
> On Aug 16, 2010, at 10:46 PM, David Read wrote:
>
>> Friedrich,
>>
>> Many thanks for looking into this.
>>
>> I noticed you did some stuff related to loggers. Did you get logging
>> working in CKAN? Rufus and I tried a while ago with the ini settings
>> and never managed to get it to produce a log file.
>>
>> If so, I'd like to log in CKAN when these notifications are sent, to
>> see if this is a problem on the CKAN side. I can reproduce creating
>> these packages, and hopefully the problem.
>>
>> David
>>
>> On 16 August 2010 13:57, Friedrich Lindenberg <friedrich at pudo.org> wrote:
>>> Hi David,
>>>
>>> thanks for the notice - this is obviously a problem with the Solr index [1]. I have no theory on what caused these packages not to be indexed other than the really bad performance on eu3 this morning. The indexer's log [2] is clean, and CKAN only has the postgres-related crashes.
>>>
>>> If you notice any pattern in this, please let me know. I'll try to write some consistency tests for the indexer that can be run against a live instance to check for the validity of its results.
>>>
>>> Friedrich
>>>
>>> [1] http://solr.okfn.org/solr/ckan/select/?q=tags:hesa&version=2.2&start=0&rows=10&indent=on&fq=site_id:test.ckan.net
>>> [2] /home/okfn/var/ckanext/ckanext/solrindexer/stdout.test.log on eu4
>>>
>>> On Aug 16, 2010, at 1:31 PM, David Read wrote:
>>>
>>>> Hi Friedrich,
>>>>
>>>> I'm getting confused with the new search running on test.ckan.net...
>>>> This morning I've loaded in 5 packages via the API which all have the
>>>> hesa tag. When I search for hesa it only comes up with one of those
>>>> packages. It's like only one of the five is indexed. I see from the
>>>> config that it's using SOLR search. Is it me or is there something not
>>>> quite right here?
>>>>
>>>> Dave
>>>
>>>
>
>