[ckan-dev] Hello

Wed Dec 8 13:15:35 UTC 2010

On 8 December 2010 12:42, Rufus Pollock <rufus.pollock at okfn.org> wrote:
> On 8 December 2010 12:08, David Read <david.read at okfn.org> wrote:
>> Seb,
>>
>> Welcome!
>>
>> +1 to ploughing on getting vdm working in sqlalchemy 0.5.
>
> vdm works with sqlalchemy 0.5 and 0.6 (and has done for a few months) :)
>
>> Yes there is a warning to do with sqla querying polymorphism that
>> crops up in the CKAN tests. I had a look at this quickly before and
>> couldn't work out how to solve it, so if another pair of eyes on that
>> would be great.
>>
>> The reason CKAN specifies postgres, as opposed to being db neutral, is
>> because of use of the pg's full text searching. There is talk of
>> ditching pg search in favour of SOLR. But requiring SOLR to be
>> installed seems a big hurdle for anyone wanting to install CKAN
>> (indeed I think it could be a faff for every developer running CKAN
>> locally too). I don't think we've gone down this path yet. Maybe it is
>> worth having an option to use SQLite with basic search (as opposed to
>> full text) and skip tests that use full text search.
>
> I definitely think a basic versus complex search is very useful. I'd
> earlier proposed we could get most of this via abstracting the
> SearchQuery interface. I also propose we converge on a search
> parameters that are modelled on solr.
>
>> I agree the tests take too long. Last time I took a look at it (June)
>> I got the time down from 475s to 275s - I'll forward you the info. FYI
>> tests take 793s (13 minutes) on my machine now. If you're getting 40
>> minutes then maybe you are hitting the ubuntu version bug James hits.
>> Or maybe the profiling you have setup is taking the extra time?
>>
>> I think there are still areas where we use setup when setup_class
>> would be possible and faster. Also I wonder if we can reduce the
>> number of times new ckan instances are started ( _start_ckan_server )
>> as these are very slow.
>
> Those _start_ckan_server come from the Changesets
> (model/changesets.py) tests I believe. Changesets are currently not
> being used in CKAN system at all and I would propose these moved out
> of core (if we want them back in later we can reintegrate).

CO have requested with 'medium-high' priority to use changesets (as I
mentioned two weeks ago) so I suggest we keep testing it and look at
making them faster.
https://trac.dataco.coi.gov.uk/projects/datagov/ticket/675
CKAN instances are also created in the harvester tests I believe.

Dave

>
> Rufus
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>