[ckan-dev] Hello

Wed Dec 8 14:31:22 UTC 2010

Hi all, 

great to have you, Seb! 

On Dec 8, 2010, at 2:15 PM, David Read wrote:
>>> The reason CKAN specifies postgres, as opposed to being db neutral, is
>>> because of use of the pg's full text searching. There is talk of
>>> ditching pg search in favour of SOLR. But requiring SOLR to be
>>> installed seems a big hurdle for anyone wanting to install CKAN
>>> (indeed I think it could be a faff for every developer running CKAN
>>> locally too). I don't think we've gone down this path yet. Maybe it is
>>> worth having an option to use SQLite with basic search (as opposed to
>>> full text) and skip tests that use full text search.
>> 
>> I definitely think a basic versus complex search is very useful. I'd
>> earlier proposed we could get most of this via abstracting the
>> SearchQuery interface. I also propose we converge on a search
>> parameters that are modelled on solr.

I fully agree, it would be fairly easy to create a version of the SQL query interface that does not include postgres-specific FTS and still wouldn't be much more basic in day to day use. 

>>> I agree the tests take too long. Last time I took a look at it (June)
>>> I got the time down from 475s to 275s - I'll forward you the info. FYI
>>> tests take 793s (13 minutes) on my machine now. If you're getting 40
>>> minutes then maybe you are hitting the ubuntu version bug James hits.
>>> Or maybe the profiling you have setup is taking the extra time?
>>> 
>>> I think there are still areas where we use setup when setup_class
>>> would be possible and faster. Also I wonder if we can reduce the
>>> number of times new ckan instances are started ( _start_ckan_server )
>>> as these are very slow.
>> 
>> Those _start_ckan_server come from the Changesets
>> (model/changesets.py) tests I believe. Changesets are currently not
>> being used in CKAN system at all and I would propose these moved out
>> of core (if we want them back in later we can reintegrate).
> 
> CO have requested with 'medium-high' priority to use changesets (as I
> mentioned two weeks ago) so I suggest we keep testing it and look at
> making them faster.
> https://trac.dataco.coi.gov.uk/projects/datagov/ticket/675
> CKAN instances are also created in the harvester tests I believe.

I'd love to learn more about this, but my access to this site seems to have been revoked. Could someone possible forward the document? 

Thanks, 

 Friedrich 