[ckan-dev] Hello

[David Read]

Seb,

Welcome!

+1 to ploughing on getting vdm working in sqlalchemy 0.5.

Yes there is a warning to do with sqla querying polymorphism that
crops up in the CKAN tests. I had a look at this quickly before and
couldn't work out how to solve it, so if another pair of eyes on that
would be great.

The reason CKAN specifies postgres, as opposed to being db neutral, is
because of use of the pg's full text searching. There is talk of
ditching pg search in favour of SOLR. But requiring SOLR to be
installed seems a big hurdle for anyone wanting to install CKAN
(indeed I think it could be a faff for every developer running CKAN
locally too). I don't think we've gone down this path yet. Maybe it is
worth having an option to use SQLite with basic search (as opposed to
full text) and skip tests that use full text search.

I agree the tests take too long. Last time I took a look at it (June)
I got the time down from 475s to 275s - I'll forward you the info. FYI
tests take 793s (13 minutes) on my machine now. If you're getting 40
minutes then maybe you are hitting the ubuntu version bug James hits.
Or maybe the profiling you have setup is taking the extra time?

I think there are still areas where we use setup when setup_class
would be possible and faster. Also I wonder if we can reduce the
number of times new ckan instances are started ( _start_ckan_server )
as these are very slow.

Hope this helps,
Dave

On 8 December 2010 11:30, William Waites <ww at eris.okfn.org> wrote:
> (changing the cc to ckan-dev, best to have this type of thing in the
> public archive)
>
> * [2010-12-08 11:12:44 +0000] Seb Bacon <seb.bacon at gmail.com> écrit:
>
> ] Hopefully by now you know I've joined the team.
>
> Hi Seb, welcome to the team! (I didn't know you had joined)
>
> ]  To get it to work, I've needed to upgrade SQLAlchemy to a
> ] recent version, despite the fact that it's pinned at a version <=
> ] 0.4.99.
> ]
> ] However, that appears to break VDM.  I've just started looking into
> ] why, but thought I would drop the list a line to check if anyone has
> ] any information about the reasons for pinning SQLAlchemy to an earlier
> ] version.
>
> I had a look at this a while back when I was writing the SQLAlchemy
> back-end for Virtuoso (http://packages.python.org/virtuoso). One of
> our potential implementation strategies for LOD2 involves using
> Virtuoso instead of PostgreSQL. Since SQLAlchemy changed the structure
> of the back ends between 0.5 and 0.6 I was led down the path that you
> are now treading.
>
> At the time, I think I was able to get all tests for VDM passing with
> versions 0.5 and 0.6. It may be that something (hopefully simple) has
> changed between then and now. I imagine this will be simple to sort
> out.
>
> More difficult, I found, was with CKAN itself. There are some places
> which I think David Read may know more about that cause a warning
> relating to use of polymorphism -- you should see these in the logs
> running CKAN normally. Upgrading to SQLAlchemy 0.5 causes these
> warnings to turn into exceptions. That's where I left it.
>
> I think it would be extremely useful if you were able to get CKAN
> properly running with SQLAlchemy 0.6 -- it would give us quite a lot
> of flexibility with the database back end.
>
> Cheers,
> -w
> --
> William Waites
> http://eris.okfn.org/ww/foaf#i
> 9C7E F636 52F6 1004 E40A  E565 98E3 BBF3 8320 7664
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
>