[wdmmg-dev] wdmmg dev setup

[Jackson Pauls]

Hi Carsten,

Thank you very much for your help. Sorry, I should have mentioned that
156 tests ran, and 153 passed.

I did manage to get solr and test.ini set up as you suggest (was
getting many more test errors before figuring that out), but still I
get the two errors I mentioned below.

I think that even with the contents of cofog_cra.tar.gz extracted into
getdata I'm still missing this file:
pylons_data/getdata/ukgov-finances-cra/cofog_map.json
as "paster load cra2010" still fails in the same way?

Cheers, Jackson


On 9 June 2011 22:50, Carsten Senger <senger at rehfisch.de> wrote:
> Hi Jackson,
>
> --On Donnerstag, Juni 09, 2011 14:46:57 +0100 Jackson Pauls
> <jacksonpauls at gmail.com> wrote:
>
>> Hi,
>>
>> I'm trying to set up wdmmg on my development machine to allow me to
>> contribute some code/effort to the project. I'm following the
>> instructions here:
>> https://bitbucket.org/okfn/wdmmg/src/5d60b4d8f5d1/doc/install.rst
>>
>> It all seems ok until line 135. Running the nostests, I get 2 errors
>> and 1 failure. The failure seems related to this:
>> http://trac.openspending.org/ticket/170. The errors are as follow:
>
> The failing test is intentional and you can ignore it.
> It's a specific missing feature that does not affect your installation.
>
>
> ======================================================================
>>
>> ERROR: test suite for test_search.TestSearch
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> [...]
>>  File "/home/j/src/wdmmgenv/lib/python2.7/site-packages/solr/core.py",
>> line 639, in _post
>>    return check_response_status(self.conn.getresponse())
>>  File "/home/j/src/wdmmgenv/lib/python2.7/site-packages/solr/core.py",
>> line 1096, in check_response_status
>>    raise ex
>> SolrException: HTTP code=404, reason=Not Found
>
> I think our install instructions are not complete here.
> For complete tests, you need to copy the test.example.ini to test.ini
> and configure mongodb and solr in test.ini.
> The solr to run the tests should be different from the solr you use
> during development. It used to be possible to run the test
> with "sorl.url = stub" in test.ini, but that does not work anymore.
>
> A second solr could be a second core in a multi core solr
> installation or a dedicated solr server (I'm not that familar with solr,
> so the terminology maybe isn't completely correct).
> For my development setup, I downloaded a solr tarball. It contains a
> directory "example" from which you can start a solr with
> `java -jar start.jar`. I just linked  solr/wdmmg_schema.xml from the
> wdmmg repository as example/solr/conf/schema.xml and start solr from
> the command line when I want to run the tests and use
> http://localhost:8983/solr as the solr url.
>
>
>> ======================================================================
>> ERROR: Checks that the CRA datapkg is available and has all the files
>> we expect it
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> [...]
>>  File "/home/j/src/wdmmgenv/src/datapkg/datapkg/package.py", line 161,
>> in load     dist = datapkg.distribution.load(path)
>>  File
>> "/home/j/src/wdmmgenv/src/datapkg/datapkg/distribution/__init__.py", line
>> 44, in load
>>    raise DatapkgException(msg)
>> DatapkgException: Failed to load distribution from
>> /home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra
>> ['File/Directory at
>> /home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra does
>> not look like a Distribution', "[Errno 2] No such file or directory:
>> '/home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra/packag
>> e.json'"]
>>
>> ======================================================================
>>
>>
>> If I skip past that, I have a file that fails to download if I run the
>> command at line 159 (all the others download ok):
>> (wdmmgenv)[j at lianli wdmmg]$ datapkg download ckan://ukgov-finances-cra
>> ./pylons_data/getdata
>> [...]
>> Downloading package resource: http://bitbucket.org/okfn/ukgov-finances-cra
>> Total size: -1b
>> [...]
>
> I had a similar problem when I installed wdmmg first. I got the files
> manually from a working wdmmg installation I think.
>
> Maybe somebody familar with ckan knows the reason. I've temporarily upload
> a tarball to <http://rehfisch.de/okfn/cofog_cra.tar.gz> which should
> contain the necessary files to load cofog, cra and cra2010. Unpack it to
> .../getdata.
>
>> And if I ignore that I get the following related error when trying to
>> load the data into the database:
>> (wdmmgenv)[j at lianli wdmmg]$ paster load cra2010
>> [...]
>> IOError: [Errno 2] No such file or directory:
>> '/home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra/cofog_
>> map.json'
>
> This should go away then.
>
>
>> Am in relatively unfamiliar territory with this software stack, would
>> be very grateful for any pointers to what changes are needed, or where
>> to look for answers.
>
> If you run the tests check that it runs all wdmmg tests. It should be
> ~150-160.
>
> ..Carsten
>
>