[wdmmg-dev] wdmmg dev setup

Fri Jun 10 11:17:54 UTC 2011

Hi Jackson,

--On Freitag, Juni 10, 2011 11:57:19 +0100 Jackson Pauls 
<jacksonpauls at gmail.com> wrote:

> Hi Carsten,
>
> Thank you very much for your help. Sorry, I should have mentioned that
> 156 tests ran, and 153 passed.
>
> I did manage to get solr and test.ini set up as you suggest (was
> getting many more test errors before figuring that out), but still I
> get the two errors I mentioned below.
>
> I think that even with the contents of cofog_cra.tar.gz extracted into
> getdata I'm still missing this file:
> pylons_data/getdata/ukgov-finances-cra/cofog_map.json
> as "paster load cra2010" still fails in the same way?

Oh, I missed that file and attached it now. I hope it's the last
one missing.

..Carsten

> Cheers, Jackson
>
>
> On 9 June 2011 22:50, Carsten Senger <senger at rehfisch.de> wrote:
>> Hi Jackson,
>>
>> --On Donnerstag, Juni 09, 2011 14:46:57 +0100 Jackson Pauls
>> <jacksonpauls at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to set up wdmmg on my development machine to allow me to
>>> contribute some code/effort to the project. I'm following the
>>> instructions here:
>>> https://bitbucket.org/okfn/wdmmg/src/5d60b4d8f5d1/doc/install.rst
>>>
>>> It all seems ok until line 135. Running the nostests, I get 2 errors
>>> and 1 failure. The failure seems related to this:
>>> http://trac.openspending.org/ticket/170. The errors are as follow:
>>
>> The failing test is intentional and you can ignore it.
>> It's a specific missing feature that does not affect your installation.
>>
>>
>> ======================================================================
>>>
>>> ERROR: test suite for test_search.TestSearch
>>> ----------------------------------------------------------------------
>>> Traceback (most recent call last):
>>> [...]
>>>  File "/home/j/src/wdmmgenv/lib/python2.7/site-packages/solr/core.py",
>>> line 639, in _post
>>>    return check_response_status(self.conn.getresponse())
>>>  File "/home/j/src/wdmmgenv/lib/python2.7/site-packages/solr/core.py",
>>> line 1096, in check_response_status
>>>    raise ex
>>> SolrException: HTTP code=404, reason=Not Found
>>
>> I think our install instructions are not complete here.
>> For complete tests, you need to copy the test.example.ini to test.ini
>> and configure mongodb and solr in test.ini.
>> The solr to run the tests should be different from the solr you use
>> during development. It used to be possible to run the test
>> with "sorl.url = stub" in test.ini, but that does not work anymore.
>>
>> A second solr could be a second core in a multi core solr
>> installation or a dedicated solr server (I'm not that familar with solr,
>> so the terminology maybe isn't completely correct).
>> For my development setup, I downloaded a solr tarball. It contains a
>> directory "example" from which you can start a solr with
>> `java -jar start.jar`. I just linked  solr/wdmmg_schema.xml from the
>> wdmmg repository as example/solr/conf/schema.xml and start solr from
>> the command line when I want to run the tests and use
>> http://localhost:8983/solr as the solr url.
>>
>>
>>> ======================================================================
>>> ERROR: Checks that the CRA datapkg is available and has all the files
>>> we expect it
>>> ----------------------------------------------------------------------
>>> Traceback (most recent call last):
>>> [...]
>>>  File "/home/j/src/wdmmgenv/src/datapkg/datapkg/package.py", line 161,
>>> in load     dist = datapkg.distribution.load(path)
>>>  File
>>> "/home/j/src/wdmmgenv/src/datapkg/datapkg/distribution/__init__.py",
>>> line 44, in load
>>>    raise DatapkgException(msg)
>>> DatapkgException: Failed to load distribution from
>>> /home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra
>>> ['File/Directory at
>>> /home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra does
>>> not look like a Distribution', "[Errno 2] No such file or directory:
>>> '/home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra/pack
>>> ag e.json'"]
>>>
>>> ======================================================================
>>>
>>>
>>> If I skip past that, I have a file that fails to download if I run the
>>> command at line 159 (all the others download ok):
>>> (wdmmgenv)[j at lianli wdmmg]$ datapkg download ckan://ukgov-finances-cra
>>> ./pylons_data/getdata
>>> [...]
>>> Downloading package resource:
>>> http://bitbucket.org/okfn/ukgov-finances-cra Total size: -1b
>>> [...]
>>
>> I had a similar problem when I installed wdmmg first. I got the files
>> manually from a working wdmmg installation I think.
>>
>> Maybe somebody familar with ckan knows the reason. I've temporarily
>> upload a tarball to <http://rehfisch.de/okfn/cofog_cra.tar.gz> which
>> should contain the necessary files to load cofog, cra and cra2010.
>> Unpack it to .../getdata.
>>
>>> And if I ignore that I get the following related error when trying to
>>> load the data into the database:
>>> (wdmmgenv)[j at lianli wdmmg]$ paster load cra2010
>>> [...]
>>> IOError: [Errno 2] No such file or directory:
>>> '/home/j/src/wdmmgenv/wdmmg/pylons_data/getdata/ukgov-finances-cra/cofo
>>> g_ map.json'
>>
>> This should go away then.
>>
>>
>>> Am in relatively unfamiliar territory with this software stack, would
>>> be very grateful for any pointers to what changes are needed, or where
>>> to look for answers.
>>
>> If you run the tests check that it runs all wdmmg tests. It should be
>> ~150-160.
>>
>> ..Carsten
>>
>>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cofog_map.json
Type: application/octet-stream
Size: 8963 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20110610/3e576b35/attachment-0001.obj>