[ckan-dev] DataStorer extension location question

Wed Jun 13 23:07:04 UTC 2012

Here is the results of nosetests tests within the ckanext-datastorer
folder (failed):

(pyenv)backadmin at ckansrv:~/pyenv/src/ckanext-datastorer$ nosetests tests
E
======================================================================
ERROR: Failure: KeyError ('__file__')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/backadmin/pyenv/lib/python2.6/site-packages/nose/loader.py",
line 379, in loadTestsFromName
    module = resolve_name(addr.module)
  File "/home/backadmin/pyenv/lib/python2.6/site-packages/nose/util.py",
line 321, in resolve_name
    module = __import__('.'.join(parts_copy))
  File "/home/backadmin/pyenv/src/ckan/ckan/tests/__init__.py", line
50, in <module>
    SetupCommand('setup-app').run([config['__file__']])
  File "/home/backadmin/pyenv/lib/python2.6/site-packages/paste/registry.py",
line 146, in __getitem__
    return self._current_obj()[key]
KeyError: "'__file__'\n-------------------- >> begin captured logging
<< --------------------\npca: DEBUG: Creating PluginEnvironment
'pca'\n<default>: DEBUG: Creating PluginEnvironment
'<default>'\n<default>: DEBUG: Pushing environment '<default>' on the
PluginGlobals stack\npca: DEBUG: Pushing environment 'pca' on the
PluginGlobals stack\npca: DEBUG: Pushing environment 'pca' on the
PluginGlobals stack\npca: INFO: Adding service IgnorePluginPlugins to
environment pca\npca: INFO: Adding service IgnorePluginPlugins to
environment pca\npca: DEBUG: Popping environment 'pca' from the
PluginGlobals stack\nvdm: INFO: Skipping adding property
Package.package_tag_all to revisioned object\nvdm: INFO: Skipping
adding property Package.all_revisions_unordered to revisioned
object\nvdm: INFO: Skipping adding property
PackageTag.all_revisions_unordered to revisioned object\nvdm: INFO:
Skipping adding property Group.all_revisions_unordered to revisioned
object\nvdm: INFO: Skipping adding property
Member.all_revisions_unordered to revisioned object\nvdm: INFO:
Skipping adding property GroupExtra.all_revisions_unordered to
revisioned object\nvdm: INFO: Skipping adding property
PackageExtra.all_revisions_unordered to revisioned object\nvdm: INFO:
Skipping adding property Resource.all_revisions_unordered to
revisioned object\nvdm: INFO: Skipping adding property
ResourceGroup.resources_all to revisioned object\nvdm: INFO: Skipping
adding property ResourceGroup.all_revisions_unordered to revisioned
object\nvdm: INFO: Skipping adding property
PackageRelationship.all_revisions_unordered to revisioned object\npca:
INFO: Adding service SynchronousSearchPlugin to environment pca\npca:
INFO: Adding service SynchronousSearchPlugin to environment
pca\n--------------------- >> end captured logging <<
---------------------"

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

Any suggestions most welcome!

Thanks,
Jim

On Wed, Jun 13, 2012 at 4:02 PM, Jim Craner <jim at codeforamerica.org> wrote:
> Hi Adrià,
>
> Thanks for the suggestions -- every command ran successfully until I
> restarted Apache.  I'm still getting a 500 error when I try to access
> CKAN, and the "PluginNotFoundException: datastorer" in the error logs.
>
> Would the rest of the Traceback help you troubleshoot?  Any other suggestions?
>
> Thanks again,
> Jim
>
> [Wed Jun 13 00:20:54 2012] [error] [client 127.0.0.1]
> PluginNotFoundException: datastorer
>
>
> On Wed, Jun 13, 2012 at 8:53 AM, Adrià Mercader <amercadero at gmail.com> wrote:
>> Hi Jim,
>>
>> Sorry to hear that you are having trouble with the datastorer. Trust
>> me, it definitely works on different projects :)
>>
>> I've improved the install instructions on the README [1], which I
>> agree could be confusing, so I suggest we start all over again.
>>
>> 1. Remove any datastorer folder that you have right now
>> 2. Activate your virtualenv
>> 3. Run pip install -e
>> git+git://github.com/okfn/ckanext-datastorer.git#egg=ckanext-datastorer
>> 4. Run pip install -r ckanext-datastorer/requires.txt
>>
>> At this point you should have a ~/pyenv/src/ckanext-datastorer folder
>> and running this command should not return any errors or prompts:
>>
>> python -c "import ckanext.datastorer"
>>
>> After this point you need to add this to your ini file:
>>
>> ckan.plugins = datastorer <rest of plugins>...
>>
>>
>> Let me know if this works for you.
>>
>> Adrià
>>
>>
>> [1] https://github.com/okfn/ckanext-datastorer/blob/master/README.rst
>>
>>
>> On 13 June 2012 16:30, Jim Craner <jim at codeforamerica.org> wrote:
>>> Hi Sean,
>>>
>>> I really appreciate the suggestions, but I think at this point I need
>>> explicit "You should do X, Y, and Z" instructions.
>>>
>>> I'm pretty sure I've tried a hundred variations on everything you
>>> suggested over the past couple of weeks, I *have* been referring to
>>> that "how to install an extension" doc, I've tried every combination
>>> of file structure for the extensions folder and I'm still getting
>>> nothing.
>>>
>>> Since this (allegedly :-) works on the Data Hub, could someone with
>>> access to that server just go in and tell me how the "datastorer"
>>> extension is set up in the CKAN source folder structure?  And what the
>>> plugins line is?  (I've seen "datastore archiver" and "datastorer
>>> archive" but never know which was correct).
>>>
>>> My offer to help improve this part of the documentation -- assuming I
>>> can ever get this figured out -- still stands as well :-)
>>>
>>> Thanks again!
>>> Jim
>>>
>>>
>>>
>>> On Wed, Jun 13, 2012 at 5:01 AM, Sean Hammond <sean.hammond at okfn.org> wrote:
>>>>> I'm (still) trying to get the DataStorer functionality working.  I
>>>>> have a fresh install of CKAN 1.7 from source running on a local Ubuntu
>>>>> 10.04 LTS virtual environment.
>>>>>
>>>>> I got CKAN running, the FileStore running, everything is looking good.
>>>>>
>>>>> I really need some help clarifying the instructions found here:
>>>>>
>>>>> https://github.com/okfn/ckanext-datastorer
>>>>>
>>>>> These instructions *seem* to work, as in no errors are generated
>>>>> during the installation, but I'm really not sure where the files are
>>>>> supposed to end up.  When I try to access CKAN after installing this,
>>>>> I get a 500 error and the logs report "PluginNotFoundException:
>>>>> datastore."
>>>>>
>>>>> My CKAN folder is in my server's user's home folder, ~/pyenv/src/ckan.
>>>>>  The config file is ~/pyenv/src/ckan/datasc.ini.  The installation
>>>>> instructions instruct me to download a zip file which becomes an
>>>>> okfn-ckanext-datastorer folder (which I've tried putting in ~ and in
>>>>> ~/pyenv/src/ckan).  When I run the installation, it generates a
>>>>> subfolder "ckanext" which has its own folder within it called
>>>>> "datastorer".  I tried putting that final "datastorer" folder directly
>>>>> in my existing ~/pyenv/src/ckan/ckanext folder and got the same
>>>>> results.  So at that point I'm completely lost.
>>>>>
>>>>> Can someone who has DataStorer running successfully tell me a) what to
>>>>> put in my CONFIG.ini file's plugins line (e.g., "datastore archiver"
>>>>> or "datastorer archiver" or something else)?  And b) what should the
>>>>> correct file structure be for the datastorer extension folder(s)
>>>>> relative to the root of my CKAN installation?
>>>>
>>>> Hi Jim,
>>>>
>>>> I haven't installed datastorer before myself, but I would try reading
>>>> the 'Installing an Extension' docs here:
>>>>
>>>> http://docs.ckan.org/en/ckan-1.7/extensions.html#installing-an-extension
>>>>
>>>> and then you might understand the ckanext-datastorer README a little
>>>> better. For example, I would probably download ckanext-datastorer by
>>>> activating my virtualenv and running a command like:
>>>>
>>>>    pip install -e git+https://github.com/okfn/ckanext-datastorer.git#egg=ckanext-datastorer
>>>>
>>>> this should download and install ckanext-datastorer in a directory like:
>>>>
>>>>    pyenv/src/ckanext-datastorer
>>>>
>>>> You might still need to run pip install -r on the requires.txt file in
>>>> the ckanext-datastorer directory.
>>>>
>>>> In the plugins line of your CKAN ini file you should add something like:
>>>>
>>>>    plugins = datastorer
>>>>
>>>> A couple of things you can do to test whether datastorer is working
>>>> properly are to run CKAN with `paster serve development.ini` and see if
>>>> it crashes with something like 'Plugin not found: datastorer', run the
>>>> datastorer tests, something like:
>>>>
>>>>    nosetests tests
>>>>
>>>> from the ckanext-datastorer directory, and to try out the
>>>> `paster datastorer` command (from the pyenv/src/ckan directory).
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>
>>>
>>>
>>> --
>>> -----
>>> Jim Craner, 2012 Fellow
>>> Code for America
>>> jim at codeforamerica.org
>>> (+1) 773-809-4546
>>> http://codeforamerica.org
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>
>
> --
> -----
> Jim Craner, 2012 Fellow
> Code for America
> jim at codeforamerica.org
> (+1) 773-809-4546
> http://codeforamerica.org

-- 
-----
Jim Craner, 2012 Fellow
Code for America
jim at codeforamerica.org
(+1) 773-809-4546
http://codeforamerica.org