[ckan-dev] Harvestor extension for CKAN

• Previous message: [ckan-dev] enabling datastore & data API breaks recline
• Next message: [ckan-dev] Harvestor extension for CKAN
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi again.

I have been looking in the log files, and I got the following:

2013-01-04 15:45:06,336 INFO  [ckanext.harvest.queue] Received harvest
object id: 8741a64e-1673-4f29-a713-f9a41d8c9732
2013-01-04 15:45:06,348 DEBUG [ckanext.harvest.harvesters.ckanharvester] In
CKANHarvester fetch_stage
2013-01-04 15:45:06,353 DEBUG [ckanext.harvest.harvesters.ckanharvester]
Using config: {u'default_groups': [u'mygroup'], u'user': u'admin',
u'api_version': u'1'}
2013-01-04 15:45:06,577 DEBUG [ckanext.harvest.harvesters.ckanharvester] In
CKANHarvester import_stage
2013-01-04 15:45:06,579 DEBUG [ckanext.harvest.harvesters.ckanharvester]
Using config: {u'default_groups': [u'mygroup'], u'user': u'admin',
u'api_version': u'1'}
2013-01-04 15:45:06,600 INFO  [ckanext.harvest.harvesters.base] Package
with GUID test does not exist, let's create it
2013-01-04 15:45:06,852 ERROR [ckan.model.modification] 'thread._local'
object has no attribute 'host'
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/ckan/model/modification.py", line 68,
in notify
    observer.notify(entity, operation)
  File
"/var/lib/ckan/std/pyenv/src/ckanext-datastorer/ckanext/datastorer/plugin.py",
line 30, in notify
    self._create_datastorer_task(entity)
  File
"/var/lib/ckan/std/pyenv/src/ckanext-datastorer/ckanext/datastorer/plugin.py",
line 42, in _create_datastorer_task
    'site_url': h.url_for_static('/', qualified=True),
  File "/usr/lib/pymodules/python2.6/ckan/lib/helpers.py", line 87, in
url_for_static
    my_url = _routes_default_url_for(*args, **kw)
  File "/usr/lib/pymodules/python2.6/ckan/routes/util.py", line 257, in
url_for
    host = config.host
  File "/usr/lib/pymodules/python2.6/ckan/routes/__init__.py", line 14, in
__getattr__
    return getattr(self.__shared_state, name)
AttributeError: 'thread._local' object has no attribute 'host'
2013-01-04 15:45:06,854 ERROR [ckanext.harvest.harvesters.base]
'thread._local' object has no attribute 'host'
Traceback (most recent call last):
  File "/home/ckan1/ckanext-harvest/ckanext/harvest/harvesters/base.py",
line 165, in _create_or_update_package
    new_package = get_action('package_create_rest')(context, package_dict)
  File "/usr/lib/pymodules/python2.6/ckan/logic/action/create.py", line
723, in package_create_rest
    dictized_after = _get_action('package_create')(context,
dictized_package)
  File "/usr/lib/pymodules/python2.6/ckan/logic/action/create.py", line
159, in package_create
    model.repo.commit()
  File "/usr/lib/pymodules/python2.6/vdm/sqlalchemy/tools.py", line 102, in
commit
    self.session.commit()
  File "/usr/lib/pymodules/python2.6/ckan/sqlalchemy/orm/scoping.py", line
114, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/ckan/sqlalchemy/orm/session.py", line
656, in commit
    self.transaction.commit()
  File "/usr/lib/pymodules/python2.6/ckan/sqlalchemy/orm/session.py", line
314, in commit
    self._prepare_impl()
  File "/usr/lib/pymodules/python2.6/ckan/sqlalchemy/orm/session.py", line
290, in _prepare_impl
    self.session.dispatch.before_commit(self.session)
  File "/usr/lib/pymodules/python2.6/ckan/sqlalchemy/event.py", line 291,
in __call__
    fn(*args, **kw)
  File "/usr/lib/pymodules/python2.6/ckan/model/extension.py", line 103, in
before_commit
    methodcaller('before_commit', session)
  File "/usr/lib/pymodules/python2.6/ckan/model/extension.py", line 38, in
notify_observers
    func(observer)
  File "/usr/lib/pymodules/python2.6/ckan/model/modification.py", line 37,
in before_commit
    self.notify(obj, domain_object.DomainObjectOperation.new)
  File "/usr/lib/pymodules/python2.6/ckan/model/modification.py", line 68,
in notify
    observer.notify(entity, operation)
  File
"/var/lib/ckan/std/pyenv/src/ckanext-datastorer/ckanext/datastorer/plugin.py",
line 30, in notify
    self._create_datastorer_task(entity)
  File
"/var/lib/ckan/std/pyenv/src/ckanext-datastorer/ckanext/datastorer/plugin.py",
line 42, in _create_datastorer_task
    'site_url': h.url_for_static('/', qualified=True),
  File "/usr/lib/pymodules/python2.6/ckan/lib/helpers.py", line 87, in
url_for_static
    my_url = _routes_default_url_for(*args, **kw)
  File "/usr/lib/pymodules/python2.6/ckan/routes/util.py", line 257, in
url_for
    host = config.host
  File "/usr/lib/pymodules/python2.6/ckan/routes/__init__.py", line 14, in
__getattr__
    return getattr(self.__shared_state, name)
AttributeError: 'thread._local' object has no attribute 'host'
2013-01-04 15:45:06,881 ERROR [ckanext.harvest.harvesters.base]
AttributeError("'thread._local' object has no attribute 'host'",)

Does this make sense to anyone :-)

/Lasse


2013/1/3 Lasse Vestergaard <ibbernik at gmail.com>

> Hi Adriá Mercader.
>
> My version of CKAN is 1.8 and the ckanext-harvest is the one I get when I
> follow the tutorial at this page: https://github.com/okfn/ckanext-harvest.
> I cloned the repository today (3. january).
>
> /Lasse
>
>
> 2013/1/3 Adrià Mercader <adria.mercader at okfn.org>
>
>> Hi Lesse,
>>
>> You are right, you don't need to specify the json config option unless
>> you need it.
>> What versions of CKAN core and ckanext-harvest are you using?
>>
>>
>> Adrià
>>
>> On 3 January 2013 14:49, Lasse Vestergaard <ibbernik at gmail.com> wrote:
>> > Hej Peder :-)
>> >
>> > I wrote this in the configuration field:
>> >
>> > {
>> >  "api_version":"1",
>> >  "default_groups":["mygroup"],
>> >  "user":"my-username"
>> > }
>> >
>> > The group exists and I can easily create the harvest source. The error
>> > occurs when the harvest process runs. As I read the documentation, I
>> don't
>> > need to put in all configuration options. Is that correct?
>> >
>> > /Lasse
>> >
>> >
>> >
>> > 2013/1/3 Peder Jakobsen | gmail <pjakobsen at gmail.com>
>> >>
>> >> HI Lasse,
>> >>
>> >> Did you enter your configuration details in the form of json in the UI?
>> >>
>> >> Venlig hilsen,
>> >>
>> >> Peder :)
>> >>
>> >> On Thu, Jan 3, 2013 at 3:56 AM, Lasse Vestergaard <ibbernik at gmail.com>
>> >> wrote:
>> >> > Hi all.
>> >> >
>> >> > I am trying to get the harvestor extension (ckanext-harvest-) work
>> for
>> >> > CKAN.
>> >> > All in all it seems like, I have managed to install it. I am
>> currently
>> >> > trying to harvest data form an other CKAN instance (remote computer
>> on
>> >> > an
>> >> > other network). I only provided the url to the CKAN frontend:
>> >> > ckan.odaa.dk.
>> >> > When i run the harvesting process I get the following error:
>> >> >
>> >> > Last Harvest Errors: 12
>> >> > Object errors
>> >> >
>> >> > GUID fredede-bygninger-og-fortidsminder
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID blog-p-bibliotekernes-hjemmeside
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID arrangementer-p-bibliotekerne
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID ejendomsfortegnelse
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID vejnavne-og-vejkoder
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID parkering
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID bibliotekstransaktioner
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID beholdning-ved-aarhus-kommunes-biblioteker
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID koordinats-tning-af-aarhus-adresser
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID cvr-produktionsenheder
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID bevaringsv-rdige-bygninger-i-aarhus-kommune
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> > GUID bbr-ejendomme-og-bbr-bygninger
>> >> > AttributeError("'thread._local' object has no attribute 'host'",)
>> >> >
>> >> > There are only 12 datasets in the CKAN instance I try to harvest from
>> >> > (it's
>> >> > a test instance). I have tried to Google the error, but I don't get
>> any
>> >> > relevant suggestions, so I hope you guys can point me in some
>> direction.
>> >> >
>> >> > Regards
>> >> >
>> >> > Lasse Vestergaard
>> >> >
>> >> > _______________________________________________
>> >> > ckan-dev mailing list
>> >> > ckan-dev at lists.okfn.org
>> >> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>> >> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>> >> >
>> >>
>> >> _______________________________________________
>> >> ckan-dev mailing list
>> >> ckan-dev at lists.okfn.org
>> >> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> >> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > ckan-dev mailing list
>> > ckan-dev at lists.okfn.org
>> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>> > Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>> >
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20130107/be71df34/attachment-0001.html>

• Previous message: [ckan-dev] enabling datastore & data API breaks recline
• Next message: [ckan-dev] Harvestor extension for CKAN
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]