[ckan-dev] user without name

Hildegard Gerlach hildegard.gerlach at jrc.ec.europa.eu
Wed Nov 27 11:52:11 UTC 2013


Dear all,

I started again up with an empty database and did the db init command.

paster --plugin=ckan db init -c /etc/ckan/production.ini

Immediately after that I see two users in the table user in PostgreSQL, 
visitor and logged_in.

Then I do this and see three

  paster --plugin=ckan user list -c /etc/ckan/production.ini
Users:
count = 3
name=
name=logged_in
name=visitor

A minute afterwards I see the third user without name also in  postgres 
table user, created half a minute later than the other 2.


Is there any explanation for that  ?

Hilde




On 11/27/2013 9:52 AM, Hildegard Gerlach wrote:
> Dear Joe,
>
> thanks for answering.
> I have set the branch of ckanext-harvest to stable
> git branch
>   master
> * stable
>
>
> I have done an upgrade of CKAN to 2.1.1 some days ago and also the 
> extensions (stable), but the error was there before and nothing changed.
>
> I don't know what git  show (in ckanext-harvest) should do, I get the 
> following:
> git show
> commit cb724b81ead8529bf109b4df0f8e348cf7b835b0
> Author: amercader <amercadero at gmail.com>
> Date:   Thu Oct 24 12:33:44 2013 +0100
>
>     Improve organizations dropdown on source form
>
> diff --git a/ckanext/harvest/templates_new/source/new_source_form.html 
> b/ckanext/harvest/templates_new/source/new_source_form.html
> index 8f79c34..5e3f4c9 100644
> --- a/ckanext/harvest/templates_new/source/new_source_form.html
> +++ b/ckanext/harvest/templates_new/source/new_source_form.html
> @@ -50,24 +50,28 @@
>    {% if data.group_id %}
>      <input type="hidden" name="groups__0__id" value="{{ data.group_id 
> }}" />
>    {% endif %}
> -  {% set existing_org = data.owner_org or data.group_id %}
> -  {% if h.check_access('sysadmin') or data.get('state', 
> 'draft').startswith('draft') or data.get('state', 'none') == 'none' %}
> -    {% set organizations_available = 
> h.organizations_available('create_dataset') %}
> -    {% if organizations_available %}
> -      <div class="control-group">
> -        <label for="field-organizations" class="control-label">{{ 
> _('Organization') }}</label>
> -        <div class="controls">
> -          <select id="field-organizations" name="owner_org" 
> data-module="autocomplete">
> -            <option value="">{{ _('Select an organization...') 
> }}</option>
> -            {% for organization in organizations_available %}
> -              {# get out first org from users list only if there is 
> not an existing org #}
> -              {% set selected_org = (existing_org and existing_org == 
> organization.id) or (not existing_org and organization.id == 
> organizations_available[
> -              <option value="{{ organization.id }}" {% if 
> selected_org %} selected="selected" {% endif %}>{{ organization.name 
> }}</option>
> -            {% endfor %}
> -          </select>
> -        </div>
> +
> +  {% set dataset_is_draft = data.get('state', 
> 'draft').startswith('draft') or data.get('state', 'none') == 'none' %}
> +  {% set dataset_has_organization = data.owner_org or data.group_id %}
> +  {% set organizations_available = 
> h.organizations_available('create_dataset') %}
> +  {% set user_is_sysadmin = h.check_access('sysadmin') %}
> +  {% set show_organizations_selector = organizations_available and 
> (user_is_sysadmin or dataset_is_draft) %}
> +
> +  {% if show_organizations_selector %}
> +    {% set existing_org = data.owner_org %}
> +    <div class="control-group">
> +      <label for="field-organizations" class="control-label">{{ 
> _('Organization') }}</label>
> +      <div class="controls">
>
> paster harvester purge_queues  didn't change anything either.
>
> I really think the user is wrong (actually it seems to be empty) but I 
> don't understand why. In the interface I use the user admin, and with 
> the paster command I don't set any user (in ckan virtualenv) when 
> using the paster command and in the ckan configuration file.
>
> I have added the following code to the file update.py which gives the 
> error message
> File 
> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/update.py", 
> line 28, in harvest_source_update
>     raise pt.ObjectNotFound(pt._('Harvest source not found'))
> ckan.logic.NotFound: Harvest source not found
>
>     f = open('/usr/local/ckan/Hildetest/Hilde', 'w')
>     f.write("user "  + user)
>     f.write(os.linesep)
>     f.write(pt._('User {0} blabla {1}').format(user, source_id))
>     f.close()
>
> and the file Hilde created  there contains
>
> more Hildetest/Hilde
> user
> User  blabla None
>
> I have already written in the earlier posts of this thread that we 
> have this mysterious empty user and cannot delete it.
>
> paster --plugin=ckan user list -c /etc/ckan/production.ini
> Users:
> count = 4
> name=
> name=admin
> name=logged_in
> name=visitor
>
> When I did initdb for ckan (not the extensions) with an empty database 
> this user was created.
> If  I look at the sysadmin users in the administration interface, I see
> /user/640f0c2d-4005-4bab-b326-c21ae9c491c3 
> <http://drdsi-data.ies.jrc.it/user/640f0c2d-4005-4bab-b326-c21ae9c491c3>
> admin
>
> I don't know what the first one means.
>
> On the second machine which had been installed earlier (always Redhat) 
> this empty user doesn't appear.
>
> Could there be an issue using PostgreSQL 9.2 ?
>
> Hilde
>
>
> On 11/27/2013 8:50 AM, Joe Tsoi wrote:
>> Hi,
>>
>> The harvest source not found error looks like it might not be the
>> harvest user as it could be bailing out before it gets to the point
>> where it uses it.
>>
>> I think there are two seperate things that might, help. Either you
>> harvest extension isn't up to date, could you do a git show on
>> ckanext-harvest directory? Just so I can know which version of the
>> code we're dealing with here.
>>
>> Or you've got jobs left over from a previous harvest job before you
>> wiped the database. In which case could you do a paster harvester
>> purge_queues and then start again with the empty database
>>
>>
>>
>> On 26 November 2013 17:45, Hildegard Gerlach
>> <hildegard.gerlach at jrc.ec.europa.eu> wrote:
>>> Hi again,
>>>
>>> I did another test. I created a new database, initialized it for 
>>> ckan, the
>>> harvest and spatial extension and created the sysadmin user admin.
>>> Anyway, an empty user (name '') is created again when initializing the
>>> database for CKAN (version 2.1.1)
>>> When I try to harvest I get the same error
>>>
>>>    File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/update.py", 
>>>
>>> line 28, in harvest_source_update
>>>      raise pt.ObjectNotFound(pt._('Harvest source not found'))
>>> ckan.logic.NotFound: Harvest source not found
>>>
>>> and in the logfile I have the following:
>>>
>>> 2013-11-26 16:19:35,913 DEBUG [ckanext.harvest.model] Harvest tables 
>>> defined
>>> in memory
>>> 2013-11-26 16:19:35,917 DEBUG [ckanext.harvest.model] Harvest tables 
>>> already
>>> exist
>>> 2013-11-26 16:19:35,938 DEBUG [ckanext.spatial.model.package_extent] 
>>> Spatial
>>> tables defined in memory
>>> 2013-11-26 16:19:35,944 DEBUG [ckanext.spatial.model.package_extent] 
>>> Spatial
>>> tables already exist
>>> 2013-11-26 16:19:35,960 DEBUG [ckanext.harvest.queue] pika 
>>> connection using
>>> {'retry_delay': 2.0, 'frame_max': 10000, 'channel_max': 0, 'locale'  :
>>> 'en_US', 'socket_timeout': 0.25, 'ssl': False, 'host': 'localhost',
>>> 'ssl_options': {}, 'virtual_host': '/', 'heartbeat': 0, 'credentials':
>>> <pika.credentials.PlainCredentials object at 0x3949a50>,
>>> 'backpressure_detection': False, 'port': 5672, 
>>> 'connection_attempts': 1}
>>> 2013-11-26 16:19:37,008 DEBUG [ckanext.harvest.queue] Gather queue 
>>> consumer
>>> registered
>>> 2013-11-26 16:20:17,507 DEBUG [ckanext.harvest.model] Harvest tables 
>>> defined
>>> in memory
>>> 2013-11-26 16:20:17,511 DEBUG [ckanext.harvest.model] Harvest tables 
>>> already
>>> exist
>>> 2013-11-26 16:20:17,532 DEBUG [ckanext.spatial.model.package_extent] 
>>> Spatial
>>> tables defined in memory
>>> 2013-11-26 16:20:17,540 DEBUG [ckanext.spatial.model.package_extent] 
>>> Spatial
>>> tables already exist
>>> 2013-11-26 16:20:17,555 DEBUG [ckanext.harvest.queue] pika 
>>> connection using
>>> {'retry_delay': 2.0, 'frame_max': 10000, 'channel_max': 0, 'locale' :
>>> 'en_US', 'socket_timeout': 0.25, 'ssl': False, 'host': 'localhost',
>>> 'ssl_options': {}, 'virtual_host': '/', 'heartbeat': 0, 'credentials':
>>> <pika.credentials.PlainCredentials object at 0x39be990>,
>>> 'backpressure_detection': False, 'port': 5672, 
>>> 'connection_attempts': 1}
>>> 2013-11-26 16:20:18,602 DEBUG [ckanext.harvest.queue] Fetch queue 
>>> consumer
>>> registered
>>> 2013-11-26 16:20:32,927 DEBUG [ckanext.harvest.model] Harvest tables 
>>> defined
>>> in memory
>>> 2013-11-26 16:20:32,931 DEBUG [ckanext.harvest.model] Harvest tables 
>>> already
>>> exist
>>> 2013-11-26 16:20:32,952 DEBUG [ckanext.spatial.model.package_extent] 
>>> Spatial
>>> tables defined in memory
>>> 2013-11-26 16:20:32,959 DEBUG [ckanext.spatial.model.package_extent] 
>>> Spatial
>>> tables already exist
>>> 2013-11-26 16:20:32,974 INFO [ckanext.harvest.logic.action.update] 
>>> Harvest
>>> job run: {}
>>> 2013-11-26 16:21:03,016 WARNI [ckan.lib.maintain] Function 
>>> nav_named_link()
>>> in module ckan.lib.helpers has been deprecated and will be removed in a
>>> later release of ckan. h.nav_named_link is deprecated please use 
>>> h.nav_link
>>> NOTE: you will need to pass the route_name as a named parameter
>>>
>>>
>>> We are using solr as a backend
>>> ckanext.spatial.search_backend = solr
>>>
>>> and have PostgreSQL 9.2 with PostGIS 2.1.0
>>>
>>> ckandb=> SELECT PostGIS_full_version();
>>> postgis_full_version
>>> ----------------------------------------------------------------------
>>>   POSTGIS="2.1.0 r11822" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 
>>> 4.8.0, 6
>>> March 2012" GDAL="GDAL 1.9.2, released 2012/10/08"
>>> LIBXML="2.7.6" LIBJSON="UNKNOWN" TOPOLOGY RASTER
>>> (1 row)
>>>
>>> and the table package_extent is slightly different as to an older 
>>> version of
>>> PostgreSQL/PostGIS
>>>
>>> ckandb=> \d package_extent
>>>            Table "public.package_extent"
>>>     Column   |          Type           | Modifiers
>>> ------------+-------------------------+-----------
>>>   package_id | text                    | not null
>>>   the_geom   | geometry(Geometry,4326) |
>>> Indexes:
>>>      "package_extent_pkey" PRIMARY KEY, btree (package_id)
>>>      "idx_package_extent_the_geom" gist (the_geom)
>>>
>>>
>>> I also cannot clean the database, I get the following error:
>>>
>>> paster db clean -c /etc/ckan/production.ini
>>> /usr/local/ckan/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/reflection.py:47: 
>>>
>>> SAWarning: Did not recognize type 'geometry(Geometry,4326)' of column
>>> 'the_geom'
>>>    ret = fn(self, con, *args, **kw)
>>>
>>> Is this version of PostgreSQL/PostGIS not yet supported by CKAN ?
>>>
>>> I really don't know what to try any more.
>>>
>>> Thanks
>>>
>>> Hilde
>>>
>>>
>>>
>>> On 26.11.2013, at 12:09, Hildegard Gerlach
>>> <hildegard.gerlach at jrc.ec.europa.eu> wrote:
>>>
>>> Hi everyone,
>>>
>>>
>>> http://docs.ckan.org/projects/ckanext-spatial/en/latest/harvesters.html 
>>> says
>>>
>>> By default the harvesting actions (eg creating or updating datasets) 
>>> will be
>>> performed by the internal site admin user. This is the recommended 
>>> setting,
>>> but if necessary, it can be overridden with the
>>> ckanext.spatial.harvest.user_name config option, eg to support the old
>>> hardcoded ‘harvest’ user:
>>>
>>> ckanext.spatial.harvest.user_name  =  harvest
>>>
>>> We have another instance of CKAN (both on Redhat) on another machine 
>>> where
>>> the harvester works without having a user harvest.
>>>
>>> I am working with Elena and we have obviously an authorization problem
>>> running the harvesting.
>>>
>>> paster --plugin=ckanext-harvest harvester run -c 
>>> /etc/ckan/production.ini
>>>
>>> Traceback (most recent call last):
>>> File "/usr/local/ckan/pyenv/bin/paster", line 9, in <module>
>>> load_entry_point('PasteScript==1.7.5', 'console_scripts', 'paster')()
>>> File
>>> "/usr/local/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py", 
>>>
>>> line 104, in run
>>> invoke(command, command_name, options, args[1:])
>>> File
>>> "/usr/local/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py", 
>>>
>>> line 143, in invoke
>>> exit_code = runner.run(args)
>>> File
>>> "/usr/local/ckan/pyenv/lib/python2.6/site-packages/paste/script/command.py", 
>>>
>>> line 238, in run
>>> result = self.command()
>>> File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", 
>>>
>>> line 113, in command
>>> self.run_harvester()
>>> File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", 
>>>
>>> line 278, in run_harvester
>>> jobs = get_action('harvest_jobs_run')(context,{})
>>> File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 
>>> 329, in
>>> wrapped
>>> return _action(context, data_dict, **kw)
>>> File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/action/update.py", 
>>>
>>> line 291, in harvest_jobs_run
>>> jobs = 
>>> harvest_job_list(context,{'source_id':source_id,'status':u'Running'})
>>> File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 
>>> 386, in
>>> wrapper
>>> return action(context, data_dict)
>>> File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/action/get.py", 
>>>
>>> line 226, in harvest_job_list
>>> check_access('harvest_job_list',context,data_dict)
>>> File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 
>>> 207, in
>>> check_access
>>> logic_authorization = new_authz.is_authorized(action, context, 
>>> data_dict)
>>> File "/usr/local/ckan/pyenv/src/ckan/ckan/new_authz.py", line 82, in
>>> is_authorized
>>> return auth_function(context, data_dict)
>>> File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/get.py", 
>>>
>>> line 85, in harvest_job_list
>>> {'id': source_id})
>>> File "/usr/local/ckan/pyenv/src/ckan/ckan/logic/__init__.py", line 
>>> 207, in
>>> check_access
>>> logic_authorization = new_authz.is_authorized(action, context, 
>>> data_dict)
>>> File "/usr/local/ckan/pyenv/src/ckan/ckan/new_authz.py", line 82, in
>>> is_authorized
>>> return auth_function(context, data_dict)
>>> File
>>> "/usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/update.py", 
>>>
>>> line 28, in harvest_source_update
>>> raise pt.ObjectNotFound(pt._('Harvest source not found'))
>>> ckan.logic.NotFound: Harvest source not found
>>>
>>> I changed the
>>> /usr/local/ckan/pyenv/src/ckanext-harvest/ckanext/harvest/logic/auth/update.py 
>>>
>>> to output some data, and I get an empty user (no value)
>>> and source_id: None
>>>
>>> Now I tried to set
>>>
>>> ckanext.spatial.harvest.user_name  =  admin
>>>
>>> but this didn't change anything.
>>>
>>> Then we have this empty user (name=  ) which might cause a problem, 
>>> but we
>>> don't know hot to get rid of it (as already described).
>>>
>>> paster --plugin=ckan user list -c /etc/ckan/production.ini
>>> Users:
>>> count = 9
>>> name=
>>> name=admin
>>> name=annafan
>>> name=joeadmin
>>> name=logged_in
>>> name=russianfan
>>> name=tester
>>> name=testsysadmin
>>> name=visitor
>>>
>>> Any help would be appreciated.
>>>
>>> Otherwise is there a way to uninstall and reinstall the harvesting 
>>> extension
>>> ? We did an upgrade to 2.1.1 of CKAN and extensions to stable, but 
>>> nothing
>>> changed.
>>>
>>> Kind regards
>>>
>>> Hilde
>>>
>>>
>>>
>>> On 11/25/2013 9:05 PM, Vitor Baptista wrote:
>>>
>>> Hi Elena,
>>>
>>> By default, ckanext-harvest use the user "harvest". You'll have some 
>>> poblems
>>> if it doesn't exist (or you haven't changed it to another username). To
>>> solve it, you can simply add this user as a sysadmin. The password 
>>> doesn't
>>> matter.
>>>
>>> Cheers,
>>>
>>>
>>> 2013/11/25 Elena Camossi <elena.camossi at ext.jrc.ec.europa.eu
>>> <mailto:elena.camossi at ext.jrc.ec.europa.eu>>
>>>
>>>     Hi everyone,
>>>
>>>     we are experiencing some problem with harvesting (CKAN 2.1, on
>>>     RedHat),
>>>     doing some debugging we discovered that harvesting seems to be
>>>     runned by a
>>>     user with no name.
>>>
>>>     Getting the list of users with "paster user list", we discovered
>>>     that one of
>>>     the users has indeed no name, and table user within Postgres has
>>>     indeed a
>>>     row
>>>     column name having value ' '.
>>>
>>>     We have tried to delete this user with "paster user remove". 
>>> Command
>>>     succeeds but user still exists. We have tried deleting it from the
>>>     Postgresql database, the row is removed but it reappears after
>>>     running again
>>>     "paster user list"...
>>>
>>>     Is there any other way to delete a CKAN user?
>>>
>>>     Does harvesting use some default CKAN user?
>>>
>>>     Thanks a lot for your help.
>>>
>>>     Kind regards,
>>>     -Elen a
>>>
>>>
>>>     _______________________________________________
>>>     ckan-dev mailing list
>>>     ckan-dev at lists.okfn.org <mailto:ckan-dev at lists.okfn.org>
>>>     http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>     Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>>>
>>>
>>> -- 
>>>
>>> Vítor Baptista
>>>
>>> Developer |http://vitorbaptista.com | LinkedIn
>>> <http://www.linkedin.com/in/vitorbaptista> | @vitorbaptista
>>> <http://twitter.com/vitorbaptista>
>>>
>>> The Open Knowledge Foundation <http://okfn.org>
>>>
>>> /Empowering through Open Knowledge/
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev





More information about the ckan-dev mailing list