[ckan-dev] Upgrading to 2.7.2 - error building search index

[McDonaldR]

Well, after some more reading I ran the paster command with the -i and -e switches to force the index rebuild and my layers reappeared.

So now I can see the database tables were updated and with the new search index everything works.

Ross

From: ckan-dev [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of McDonaldR
Sent: 17 October 2017 16:07
To: ckan-dev at lists.okfn.org
Subject: [ckan-dev] Upgrading to 2.7.2 - error building search index

Hi List

Thanks for all the work going into new versions of CKAN.  I've just upgraded to 2.7.2 on Ubuntu 14.04.  I run the web application on one server and the database on another.  I have got almost everything working correctly again but there are a couple of issues I have.

My site (http://opendata.angus.gov.uk) is reading the customised theme and gets the list of organisations and groups but not the datasets.

I've tried to rebuild the search index and get the following error:

# paster search-index rebuild -c /etc/ckan/default/production.ini

2017-10-17 15:25:42,627 WARNI [ckan.lib.celery_app] ckan.lib.celery_app is deprecated, use ckan.lib.jobs instead.
2017-10-17 15:25:42,940 INFO  [ckan.lib.search] Rebuilding search index...
2017-10-17 15:25:42,941 INFO  [ckan.lib.search] Rebuilding the whole index...
Indexing dataset 9/482017-10-17 15:25:43,221 ERROR [ckan.lib.search] Error while indexing dataset 2b4f7001-b2f2-4948-9b8a-8e5ec4572579: TypeError("unsupported operand type(s) for +=: 'NoneType' and 'str'",)
Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 554, in command
    self.rebuild()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 577, in rebuild
    quiet=self.options.quiet)
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/__init__.py", line 194, in rebuild
    defer_commit
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 101, in update_dict
    self.index_package(pkg_dict, defer_commit)
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/search/index.py", line 256, in index_package
    pkg_dict['metadata_created'] += 'Z'
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Line 101 in search/index.py self.index_package(pkg_dict, defer_commit)

Line 256 in search/index.py pkg_dict['metadata_created'] += 'Z'

I can get to http://opendata.angus.gov.uk/stats/ and http://opendata.angus.gov.uk/report/ and from there I can see datasets and resources like http://opendata.angus.gov.uk/dataset/angus-council-conservation-areas and http://opendata.angus.gov.uk/dataset/angus-council-conservation-areas/resource/f2ce10ae-2571-4f27-8904-2304fc8efdd1 so it is reading the database and pulling out the correct information.

The other issue I have is logging in - I have a valid username / password  but I keep getting an error.  Are these credentials stored in the PostgreSQL database?  I upgraded the database with

# paster db upgrade -c /etc/ckan/default/production.ini

But got returned to the prompt after a few seconds with no indication of success or failure.

I also set the datastore permissions.

So, I guess my questions are 1) has my database been upgraded to work with 2.7.2 and 2) is that why my search index rebuild fails?

As an aside, the warning messages about Celery being deprecated and to use ckan.lib.jobs instead that keep popping up, are these for me or for extension developers?  Is the upgrade to 2.8.0 going to change the backend even more?

Thanks in advance for any pointers

Ross

Ross McDonald | GIS Data Coordinator | Angus Council, People, IT | Angus House, Orchardbank Business Park, Sylvie Way, Forfar DD8 1AT | t: 01307 476419


This message is strictly confidential. If you have received this in error, please inform the sender and remove it from your system. If received in error you may not copy, print, forward or use it or any attachment in any way. This message is not capable of creating a legal contract or a binding representation and does not represent the views of Angus Council. Emails may be monitored for security and network management reasons. Messages containing inappropriate content may be intercepted. Angus Council does not accept any liability for any harm that may be caused to the recipient system or data on it by this message or any attachment.

This message is strictly confidential. If you have received this in error, please inform the sender and remove it from your system. If received in error you may not copy, print, forward or use it or any attachment in any way. This message is not capable of creating a legal contract or a binding representation and does not represent the views of Angus Council. Emails may be monitored for security and network management reasons. Messages containing inappropriate content may be intercepted. Angus Council does not accept any liability for any harm that may be caused to the recipient system or data on it by this message or any attachment.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20171018/74cdba19/attachment-0003.html>