[ckan-changes] [ckan/ckan] aa2a57: [#4801] Catch exception outside request context

Thu Dec 19 09:44:45 UTC 2019

  Branch: refs/heads/master
  Home:   https://github.com/ckan/ckan
  Commit: aa2a57ef656ae0349d14ef555f46ccbc82161880
      https://github.com/ckan/ckan/commit/aa2a57ef656ae0349d14ef555f46ccbc82161880
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/logic/__init__.py

  Log Message:
  -----------
  [#4801] Catch exception outside request context

  Commit: 690a33f9da2df204acf99aa46550b2514342a8d5
      https://github.com/ckan/ckan/commit/690a33f9da2df204acf99aa46550b2514342a8d5
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/lib/fanstatic_resources.py
    M scripts/4042_fix_resource_extras.py

  Log Message:
  -----------
  [#4801] Rename configparser module

  Commit: 9851dd372a71af3097971893ea6825f70e7e65d0
      https://github.com/ckan/ckan/commit/9851dd372a71af3097971893ea6825f70e7e65d0
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M requirements.in
    M requirements.txt

  Log Message:
  -----------
  [#4801] Upgrade fanstatic on to a py3 compatible version (only in py3)

  Commit: 5a6e7ad0888bfab6b28fa64819a63db6ebc9fada
      https://github.com/ckan/ckan/commit/5a6e7ad0888bfab6b28fa64819a63db6ebc9fada
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/views/admin.py
    M ckan/views/dataset.py
    M ckan/views/home.py
    M ckan/views/resource.py

  Log Message:
  -----------
  [#4801] Don't import stuff from the old controllers

  Commit: 0018cc2fec696f65080bbe42cbc15c1c904d3135
      https://github.com/ckan/ckan/commit/0018cc2fec696f65080bbe42cbc15c1c904d3135
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/lib/app_globals.py

  Log Message:
  -----------
  [#4801] Don't decode str in py3

  Commit: 1640a46fac313c389d790966562f58cba0bc7bd9
      https://github.com/ckan/ckan/commit/1640a46fac313c389d790966562f58cba0bc7bd9
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/plugins/toolkit.py

  Log Message:
  -----------
  [#4801] Avoid using h.url_for_static when adding a public folder

As at this point there isn't a Flask app context yet and we can't call
the url_for function. But if I understand the code correctly we just
want a link to the root of the site, taking into account stuff like root
path settings etc

  Commit: 3d9c30d5a33d82d6ba39d392f9ca48a813cd8623
      https://github.com/ckan/ckan/commit/3d9c30d5a33d82d6ba39d392f9ca48a813cd8623
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/lib/authenticator.py
    M ckan/lib/repoze_plugins/friendly_form.py

  Log Message:
  -----------
  [#4801] Use the implementer decorator from zope.interface

Rather than the implements method, as this is only supported on py2

https://twistedmatrix.com/pipermail/twisted-python/2013-January/026414.html

  Commit: 26a790c5100796363935d66684ad6201ffb7a137
      https://github.com/ckan/ckan/commit/26a790c5100796363935d66684ad6201ffb7a137
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/controllers/error.py
    M ckan/lib/base.py
    M ckan/lib/helpers.py
    A ckan/lib/pagination.py
    M ckan/plugins/toolkit.py
    M ckan/tests/legacy/functional/api/base.py
    M ckan/tests/legacy/functional/api/test_package_search.py
    M ckan/tests/legacy/lib/test_helpers.py
    M ckan/tests/lib/test_helpers.py
    M ckan/views/feed.py
    M requirements-py2.in
    M requirements-py2.txt
    M requirements.in
    M requirements.txt

  Log Message:
  -----------
  Merge branch 'master' into py3-general-support-2

  Commit: d1ef3fa53fea632c751d45bc87438d65157581b6
      https://github.com/ckan/ckan/commit/d1ef3fa53fea632c751d45bc87438d65157581b6
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/views/feed.py

  Log Message:
  -----------
  Remove webhelpers based class

  Commit: 01605f1d436cfb2ed5ebc3509e36ee2d98d79c86
      https://github.com/ckan/ckan/commit/01605f1d436cfb2ed5ebc3509e36ee2d98d79c86
  Author: amercader <amercadero at gmail.com>
  Date:   2019-11-28 (Thu, 28 Nov 2019)

  Changed paths:
    M ckan/views/group.py

  Log Message:
  -----------
  [#4801] Remove unused imports (including pylons one) in group view

  Commit: f93fc060f3dcfd0aa0f93d3a70f23ddb92d3486e
      https://github.com/ckan/ckan/commit/f93fc060f3dcfd0aa0f93d3a70f23ddb92d3486e
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-03 (Tue, 03 Dec 2019)

  Changed paths:
    M ckan/cli/cli.py
    A ckan/cli/sysadmin.py
    A ckan/cli/user.py
    M ckan/public/base/less/dataset.less
    M doc/maintaining/getting-started.rst

  Log Message:
  -----------
  Merge branch 'master' into py3-general-support-2

  Commit: 99fcae46c5240aa8da15332f43ebae5fd22c0b54
      https://github.com/ckan/ckan/commit/99fcae46c5240aa8da15332f43ebae5fd22c0b54
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-03 (Tue, 03 Dec 2019)

  Changed paths:
    M ckan/lib/fanstatic_resources.py
    M scripts/4042_fix_resource_extras.py

  Log Message:
  -----------
  [#4801] Fix configparser imports

  Commit: 126f862293ab145f516bcd4e3d85f613278c7eed
      https://github.com/ckan/ckan/commit/126f862293ab145f516bcd4e3d85f613278c7eed
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/config/middleware/flask_app.py
    M ckan/views/__init__.py

  Log Message:
  -----------
  [#4801] Add routes.named_routes config option

  Commit: 92326d1b357e128bdd08bcb26a1eda255675830c
      https://github.com/ckan/ckan/commit/92326d1b357e128bdd08bcb26a1eda255675830c
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/lib/repoze_plugins/friendly_form.py

  Log Message:
  -----------
  [#4801] Handle webob py2/3 differences in friendly form

  Commit: 15440ebba7ed96ed1e9e93c93a6216e0a643395b
      https://github.com/ckan/ckan/commit/15440ebba7ed96ed1e9e93c93a6216e0a643395b
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/lib/navl/dictization_functions.py

  Log Message:
  -----------
  [#4801] Don't modify dict while in loop

  Commit: 3f44157c7efdb73ff1f5356224e02eb9988091f6
      https://github.com/ckan/ckan/commit/3f44157c7efdb73ff1f5356224e02eb9988091f6
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/model/license.py

  Log Message:
  -----------
  [#4801] Don't decode to unicode on py3, mark regexp in license model

  Commit: 621b008c9b2e7e07aef7410f85a61fec94154df0
      https://github.com/ckan/ckan/commit/621b008c9b2e7e07aef7410f85a61fec94154df0
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/lib/jinja_extensions.py

  Log Message:
  -----------
  [#4801] Use buitin next() function rather than the method

  Commit: 976b7072280a694e59de35f9095b9985bef882bc
      https://github.com/ckan/ckan/commit/976b7072280a694e59de35f9095b9985bef882bc
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/lib/helpers.py

  Log Message:
  -----------
  [#4801] No paster cli in py3

  Commit: b678bdc6b5c8327ae504ffdc58927d43833be46b
      https://github.com/ckan/ckan/commit/b678bdc6b5c8327ae504ffdc58927d43833be46b
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/lib/webassets_tools.py

  Log Message:
  -----------
  [#4801] Webassets requires a proper boolean for debug in py3

  Commit: 7da99d248b8e5760ffbb9cb68d984a786b7b0a83
      https://github.com/ckan/ckan/commit/7da99d248b8e5760ffbb9cb68d984a786b7b0a83
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    M ckan/lib/helpers.py

  Log Message:
  -----------
  [#4801] Handle local URLs in url_for

In some cases we call `helpers.url_for()` with an actual URL path rather
than an endpoint name or controller/action kwargs. The main use case for
this is building static URLs that take into account the root path, etc

    h.url_for('/base/css/main.css')

The Pylons router (`routes`) was find with this, but the Flask one will
fail because it requires the first arg to be an actual `endpoint` name
like `dataset.new`. Up until now the Flask router was failing and the
url was then picked up by the fallback Pylons one, which returned the
expected URL.

On python 3 Pylons and its router is gone, so the generation of these
URLs (eg during the initialization of webassets) raises a
`FlaskRouteBuildError` exception.

This commit adds the routes behaviour to our wrapper if the Flask router
fails and the URL starts with `/`. The URL will get prepended with
`environ['SCRIPT_NAME']` if present, and any keyword args will be added
as query string (which is unlikely that will get used but will preserve
the current behaviour)

  Commit: ebb8e0ffe3161da918e4078b6afa7f69c523119e
      https://github.com/ckan/ckan/commit/ebb8e0ffe3161da918e4078b6afa7f69c523119e
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-06 (Fri, 06 Dec 2019)

  Changed paths:
    R ckan/tests/migration/__init__.py
    R ckan/tests/migration/revision_legacy_code_tests.py
    M doc/contributing/testing.rst

  Log Message:
  -----------
  Merge branch 'py3-general-support' into py3-general-support-2

  Commit: a63207806c9474ed736c1a4fc774155042a45a51
      https://github.com/ckan/ckan/commit/a63207806c9474ed736c1a4fc774155042a45a51
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-09 (Mon, 09 Dec 2019)

  Changed paths:
    M ckan/lib/navl/dictization_functions.py

  Log Message:
  -----------
  [#4801] Refactor processing of errors in NAVL

At the end of the `validate` function run the errors are processed and
return as a dict. The current implementation was complex and
incompatible with Python 3, as it changed dictionaries inside loops.
Also it returned unnecessary key, eg before the changes:

    {
      'resources': [
        {'position': [u'Missing value']}, {}],
      'tags': [{}, {}, {}],
      'extras': [{}, {}]
    }

After the changes:

    {
      'resources: [
        {'position': [u'Missing value']}
      ]
    }

  Commit: c8954433e0006097d9c388e776147bc5b5accf48
      https://github.com/ckan/ckan/commit/c8954433e0006097d9c388e776147bc5b5accf48
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-09 (Mon, 09 Dec 2019)

  Changed paths:
    M ckan/lib/helpers.py
    M ckan/lib/navl/dictization_functions.py

  Log Message:
  -----------
  [#4801] Fix bug in url_for

  Commit: 0f99b93d1e12a4fa214f8d5810a4733069e43e8c
      https://github.com/ckan/ckan/commit/0f99b93d1e12a4fa214f8d5810a4733069e43e8c
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M ckan/lib/repoze_plugins/friendly_form.py

  Log Message:
  -----------
  [#4801] Use Webob Request.queryvars in friendly form plugin

Request.params exists in webob 1.0.8 but itchanges the wsgi
environ and makes submitted form params not available later on.
This will probably affect py3 as well but we are not yet at a stage
where it can be tested, so we'll need to revisit it.

  Commit: ede7a1f41ca2ad0ffbd52fd81a716797cf0da389
      https://github.com/ckan/ckan/commit/ede7a1f41ca2ad0ffbd52fd81a716797cf0da389
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M ckan/lib/navl/dictization_functions.py

  Log Message:
  -----------
  [#4801] Iterate over a copy of the dict to modify it

  Commit: da340eddaa36f3ac8b568c6d915f403998dcbcb7
      https://github.com/ckan/ckan/commit/da340eddaa36f3ac8b568c6d915f403998dcbcb7
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M ckan/lib/helpers.py

  Log Message:
  -----------
  [#4801] Don't rely on routes router to build root URL

  Commit: 0dae7eae36c373cb6bf24ccf9ad61b12c6350615
      https://github.com/ckan/ckan/commit/0dae7eae36c373cb6bf24ccf9ad61b12c6350615
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M ckan/config/middleware/__init__.py
    M ckan/config/middleware/common_middleware.py
    M ckan/config/middleware/flask_app.py
    M ckan/views/__init__.py

  Log Message:
  -----------
  [#4801] Move the handle_i18n function to somewhere the Flask app can access it

  Commit: 8340f558f5451cf3cbdb5bdab27d20997bebd26e
      https://github.com/ckan/ckan/commit/8340f558f5451cf3cbdb5bdab27d20997bebd26e
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-12 (Thu, 12 Dec 2019)

  Changed paths:
    M ckan/views/__init__.py

  Log Message:
  -----------
  [#4801] Keep unicode logic in py2

  Commit: 7b6702b6c17efbe0b2b7eb46ab46da86e387c55c
      https://github.com/ckan/ckan/commit/7b6702b6c17efbe0b2b7eb46ab46da86e387c55c
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-13 (Fri, 13 Dec 2019)

  Changed paths:
    M ckan/lib/navl/dictization_functions.py
    M ckan/tests/legacy/lib/test_navl.py

  Log Message:
  -----------
  [#4801] Simplify even more the erros handling in navl

  Commit: a076ccd92497a7505e19210472d658ecc676ea51
      https://github.com/ckan/ckan/commit/a076ccd92497a7505e19210472d658ecc676ea51
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-13 (Fri, 13 Dec 2019)

  Changed paths:
    M ckan/views/__init__.py
    M ckan/views/home.py

  Log Message:
  -----------
  Coding standards

  Commit: 00ee56030e6b2a6ab2a2908e25b19f889cda4421
      https://github.com/ckan/ckan/commit/00ee56030e6b2a6ab2a2908e25b19f889cda4421
  Author: amercader <amercadero at gmail.com>
  Date:   2019-12-13 (Fri, 13 Dec 2019)

  Changed paths:
    M ckan/lib/helpers.py

  Log Message:
  -----------
  [#4801] Fix url_for_static logic

  Commit: 467ceebb6b147e9ed7c928225c2dfbcdb2179dae
      https://github.com/ckan/ckan/commit/467ceebb6b147e9ed7c928225c2dfbcdb2179dae
  Author: Sergey <tonnydexter at gmail.com>
  Date:   2019-12-19 (Thu, 19 Dec 2019)

  Changed paths:
    M ckan/config/middleware/__init__.py
    M ckan/config/middleware/common_middleware.py
    M ckan/config/middleware/flask_app.py
    M ckan/lib/app_globals.py
    M ckan/lib/authenticator.py
    M ckan/lib/helpers.py
    M ckan/lib/jinja_extensions.py
    M ckan/lib/navl/dictization_functions.py
    M ckan/lib/repoze_plugins/friendly_form.py
    M ckan/lib/webassets_tools.py
    M ckan/model/license.py
    M ckan/plugins/toolkit.py
    M ckan/tests/legacy/lib/test_navl.py
    M ckan/views/__init__.py
    M ckan/views/admin.py
    M ckan/views/dataset.py
    M ckan/views/feed.py
    M ckan/views/group.py
    M ckan/views/home.py
    M ckan/views/resource.py

  Log Message:
  -----------
  Merge pull request #5107 from ckan/py3-general-support-2

PY3 General Support - Part 2

Compare: https://github.com/ckan/ckan/compare/a93b5068b392...467ceebb6b14