[ckan-changes] commit/ckan: 4 new changesets
Bitbucket
commits-noreply at bitbucket.org
Wed Oct 26 16:14:23 UTC 2011
4 new commits in ckan:
https://bitbucket.org/okfn/ckan/changeset/a34e584122a9/
changeset: a34e584122a9
user: dread
date: 2011-10-26 16:23:01
summary: [merge] from release 1.5.
affected #: 41 files
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 LICENSE.txt
--- /dev/null
+++ b/LICENSE.txt
@@ -0,0 +1,117 @@
+License
++++++++
+
+CKAN - Data Catalogue Software
+Copyright (C) 2007 Open Knowledge Foundation
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as
+published by the Free Software Foundation, either version 3 of the
+License, or (at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+Note
+====
+
+CKAN is sometimes packaged directly with other software (listed in
+requires/lucid_conflict.txt). In these cases, we are required to list the
+licenses of the packaged softare too. They are all AGPL compatible and read as
+follows in the next sections.
+
+WebHelpers
+----------
+
+Portions of WebHelpers covered by the following license::
+
+Copyright (c) 2005, the Lawrence Journal-World
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+ 1. Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+
+ 2. Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+
+ 3. Neither the name of Django nor the names of its contributors may be used
+ to endorse or promote products derived from this software without
+ specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Genshi
+------
+
+Copyright © 2006-2007 Edgewall Software
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+ 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+ 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+ 3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Migrate
+-------
+
+Open Source Initiative OSI - The MIT License:Licensing
+[OSI Approved License]
+
+The MIT License
+
+Copyright (c) <year><copyright holders>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+SQLAlchemy
+----------
+
+This is the MIT license: http://www.opensource.org/licenses/mit-license.php
+
+Copyright (c) 2005-2011 Michael Bayer and contributors. SQLAlchemy is a trademark of Michael Bayer.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 MANIFEST.in
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -1,8 +1,9 @@
include ckan/config/deployment.ini_tmpl
recursive-include ckan/public *
recursive-include ckan/config *.ini
+recursive-include ckan/config *.xml
recursive-include ckan/templates *
recursive-include ckan *.ini
prune .hg
include CHANGELOG.txt
-include ckan/migration/migrate.cfg
\ No newline at end of file
+include ckan/migration/migrate.cfg
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/config/deployment.ini_tmpl
--- a/ckan/config/deployment.ini_tmpl
+++ b/ckan/config/deployment.ini_tmpl
@@ -143,6 +143,8 @@
## Favicon (default is the CKAN software favicon)
ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico
+## Solr support
+#solr_url = http://127.0.0.1:8983/solr
## An 'id' for the site (using, for example, when creating entries in a common search index)
## If not specified derived from the site_url
@@ -182,6 +184,14 @@
#ckan.recaptcha.publickey =
#ckan.recaptcha.privatekey =
+# Locale/languages
+ckan.locale_default = en
+#ckan.locales_offered =
+# Default order is roughly by number of people speaking it in Europe:
+# http://en.wikipedia.org/wiki/Languages_of_the_European_Union#Knowledge
+ckan.locale_order = en de fr it es pl ru nl sv no cs_CZ hu pt_BR fi bg ca sq
+ckan.locales_filtered_out = el
+
# Logging configuration
[loggers]
keys = root, ckan, ckanext
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/controllers/home.py
--- a/ckan/controllers/home.py
+++ b/ckan/controllers/home.py
@@ -2,13 +2,14 @@
import sys
from pylons import cache, config
+from pylons.i18n import set_lang
from genshi.template import NewTextTemplate
import sqlalchemy.exc
from ckan.authz import Authorizer
from ckan.logic import NotAuthorized
from ckan.logic import check_access, get_action
-from ckan.i18n import set_session_locale, set_lang
+from ckan.lib.i18n import set_session_locale, get_lang
from ckan.lib.search import query_for, QueryOptions, SearchError
from ckan.lib.cache import proxy_cache, get_cache_expires
from ckan.lib.base import *
@@ -41,16 +42,13 @@
@staticmethod
- def _home_cache_key(latest_revision_id=None):
- '''Calculate the etag cache key for the home page. If you have
- the latest revision id then supply it as a param.'''
- if not latest_revision_id:
- latest_revision_id = model.repo.youngest_revision().id
+ def _home_cache_key():
+ '''Calculate the etag cache key for the home page.'''
+ # a change to the data means the group package amounts may change
+ latest_revision_id = model.repo.youngest_revision().id
user_name = c.user
- if latest_revision_id:
- cache_key = str(hash((latest_revision_id, user_name)))
- else:
- cache_key = 'fresh-install'
+ language = get_lang()
+ cache_key = str(hash((user_name, language, latest_revision_id)))
return cache_key
@proxy_cache(expires=cache_expires)
@@ -60,8 +58,9 @@
try:
query = query_for(model.Package)
- query.run({'q': '*:*'})
+ query.run({'q': '*:*', 'facet.field': g.facets})
c.package_count = query.count
+ c.facets = query.facets # used by the 'tag cloud' recipe
q = model.Session.query(model.Group).filter_by(state='active')
c.groups = sorted(q.all(), key=lambda g: len(g.packages), reverse=True)[:6]
except SearchError, se:
@@ -86,6 +85,11 @@
abort(400, _('Invalid language specified'))
try:
set_lang(locale)
+ # NOTE: When translating this string, substitute the word
+ # 'English' for the language being translated into.
+ # We do it this way because some Babel locales don't contain
+ # a display_name!
+ # e.g. babel.Locale.parse('no').get_display_name() returns None
h.flash_notice(_("Language has been set to: English"))
except:
h.flash_notice("Language has been set to: English")
@@ -96,8 +100,6 @@
# no need for error, just don't redirect
return
return_to += '&' if '?' in return_to else '?'
- # hack to prevent next page being cached
- return_to += '__cache=%s' % int(random.random()*100000000)
redirect_to(return_to)
def cache(self, id):
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/controllers/package.py
--- a/ckan/controllers/package.py
+++ b/ckan/controllers/package.py
@@ -7,7 +7,7 @@
from sqlalchemy.orm import eagerload_all
import genshi
from pylons import config, cache
-from pylons.i18n import get_lang, _
+from pylons.i18n import _
from autoneg.accept import negotiate
from babel.dates import format_date, format_datetime, format_time
@@ -25,6 +25,7 @@
from ckan.logic import NotFound, NotAuthorized, ValidationError
from ckan.logic import tuplize_dict, clean_dict, parse_params, flatten_to_string_key
from ckan.lib.dictization import table_dictize
+from ckan.lib.i18n import get_lang
import ckan.forms
import ckan.authz
import ckan.rating
@@ -179,7 +180,8 @@
def _pkg_cache_key(pkg):
# note: we need pkg.id in addition to pkg.revision.id because a
# revision may have more than one package in it.
- return str(hash((pkg.id, pkg.latest_related_revision.id, c.user, pkg.get_average_rating())))
+ language = get_lang()
+ return str(hash((pkg.id, pkg.latest_related_revision.id, c.user, pkg.get_average_rating(), language)))
@proxy_cache()
def read(self, id):
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/i18n/__init__.py
--- a/ckan/i18n/__init__.py
+++ b/ckan/i18n/__init__.py
@@ -1,65 +0,0 @@
-from pylons.i18n import _, add_fallback, get_lang, set_lang, gettext
-from babel import Locale
-
-
-# TODO: Figure out a nicer way to get this. From the .ini?
-# Order these by number of people speaking it in Europe:
-# http://en.wikipedia.org/wiki/Languages_of_the_European_Union#Knowledge
-# (or there abouts)
-_KNOWN_LOCALES = ['en',
- 'de',
- 'fr',
- 'it',
- 'es',
- 'pl',
- 'ru',
- 'nl',
- 'sv', # Swedish
- 'no',
-# 'el', # Greek
- 'cs_CZ',
- 'hu',
- 'pt_BR',
- 'fi',
- 'bg',
- 'ca',
- 'sq',
- ]
-
-def get_available_locales():
- return map(Locale.parse, _KNOWN_LOCALES)
-
-def get_default_locale():
- from pylons import config
- return Locale.parse(config.get('ckan.locale')) or \
- Locale.parse('en')
-
-def set_session_locale(locale):
- if locale not in _KNOWN_LOCALES:
- raise ValueError
- from pylons import session
- session['locale'] = locale
- session.save()
-
-def handle_request(request, tmpl_context):
- from pylons import session
-
- tmpl_context.language = locale = None
- if 'locale' in session:
- locale = Locale.parse(session.get('locale'))
- else:
- requested = [l.replace('-', '_') for l in request.languages]
- locale = Locale.parse(Locale.negotiate(_KNOWN_LOCALES, requested))
-
- if locale is None:
- locale = get_default_locale()
-
- options = [str(locale), locale.language, str(get_default_locale()),
- get_default_locale().language]
- for language in options:
- try:
- set_lang(language)
- tmpl_context.language = language
- except: pass
-
-
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/i18n/ckan.pot
--- a/ckan/i18n/ckan.pot
+++ b/ckan/i18n/ckan.pot
@@ -214,6 +214,7 @@
msgid "Invalid language specified"
msgstr ""
+#. NOTE: Substitute 'English' for the language being translated into.
#: ckan/controllers/home.py:89
msgid "Language has been set to: English"
msgstr ""
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/lib/base.py
--- a/ckan/lib/base.py
+++ b/ckan/lib/base.py
@@ -20,7 +20,7 @@
import ckan
from ckan import authz
-from ckan import i18n
+from ckan.lib import i18n
import ckan.lib.helpers as h
from ckan.plugins import PluginImplementations, IGenshiStreamFilter
from ckan.lib.helpers import json
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/lib/cache.py
--- a/ckan/lib/cache.py
+++ b/ckan/lib/cache.py
@@ -7,6 +7,7 @@
from pylons.decorators.util import get_pylons
from pylons.controllers.util import etag_cache as pylons_etag_cache
import pylons.config
+from ckan.lib.helpers import are_there_flash_messages
__all__ = ["ckan_cache", "get_cache_expires"]
@@ -210,5 +211,6 @@
return cache_expires
def etag_cache(page_hash):
- if cache_validation_enabled:
+ # don't cache if there are flash messages to show (#1321)
+ if cache_validation_enabled and not are_there_flash_messages():
pylons_etag_cache(page_hash)
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/lib/dictization/model_dictize.py
--- a/ckan/lib/dictization/model_dictize.py
+++ b/ckan/lib/dictization/model_dictize.py
@@ -186,7 +186,6 @@
del result_dict['password']
result_dict['display_name'] = user.display_name
- result_dict['email_hash'] = user.email_hash
result_dict['number_of_edits'] = user.number_of_edits()
result_dict['number_administered_packages'] = user.number_administered_packages()
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/lib/helpers.py
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -19,7 +19,7 @@
from routes import url_for, redirect_to
from alphabet_paginate import AlphaPage
from lxml.html import fromstring
-from ckan.i18n import get_available_locales
+from i18n import get_available_locales
@@ -105,6 +105,10 @@
session.save()
return [Message(*m) for m in messages]
+ def are_there_messages(self):
+ from pylons import session
+ return bool(session.get(self.session_key))
+
_flash = _Flash()
def flash_notice(message, allow_html=False):
@@ -116,6 +120,9 @@
def flash_success(message, allow_html=False):
_flash(message, category='success', allow_html=allow_html)
+def are_there_flash_messages():
+ return _flash.are_there_messages()
+
# FIXME: shouldn't have to pass the c object in to this.
def nav_link(c, text, controller, **kwargs):
highlight_actions = kwargs.pop("highlight_actions",
@@ -215,12 +222,6 @@
def icon(name, alt=None):
return literal('<img src="%s" height="16px" width="16px" alt="%s" /> ' % (icon_url(name), alt))
-def gravatar(email_hash, size=100):
- return literal('''<a href="http://gravatar.com" target="_blank">
- <img src="http://gravatar.com/avatar/%s?s=%d&d=mm" />
- </a>''' % (email_hash, size))
-
-
class Page(paginate.Page):
# Curry the pager method of the webhelpers.paginate.Page class, so we have
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/lib/i18n.py
--- /dev/null
+++ b/ckan/lib/i18n.py
@@ -0,0 +1,201 @@
+import os
+
+import pylons
+from pylons.i18n import _, add_fallback, set_lang, gettext, LanguageError
+from pylons.i18n.translation import _get_translator
+from babel import Locale, localedata
+from babel.core import LOCALE_ALIASES
+
+import ckan.i18n
+
+def singleton(cls):
+ instances = {}
+ def getinstance():
+ if cls not in instances:
+ instances[cls] = cls()
+ return instances[cls]
+ return getinstance
+
+i18n_path = os.path.dirname(ckan.i18n.__file__)
+
+ at singleton
+class Locales(object):
+ def __init__(self):
+ from pylons import config
+
+ # Get names of the locales
+ # (must be a better way than scanning for i18n directory?)
+ known_locales = ['en'] + [locale_name for locale_name in os.listdir(i18n_path) \
+ if localedata.exists(locale_name)]
+ self._locale_names, self._default_locale_name = self._work_out_locales(
+ known_locales, config)
+ self._locale_objects = map(Locale.parse, self._locale_names)
+ self._default_locale_object = Locale.parse(self._default_locale_name)
+
+ self._aliases = LOCALE_ALIASES
+ self._aliases['pt'] = 'pt_BR' # Default Portuguese language to
+ # Brazilian territory, since
+ # we don't have a Portuguese territory
+ # translation currently.
+
+ def _work_out_locales(self, known_locales, config_dict):
+ '''Work out the locale_names to offer to the user and the default locale.
+ All locales in this method are strings, not Locale objects.'''
+ # pass in a config_dict rather than ckan.config to make this testable
+
+ # Get default locale
+ assert not config_dict.get('lang'), \
+ '"lang" config option not supported - please use ckan.locale_default instead.'
+ default_locale = config_dict.get('ckan.locale_default') or \
+ config_dict.get('ckan.locale') or \
+ None # in this case, set it later on
+ if default_locale:
+ assert default_locale in known_locales
+
+ # Filter and reorder locales by config options
+ def get_locales_in_config_option(config_option):
+ locales_ = config_dict.get(config_option, '').split()
+ if locales_:
+ unknown_locales = set(locales_) - set(known_locales)
+ assert not unknown_locales, \
+ 'Bad config option %r - locales not found: %s' % \
+ (config_option, unknown_locales)
+ return locales_
+ only_locales_offered = get_locales_in_config_option('ckan.locales_offered')
+ if only_locales_offered:
+ locales = only_locales_offered
+ else:
+ locales = known_locales
+
+ def move_locale_to_start_of_list(locale_):
+ if locale_ not in locales:
+ raise ValueError('Cannot find locale "%s" in locales offered.' % locale_)
+ locales.pop(locales.index(locale_))
+ locales.insert(0, locale_)
+
+ locales_filtered_out = get_locales_in_config_option('ckan.locales_filtered_out')
+ for locale in locales_filtered_out:
+ try:
+ locales.pop(locales.index(locale))
+ except ValueError, e:
+ raise ValueError('Could not filter out locale "%s" from offered locale list "%s": %s') % \
+ (locale, locales, e)
+
+ locale_order = get_locales_in_config_option('ckan.locale_order')
+ if locale_order:
+ for locale in locale_order[::-1]:
+ # bring locale_name to the front
+ try:
+ move_locale_to_start_of_list(locale)
+ except ValueError, e:
+ raise ValueError('Could not process ckan.locale_order options "%s" for offered locale list "%s": %s' % \
+ (locale_order, locales, e))
+ elif default_locale:
+ if default_locale not in locales:
+ raise ValueError('Default locale "%s" is not amongst locales offered: %s' % \
+ (default_locale, locales))
+ # move the default locale to the start of the list
+ try:
+ move_locale_to_start_of_list(default_locale)
+ except ValueError, e:
+ raise ValueError('Could not move default locale "%s" to the start ofthe list of offered locales "%s": %s' % \
+ (default_locale, locales, e))
+
+ assert locales
+
+ if not default_locale:
+ default_locale = locales[0]
+ assert default_locale in locales
+
+ return locales, default_locale
+
+ def get_available_locales(self):
+ '''Returns a list of the locale objects for which translations are
+ available.'''
+ return self._locale_objects
+
+ def get_available_locale_names(self):
+ '''Returns a list of the locale strings for which translations are
+ available.'''
+ return self._locale_names
+
+ def get_default_locale(self):
+ '''Returns the default locale/language as specified in the CKAN
+ config. It is a locale object.'''
+ return self._default_locale_object
+
+ def get_aliases(self):
+ '''Returns a mapping of language aliases, like the Babel LOCALE_ALIASES
+ but with hacks for specific CKAN issues.'''
+ return self._aliases
+
+ def negotiate_known_locale(self, preferred_locales):
+ '''Given a list of preferred locales, this method returns the best
+ match locale object from the known ones.'''
+ assert isinstance(preferred_locales, (tuple, list))
+ preferred_locales = [str(l).replace('-', '_') for l in preferred_locales]
+ return Locale.parse(Locale.negotiate(preferred_locales,
+ self.get_available_locale_names(),
+ aliases=self.get_aliases()
+ ))
+
+def get_available_locales():
+ return Locales().get_available_locales()
+
+def set_session_locale(locale):
+ if locale not in get_available_locales():
+ raise ValueError
+ from pylons import session
+ session['locale'] = locale
+ session.save()
+
+def handle_request(request, tmpl_context):
+ from pylons import session
+
+ # Work out what language to show the page in.
+ locales = [] # Locale objects. Ordered highest preference first.
+ tmpl_context.language = None
+ if session.get('locale'):
+ # First look for locale saved in the session (by home controller)
+ locales.append(Locale.parse(session.get('locale')))
+ else:
+ # Next try languages in the HTTP_ACCEPT_LANGUAGE header
+ locales.append(Locales().negotiate_known_locale(request.languages))
+
+ # Next try the default locale in the CKAN config file
+ locales.append(Locales().get_default_locale())
+
+ locale = set_lang_list(locales)
+ tmpl_context.language = locale.language
+ return locale
+
+def set_lang_list(locales):
+ '''Takes a list of locales (ordered by reducing preference) and tries
+ to set them in order. If one fails then it puts up a flash message and
+ tries the next.'''
+ import ckan.lib.helpers as h
+ failed_locales = set()
+ for locale in locales:
+ # try locales in order of preference until one works
+ try:
+ if str(locale) == 'en':
+ # There is no language file for English, so if we set_lang
+ # we would get an error. Just don't set_lang and finish.
+ break
+ set_lang(str(locale))
+ break
+ except LanguageError, e:
+ if str(locale) not in failed_locales:
+ h.flash_error('Could not change language to %r: %s' % \
+ (str(locale), e))
+ failed_locales.add(str(locale))
+ return locale
+
+def get_lang():
+ '''Returns the current language. Based on babel.i18n.get_lang but works
+ when set_lang has not been run (i.e. still in English).'''
+ langs = pylons.i18n.get_lang()
+ if langs:
+ return langs[0]
+ else:
+ return 'en'
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/lib/munge.py
--- a/ckan/lib/munge.py
+++ b/ckan/lib/munge.py
@@ -19,6 +19,33 @@
name = re.sub('[^a-zA-Z0-9-_]', '', name).lower()
return name
+def munge_title_to_name(name):
+ '''Munge a title into a name.
+ '''
+ # remove foreign accents
+ if isinstance(name, unicode):
+ name = substitute_ascii_equivalents(name)
+ # convert spaces and separators
+ name = re.sub('[ .:/]', '-', name)
+ # take out not-allowed characters
+ name = re.sub('[^a-zA-Z0-9-_]', '', name).lower()
+ # remove doubles
+ name = re.sub('--', '-', name)
+ # remove leading or trailing hyphens
+ name = name.strip('-')
+ # if longer than max_length, keep last word if a year
+ max_length = model.PACKAGE_NAME_MAX_LENGTH - 5
+ # (make length less than max, in case we need a few for '_' chars
+ # to de-clash names.)
+ if len(name) > max_length:
+ year_match = re.match('.*?[_-]((?:\d{2,4}[-/])?\d{2,4})$', name)
+ if year_match:
+ year = year_match.groups()[0]
+ name = '%s-%s' % (name[:(max_length-len(year)-1)], year)
+ else:
+ name = name[:max_length]
+ return name
+
def substitute_ascii_equivalents(text_unicode):
# Method taken from: http://code.activestate.com/recipes/251871/
"""This takes a UNICODE string and replaces Latin-1 characters with
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/model/user.py
--- a/ckan/model/user.py
+++ b/ckan/model/user.py
@@ -49,14 +49,6 @@
if self.fullname is not None and len(self.fullname.strip()) > 0:
return self.fullname
return self.name
-
- @property
- def email_hash(self):
- import hashlib
- e = ''
- if self.email:
- e = self.email.strip().lower()
- return hashlib.md5(e).hexdigest()
def get_reference_preferred_for_uri(self):
'''Returns a reference (e.g. name, id, openid) for this user
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/public/css/style.css
--- a/ckan/public/css/style.css
+++ b/ckan/public/css/style.css
@@ -561,14 +561,6 @@
/* ================== */
/* = User Read page = */
/* ================== */
-.gravatar {
- border: 1px solid #777;
- padding: 1px;
- width: 120px;
- height: 120px;
- margin: 0 auto 10px auto;
-}
-
/* ========================= */
/* = Dataset Snapshot View = */
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/templates/user/layout.html
--- a/ckan/templates/user/layout.html
+++ b/ckan/templates/user/layout.html
@@ -6,26 +6,11 @@
><py:match path="minornavigation">
- <py:if test="c.is_myself">
- <ul class="tabbed">
- <li py:attrs="{'class':'current-tab'} if c.action=='read' else {}"><a href="${h.url_for(controller='user', action='read')}">My Profile</a></li>
- <li py:attrs="{'class':'current-tab'} if c.action=='edit' else {}"><a href="${h.url_for(controller='user', action='edit')}">Edit Profile</a></li>
- <li><a href="${h.url_for('/user/logout')}">Log out</a></li>
- </ul>
- </py:if>
- <py:if test="not c.is_myself">
- <py:if test="c.id">
- <ul class="tabbed">
- <li py:attrs="{'class':'current-tab'} if c.action=='read' else {}"><a href="${h.url_for(controller='user', action='read')}">View Profile</a></li>
- </ul>
- </py:if>
- <py:if test="not c.id">
- <ul class="tabbed">
- <li py:attrs="{'class':'current-tab'} if c.action=='login' else {}"><a href="${h.url_for(controller='user', action='login')}">Login</a></li>
- <li py:attrs="{'class':'current-tab'} if c.action=='register' else {}"><a href="${h.url_for('register')}">Register Account</a></li>
- </ul>
- </py:if>
- </py:if>
+ <ul class="tabbed" py:if="c.is_myself">
+ <li py:attrs="{'class':'current-tab'} if c.action=='read' else {}"><a href="${h.url_for(controller='user', action='read')}">My Profile</a></li>
+ <li py:attrs="{'class':'current-tab'} if c.action=='edit' else {}"><a href="${h.url_for(controller='user', action='edit')}">Edit Profile</a></li>
+ <li><a href="${h.url_for('/user/logout')}">Log out</a></li>
+ </ul></py:match>
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/templates/user/login.html
--- a/ckan/templates/user/login.html
+++ b/ckan/templates/user/login.html
@@ -34,9 +34,10 @@
<input type="hidden" name="remember" value="1576800000" /><br/></fieldset>
- <input name="s" id="s" type="submit" class="pretty-button primary" value="Sign In"/>
- —
+ ${h.submit('s', _('Login'))} —
<a href="${h.url_for('reset')}">Forgot your password?</a>
+ —
+ ${h.link_to(_('Not yet registered?'), h.url_for(action='register'))}
</form><br/><!-- Simple OpenID Selector -->
@@ -65,7 +66,7 @@
</p></div></fieldset>
- <input id="openid_submit" type="submit" class="pretty-button primary" value="Sign in with OpenID"/>
+ <input id="openid_submit" type="submit" value="Sign in"/></form></div><xi:include href="layout.html" />
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/templates/user/logout.html
--- a/ckan/templates/user/logout.html
+++ b/ckan/templates/user/logout.html
@@ -4,10 +4,8 @@
<py:def function="page_title">Logout - User</py:def>
- <py:def function="page_title">Logout</py:def>
- <py:def function="page_heading">Logout from ${g.site_title}</py:def>
-
<div py:match="content">
+ <h2>Logout</h2><p>You have logged out successfully.</p></div>
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/templates/user/new_user_form.html
--- a/ckan/templates/user/new_user_form.html
+++ b/ckan/templates/user/new_user_form.html
@@ -45,5 +45,5 @@
</dd></dl>
- <input id="save" name="save" type="submit" class="pretty-button primary" value="Register now »" />
+ <input id="save" name="save" type="submit" value="Register now »" /></form>
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/templates/user/read.html
--- a/ckan/templates/user/read.html
+++ b/ckan/templates/user/read.html
@@ -7,9 +7,6 @@
<py:def function="body_class">user-view</py:def><py:match path="primarysidebar">
- <div class="gravatar">
- ${h.gravatar(c.user_dict['email_hash'],120)}
- </div><li class="widget-container widget_text" py:if="not c.hide_welcome_message"><h3>Activity</h3><ul>
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/templates/user/request_reset.html
--- a/ckan/templates/user/request_reset.html
+++ b/ckan/templates/user/request_reset.html
@@ -14,7 +14,7 @@
<input type="text" name="user" value="" /><br/></fieldset><div>
- <input type="submit" id="reset" name="reset" class="pretty-button primary" value="Reset Password" />
+ ${h.submit('reset', _('Reset password'))}
</div></form></div>
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/tests/functional/test_home.py
--- a/ckan/tests/functional/test_home.py
+++ b/ckan/tests/functional/test_home.py
@@ -1,4 +1,5 @@
-from pylons import c
+from pylons import c, session
+from pylons.i18n import set_lang
from ckan.lib.create_test_data import CreateTestData
from ckan.controllers.home import HomeController
@@ -21,28 +22,44 @@
def teardown_class(self):
model.repo.rebuild_db()
+ def clear_language_setting(self):
+ self.app.cookies = {}
+
def test_home_page(self):
offset = url_for('home')
res = self.app.get(offset)
- print res
assert 'Add a dataset' in res
+ assert 'Could not change language' not in res
def test_calculate_etag_hash(self):
+ # anything that changes the home page appearance should change the
+ # etag hash
c.user = 'test user'
get_hash = HomeController._home_cache_key
- hash_1 = get_hash()
- hash_2 = get_hash()
- self.assert_equal(hash_1, hash_2)
+ hashes = [get_hash(), get_hash()]
+ self.assert_equal(hashes[0], hashes[1])
+ def assert_hash_changed(hashes):
+ current_hash = get_hash()
+ assert current_hash != hashes[-1]
+ hashes.append(current_hash)
+
+ # login as a different user
c.user = 'another user'
- hash_3 = get_hash()
- assert hash_2 != hash_3
+ assert_hash_changed(hashes)
- model.repo.new_revision()
- model.Session.add(model.Package(name=u'test_etag'))
+ # add a package to a group
+ rev = model.repo.new_revision()
+ model.Group.by_name(u'roger').add_package_by_name(u'warandpeace')
model.repo.commit_and_remove()
- hash_4 = get_hash()
- assert hash_3 != hash_4
+ assert_hash_changed(hashes)
+
+ # flash message is not cached, but this is done in ckan/lib/cache
+
+ # I can't get set_lang to work and deliver correct
+ # result to get_lang, so leaving it commented
+## set_lang('fr')
+## assert_hash_changed(hashes)
@search_related
def test_packages_link(self):
@@ -68,6 +85,36 @@
res = self.app.get(offset)
assert '<strong>TEST TEMPLATE_FOOTER_END TEST</strong>'
+ def test_locale_detect(self):
+ offset = url_for('home')
+ self.clear_language_setting()
+ res = self.app.get(offset, headers={'Accept-Language': 'de,pt-br,en'})
+ try:
+ assert 'Willkommen' in res.body, res.body
+ finally:
+ self.clear_language_setting()
+
+ def test_locale_negotiate(self):
+ offset = url_for('home')
+ self.clear_language_setting()
+ res = self.app.get(offset, headers={'Accept-Language': 'fr-ca'})
+ # Request for French with Canadian territory should negotiate to
+ # just 'fr'
+ try:
+ assert 'propos' in res.body, res.body
+ finally:
+ self.clear_language_setting()
+
+ def test_locale_negotiate_pt(self):
+ offset = url_for('home')
+ self.clear_language_setting()
+ res = self.app.get(offset, headers={'Accept-Language': 'pt'})
+ # Request for Portuguese should find pt_BR because of our alias hack
+ try:
+ assert 'Bem-vindo' in res.body, res.body
+ finally:
+ self.clear_language_setting()
+
def test_locale_change(self):
offset = url_for('home')
res = self.app.get(offset)
@@ -76,7 +123,7 @@
res = res.follow()
assert 'Willkommen' in res.body
finally:
- res = res.click('English')
+ self.clear_language_setting()
def test_locale_change_invalid(self):
offset = url_for(controller='home', action='locale', locale='')
@@ -111,9 +158,7 @@
res = res.goto(href)
assert res.status == 200, res.status # doesn't redirect
finally:
- offset = url_for('home')
- res = self.app.get(offset)
- res = res.click('English')
+ self.clear_language_setting()
class TestDatabaseNotInitialised(TestController):
@classmethod
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/tests/lib/test_helpers.py
--- a/ckan/tests/lib/test_helpers.py
+++ b/ckan/tests/lib/test_helpers.py
@@ -50,15 +50,3 @@
two_months_ago_str = h.datetime_to_date_str(two_months_ago)
res = h.time_ago_in_words_from_str(two_months_ago_str)
assert_equal(res, '2 months')
-
- def test_gravatar(self):
- email = 'zephod at gmail.com'
- expected =['<a href="http://gravatar.com" target="_blank">', '<img src="http://gravatar.com/avatar/7856421db6a63efa5b248909c472fbd2?s=200&d=mm" />', '</a>']
- # Hash the email address
- import hashlib
- email_hash = hashlib.md5(email).hexdigest()
- res = h.gravatar(email_hash, 200)
- for e in expected:
- assert e in res, e
-
-
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan/tests/lib/test_i18n.py
--- /dev/null
+++ b/ckan/tests/lib/test_i18n.py
@@ -0,0 +1,154 @@
+from nose.tools import assert_equal, assert_raises
+from babel import Locale
+from pylons import config, session
+import pylons
+from pylons.i18n import get_lang
+
+from ckan.lib.i18n import Locales, set_session_locale, set_lang
+import ckan.lib.i18n
+
+from ckan.tests.pylons_controller import PylonsTestCase, TestSession
+
+class TestLocales:
+ def test_work_out_locales__thedatahub(self):
+ # as it is (roughly) on thedatahub.org
+ locales, default = Locales()._work_out_locales(
+ ['en', 'fr', 'de'],
+ {'ckan.locale': 'en'})
+ assert_equal(locales, ['en', 'fr', 'de'])
+ assert_equal(default, 'en')
+
+ def test_work_out_locales__france(self):
+ # as it is (roughly) on a foreign language site
+ locales, default = Locales()._work_out_locales(
+ ['en', 'fr', 'de'],
+ {'ckan.locale': 'fr'})
+ # fr moved to start of the list
+ assert_equal(locales, ['fr', 'en', 'de'])
+ assert_equal(default, 'fr')
+
+ def test_work_out_locales__locales_offered(self):
+ locales, default = Locales()._work_out_locales(
+ ['en', 'fr', 'de'],
+ {'ckan.locales_offered': 'fr de'})
+ assert_equal(locales, ['fr', 'de'])
+ assert_equal(default, 'fr')
+
+ def test_work_out_locales__locales_order(self):
+ locales, default = Locales()._work_out_locales(
+ ['en', 'fr', 'de'],
+ {'ckan.locale': 'fr',
+ 'ckan.locale_order': 'de fr en'})
+ assert_equal(locales, ['de', 'fr', 'en'])
+ assert_equal(default, 'fr')
+
+ def test_work_out_locales__locales_filtered_out(self):
+ locales, default = Locales()._work_out_locales(
+ ['en', 'fr', 'de'],
+ {'ckan.locale': 'fr',
+ 'ckan.locales_filtered_out': 'de'})
+ assert_equal(locales, ['fr', 'en'])
+ assert_equal(default, 'fr')
+
+ def test_work_out_locales__default(self):
+ # don't specify default lang and it is not en,
+ # so default to next in list.
+ locales, default = Locales()._work_out_locales(
+ ['en', 'fr', 'de'],
+ {'ckan.locale': 'fr',
+ 'ckan.locales_filtered_out': 'en'})
+ assert_equal(locales, ['fr', 'de'])
+ assert_equal(default, 'fr')
+
+ def test_work_out_locales__bad_default(self):
+ assert_raises(ValueError, Locales()._work_out_locales,
+ ['en', 'fr', 'de'],
+ {'ckan.locale': 'en',
+ 'ckan.locales_offered': 'fr de'})
+
+ def test_get_available_locales(self):
+ locales = Locales().get_available_locales()
+ assert len(locales) > 5, locales
+ locale = locales[0]
+ assert isinstance(locale, Locale)
+
+ locales_str = set([str(locale) for locale in locales])
+ langs = set([locale.language for locale in locales])
+ assert set(('en', 'de', 'cs_CZ')) < locales_str, locales_str
+ assert set(('en', 'de', 'cs')) < langs, langs
+
+ def test_default_locale(self):
+ # This should be setup in test-core.ini
+ assert_equal(config.get('ckan.locale_default'), 'en')
+ default_locale = Locales().get_default_locale()
+ assert isinstance(default_locale, Locale)
+ assert_equal(default_locale.language, 'en')
+
+ def test_negotiate_known_locale(self):
+ # check exact matches always work
+ locales = Locales().get_available_locales()
+ for locale in locales:
+ result = Locales().negotiate_known_locale([locale])
+ assert_equal(result, locale)
+
+ assert_equal(Locales().negotiate_known_locale(['en_US']), 'en')
+ assert_equal(Locales().negotiate_known_locale(['en_AU']), 'en')
+ assert_equal(Locales().negotiate_known_locale(['es_ES']), 'es')
+ assert_equal(Locales().negotiate_known_locale(['pt']), 'pt_BR')
+
+class TestI18n(PylonsTestCase):
+ def test_set_session_locale(self):
+ set_session_locale('en')
+ assert_equal(session['locale'], 'en')
+
+ set_session_locale('fr')
+ assert_equal(session['locale'], 'fr')
+
+ def handle_request(self, session_language=None, languages_header=[]):
+ session['locale'] = session_language
+ class FakePylons:
+ translator = None
+ class FakeRequest:
+ # Populated from the HTTP_ACCEPT_LANGUAGE header normally
+ languages = languages_header
+ # Stores details of the translator
+ environ = {'pylons.pylons': FakePylons()}
+ request = FakeRequest()
+ real_pylons_request = pylons.request
+ try:
+ pylons.request = request # for set_lang to work
+ class FakeTmplContext:
+ language = None # gets filled in by handle_request
+ tmpl_context = FakeTmplContext()
+ ckan.lib.i18n.handle_request(request, tmpl_context)
+ return tmpl_context.language # the language that got set
+ finally:
+ pylons.request = real_pylons_request
+
+ def test_handle_request__default(self):
+ assert_equal(self.handle_request(),
+ 'en')
+
+ def test_handle_request__session(self):
+ assert_equal(self.handle_request(session_language='fr'),
+ 'fr')
+
+ def test_handle_request__header(self):
+ assert_equal(self.handle_request(languages_header=['de']),
+ 'de')
+
+ def test_handle_request__header_negotiate(self):
+ # Language so is not an option, so reverts to next one
+ assert_equal(self.handle_request(languages_header=['so_KE', 'de']),
+ 'de')
+
+ def test_handle_request__header_but_defaults(self):
+ # Language so is not an option, so reverts to default
+ assert_equal(self.handle_request(languages_header=['so_KE']),
+ 'en')
+
+ def test_handle_request__header_territory(self):
+ # Request for specific version of German ends up simply as de.
+ assert_equal(self.handle_request(languages_header=['fr_CA', 'en']),
+ 'fr')
+
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan_deb/DEBIAN/control.template
--- a/ckan_deb/DEBIAN/control.template
+++ b/ckan_deb/DEBIAN/control.template
@@ -5,7 +5,7 @@
Maintainer: James Gardner <james.gardner at okfn.org>
Section: main/web
Priority: extra
-Depends: python-ckan, postgresql-8.4, apache2, libapache2-mod-wsgi, python-apachemiddleware, python-virtualenv, python-pip
+Depends: python-ckan, postgresql-8.4, apache2, libapache2-mod-wsgi, python-virtualenv, python-pip, solr-jetty
Homepage: http://ckan.org
Description: ckan
Common files useful for managing CKAN installations
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan_deb/usr/bin/ckan-create-instance
--- a/ckan_deb/usr/bin/ckan-create-instance
+++ b/ckan_deb/usr/bin/ckan-create-instance
@@ -43,6 +43,8 @@
echo "Ensuring users and groups are set up correctly ..."
ckan_ensure_users_and_groups ${INSTANCE}
+chown ckan${INSTANCE}:ckan${INSTANCE} /etc/ckan/${INSTANCE}/install_settings.sh
+
echo "Ensuring directories exist for ${INSTANCE} CKAN INSTANCE ..."
ckan_make_ckan_directories ${INSTANCE}
@@ -75,6 +77,17 @@
echo "Setting the password of the ${INSTANCE} user in PostgreSQL"
ckan_add_or_replace_database_user ${INSTANCE} ${CKAN_DB_PASSWORD}
+# Solr support
+echo "Setting up Solr ..."
+sed \
+ -e "s,NO_START=1,NO_START=0," \
+ -e "s,#JETTY_HOST=\$(uname -n),JETTY_HOST=127.0.0.1," \
+ -e "s,#JETTY_PORT=8080,JETTY_PORT=8983," \
+ -i /etc/default/jetty
+mv /usr/share/solr/conf/schema.xml /usr/share/solr/conf/schema.xml.`date --utc "+%Y-%m-%d_%T"`.bak
+ln -s /usr/lib/pymodules/python2.6/ckan/config/schema.xml /usr/share/solr/conf/schema.xml
+service jetty start
+
# Create the config file
echo "Creating/overwriting the config for CKAN ... "
# We use the actual password in PostgreSQL in case any non-sense has gone on
@@ -121,7 +134,7 @@
# Install the new crontab
echo "Enabling crontab for the ckan${INSTANCE} user ..."
PACKAGED_CRONJOB="/tmp/${INSTANCE}-cronjob"
-cat <<EOF > ${PACKAGED_CRONJOB}
+cat << EOF > ${PACKAGED_CRONJOB}
# WARNING: Do not edit these cron tabs, they will be overwritten any time
# the ckan INSTANCE package is upgraded
# QUESTION: Should email reports be sent to root?
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 ckan_deb/usr/lib/ckan/common.sh
--- a/ckan_deb/usr/lib/ckan/common.sh
+++ b/ckan_deb/usr/lib/ckan/common.sh
@@ -45,12 +45,12 @@
INSTANCE=$1
COMMAND_OUTPUT=`cat /etc/group | grep "ckan${INSTANCE}:"`
if ! [[ "$COMMAND_OUTPUT" =~ "ckan${INSTANCE}:" ]] ; then
- echo "Crating the 'ckan${INSTANCE}' group ..."
+ echo "Creating the 'ckan${INSTANCE}' group ..."
sudo groupadd --system "ckan${INSTANCE}"
fi
COMMAND_OUTPUT=`cat /etc/passwd | grep "ckan${INSTANCE}:"`
if ! [[ "$COMMAND_OUTPUT" =~ "ckan${INSTANCE}:" ]] ; then
- echo "Crating the 'ckan${INSTANCE}' user ..."
+ echo "Creating the 'ckan${INSTANCE}' user ..."
sudo useradd --system --gid "ckan${INSTANCE}" --home /var/lib/ckan/${INSTANCE} -M --shell /usr/sbin/nologin ckan${INSTANCE}
fi
}
@@ -84,6 +84,7 @@
cp -n /usr/share/pyshared/ckan/config/who.ini /etc/ckan/${INSTANCE}/who.ini
sed -e "s,%(here)s,/var/lib/ckan/${INSTANCE}," \
-i /etc/ckan/${INSTANCE}/who.ini
+ chown ckan${INSTANCE}:ckan${INSTANCE} /etc/ckan/${INSTANCE}/who.ini
fi
fi
}
@@ -109,7 +110,11 @@
-e "s,^\(sqlalchemy.url\)[ =].*,\1 = postgresql://${INSTANCE}:${password}@localhost/${INSTANCE}," \
-e "s,ckan\.site_logo,\#ckan.site_logo," \
-e "s,ckan\.log,/var/log/ckan/${INSTANCE}/${INSTANCE}.log," \
+ -e "s,ckan\.site_id,${INSTANCE}," \
+ -e "s,ckan\.site_description,${INSTANCE}," \
+ -e "s,#solr_url = http://127.0.0.1:8983/solr,solr_url = http://127.0.0.1:8983/solr," \
-i /etc/ckan/${INSTANCE}/${INSTANCE}.ini
+ sudo chown ckan${INSTANCE}:ckan${INSTANCE} /etc/ckan/${INSTANCE}/${INSTANCE}.ini
fi
}
@@ -147,6 +152,7 @@
if ! [[ "$COMMAND_OUTPUT" =~ ${INSTANCE} ]] ; then
echo "Creating the database ..."
sudo -u postgres createdb -O ${INSTANCE} ${INSTANCE}
+ paster --plugin=ckan db init --config=/etc/ckan/${INSTANCE}/${INSTANCE}.ini
fi
fi
}
@@ -161,7 +167,11 @@
INSTANCE=$1
if [ ! -f "/var/lib/ckan/${INSTANCE}/wsgi.py" ]
then
- sudo virtualenv --setuptools /var/lib/ckan/${INSTANCE}/pyenv
+ sudo mkdir /var/lib/ckan/${INSTANCE}/pyenv
+ sudo chown -R ckan${INSTANCE}:ckan${INSTANCE} /var/lib/ckan/${INSTANCE}/pyenv
+ sudo -u ckan${INSTANCE} virtualenv --setuptools /var/lib/ckan/${INSTANCE}/pyenv
+ echo "Attempting to install Pip 1.0 from pypi.python.org into pyenv to be used for extensions ..."
+ sudo -u ckan${INSTANCE} /var/lib/ckan/${INSTANCE}/pyenv/bin/easy_install pip -U "pip>=1.0" "pip<=1.0.99"
cat <<- EOF > /var/lib/ckan/${INSTANCE}/wsgi.py
import os
instance_dir = '/var/lib/ckan/${INSTANCE}'
@@ -235,6 +245,10 @@
# pass authorization info on (needed for rest api)
WSGIPassAuthorization On
+ # Deploy as a daemon (avoids conflicts between CKAN instances)
+ # WSGIDaemonProcess ${INSTANCE} display-name=${INSTANCE} processes=4 threads=15 maximum-requests=10000
+ # WSGIProcessGroup ${INSTANCE}
+
ErrorLog /var/log/apache2/${INSTANCE}.error.log
CustomLog /var/log/apache2/${INSTANCE}.custom.log combined
EOF
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/api.rst
--- a/doc/api.rst
+++ b/doc/api.rst
@@ -556,4 +556,4 @@
Action API
~~~~~~~~~~
-See:
\ No newline at end of file
+See: :doc:`apiv3`
\ No newline at end of file
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/common-error-messages.rst
--- /dev/null
+++ b/doc/common-error-messages.rst
@@ -0,0 +1,94 @@
+Common error messages
+---------------------
+
+Whether a developer runs CKAN using paster or going through CKAN test suite, there are a number of error messages seen that are the result of setup problems. As people experience them, please add them to the list here.
+
+These instructions assume you have the python virtual environment enabled (``. pyenv/bin/activate``) and the current directory is the top of the ckan source, which is probably: ``../pyenv/src/ckan/``.
+
+``nose.config.ConfigError: Error reading config file 'setup.cfg': no such option 'with-pylons'``
+================================================================================================
+
+ This error can result when you run nosetests for two reasons:
+
+ 1. Pylons nose plugin failed to run. If this is the case, then within a couple of lines of running `nosetests` you'll see this warning: `Unable to load plugin pylons` followed by an error message. Fix the error here first.
+
+ 2. The Python module 'Pylons' is not installed into you Python environment. Confirm this with::
+
+ python -c "import pylons"
+
+``OperationalError: (OperationalError) no such function: plainto_tsquery ...``
+==============================================================================
+
+ This error usually results from running a test which involves search functionality, which requires using a PostgreSQL database, but another (such as SQLite) is configured. The particular test is either missing a `@search_related` decorator or there is a mixup with the test configuration files leading to the wrong database being used.
+
+``ImportError: No module named worker``
+=======================================
+
+ The python entry point for the worker has not been generated. This occurs during the 'pip install' of the CKAN source, and needs to be done again if switching from older code that didn't have it. To recitify it::
+
+ python setup.py egg_info
+
+``ImportError: cannot import name get_backend``
+===============================================
+
+ This can be caused by an out of date pyc file. Delete all your pyc files and start again::
+
+ find . -name "*.pyc" | xargs rm
+
+``ImportError: cannot import name UnicodeMultiDict``
+====================================================
+
+ This is caused by using a version of WebOb that is too new (it has deprecated UnicodeMultiDict). Check the version like this (ensure you have activated your python environment first)::
+
+ pip freeze | grep -i webob
+
+ Now install the version specified in requires/lucid_present.txt. e.g.::
+
+ pip install webob==1.0.8
+
+``nosetests: error: no such option: --ckan``
+============================================
+
+ Nose is either unable to find ckan/ckan_nose_plugin.py in the python environment it is running in, or there is an error loading it. If there is an error, this will surface it::
+
+ nosetests --version
+
+ There are a few things to try to remedy this:
+
+ Commonly this is because the nosetests isn't running in the python environment. You need to have nose actually installed in the python environment. To see which you are running, do this::
+
+ which nosetests
+
+ If you have activated the environment and this still reports ``/usr/bin/nosetests`` then you need to::
+
+ pip install --ignore-installed nose
+
+ If ``nose --version`` still fails, ensure that ckan is installed in your environment::
+
+ cd pyenv/src/ckan
+ python setup.py develop
+
+ One final check - the version of nose should be at least 1.0. Check with::
+
+ pip freeze | grep -i nose
+
+``AttributeError: 'unicode' object has no attribute 'items'`` (Cookie.py)
+=========================================================================
+
+This can be caused by using repoze.who version 1.0.18 when 1.0.19 is required. Check what you have with::
+
+ pip freeze | grep -i repoze.who=
+
+See what version you need with::
+
+ grep -f requires/*.txt |grep repoze\.who=
+
+Then install the version you need (having activated the environment)::
+
+ pip install repoze.who==1.0.19
+
+``AttributeError: 'module' object has no attribute 'BigInteger'``
+=================================================================
+
+The sqlalchemy module version is too old.
+
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/configuration.rst
--- a/doc/configuration.rst
+++ b/doc/configuration.rst
@@ -194,18 +194,54 @@
-----------------------------
.. index::
- single: lang
+ single: ckan.locale_default
-lang
-^^^^
+ckan.locale_default
+^^^^^^^^^^^^^^^^^^^
Example::
- lang=de
+ ckan.locale_default=de
Default value: ``en`` (English)
-Use this to specify the language of the text displayed in the CKAN web UI. This requires a suitable `mo` file installed for the language. For more information on internationalization, see :doc:`i18n`.
+Use this to specify the locale (language of the text) displayed in the CKAN Web UI. This requires a suitable `mo` file installed for the locale in the ckan/i18n. For more information on internationalization, see :doc:`i18n`. If you don't specify a default locale, then it will default to the first locale offered, which is by default English (alter that with `ckan.locales_offered` and `ckan.locales_filtered_out`.
+
+.. note: In versions of CKAN before 1.5, the settings used for this was variously `lang` or `ckan.locale`, which have now been deprecated in favour of `ckan.locale_default`.
+
+ckan.locales_offered
+^^^^^^^^^^^^^^^^^^^^
+
+Example::
+
+ ckan.locales_offered=en de fr
+
+Default value: (none)
+
+By default, all locales found in the ckan/i18n directory will be offered to the user. To only offer a subset of these, list them under this option. The ordering of the locales is preserved when offered to the user.
+
+ckan.locales_filtered_out
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Example::
+
+ ckan.locales_filtered_out=pl ru
+
+Default value: (none)
+
+If you want to not offer particular locales to the user, then list them here to have them removed from the options.
+
+ckan.locale_order
+^^^^^^^^^^^^^^^^^
+
+Example::
+
+ ckan.locale_order=fr de
+
+Default value: (none)
+
+If you want to specify the ordering of all or some of the locales as they are offered to the user, then specify them here in the required order. Any locales that are available but not specified in this option, will still be offered at the end of the list.
+
Theming Settings
----------------
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/i18n.rst
--- a/doc/i18n.rst
+++ b/doc/i18n.rst
@@ -2,16 +2,18 @@
Internationalize CKAN
=====================
-CKAN is used in many countries, and adding a new language is a simple process.
+CKAN is used in many countries, and adding a new language to the web interface is a simple process.
+
+.. Note: Storing metadata field values in more than one language is a separate topic. This is achieved by storing the translations in extra fields. A custom dataset form and dataset display template are recommended. Ask the CKAN team for more information.
Supported Languages
===================
CKAN already supports numerous languages. To check whether your language is supported, look in the source at ``ckan/i18n`` for translation files. Languages are named using two-letter ISO language codes (e.g. ``es``, ``de``).
-If your language is present, you can switch language simply by setting the ``lang`` option in your CKAN config file, as described in :ref:`config-i18n`. For example, to switch to German::
+If your language is present, you can switch the default language simply by setting the ``ckan.locale_default`` option in your CKAN config file, as described in :ref:`config-i18n`. For example, to switch to German::
- lang=de
+ ckan.locale_default=de
If your language is not supported yet, the remainder of this section section provides instructions on how to prepare a translation file and add it to CKAN.
@@ -20,8 +22,36 @@
If you want to add an entirely new language to CKAN, you have two options.
-* :ref:`i18n-manual`. Creating translation files manually.
* :ref:`i18n-transifex`. Creating translation files using Transifex, the open source translation software.
+* :ref:`i18n-manual`. Creating translation files manually.
+
+
+.. _i18n-transifex:
+
+Transifex Setup
+---------------
+
+Transifex, the open translation platform, provides a simple web interface for writing translations and is widely used for CKAN internationalization.
+
+Using Transifex makes it easier to handle collaboration, with an online editor that makes the process more accessible.
+
+Existing CKAN translation projects can be found at: https://www.transifex.net/projects/p/ckan/teams/
+
+Updated translations are automatically pushed to https://bitbucket.org/bboissin/ckan-i18n and these can be compiled and placed on CKAN servers by the server administrators.
+
+Transifex Administration
+++++++++++++++++++++++++
+
+The Transifex workflow is as follows:
+
+* Install transifex command-line utilities
+* ``tx init`` in CKAN to connect to Transifex
+* Run ``python setup.py extract_messages`` on the CKAN source
+* Upload the local .pot file via command-line ``tx push``
+* Get people to complete translations on Transifex
+* Pull locale .po files via ``tx pull``
+* ``python setup.py compile_catalog``
+* Mercurial Commit and push po and mo files
.. _i18n-manual:
@@ -29,7 +59,9 @@
Manual Setup
------------
-If you prefer not to use Transifex, you can create translation files manually.
+If you prefer not to use Transifex, you can create translation files manually.
+
+.. note:: Please keep the CKAN core developers aware of new languages created in this way.
All the English strings in CKAN are extracted into the ``ckan.pot`` file, which can be found in ``ckan/i18n``.
@@ -112,32 +144,5 @@
6. Configure the Language
+++++++++++++++++++++++++
-Finally, once the mo file is in place, you can switch between the installed languages using the ``lang`` option in the CKAN config file, as described in :ref:`config-i18n`.
+Finally, once the mo file is in place, you can switch between the installed languages using the ``ckan.locale`` option in the CKAN config file, as described in :ref:`config-i18n`.
-
-.. _i18n-transifex:
-
-Transifex Setup
----------------
-
-Transifxes, the open translation platform, provides a simple web interface for writing translations and is widely used for CKAN internationalization.
-
-Using Transifex makes it easier to handle collaboration, with an online editor that makes the process more accessible.
-
-Existing CKAN translation projects can be found at: https://www.transifex.net/projects/p/ckan/teams/
-
-Updated translations are automatically pushed to https://bitbucket.org/bboissin/ckan-i18n and these can be compiled and placed on CKAN servers by the server administrators.
-
-Transifex Administration
-++++++++++++++++++++++++
-
-The Transifex workflow is as follows:
-
-* Install transifex command-line utilities
-* ``tx init`` in CKAN to connect to Transifex
-* Run ``python setup.py extract_messages`` on the CKAN source
-* Upload the local .pot file via command-line ``tx push``
-* Get people to complete translations on Transifex
-* Pull locale .po files via ``tx pull``
-* ``python setup.py compile_catalog``
-* Commit and push po and mo files
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/index.rst
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -33,6 +33,7 @@
configuration
api
test
+ common-error-messages
buildbot
about
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/install-from-source.rst
--- a/doc/install-from-source.rst
+++ b/doc/install-from-source.rst
@@ -295,3 +295,8 @@
Finally, make sure that tests pass, as described in :ref:`basic-tests`.
You can now proceed to :doc:`post-installation`.
+
+Common error messages
+---------------------
+
+Consult :doc:`common-error-messages` for solutions to a range of error messages seen during setup.
\ No newline at end of file
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 doc/test.rst
--- a/doc/test.rst
+++ b/doc/test.rst
@@ -99,66 +99,9 @@
A common error when wanting to run tests against a particular database is to change ``sqlalchemy.url`` in ``test.ini`` or ``test-core.ini``. The problem is that these are versioned files and people have checked in these by mistake, creating problems for other developers and the CKAN buildbot. This is easily avoided by only changing ``sqlalchemy.url`` in your local ``development.ini`` and testing ``--with-pylons=test-core.ini``.
-Common problems running tests
------------------------------
+Common error messages
+---------------------
-* `nose.config.ConfigError: Error reading config file 'setup.cfg': no such option 'with-pylons'`
+Often errors are due to set-up errors. Always refer to the CKAN buildbot as the canonical build.
- This error can result when you run nosetests for two reasons:
-
- 1. Pylons nose plugin failed to run. If this is the case, then within a couple of lines of running `nosetests` you'll see this warning: `Unable to load plugin pylons` followed by an error message. Fix the error here first.
-
- 2. The Python module 'Pylons' is not installed into you Python environment. Confirm this with::
-
- python -c "import pylons"
-
-* `OperationalError: (OperationalError) no such function: plainto_tsquery ...`
-
- This error usually results from running a test which involves search functionality, which requires using a PostgreSQL database, but another (such as SQLite) is configured. The particular test is either missing a `@search_related` decorator or there is a mixup with the test configuration files leading to the wrong database being used.
-
-* `ImportError: No module named worker`
-
- The python entry point for the worker has not been generated. This occurs during the 'pip install' of the CKAN source, and needs to be done again if switching from older code that didn't have it. To recitify it::
-
- python setup.py egg_info
-
-* `ImportError: cannot import name get_backend`
-
- This can be caused by an out of date pyc file. Delete all your pyc files and start again::
-
- find . -name "*.pyc" | xargs rm
-
-* `ImportError: cannot import name UnicodeMultiDict`
-
- This is caused by using a version of WebOb that is too new (it has deprecated UnicodeMultiDict). Check the version like this (ensure you have activated your python environment first)::
-
- pip freeze | grep -i webob
-
- Now install the version specified in requires/lucid_present.txt. e.g.::
-
- pip install webob==1.0.8
-
-* `nosetests: error: no such option: --ckan`
-
- Nose is either unable to find ckan/ckan_nose_plugin.py in the python environment it is running in, or there is an error loading it. If there is an error, this will surface it::
-
- nosetests --version
-
- There are a few things to try to remedy this:
-
- Commonly this is because the nosetests isn't running in the python environment. You need to have nose actually installed in the python environment. To see which you are running, do this::
-
- which nosetests
-
- If you have activated the environment and this still reports ``/usr/bin/nosetests`` then you need to::
-
- pip install --ignore-installed nose
-
- If ``nose --version`` still fails, ensure that ckan is installed in your environment::
-
- cd pyenv/src/ckan
- python setup.py develop
-
- One final check - the version of nose should be at least 1.0. Check with::
-
- pip freeze | grep -i nose
\ No newline at end of file
+Consult :doc:`common-error-messages` for solutions to a range of setup problems.
\ No newline at end of file
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 requires/lucid_missing.txt
--- a/requires/lucid_missing.txt
+++ b/requires/lucid_missing.txt
@@ -8,17 +8,17 @@
# vdm>=0.9,<0.9.99
-e hg+https://bitbucket.org/okfn/vdm@vdm-0.9#egg=vdm
# markupsafe==0.9.2 required by webhelpers==1.2 required by formalchemy with SQLAlchemy 0.6
--e git+https://github.com/mitsuhiko/markupsafe.git@0.9.2#egg=markupsafe
+markupsafe==0.9.2
# autoneg>=0.5
-e git+https://github.com/wwaites/autoneg.git@b4c727b164f411cc9d60#egg=autoneg
# flup>=0.5
-e hg+http://hg.saddi.com/flup@301a58656bfb#egg=flup
# solrpy == 0.9.4
solrpy==0.9.4
-# All the conflicting dependencies from the lucid_conflict.txt file
--e hg+https://bitbucket.org/okfn/ckan-deps@6287665a1965#egg=ckan-deps
# FormAlchemy
--e git+https://github.com/FormAlchemy/formalchemy.git@1.3.9#egg=formalchemy
+formalchemy==1.3.9
+# Apachemiddleware
+-e hg+https://hg.3aims.com/public/ApacheMiddleware@tip#egg=apachemiddleware
# NOTE: Developers, our build script for the Debian packages relies on the
# requirements above being specified as editable resources with their
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 setup.py
--- a/setup.py
+++ b/setup.py
@@ -20,7 +20,6 @@
install_requires=[
],
extras_require = {
- 'solr': ['solrpy==0.9.4'],
},
packages=find_packages(exclude=['ez_setup']),
include_package_data=True,
diff -r f600e0d37f6cbe3963b0ad610c08bd868f1d1f62 -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 test-core.ini
--- a/test-core.ini
+++ b/test-core.ini
@@ -56,6 +56,7 @@
test_smtp_server = localhost:6675
ckan.mail_from = info at test.ckan.net
+ckan.locale_default = en
# Logging configuration
[loggers]
https://bitbucket.org/okfn/ckan/changeset/54fc460602d2/
changeset: 54fc460602d2
user: dread
date: 2011-10-26 17:54:10
summary: [controllers,lib,config]: #1422 Removed etag headers and caching decorators etc.
affected #: 12 files
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/config/deployment.ini_tmpl
--- a/ckan/config/deployment.ini_tmpl
+++ b/ckan/config/deployment.ini_tmpl
@@ -53,24 +53,6 @@
# space separated list ...
auth.admins =
-# CKAN caching
-ckan.cache_enabled = False
-ckan.cache_validation_enabled = True
-# cache to persistent files
-beaker.cache.type = file
-# default expiry time for the cache (where not specified specifically)
-ckan.cache.default_expires = -1
-# cache expiry settings for specific controllers
-# by default these are disabled, enable as needed to a time in seconds
-ckan.controllers.expires = -1
-ckan.controllers.home.expires = -1
-ckan.controllers.package.list.expires = -1
-ckan.controllers.tag.read.expires = -1
-ckan.controllers.apiv1.package.list.expires = -1
-ckan.controllers.apiv1.package.show.expires = -1
-ckan.controllers.apiv2.package.list.expires = -1
-ckan.controllers.apiv2.package.show.expires = -1
-
# CKAN QoS monitoring
ckan.enable_call_timing = false
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/controllers/home.py
--- a/ckan/controllers/home.py
+++ b/ckan/controllers/home.py
@@ -1,7 +1,7 @@
import random
import sys
-from pylons import cache, config
+from pylons import config
from pylons.i18n import set_lang
from genshi.template import NewTextTemplate
import sqlalchemy.exc
@@ -11,13 +11,10 @@
from ckan.logic import check_access, get_action
from ckan.lib.i18n import set_session_locale, get_lang
from ckan.lib.search import query_for, QueryOptions, SearchError
-from ckan.lib.cache import proxy_cache, get_cache_expires
from ckan.lib.base import *
import ckan.lib.stats
from ckan.lib.hash import get_redirect
-cache_expires = get_cache_expires(sys.modules[__name__])
-
class HomeController(BaseController):
repo = model.repo
@@ -41,21 +38,7 @@
raise
- @staticmethod
- def _home_cache_key():
- '''Calculate the etag cache key for the home page.'''
- # a change to the data means the group package amounts may change
- latest_revision_id = model.repo.youngest_revision().id
- user_name = c.user
- language = get_lang()
- cache_key = str(hash((user_name, language, latest_revision_id)))
- return cache_key
-
- @proxy_cache(expires=cache_expires)
def index(self):
- cache_key = self._home_cache_key()
- etag_cache(cache_key)
-
try:
query = query_for(model.Package)
query.run({'q': '*:*', 'facet.field': g.facets})
@@ -67,14 +50,13 @@
c.package_count = 0
c.groups = []
- return render('home/index.html', cache_key=cache_key,
- cache_expire=cache_expires)
+ return render('home/index.html')
def license(self):
- return render('home/license.html', cache_expire=cache_expires)
+ return render('home/license.html')
def about(self):
- return render('home/about.html', cache_expire=cache_expires)
+ return render('home/about.html')
def locale(self):
locale = request.params.get('locale')
@@ -105,7 +87,7 @@
def cache(self, id):
'''Manual way to clear the caches'''
if id == 'clear':
- wui_caches = ['tag_counts', 'search_results', 'stats']
+ wui_caches = ['stats']
for cache_name in wui_caches:
cache_ = cache.get_cache(cache_name, type='dbm')
cache_.clear()
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/controllers/package.py
--- a/ckan/controllers/package.py
+++ b/ckan/controllers/package.py
@@ -6,7 +6,7 @@
from sqlalchemy.orm import eagerload_all
import genshi
-from pylons import config, cache
+from pylons import config
from pylons.i18n import _
from autoneg.accept import negotiate
from babel.dates import format_date, format_datetime, format_time
@@ -15,10 +15,9 @@
from ckan.logic.schema import package_form_schema
from ckan.lib.helpers import date_str_to_datetime
from ckan.lib.base import request, c, BaseController, model, abort, h, g, render
-from ckan.lib.base import etag_cache, response, redirect, gettext
+from ckan.lib.base import response, redirect, gettext
from ckan.authz import Authorizer
from ckan.lib.search import SearchIndexError, SearchError
-from ckan.lib.cache import proxy_cache
from ckan.lib.package_saver import PackageSaver, ValidationException
from ckan.lib.navl.dictization_functions import DataError, unflatten, validate
from ckan.lib.helpers import json
@@ -176,14 +175,6 @@
return render('package/search.html')
- @staticmethod
- def _pkg_cache_key(pkg):
- # note: we need pkg.id in addition to pkg.revision.id because a
- # revision may have more than one package in it.
- language = get_lang()
- return str(hash((pkg.id, pkg.latest_related_revision.id, c.user, pkg.get_average_rating(), language)))
-
- @proxy_cache()
def read(self, id):
context = {'model': model, 'session': model.Session,
'user': c.user or c.author, 'extras_as_string': True,
@@ -217,9 +208,6 @@
except NotAuthorized:
abort(401, _('Unauthorized to read package %s') % id)
- cache_key = self._pkg_cache_key(c.pkg)
- etag_cache(cache_key)
-
#set a cookie so we know whether to display the welcome message
c.hide_welcome_message = bool(request.cookies.get('hide_welcome_message', False))
response.set_cookie('hide_welcome_message', '1', max_age=3600) #(make cross-site?)
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/controllers/revision.py
--- a/ckan/controllers/revision.py
+++ b/ckan/controllers/revision.py
@@ -8,8 +8,6 @@
from ckan.lib.base import *
from ckan.lib.helpers import Page
import ckan.authz
-from ckan.lib.cache import proxy_cache, get_cache_expires
-cache_expires = get_cache_expires(sys.modules[__name__])
class RevisionController(BaseController):
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/controllers/tag.py
--- a/ckan/controllers/tag.py
+++ b/ckan/controllers/tag.py
@@ -4,7 +4,6 @@
from ckan.lib.base import *
from ckan.lib.search import query_for
-from ckan.lib.cache import proxy_cache
from ckan.lib.helpers import AlphaPage, Page
from ckan.logic import NotFound, NotAuthorized
@@ -57,7 +56,6 @@
return render('tag/index.html')
- @proxy_cache()
def read(self, id):
context = {'model': model, 'session': model.Session,
'user': c.user or c.author}
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/lib/base.py
--- a/ckan/lib/base.py
+++ b/ckan/lib/base.py
@@ -25,11 +25,6 @@
from ckan.plugins import PluginImplementations, IGenshiStreamFilter
from ckan.lib.helpers import json
import ckan.model as model
-from ckan.lib.cache import etag_cache
-
-# nuke cache
-#from pylons import cache
-#cache.clear()
PAGINATE_ITEMS_PER_PAGE = 50
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/lib/cache.py
--- a/ckan/lib/cache.py
+++ /dev/null
@@ -1,216 +0,0 @@
-import inspect
-from time import gmtime, strftime, time
-from calendar import timegm
-from decorator import decorator
-from paste.deploy.converters import asbool
-from pylons.decorators.cache import beaker_cache, create_cache_key, _make_dict_from_args
-from pylons.decorators.util import get_pylons
-from pylons.controllers.util import etag_cache as pylons_etag_cache
-import pylons.config
-from ckan.lib.helpers import are_there_flash_messages
-
-__all__ = ["ckan_cache", "get_cache_expires"]
-
-cache_enabled = asbool(pylons.config.get('cache_enabled', 'False')) or \
- asbool(pylons.config.get('ckan.cache_enabled', 'False'))
-cache_validation_enabled = asbool(pylons.config.get('ckan.cache_validation_enabled', 'True'))
-
-def ckan_cache(test=lambda *av, **kw: 0,
- key="cache_default",
- expires=None,
- type=None,
- query_args=False,
- cache_headers=('content-type', 'content-length',),
- **cache_kwargs):
- """
- This is a specialised cache decorator that borrows much of its functionality
- from the func:`pylons.decorators.cache.beaker_cache`. The key differences are
-
- :param expires: is not the expiry of the local disk or memory cache but the
- expiry that gets set in the max-age Cache-Control header. The default is
- not to set Cache-Control
-
- :param test: is a function that takes the same arguments as the wrapped
- controller and returns an numeric value in seconds from the epoch GMT.
-
- The ''test'' function is crucial for cache expiry. The decorator keeps a
- timestamp for the last time the cache was updated. If the value returned by
- ''test()'' is greater than the timestamp, the cache will be purged and the
- document re-rendered.
-
- This decorator sets the ''Last-Modified'', ''ETag'' and ''Cache-Control''
- HTTP headers in the response according to the remembered timestamp and the
- given ''expires'' parameter.
-
- Other parameters as supported by the beaker cache are supported here in the
- same way.
-
- Some examples:
-
- .. code-block:: python
-
- # defaults
- @cache()
- def controller():
- return "I never expire, last-modified is the epoch"
-
- from time import timegm, gmtime
- @cache(test=lambda *av, **kw: timegm(gmtime()))
- def controller():
- return "I am never cached locally but set cache-control headers"
-
- @cache(query_args=True)
- def controller():
- return "I cache each new combination of GET parameters separately"
-
- """
- cache_headers = set(cache_headers)
- log = __import__("logging").getLogger("ckan_cache")
-
- def wrapper(func, *args, **kwargs):
- pylons = get_pylons(args)
- if not cache_enabled:
- log.debug("Caching disabled, skipping cache lookup")
- return func(*args, **kwargs)
-
- cfg_expires = "%s.expires" % _func_cname(func)
-
- # this section copies entirely too much from beaker cache
- if key:
- if query_args:
- key_dict = pylons.request.GET.mixed()
- else:
- key_dict = kwargs.copy()
- # beaker only does this if !query_args, we do it in
- # all cases to support both query args and method args
- # in the controller
- key_dict.update(_make_dict_from_args(func, args))
- if key != "cache_default":
- if isinstance(key, list):
- key_dict = dict((k, key_dict[k]) for k in key)
- else:
- key_dict = {key: key_dict[key]}
- else:
- key_dict = None
-
- self = None
- if args:
- self = args[0]
-
- namespace, cache_key = create_cache_key(func, key_dict, self)
-
- if type:
- cache_kwargs["type"] = type
- my_cache = pylons.cache.get_cache(namespace, **cache_kwargs)
-
- ## end copy from beaker_cache
-
- last_modified = test(*args, **kwargs)
- cache_miss = list()
-
- def render():
- log.debug("Creating new cache copy with key: %s, type: %s",
- cache_key, type)
- result = func(*args, **kwargs)
- glob_response = pylons.response
- headers = dict(glob_response.headerlist)
- status = glob_response.status
- full_response = dict(headers=headers, status=status,
- cookies=None, content=result,
- timestamp=last_modified)
- cache_miss.append(True)
- return full_response
-
- response = my_cache.get_value(cache_key, createfunc=render)
- timestamp = response["timestamp"]
- if timestamp < last_modified:
- my_cache.remove(cache_key)
- response = my_cache.get_value(cache_key, createfunc=render)
-
- glob_response = pylons.response
-
- if response["status"][0] in ("4", "5"): # do not cache 4XX, 5XX
- my_cache.remove(cache_key)
- else:
- headers = dict(glob_response.headerlist)
- headers.update(header for header in response["headers"].items()
- if header[0].lower() in cache_headers)
-
- headers["Last-Modified"] = strftime("%a, %d %b %Y %H:%M:%S GMT", gmtime(last_modified))
- headers["ETag"] = str(last_modified)
- if cache_miss:
- headers["X-CKAN-Cache"] = "MISS"
- else:
- headers["X-CKAN-Cache"] = "HIT"
-
- if expires:
- if "Pragma" in headers: del headers["Pragma"]
- if "Cache-Control" in headers: del headers["Cache-Control"]
- else:
- headers["Pragma"] = "no-cache"
- headers["Cache-Control"] = "no-cache"
-
- glob_response.headerlist = headers.items()
-
- if expires:
- glob_response.cache_expires(seconds=expires)
- cc = glob_response.headers["Cache-Control"]
- glob_response.headers["Cache-Control"] = "%s, must-revalidate" % cc
-
- glob_response.status = response['status']
- return response["content"]
-
- return decorator(wrapper)
-
-def proxy_cache(expires=None):
- log = __import__("logging").getLogger("proxy_cache")
- def wrapper(func, *args, **kwargs):
- result = func(*args, **kwargs)
-
- pylons = get_pylons(args)
- if not cache_enabled:
- log.debug("Caching disabled, skipping cache lookup")
- return result
-
- cfg_expires = "%s.expires" % _func_cname(func)
- cache_expires = expires if expires else int(pylons.config.get(cfg_expires, 0))
-
- headers = pylons.response.headers
- status = pylons.response.status
- if pylons.response.status[0] not in ("4", "5"):
- if "Pragma" in headers: del headers["Pragma"]
- if "Cache-Control" in headers: del headers["Cache-Control"]
- headers["Last-Modified"] = strftime("%a, %d %b %Y %H:%M:%S GMT", gmtime())
- pylons.response.cache_expires(seconds=cache_expires)
- return result
- return decorator(wrapper)
-
-def _func_cname(func):
- if inspect.ismodule(func):
- base = '%s' % func.__name__
- func_name = ''
- else:
- if hasattr(func, "im_class"):
- base = "%s.%s" % (func.im_class.__module__, func.im_class.__name__)
- func_name = '.' + func.im_func.func_name
- else:
- base = func.__module__
- func_name = ''
- return "%s%s" % (base, func_name)
-
-
-def get_cache_expires(module_or_func):
- # very weird experience (in tests): at module level it does not appear that
- # config option is defined but it is defined here (must be to do with when
- # config is loaded ...)
- default_expires = pylons.config.get('ckan.cache.default_expires', -1)
- if not cache_enabled:
- return -1
- cfg_expires = '%s.expires' % _func_cname(module_or_func)
- cache_expires = int(pylons.config.get(cfg_expires, default_expires))
- return cache_expires
-
-def etag_cache(page_hash):
- # don't cache if there are flash messages to show (#1321)
- if cache_validation_enabled and not are_there_flash_messages():
- pylons_etag_cache(page_hash)
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/lib/stats.py
--- a/ckan/lib/stats.py
+++ b/ckan/lib/stats.py
@@ -1,10 +1,14 @@
import datetime
-from pylons import config
+from paste.deploy.converters import asbool
+import pylons
from sqlalchemy import *
from ckan import model
-from ckan.lib.cache import cache_enabled
+
+# Since #1422 the cache should always be enabled apart from when
+# running in tests. Therefore don't document this config option.
+cache_enabled = asbool(pylons.config.get('ckan.cache_enabled', 'True'))
if cache_enabled:
from pylons import cache
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/tests/functional/test_cache.py
--- a/ckan/tests/functional/test_cache.py
+++ /dev/null
@@ -1,157 +0,0 @@
-from ckan.tests import *
-from pylons import config
-from ckan.lib.base import BaseController
-import ckan.lib.cache
-from ckan.lib.cache import ckan_cache, get_cache_expires
-from time import gmtime, time, strptime, sleep
-from calendar import timegm
-import sys
-
-def now():
- return timegm(gmtime())
-start = now()
-
-class TestCacheBasics:
- def setup(self):
- self.cache_enabled = ckan.lib.cache.cache_enabled
-
- def teardown(self):
- ckan.lib.cache.cache_enabled = self.cache_enabled
-
- def test_get_cache_expires(self):
- # cache enabled disabled by default
- out = get_cache_expires(sys.modules[__name__])
- assert out == -1, out
-
- ckan.lib.cache.cache_enabled = True
- out = get_cache_expires(sys.modules[__name__])
- assert out == 1800, out
-
- out = get_cache_expires(self.test_get_cache_expires)
- assert out == 3600, out
-
- # no match, so use config default_expires
- out = get_cache_expires(ckan.lib.cache)
- assert out == 200, out
-
-
-class CacheController(BaseController):
- """
- Dummy controller - we are testing the decorator
- not the controller
- """
- @ckan_cache()
- def defaults(self):
- return "defaults"
-
- @ckan_cache(test=lambda *av, **kw: start + 3600)
- def future(self):
- return "future"
-
- @ckan_cache(test=lambda *av, **kw: now())
- def always(self):
- return "always"
-# put the dummy controller where routes can find it
-# XXX FIXME THIS DOESN'T WORK
-sys.modules["ckan.controllers.cache"] = __import__(__name__)
-sys.modules["ckan.controllers.cache"].CacheController = CacheController
-
-from nose.plugins.skip import SkipTest
-
-class TestCacheController(TestController):
- def test_defaults(self):
- """
- Check default behaviour, cache once, never expire
- """
- raise SkipTest()
- url = url_for(controller="cache", action="defaults")
-
- resp = self.app.get(url)
- headers = dict(resp.headers)
-
- # check last modified
- last_modified = headers["Last-Modified"]
- last_modified = strptime(last_modified, "%a, %d %b %Y %H:%M:%S GMT")
- assert timegm(last_modified) == 0, last_modified
-
- # check no-cache does not appear
- assert "no-cache" not in headers["Cache-Control"], headers["Cache-Control"]
- assert "Pragma" not in headers
-
- # should have been a cache miss
- assert headers["X-CKAN-Cache"] == "MISS"
-
- resp = self.app.get(url)
- headers = dict(resp.headers)
-
- # check last modified
- last_modified = headers["Last-Modified"]
- last_modified = strptime(last_modified, "%a, %d %b %Y %H:%M:%S GMT")
- assert timegm(last_modified) == 0, last_modified
-
- # check no-cache does not appear
- assert "no-cache" not in headers["Cache-Control"], headers["Cache-Control"]
- assert "Pragma" not in headers
-
- # should have been a cache miss
- assert headers["X-CKAN-Cache"] == "HIT"
-
- def test_future(self):
- """
- Expiry in the future
-
- This should raise an exception as it is not allowed per HTTP/1.1
- """
- raise SkipTest()
- url = url_for(controller="cache", action="future")
-
- resp = self.app.get(url)
- headers = dict(resp.headers)
-
- # check last modified
- last_modified = headers["Last-Modified"]
- last_modified = strptime(last_modified, "%a, %d %b %Y %H:%M:%S GMT")
- last_modified = timegm(last_modified)
- assert last_modified == start + 3600, (start + 3600, last_modified)
-
- # should have been a cache miss
- assert headers["X-CKAN-Cache"] == "MISS"
-
- def test_always(self):
- """
- Check where last-modified is always now()
- """
- raise SkipTest()
- url = url_for(controller="cache", action="always")
-
- resp = self.app.get(url)
- headers = dict(resp.headers)
- first_modified = headers["Last-Modified"]
- first_modified = strptime(first_modified, "%a, %d %b %Y %H:%M:%S GMT")
- first_modified = timegm(first_modified)
-
- # check no-cache does not appear
- assert "no-cache" not in headers["Cache-Control"], headers["Cache-Control"]
- assert "Pragma" not in headers
-
- # should have been a cache miss
- assert headers["X-CKAN-Cache"] == "MISS"
-
- sleep(1)
-
- resp = self.app.get(url)
- headers = dict(resp.headers)
- last_modified = headers["Last-Modified"]
- last_modified = strptime(last_modified, "%a, %d %b %Y %H:%M:%S GMT")
- last_modified = timegm(last_modified)
-
- # check last-modified
- assert last_modified > first_modified, (first_modified, last_modified)
-
- # check no-cache does not appear
- assert "no-cache" not in headers["Cache-Control"], headers["Cache-Control"]
- assert "Pragma" not in headers
-
- # should have been a cache miss
- assert headers["X-CKAN-Cache"] == "MISS"
-
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/tests/functional/test_home.py
--- a/ckan/tests/functional/test_home.py
+++ b/ckan/tests/functional/test_home.py
@@ -31,36 +31,6 @@
assert 'Add a dataset' in res
assert 'Could not change language' not in res
- def test_calculate_etag_hash(self):
- # anything that changes the home page appearance should change the
- # etag hash
- c.user = 'test user'
- get_hash = HomeController._home_cache_key
- hashes = [get_hash(), get_hash()]
- self.assert_equal(hashes[0], hashes[1])
-
- def assert_hash_changed(hashes):
- current_hash = get_hash()
- assert current_hash != hashes[-1]
- hashes.append(current_hash)
-
- # login as a different user
- c.user = 'another user'
- assert_hash_changed(hashes)
-
- # add a package to a group
- rev = model.repo.new_revision()
- model.Group.by_name(u'roger').add_package_by_name(u'warandpeace')
- model.repo.commit_and_remove()
- assert_hash_changed(hashes)
-
- # flash message is not cached, but this is done in ckan/lib/cache
-
- # I can't get set_lang to work and deliver correct
- # result to get_lang, so leaving it commented
-## set_lang('fr')
-## assert_hash_changed(hashes)
-
@search_related
def test_packages_link(self):
offset = url_for('home')
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 ckan/tests/functional/test_package.py
--- a/ckan/tests/functional/test_package.py
+++ b/ckan/tests/functional/test_package.py
@@ -1454,74 +1454,6 @@
def fail_if_fragment(self, fragment):
assert fragment not in self.body, (fragment, self.body)
-
-class TestEtags(PylonsTestCase, TestPackageBase):
- @classmethod
- def setup_class(cls):
- PylonsTestCase.setup_class()
- CreateTestData.create()
-
- @classmethod
- def teardown_class(cls):
- model.repo.rebuild_db()
-
- def test_calculate_etag_hash(self):
- c.user = 'test user'
- get_hash = PackageController._pkg_cache_key
- hash_1 = get_hash(self.anna)
- hash_2 = get_hash(self.anna)
- self.assert_equal(hash_1, hash_2)
-
- test_title = u'Anna changed'
- rev = model.repo.new_revision()
- self.anna.title = test_title
- model.repo.commit_and_remove()
- hash_3 = get_hash(self.anna)
- self.assert_not_equal(hash_2, hash_3)
-
- test_url = u'http://some.com/file.xls'
- rev = model.repo.new_revision()
- self.anna.add_resource(test_url)
- model.repo.commit_and_remove()
- hash_4 = get_hash(self.anna)
- self.assert_not_equal(hash_4, hash_3)
-
- test_extra_key = u'testkey'
- test_extra_value = u'testval'
-
- rev = model.repo.new_revision()
- # XXX without the following line, we get a "stale association
- # proxy" in sqlalchemy > 0.6 when we try to access
- # self.anna.extras. "extras" is the association proxy; for
- # some reason the association's reference to its parent
- # (i.e. anna) is garbage-collected prematurely. By creating a
- # reference to anna here, we prevent the reference from having
- # its parent destroyed during garbage collection. The mystery
- # is why this should be different with different versions of
- # sqlalchemy.
- anna = self.anna
- self.anna.extras[test_extra_key] = test_extra_value
- model.repo.commit_and_remove()
- hash_5 = get_hash(self.anna)
- self.assert_not_equal(hash_5, hash_4)
-
- set_rating(u'random user', self.anna, 4)
- # (commit_and_remove not required for rating)
- hash_6 = get_hash(self.anna)
- self.assert_not_equal(hash_6, hash_5)
-
- c.user = 'another user'
- hash_7 = get_hash(self.anna)
- self.assert_not_equal(hash_7, hash_6)
-
- def test_etags_in_response(self):
- c.user = 'annafan'
- c.userobj = model.User.by_name(u'annafan')
- res = self.app.get('/dataset/annakarenina',
- extra_environ={'REMOTE_USER':c.user})
- anna_hash = str(PackageController._pkg_cache_key(self.anna))
- self.assert_equal(res.header_dict['ETag'], anna_hash)
-
class TestAutocomplete(PylonsTestCase, TestPackageBase):
@classmethod
def setup_class(cls):
diff -r a34e584122a925e396ed3ac4f1cd59bbec4eabe4 -r 54fc460602d2bba9bbe71018e699470dc700ba34 doc/configuration.rst
--- a/doc/configuration.rst
+++ b/doc/configuration.rst
@@ -117,57 +117,6 @@
For more information on using dumpfiles, see :doc:`database_dumps`.
-Cache Settings
---------------
-
-.. index::
- single: cache_validation_enabled
-
-cache_validation_enabled
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-Example::
-
- ckan.cache_validation_enabled = False
-
-Default value: ``True``
-
-This option determines whether browsers (or other caching services running between the browser and CKAN) are helped to cache particular CKAN pages, by validating when the page content hasn't changed. This is achieved using ETag headers provided by CKAN, which is a hash that changes when the content has changed.
-
-Developers editing the templates should set this to False, since ETag hashes don't look for template changes.
-
-.. index::
- single: cache_enabled
-
-cache_enabled
-^^^^^^^^^^^^^
-
-Example::
-
- ckan.cache_enabled = True
-
-Default value: ``False``
-
-Setting this option to True turns on several server-side caches. When caching is on, caching can be further configured as follows.
-
-To set the type of Beaker storage::
-
- beaker.cache.type = file
-
-To set the expiry times (in seconds) for specific controllers (which use the proxy_cache) specify the methods like this::
-
- ckan.controllers.package.list.expires = 600
- ckan.controllers.tag.read.expires = 600
- ckan.controllers.apiv1.package.list.expires = 600
- ckan.controllers.apiv1.package.show.expires = 600
- ckan.controllers.apiv2.package.list.expires = 600
- ckan.controllers.apiv2.package.show.expires = 600
-
-There is also an option to set the max-age value of static files delivered by paster::
-
- ckan.static_max_age = 3600
-
-
Authentication Settings
-----------------------
https://bitbucket.org/okfn/ckan/changeset/67161d762451/
changeset: 67161d762451
branch: release-v1.5
user: dread
date: 2011-10-26 17:59:33
summary: [tests]: Add in test for munge, alongside code in db8c9f64bb3d.
affected #: 1 file
diff -r 109db2176fac5b417391201ef5cb655e0623d9c2 -r 67161d762451031829aab3970bfc74dffae5caa6 ckan/tests/lib/test_munge.py
--- a/ckan/tests/lib/test_munge.py
+++ b/ckan/tests/lib/test_munge.py
@@ -11,4 +11,4 @@
test_munge('unchanged', 'unchanged')
test_munge('bad spaces', 'bad-spaces')
test_munge('random:other%character&', 'random-othercharacter')
- test_munge(u'u with umlaut \xfc', 'u-with-umlaut-u')
+ test_munge(u'u with umlaut \xfc', 'u-with-umlaut-u')
https://bitbucket.org/okfn/ckan/changeset/c9bfdcb5c4e7/
changeset: c9bfdcb5c4e7
user: dread
date: 2011-10-26 18:06:47
summary: [merge] of default heads.
affected #: 7 files
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 ckan/controllers/home.py
--- a/ckan/controllers/home.py
+++ b/ckan/controllers/home.py
@@ -40,12 +40,22 @@
def index(self):
try:
- query = query_for(model.Package)
- query.run({'q': '*:*', 'facet.field': g.facets})
- c.package_count = query.count
- c.facets = query.facets # used by the 'tag cloud' recipe
- q = model.Session.query(model.Group).filter_by(state='active')
- c.groups = sorted(q.all(), key=lambda g: len(g.packages), reverse=True)[:6]
+ # package search
+ context = {'model': model, 'session': model.Session,
+ 'user': c.user or c.author}
+ data_dict = {
+ 'q':'*:*',
+ 'facet.field':g.facets,
+ 'rows':0,
+ 'start':0,
+ }
+ query = get_action('package_search')(context,data_dict)
+ c.package_count = query['count']
+ c.facets = query['facets']
+
+ # group search
+ data_dict = {'order_by': 'packages', 'all_fields': 1}
+ c.groups = get_action('group_list')(context, data_dict)
except SearchError, se:
c.package_count = 0
c.groups = []
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 ckan/controllers/package.py
--- a/ckan/controllers/package.py
+++ b/ckan/controllers/package.py
@@ -175,6 +175,7 @@
return render('package/search.html')
+
def read(self, id):
context = {'model': model, 'session': model.Session,
'user': c.user or c.author, 'extras_as_string': True,
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 ckan/logic/action/get.py
--- a/ckan/logic/action/get.py
+++ b/ckan/logic/action/get.py
@@ -117,21 +117,28 @@
user = context['user']
api = context.get('api_version') or '1'
ref_group_by = 'id' if api == '2' else 'name';
-
+ order_by = data_dict.get('order_by', 'name')
+ if order_by not in set(('name', 'packages')):
+ raise ValidationError('"order_by" value %r not implemented.' % order_by)
all_fields = data_dict.get('all_fields',None)
check_access('group_list',context, data_dict)
- # We need Groups for group_list_dictize
query = model.Session.query(model.Group).join(model.GroupRevision)
query = query.filter(model.GroupRevision.state=='active')
query = query.filter(model.GroupRevision.current==True)
- query = query.order_by(model.Group.name.asc())
- query = query.order_by(model.Group.title.asc())
+ if order_by == 'name':
+ query = query.order_by(model.Group.name.asc())
+ query = query.order_by(model.Group.title.asc())
groups = query.all()
+ if order_by == 'packages':
+ groups = sorted(query.all(),
+ key=lambda g: len(g.packages),
+ reverse=True)
+
if not all_fields:
group_list = [getattr(p, ref_group_by) for p in groups]
else:
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 ckan/templates/home/index.html
--- a/ckan/templates/home/index.html
+++ b/ckan/templates/home/index.html
@@ -52,13 +52,13 @@
<div py:if="c.groups" class="span-24 last whoelse"><h2>Who else is here?</h2></div>
- <py:for each="i, group in enumerate(c.groups)">
+ <py:for each="i, group_dict in enumerate(c.groups[:6])"><div class="span-8 group ${'last' if i % 3 == 2 else ''}">
- <h3><a href="${h.url_for(controller='group', action='read', id=group.name)}">${group.title}</a></h3>
+ <h3><a href="${h.url_for(controller='group', action='read', id=group_dict['name'])}">${group_dict['title']}</a></h3><p>
- ${h.markdown_extract(group.description)}
+ ${h.markdown_extract(group_dict['description'])}
</p>
- <strong>${group.title} has ${len(group.packages)} datasets.</strong>
+ <strong>${group_dict['title']} has ${group_dict['packages']} datasets.</strong></div></py:for></div>
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 ckan/tests/functional/api/test_action.py
--- a/ckan/tests/functional/api/test_action.py
+++ b/ckan/tests/functional/api/test_action.py
@@ -419,6 +419,27 @@
assert 'revision_id' in res_obj['result'][0]
assert 'state' in res_obj['result'][0]
+ def test_13_group_list_by_size(self):
+ postparams = '%s=1' % json.dumps({'order_by': 'packages'})
+ res = self.app.post('/api/action/group_list',
+ params=postparams)
+ res_obj = json.loads(res.body)
+ assert_equal(res_obj['result'], ['david',
+ 'roger'])
+
+ def test_13_group_list_by_size_all_fields(self):
+ postparams = '%s=1' % json.dumps({'order_by': 'packages',
+ 'all_fields': 1})
+ res = self.app.post('/api/action/group_list',
+ params=postparams)
+ res_obj = json.loads(res.body)
+ result = res_obj['result']
+ assert_equal(len(result), 2)
+ assert_equal(result[0]['name'], 'david')
+ assert_equal(result[0]['packages'], 2)
+ assert_equal(result[1]['name'], 'roger')
+ assert_equal(result[1]['packages'], 1)
+
def test_14_group_show(self):
postparams = '%s=1' % json.dumps({'id':'david'})
res = self.app.post('/api/action/group_show', params=postparams)
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 ckan/tests/functional/test_home.py
--- a/ckan/tests/functional/test_home.py
+++ b/ckan/tests/functional/test_home.py
@@ -30,6 +30,12 @@
res = self.app.get(offset)
assert 'Add a dataset' in res
assert 'Could not change language' not in res
+ assert "Dave's books has 2 datasets" in res, res
+ assert "Roger's books has 1 datasets" in res, res
+
+
+
+
@search_related
def test_packages_link(self):
@@ -50,6 +56,11 @@
res = self.app.get(offset)
assert 'ckan.template_head_end = <link rel="stylesheet" href="TEST_TEMPLATE_HEAD_END.css" type="text/css"> '
+ def test_template_head_end(self):
+ offset = url_for('home')
+ res = self.app.get(offset)
+ assert 'ckan.template_head_end = <link rel="stylesheet" href="TEST_TEMPLATE_HEAD_END.css" type="text/css"> '
+
def test_template_footer_end(self):
offset = url_for('home')
res = self.app.get(offset)
diff -r 54fc460602d2bba9bbe71018e699470dc700ba34 -r c9bfdcb5c4e70011b3d60f0212b727c6e50dfe13 doc/configuration.rst
--- a/doc/configuration.rst
+++ b/doc/configuration.rst
@@ -192,6 +192,7 @@
If you want to specify the ordering of all or some of the locales as they are offered to the user, then specify them here in the required order. Any locales that are available but not specified in this option, will still be offered at the end of the list.
+
Theming Settings
----------------
Repository URL: https://bitbucket.org/okfn/ckan/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the ckan-changes
mailing list