[ckan-dev] FW: Datastore setup for ckan 2.0
Stefán Þór Finnsson
stefan.thor.finnsson at reykjavik.is
Wed Apr 10 16:53:12 UTC 2013
Sending this again, dont see the attached file I sent last time with the post, the ini config is after this log error.
Here is what the log files says:
Here is what the log says:
[2013-03-22 12:47:19,014: INFO/MainProcess] Got task from broker: datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]
[2013-03-22 12:47:19,031: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:19,060: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:19,062: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:19,090: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:24,096: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:24,925: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:34,928: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:34,966: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:49,983: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:50,016: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:48:10,099: ERROR/MainProcess] Task datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415] raised exception: MissingSchema("Invalid URL u'api/action/task_status_update': No schema supplied",) Traceback (most recent call last):
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/execute/trace.py", line 47, in trace
return cls(states.SUCCESS, retval=fun(*args, **kwargs))
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/task/__init__.py", line 247, in __call__
return self.run(*args, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/__init__.py", line 175, in run
return fun(*args, **kwargs)
File "/usr/local/ckan-test/pyenv/src/ckanext-datastorer/ckanext/datastorer/tasks.py", line 118, in datastorer_upload
}, logger)
File "/usr/local/ckan-test/pyenv/src/ckanext-archiver/ckanext/archiver/tasks.py", line 474, in update_task_status
'Content-Type': 'application/json'}
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/api.py", line 98, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/safe_mode.py", line 39, in wrapped
return function(method, url, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/api.py", line 51, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/sessions.py", line 252, in request
r.send(prefetch=prefetch)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/models.py", line 490, in send
url = self.full_url
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/models.py", line 393, in full_url
raise MissingSchema("Invalid URL %r: No schema supplied" % url)
MissingSchema: Invalid URL u'api/action/task_status_update': No schema supplied
And here is the ini file without users for postgres:
#
# ckan - Pylons configuration
#
# The %(here)s variable will be replaced with the parent directory of this file
#
[DEFAULT]
# Change debug to true when doing CKAN development, it enables Pylons'
# interactive debugging tool, makes Fanstatic serve unminified JS and CSS
# files, and enables CKAN templates' debugging features.
#
# WARNING: *THIS SETTING MUST BE SET TO FALSE ON A PRODUCTION ENVIRONMENT*
# Debug mode will enable the interactive debugging tool, allowing ANYONE to
# execute malicious code after an exception is raised.
debug = false
email_to = you at yourdomain.com
error_email_from = paste at localhost
# The SMTP server to connect to when sending emails to users, with optional
# port (default: 25). For example: lavabit.com, or smtp.gmail.com:587. To
# connect to a local sendmail process enter 'localhost'.
smtp.server = localhost
# Whether or not to use STARTTLS when connecting to the SMTP server.
smtp.starttls = False
# The username and password to use to authenticate with the SMTP server.
# Optional, if these are left commented-out then CKAN will try to send email
# to the SMTP server without logging in first.
#smtp.user = your_username at gmail.com
#smtp.password = your_password
# The email address that emails sent by CKAN to users should appear to come
# from. Optional (default: None, note that the SMTP server may insert its own
# from address).
#smtp.mail_from =
[server:main]
use = egg:Paste#http
host = ckan-test
port = 5000
[app:main]
# Required
## 'Bucket' (subdirectory for file based storage) to use for file storage
ckan.storage.bucket = ck
## Optional
## maximum content size for uploads in bytes, defaults to 1Gb
# ckanext.storage.max_content_length = 1000000000
## OFS configuration
ofs.impl = pairtree
# directory on disk for data storage (should be empty)
ofs.storage_dir = /var/local/ckan
use = egg:ckan
full_stack = true
cache_dir = %(here)s/data
beaker.session.key = ckan
beaker.session.secret =
app_instance_uuid = {af3de861-f18b-48b5-99a7-b9c50a7d179b}
# List the names of CKAN extensions to activate.
# Note: This line is required to be here for packaging, even if it is empty.
# Note: Add ``pdf_preview`` to enable the resource preview for PDFs
# Add the ``resource_proxy`` plugin to enable resorce proxying and get around the same origin policy
# Add ``datastore`` to enable the CKAN DataStore extension
ckan.plugins = stats synchronous_search datastore datastorer pdf_preview
# If you'd like to fine-tune the individual locations of the cache data dirs
# for the Cache data, or the Session saves, un-comment the desired settings
# here:
#beaker.cache.data_dir = %(here)s/data/cache
#beaker.session.data_dir = %(here)s/data/sessions
# Specify the database for SQLAlchemy to use:
# * Postgres is currently required for a production CKAN deployment
# * Sqlite (memory or file) can be used as a quick alternative for testing
sqlalchemy.url = postgresql://:@localhost/ckantest
#sqlalchemy.url = sqlite:///
#sqlalchemy.url = sqlite:///%(here)s/somedb.db
# Un-comment and specify the URLs for the DataStore database.
# * Postgres is required
ckan.datastore.write_url = postgresql://:@localhost/datastore
ckan.datastore.read_url = postgresql://:@localhost/datastore
# repoze.who config
who.config_file = %(here)s/who.ini
who.log_level = warning
who.log_file = %(cache_dir)s/who_log.ini
# Location of RDF versions of datasets
#rdf_packages = http://semantic.ckan.net/record/
# Location of licenses group (defaults to cached local version of ckan group)
#licenses_group_url = http://licenses.opendefinition.org/licenses/groups/ckan.json
# Dataset form to use
package_form = standard
# Hide certain extras fields from dataset read form:
# package_hide_extras = for_search_index_only
# API configuration
#apikey_header_name = X-CKAN-API-Key
## extra places to look for templates and public files (comma separated lists)
## any templates/files found will override correspondingly named ones in
## ckan/templates/ and ckan/public
## (e.g. to override main layout template layout.html or add extra css files)
# extra_template_paths = %(here)s/my-templates
# extra_public_paths = %(here)s/my-public
# Dataset form integration
#package_edit_return_url = http://another.frontend/dataset/<NAME>
#package_new_return_url = http://another.frontend/dataset/<NAME>
# Turn on messaging with carrot, default to false
#ckan.async_notifier = true
# Messaging module used by carrot:
# * pyamqplib - AMQP (e.g. for RabbitMQ)
# * queue - native Python Queue (debugging and tests only)
#carrot_messaging_library = pyamqplib
## Perform search just using database (rather than use e.g. solr).
## In this setup search is crude and limited .e.g no full-text search, no faceting ...
## However, very useful for getting up and running quickly with CKAN
# ckan.simple_search = 1
## Title of site (using in several places including templates and <title> tag
ckan.site_title = Opin gogn hja
## Logo image to use on the home page
ckan.site_logo = /base/images/merki.jpg
## Site tagline / description (used on front page)
ckan.site_description = Opin gogn fyrir alla, til notkunar eda dreifingar
## Used in creating some absolute urls (such as rss feeds, css files) and
## dump filenames
ckan.site_url = ckan-test
## Favicon (default is the CKAN software favicon)
ckan.favicon = /images/icons/ckan.ico
## The gravatar default to use. This can be any of the pre-defined strings
## as defined on http://en.gravatar.com/site/implement/images/ (e.g. "identicon"
## or "mm"). Or it can be a url, e.g. "http://example.com/images/avatar.jpg"
ckan.gravatar_default = identicon
## Solr support
solr_url = http://127.0.0.1:8080/solr/ckan-schema-1.4
## Automatic indexing. Make all changes immediately available via the search
## after editing or creating a dataset. Default is true. If for some reason
## you need the indexing to occur asynchronously, set this option to 0.
# ckan.search.automatic_indexing = 1
## An 'id' for the site (using, for example, when creating entries in a common search index)
## If not specified derived from the site_url
ckan.site_id = ckan-test
## API url to use (e.g. in AJAX callbacks)
## Enable if the API is at a different domain
# ckan.api_url = http://www.ckan.net
## html content to be inserted just before </head> tag (e.g. extra stylesheet)
## NB: can use html e.g. <strong>blah</strong>
## NB: can have multiline strings just indent following lines
# ckan.template_head_end = <link rel="stylesheet" href="http://mysite.org/css/custom.css" type="text/css">
## html content to be inserted just before </body> tag (e.g. google analytics code)
## NB: can use html e.g. <strong>blah</strong>
## NB: can have multiline strings just indent following lines
# ckan.template_footer_end =
# These three settings (ckan.log_dir, ckan.dump_dir and ckan.backup_dir) are
# all used in cron jobs, not in CKAN itself. CKAN logging is configured
# in the logging configuration below
# Directory for logs (produced by cron scripts associated with ckan)
ckan.log_dir = %(here)s/log
# Directory for JSON/CSV dumps (must match setting in apache config)
ckan.dump_dir = %(here)s/dump
# Directory for SQL database backups
ckan.backup_dir = %(here)s/backup
# Default authorizations for new domain objects
#ckan.default_roles.Package = {"visitor": ["reader"], "logged_in": ["reader"]}
#ckan.default_roles.Group = {"visitor": ["reader"], "logged_in": ["reader"]}
#ckan.default_roles.System = {"visitor": ["reader"], "logged_in": ["editor"]}
#ckan.default_roles.AuthorizationGroup = {"visitor": ["reader"], "logged_in": ["reader"]}
## Ckan public and private recaptcha keys [localhost]
#ckan.recaptcha.publickey =
#ckan.recaptcha.privatekey =
# Locale/languages
ckan.locale_default = en
#ckan.locales_offered =
# Languages are grouped by percentage of strings in CKAN 1.8 translated
# (those with 100% first, then those with >=80%, then >=50%, then <50%) and
# within these groups roughly sorted by number of worldwide native speakers
# according to Wikipedia.
ckan.locale_order = en pt_BR ja it cs_CZ ca es fr el sv sr sr at latin no sk fi ru de pl nl bg ko_KR hu sa sl lv
ckan.locales_filtered_out =
## Atom Feeds
#
# Settings for customising the metadata provided in
# atom feeds.
#
# These settings are used to generate the <id> tags for both feeds
# and entries. The unique <id>s are created following the method
# outlined in http://www.taguri.org/ ie - they generate tagURIs, as specified
# in http://tools.ietf.org/html/rfc4151#section-2.1 :
#
# <id>tag:thedatahub.org,2012:/feeds/group/933f3857-79fd-4beb-a835-c0349e31ce76</id>
#
# Each component has the corresponding settings:
#
# "thedatahub.org" is ckan.feeds.authority_name
# "2012" is ckan.feeds.date
#
# Leave blank to use the ckan.site_url config value, otherwise set to a
# domain or email address that you own. e.g. thedatahub.org or
# admin at thedatahub.org
ckan.feeds.authority_name =
# Pick a date of the form "yyyy[-mm[-dd]]" during which the above domain was
# owned by you.
ckan.feeds.date = 2012
# If not set, then the value in `ckan.site_id` is used.
ckan.feeds.author_name =
# If not set, then the value in `ckan.site_url` is used.
ckan.feeds.author_link =
## File Store
#
# CKAN allows users to upload files directly to file storage either on the local
# file system or to online âcloudâ storage like Amazon S3 or Google Storage.
#
# If you are using local file storage, remember to set ckan.site_url.
#
# To enable cloud storage (Google or S3), first run: pip install boto
#
# @see http://docs.ckan.org/en/latest/filestore.html
# 'Bucket' to use for file storage
#ckan.storage.bucket = my-bucket-name
# To enable local file storage:
#ofs.impl = pairtree
#ofs.storage_dir = /my/path/to/storage/root/directory
# To enable Google cloud storage:
#ofs.impl = google
#ofs.gs_access_key_id =
#ofs.gs_secret_access_key =
# To enable S3 cloud storage:
#ofs.impl = s3
#ofs.aws_access_key_id = ....
#ofs.aws_secret_access_key = ....
## Previews
#
# Set the file types that should be previewed inline (e.g. images) or directly in an iframe.
ckan.preview.direct = png jpg gif
ckan.preview.loadable = html htm rdf+xml owl+xml xml n3 n-triples turtle plain atom csv tsv rss txt json
# Activity Streams
#
# Default maximum number of activities to show in an activity stream.
# ckan.activity_list_limit = 31
# Activity Streams Email Notifications
#
# Uncomment this line to enable activity streams email notifications.
# You also need to setup a cron job to send the emails, see the documentation.
#ckan.activity_streams_email_notifications = True
# Email notifications for events older than this time delta will not be sent.
# Accepted formats: '2 days', '14 days', '4:35:00' (hours, minutes, seconds),
# '7 days, 3:23:34', etc.
# ckan.email_notifications_since = 2 days
# DEBUGGING
# ckan.debug_supress_header This option can be set to suppress the debug
# information showing the controller and action recieving the request being
# shown in the header. Note: This info only shows if debug is set to true.
ckan.debug_supress_header = false
## ===================================
## Extensions
## Config option to enable the (1 day) cache for stats
## Default (if not defined) is True as stats computations are intensive
# ckanext.stats.cache_enabled = True
# Logging configuration
[loggers]
keys = root, ckan, ckanext
[handlers]
keys = console, file
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console, file
[logger_ckan]
level = INFO
handlers = console, file
qualname = ckan
propagate = 0
[logger_ckanext]
level = DEBUG
handlers = console, file
qualname = ckanext
propagate = 0
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[handler_file]
class = logging.handlers.RotatingFileHandler
formatter = generic
level = NOTSET
args = ("/var/log/ckan/ckan-test/ckan.log", "a", 20000000, 9)
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s] %(message)s
-----Original Message-----
From: Stefán Þór Finnsson
Sent: 22. mars 2013 16:43
To: 'CKAN Development Discussions'
Subject: RE: [ckan-dev] Datastore setup for ckan 2.0
Ok - changed the branch and see that the plugin.py has changed. But still get this error.
Here is what the log says:
[2013-03-22 12:47:19,014: INFO/MainProcess] Got task from broker: datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]
[2013-03-22 12:47:19,031: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:19,060: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:19,062: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:19,090: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:24,096: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:24,925: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:34,928: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:34,966: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:47:49,983: INFO/PoolWorker-1] Starting new HTTP connection (1): ckan-test
[2013-03-22 12:47:50,016: ERROR/PoolWorker-1] datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415]: Error while performing download: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',). Retrying...
[2013-03-22 12:48:10,099: ERROR/MainProcess] Task datastorer.upload[3922d5da-88bc-4d67-9348-8e3c11646415] raised exception: MissingSchema("Invalid URL u'api/action/task_status_update': No schema supplied",) Traceback (most recent call last):
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/execute/trace.py", line 47, in trace
return cls(states.SUCCESS, retval=fun(*args, **kwargs))
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/task/__init__.py", line 247, in __call__
return self.run(*args, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/__init__.py", line 175, in run
return fun(*args, **kwargs)
File "/usr/local/ckan-test/pyenv/src/ckanext-datastorer/ckanext/datastorer/tasks.py", line 118, in datastorer_upload
}, logger)
File "/usr/local/ckan-test/pyenv/src/ckanext-archiver/ckanext/archiver/tasks.py", line 474, in update_task_status
'Content-Type': 'application/json'}
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/api.py", line 98, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/safe_mode.py", line 39, in wrapped
return function(method, url, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/api.py", line 51, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/sessions.py", line 252, in request
r.send(prefetch=prefetch)
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/models.py", line 490, in send
url = self.full_url
File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/requests/models.py", line 393, in full_url
raise MissingSchema("Invalid URL %r: No schema supplied" % url)
MissingSchema: Invalid URL u'api/action/task_status_update': No schema supplied
The ini is as attachment.
-----Original Message-----
From: ckan-dev-bounces at lists.okfn.org [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Dominik Moritz
Sent: 21. mars 2013 20:51
To: CKAN Development Discussions
Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
You can try to change to the branch if you have installed the datastore via github. If not, you will need to patch `plugin.py`. There is no need to change the tasks.py since you do not need the download retry code.
On 21 Mar 2013, at 17:22, Stefán Þór Finnsson <stefan.thor.finnsson at reykjavik.is> wrote:
> Hi Dominik
>
> No worries, thanks for this?
>
> Do you want me to replace the plugin.py and make the tasks.py with the one from the pull request?
>
> -----Original Message-----
> From: ckan-dev-bounces at lists.okfn.org
> [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Dominik Moritz
> Sent: 21. mars 2013 08:47
> To: CKAN Development Discussions
> Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
>
> Hi,
>
> sorry that my answer took so long. Can you try the version from the pull request at https://github.com/okfn/ckanext-datastorer/pull/21? It looks like it could fix the problem.
>
> If not, could you send me your .ini file (without the passwords for the database or any secret keys ;-)).
>
> Cheers,
> Dominik
>
> On 14 Mar 2013, at 16:37, Stefán Þór Finnsson <stefan.thor.finnsson at reykjavik.is> wrote:
>
>> Still having this problem, did try to edit deployment.ini, but now
>> luck, if I leave the ckan.site_url = empty I see when I try to upload
>> a file that it does not add the ckan-test but then I get some authentication error, it ask me to login agin and goes in to a loop.
>>
>> Would be nice if someone could help me see the light here :)
>>
>> -----Original Message-----
>> From: ckan-dev-bounces at lists.okfn.org
>> [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Stefán Þór
>> Finnsson
>> Sent: 11. mars 2013 13:36
>> To: CKAN Development Discussions
>> Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
>>
>> Hi,
>>
>> I did see error´s now when the celery is running:
>>
>> [2013-03-11 10:36:40,829: ERROR/MainProcess] Task datastorer.upload[41e99198-a0be-43a0-b4d6-54eb4a9b9d80] raised exception: LinkHeadRequestError('URL unobtainable: Server returned HTTP 404',) Traceback (most recent call last):
>> File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/execute/trace.py", line 47, in trace
>> return cls(states.SUCCESS, retval=fun(*args, **kwargs)) File
>> "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/task/__init__.py", line 247, in __call__
>> return self.run(*args, **kwargs)
>> File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/__init__.py", line 175, in run
>> return fun(*args, **kwargs)
>> File "/usr/local/ckan-test/pyenv/src/ckanext-datastorer/ckanext/datastorer/tasks.py", line 108, in datastorer_upload
>> return _datastorer_upload(context, data, logger) File
>> "/usr/local/ckan-test/pyenv/src/ckanext-datastorer/ckanext/datastorer/tasks.py", line 123, in _datastorer_upload
>> result = download(context, resource, data_formats=DATA_FORMATS)
>> File "/usr/local/ckan-test/pyenv/src/ckanext-archiver/ckanext/archiver/tasks.py", line 103, in download
>> headers = json.loads(link_checker(link_context, link_data)) File
>> "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/task/__init__.py", line 247, in __call__
>> return self.run(*args, **kwargs)
>> File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/celery/app/__init__.py", line 175, in run
>> return fun(*args, **kwargs)
>> File "/usr/local/ckan-test/pyenv/src/ckanext-archiver/ckanext/archiver/tasks.py", line 361, in link_checker
>> raise LinkHeadRequestError(error_message)
>> LinkHeadRequestError: URL unobtainable: Server returned HTTP 404
>>
>> When I go to download for the document I get this path:
>> http://ckan-test/ckan-test/storage/f/2013-03-11T10%3A36%3A28.446Z/cka
>> n
>> test.csv
>>
>> and get 404 error, But if I take the extra ckan-test I can download the document, not sure where it gets the parameter to add this ckan-test in the url, is it from the ini file or some other config?
>>
>>
>>
>> -----Original Message-----
>> From: ckan-dev-bounces at lists.okfn.org
>> [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Dominik Moritz
>> Sent: 8. mars 2013 15:26
>> To: CKAN Development Discussions
>> Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
>>
>> Hi,
>>
>> could you go into a little bit more detail what you've done? Have you installed the datastorer properly (see https://github.com/okfn/ckanext-datastorer)? Did you wait for the datastorer to finish its work? Is the recline preview enabled in your .ini file?
>>
>> Dominik
>>
>> On 7 Mar 2013, at 15:59, Stefán Þór Finnsson <stefan.thor.finnsson at reykjavik.is> wrote:
>>
>>> The fun goes on :) When I click preview for csv or xlsx file it get this:
>>>
>>> This resource cannot be previewed. More details...
>>> No handler defined for data type: xls.
>>>
>>> When filestore is in place I can upload files and download them,
>>> that works
>>>
>>> When I have added datastore to the party and running celeryd I should see preview of the file and be able to do something with it on the page? Or not? I´m in the dark with the functions?
>>>
>>> -----Original Message-----
>>> From: ckan-dev-bounces at lists.okfn.org
>>> [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Stefán Þór
>>> Finnsson
>>> Sent: 6. mars 2013 14:19
>>> To: CKAN Development Discussions
>>> Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
>>>
>>> Great, somehow I did not see this document :) thanks :)
>>>
>>> -----Original Message-----
>>> From: ckan-dev-bounces at lists.okfn.org
>>> [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Dominik Moritz
>>> Sent: 5. mars 2013 17:28
>>> To: CKAN Development Discussions
>>> Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
>>>
>>> I hope this answers all your questions:
>>> http://docs.ckan.org/en/latest/datastore.html ;-)
>>>
>>> On 5 Mar 2013, at 16:18, Stefán Þór Finnsson <stefan.thor.finnsson at reykjavik.is> wrote:
>>>
>>>> Hi
>>>>
>>>> Thanks for this :) - I have change the ckanext/datastore/plugin.py file and now the system starts and the ckan page does work.
>>>>
>>>> But, I can find any good info about how to use the datastore compared to use the filestore, do I have to have both enabled? How do I put data to the datastore? Do I need the datastorer plugin or is there another way to use it?
>>>>
>>>> Would be nice if someone could clear this out for me :)
>>>>
>>>> -----Original Message-----
>>>> From: ckan-dev-bounces at lists.okfn.org
>>>> [mailto:ckan-dev-bounces at lists.okfn.org] On Behalf Of Dominik
>>>> Moritz
>>>> Sent: 28. febrúar 2013 18:18
>>>> To: CKAN Development Discussions
>>>> Subject: Re: [ckan-dev] Datastore setup for ckan 2.0
>>>>
>>>> Hi,
>>>>
>>>> looks like your postgres version does not have support for the concat function. I submitted a pull request [1] that replaces the concat function. You could either cherry-pick the commit, get the patch or wait for it to be in master.
>>>>
>>>> I'm sorry that we did'n catch that during our tests.
>>>>
>>>> Cheers,
>>>> Dominik
>>>>
>>>> [1] https://github.com/okfn/ckan/pull/514
>>>>
>>>> On 28 Feb 2013, at 17:53, Stefán Þór Finnsson <stefan.thor.finnsson at reykjavik.is> wrote:
>>>>
>>>>> Hello :)
>>>>>
>>>>> When I have added the datastore extension and done the config for
>>>>> it I get this error in my ckan log and the system does not work.
>>>>> Without the datastore it works fine. I´m lost in what could be
>>>>> causing this error, anyone here with a quick answer for me :)
>>>>>
>>>>> [Thu Feb 28 16:46:48 2013] [info] [client 82.112.70.95] mod_wsgi (pid=28703, process='', application='ckan-test|'): Loading WSGI script '/usr/local/ckan-test/pyenv/bin/ckan-test.py'.
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] mod_wsgi (pid=28703): Target WSGI script '/usr/local/ckan-test/pyenv/bin/ckan-test.py' cannot be loaded as Python module.
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] mod_wsgi (pid=28703): Exception occurred processing WSGI script '/usr/local/ckan-test/pyenv/bin/ckan-test.py'.
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] Traceback (most recent call last):
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/bin/ckan-test.py", line 11, in <module>
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] application = loadapp('config:%s' % config_filepath)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] return loadobj(APP, uri, name=name, **kw)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] return context.create()
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 710, in create
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] return self.object_type.invoke(self)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] return fix_call(context.object, context.global_conf, **context.local_conf)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/paste/deploy/util.py", line 56, in fix_call
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] val = callable(*args, **kw)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/src/ckan/ckan/config/middleware.py", line 54, in make_app
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] load_environment(global_conf, app_conf)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/src/ckan/ckan/config/environment.py", line 349, in load_environment
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] plugin.configure(config)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/src/ckan/ckanext/datastore/plugin.py", line 68, in configure
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] self._create_alias_table()
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/src/ckan/ckanext/datastore/plugin.py", line 206, in _create_alias_table
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] connection.execute(create_alias_table_sql)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1449, in execute
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] params)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1628, in _execute_text
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] statement, parameters
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1698, in _execute_context
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] context)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1691, in _execute_context
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] context)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] File "/usr/local/ckan-test/pyenv/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 331, in do_execute
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] cursor.execute(statement, parameters)
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] ProgrammingError: (ProgrammingError) function concat(name, name) does not exist
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] LINE 3: substr(md5(concat(dependee.relname, dependen...
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] ^
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] HINT: No function matches the given name and argument types. You might need to add explicit type casts.
>>>>> [Thu Feb 28 16:46:49 2013] [error] [client 82.112.70.95] 'CREATE OR REPLACE VIEW "_table_metadata" AS \\n SELECT DISTINCT\\n substr(md5(concat(dependee.relname, dependent.relname)), 0, 17) AS "_id",\\n dependee.relname AS name,\\n dependee.oid AS oid,\\n dependent.relname AS alias_of\\n -- dependent.oid AS oid\\n FROM\\n pg_class AS dependee\\n LEFT OUTER JOIN pg_rewrite AS r ON r.ev_class = dependee.oid\\n LEFT OUTER JOIN pg_depend AS d ON d.objid = r.oid\\n LEFT OUTER JOIN pg_class AS dependent ON d.refobjid = dependent.oid\\n WHERE\\n (dependee.oid != dependent.oid OR dependent.oid IS NULL) AND\\n (dependee.relname IN (SELECT tablename FROM pg_catalog.pg_tables)\\n OR dependee.relname IN (SELECT viewname FROM pg_catalog.pg_views)) AND\\n dependee.relnamespace = (SELECT oid FROM pg_namespace WHERE nspname=\\'public\\')\\n ORDER BY dependee.oid DESC;\\n ' {}
>>>>>
>>>>> __________________________________
>>>>> Reglur um trúnað í tölvupóstsamskiptum:
>>>>> http://www.reykjavik.is/trunadur
>>>>>
>>>>> Leiddu hugann að umhverfinu áður en þú prentar út þennan tölvupóst. Prentaðu á báðar hliðar í svarthvítu ef nauðsyn krefur.
>>>>> _______________________________________________
>>>>> ckan-dev mailing list
>>>>> ckan-dev at lists.okfn.org
>>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
_______________________________________________
ckan-dev mailing list
ckan-dev at lists.okfn.org
http://lists.okfn.org/mailman/listinfo/ckan-dev
Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
More information about the ckan-dev
mailing list