[ckan-changes] [okfn/ckan] 6652bd: [xs] allow qualified=True in url_for
GitHub
noreply at github.com
Tue Apr 24 13:05:47 UTC 2012
Branch: refs/heads/master
Home: https://github.com/okfn/ckan
Commit: 6652bd6806ccf115366c37ce719fda99b275a71e
https://github.com/okfn/ckan/commit/6652bd6806ccf115366c37ce719fda99b275a71e
Author: Toby <toby.junk at gmail.com>
Date: 2012-04-24 (Tue, 24 Apr 2012)
Changed paths:
M ckan/lib/helpers.py
Log Message:
-----------
[xs] allow qualified=True in url_for
diff --git a/ckan/lib/helpers.py b/ckan/lib/helpers.py
index dfdc0a8..78924b4 100644
--- a/ckan/lib/helpers.py
+++ b/ckan/lib/helpers.py
@@ -112,6 +112,9 @@ def _add_i18n_to_url(url_to_amend, **kw):
root = request.environ.get('SCRIPT_NAME', '')
except TypeError:
root = ''
+ if kw.get('qualified', False):
+ # if qualified is given we want the full url ie http://...
+ root = _routes_default_url_for('/', qualified=True)[:-1] + root
# ckan.root_path is defined when we have none standard language
# position in the url
root_path = config.get('ckan.root_path', None)
================================================================
Commit: 4e0b8089e7084a5d4a5e1e767dbc94dc27b47e3d
https://github.com/okfn/ckan/commit/4e0b8089e7084a5d4a5e1e767dbc94dc27b47e3d
Author: Toby <toby.junk at gmail.com>
Date: 2012-04-24 (Tue, 24 Apr 2012)
Changed paths:
M ckan/templates/user/layout.html
M doc/_themes/sphinx-theme-okfn
M doc/api.rst
A doc/contrib.rst
M doc/index.rst
M doc/using-data-api.rst
Log Message:
-----------
Merge branch 'master' of github.com:okfn/ckan
diff --git a/ckan/templates/user/layout.html b/ckan/templates/user/layout.html
index cf4d02c..2de8d35 100644
--- a/ckan/templates/user/layout.html
+++ b/ckan/templates/user/layout.html
@@ -14,7 +14,7 @@
</py:when>
<py:otherwise>
<py:if test="c.id">
- <li class="${'active' if c.action=='read' else ''}"><a href="${h.url_for(controller='user', action='me')}">View Profile</a></li>
+ <li class="${'active' if c.action=='read' else ''}"><a href="${h.url_for(controller='user', action='read', id=c.user_dict.name)}">View Profile</a></li>
</py:if>
<py:if test="not c.id">
<li class="${'active' if c.action=='login' else ''}"><a href="${h.url_for(controller='user', action='login')}">Login</a></li>
diff --git a/doc/_themes/sphinx-theme-okfn b/doc/_themes/sphinx-theme-okfn
index ef101a1..fd96c62 160000
--- a/doc/_themes/sphinx-theme-okfn
+++ b/doc/_themes/sphinx-theme-okfn
@@ -1 +1 @@
-Subproject commit ef101a18d6de959207361e233b89efd3be24e66f
+Subproject commit fd96c62b6426a3cc72b3c70314bc4af25e265cb3
diff --git a/doc/api.rst b/doc/api.rst
index 98d84dc..51deb4a 100644
--- a/doc/api.rst
+++ b/doc/api.rst
@@ -91,6 +91,8 @@ key is not authorized for the operation, or the header is somehow malformed,
then the requested operation will not be carried out and the CKAN API will
respond with status code 403.
+.. _get-api-key:
+
Obtaining an API key
--------------------
diff --git a/doc/contrib.rst b/doc/contrib.rst
new file mode 100644
index 0000000..87e1967
--- /dev/null
+++ b/doc/contrib.rst
@@ -0,0 +1,73 @@
+=================
+Contrib and Tools
+=================
+
+This is a a place for code, scripts and applications that extend CKAN in some
+way (but which aren't extensions).
+
+.. note:: Have something to add to this page? Please either email ``info [at] ckan [dot]
+ org`` or just edit this `page in the CKAN repo`_.
+
+.. _page in the CKAN repo: https://github.com/okfn/ckan/blob/master/doc/contrib.rst
+
+Google Docs Integration
+=======================
+
+Integration of Google docs spreadsheets with CKAN. Google app script script which supports:
+
+* Publish a Google docs spreadsheet as a dataset on CKAN
+* Push and pull metadata from a CKAN instance to a Google docs spreadsheet
+
+Install: via the google docs Script Gallery in the tools menu (or use the source).
+
+Repo: https://github.com/okfn/ckan-google-docs
+
+Google Refine Extension for CKAN
+================================
+
+This extension allows data of Google Refine projects to be uploaded to CKAN
+Storage and connected to a dataset on a running CKAN instance (for example
+http://thedatahub.org).
+
+* http://lab.linkeddata.deri.ie/2011/grefine-ckan/
+* Intro: http://ckan.org/2011/07/05/google-refine-extension-for-ckan/
+* Author: Fadi Maali (DERI)
+
+Repo: https://github.com/fadmaa/grefine-ckan-storage-extension
+
+Embeddable Dataset Count Widget
+===============================
+
+Simple group count widget in pure javascript for embedding in other websites.
+
+http://okfnlabs.org/ckanjs/widgets/count/
+
+Embeddable Dataset Listing Widget
+=================================
+
+http://okfnlabs.org/ckanjs/widgets/search/
+
+SPARQL Endpoint Status Checker
+==============================
+
+Status of SPARQL endpoints (in the DataHub).
+
+http://labs.mondeca.com/sparqlEndpointsStatus/index.html
+
+* Author: Mondeca
+
+PublicData.eu Data map
+======================
+
+http://publicdata.eu/map
+
+VoID metadata generator
+=======================
+
+Useful to automatically correct/maintain/enrich dataset metadata
+
+* voidGen : http://www.hpi.uni-potsdam.de/naumann/projects/btc
+* ve2 : http://lab.linkeddata.deri.ie/ve2/
+* (POC by Richard) : https://github.com/cygri/make-void
+* (POC by Pierre-Yves) : http://labs.mondeca.com/vocabuse/
+
diff --git a/doc/index.rst b/doc/index.rst
index 67e1153..b1c71f0 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -93,8 +93,9 @@ Other material
==============
.. toctree::
- :maxdepth: 1
+ :maxdepth: 2
+ contrib.rst
CHANGELOG.rst
Indices and tables
diff --git a/doc/using-data-api.rst b/doc/using-data-api.rst
index 3c6a6fe..09caf21 100644
--- a/doc/using-data-api.rst
+++ b/doc/using-data-api.rst
@@ -17,16 +17,16 @@ Furthermore, it means that what is presented below is essentially a tutorial in
Quickstart
==========
-``endpoint`` refers to the data API endpoint (or ElasticSearch index / table).
+``{{endpoint}}`` refers to the data API endpoint (or ElasticSearch index / table).
Key urls:
-* Query: ``{endpoint}/_search`` (in ElasticSearch < 0.19 this will return an
+* Query: ``{{endpoint}}/_search`` (in ElasticSearch < 0.19 this will return an
error if visited without a query parameter)
- * Query example: ``{endpoint}/_search?size=5&pretty=true``
+ * Query example: ``{{endpoint}}/_search?size=5&pretty=true``
-* Schema (Mapping): ``{endpoint}/_mapping``
+* Schema (Mapping): ``{{endpoint}}/_mapping``
Examples
--------
@@ -34,15 +34,26 @@ Examples
cURL (or Browser)
~~~~~~~~~~~~~~~~~
-The following examples utilize the <a href="http://curl.haxx.se/">cURL</a>
-command line utility. If you prefer, you you can just open the relevant urls in
-your browser::
+The following examples utilize the cURL_ command line utility. If you prefer,
+you you can just open the relevant urls in your browser::
+ // query for documents / rows with title field containing 'jones'
// added pretty=true to get the json results pretty printed
- curl {endpoint}/_search?q=title:jones&size=5&pretty=true</pre>
+ curl {{endpoint}}/_search?q=title:jones&size=5&pretty=true
+
+Adding some data (requires an :ref:`API Key <get-api-key>`)::
+
+ // requires an API key
+ // Data (argument to -d) should be a JSON document
+ curl -X POST -H "Authorization: {{YOUR-API-KEY}}" {{endpoint}} -d '{
+ "title": "jones",
+ "amount": 5.7
+ }'
+
+.. _cURL: http://curl.haxx.se/
Javascript
-~~~~~~~~~~~
+~~~~~~~~~~
A simple ajax (JSONP) request to the data API using jQuery::
@@ -51,13 +62,71 @@ A simple ajax (JSONP) request to the data API using jQuery::
q: 'title:jones' // query on the title field for 'jones'
};
$.ajax({
- url: {endpoint}/_search,
+ url: {{endpoint}}/_search,
dataType: 'jsonp',
success: function(data) {
alert('Total results found: ' + data.hits.total)
}
});
+The Data API supports CORs so you can also write to it (this requires the json2_ library for ``JSON.stringify``)::
+
+ var data = {
+ title: 'jones',
+ amount: 5.7
+ };
+ $.ajax({
+ url: {{endpoint}},
+ type: 'POST',
+ data: JSON.stringify(data),
+ success: function(data) {
+ alert('Uploaded ok')
+ }
+ });
+
+.. _json2: https://github.com/douglascrockford/JSON-js/blob/master/json2.js
+
+Python
+~~~~~~
+
+.. note:: You can also use the `DataStore Python client library`_.
+
+.. _DataStore Python client library: http://github.com/okfn/datastore-client
+
+::
+
+ import urllib2
+ import json
+
+ # =================================
+ # Store data in the DataStore table
+
+ url = '{{endpoint}}'
+ data = {
+ 'title': 'jones',
+ 'amount': 5.7
+ }
+ # have to send the data as JSON
+ data = json.dumps(data)
+ # need to add your API key (and have authorization to write to this endpoint)
+ headers = {'Authorization': 'YOUR-API-KEY'}
+
+ req = urllib2.Request(url, data, headers)
+ out = urllib2.urlopen(req)
+ print out.read()
+
+ # =========================
+ # Query the DataStore table
+
+ url = '{{endpoint}}/_search?q=title:jones&size=5'
+ req = urllib2.Request(url)
+ out = urllib2.urlopen(req)
+ data = out.read()
+ print data
+ # returned data is JSON
+ data = json.loads(data)
+ # total number of results
+ print data['hits']['total']
Querying
========
@@ -69,7 +138,7 @@ Basic queries can be done using only query string parameters in the URL. For
example, the following searches for text 'hello' in any field in any document
and returns at most 5 results::
- {endpoint}/_search?q=hello&size=5
+ {{endpoint}}/_search?q=hello&size=5
Basic queries like this have the advantage that they only involve accessing a
URL and thus, for example, can be performed just using any web browser.
@@ -99,15 +168,15 @@ options for how a query is sent to the search endpoint:
1. Either as the value of a source query parameter e.g.::
- {endpoint}/_search?source={Query-as-JSON}
+ {{endpoint}}/_search?source={Query-as-JSON}
2. Or in the request body, e.g.::
- curl -XGET {endpoint}/_search -d 'Query-as-JSON'
+ curl -XGET {{endpoint}}/_search -d 'Query-as-JSON'
For example::
- curl -XGET {endpoint}/_search -d '{
+ curl -XGET {{endpoint}}/_search -d '{
"query" : {
"term" : { "user": "kimchy" }
}
@@ -364,13 +433,54 @@ result here using a `bool query`_.
}
}
+
Facets
------
Facets provide a way to get summary information about then data in an
elasticsearch table, for example counts of distinct values.
-TODO: complete
+ElasticSearch (and hence the Data API) provides rich faceting capabilities:
+http://www.elasticsearch.org/guide/reference/api/search/facets/
+
+There are various kinds of facets available, for example (full list on the facets page):
+
+* Terms_ - counts by distinct terms (values) in a field
+* Range_ - counts for a given set of ranges in a field
+* Histogram_ and `Date Histogram`_ - counts by constant interval ranges
+* Statistical_ - statistical summary of a field (mean, sum etc)
+* `Terms Stats`_ - statistical summary on one field (stats field) for distinct
+ terms in another field. For example, spending stats per department or per
+ region.
+* `Geo Distance`_: counts by distance ranges from a given point
+
+Note that you can apply multiple facets per query.
+
+.. _Terms: http://www.elasticsearch.org/guide/reference/api/search/facets/terms-facet.html
+.. _Range: http://www.elasticsearch.org/guide/reference/api/search/facets/range-facet.html
+.. _Histogram: http://www.elasticsearch.org/guide/reference/api/search/facets/histogram-facet.html
+.. _Date Histogram: http://www.elasticsearch.org/guide/reference/api/search/facets/date-histogram-facet.html
+.. _Statistical: http://www.elasticsearch.org/guide/reference/api/search/facets/statistical-facet.html
+.. _Terms Stats: http://www.elasticsearch.org/guide/reference/api/search/facets/terms-stats-facet.html
+.. _Geo Distance: http://www.elasticsearch.org/guide/reference/api/search/facets/geo-distance-facet.html
+
+
+Adding, Updating and Deleting Data
+==================================
+
+ElasticSeach, and hence the Data API, have a standard RESTful API. Thus::
+
+ POST {{endpoint}} : INSERT
+ PUT/POST {{endpoint}}/{{id}} : UPDATE (or INSERT)
+ DELETE {{endpoint}}/{{id}} : DELETE
+
+For more on INSERT and UPDATE see the `Index API`_ documentation.
+
+.. _Index API: http://elasticsearch.org/guide/reference/api/index_.html
+
+There is also support bulk insert and updates via the `Bulk API`_.
+
+.. _Bulk API: http://elasticsearch.org/guide/reference/api/bulk.html
Schema Mapping
================================================================
Commit: e2d976a3276cd71315630dd04bd16ac6aae37a92
https://github.com/okfn/ckan/commit/e2d976a3276cd71315630dd04bd16ac6aae37a92
Author: Toby <toby.junk at gmail.com>
Date: 2012-04-24 (Tue, 24 Apr 2012)
Log Message:
-----------
Merge branch 'master' of github.com:okfn/ckan
================================================================
Compare: https://github.com/okfn/ckan/compare/c354509...e2d976a
More information about the ckan-changes
mailing list