[ckan-changes] commit/ckanext-spatial: 2 new changesets
Bitbucket
commits-noreply at bitbucket.org
Mon Oct 17 09:02:59 UTC 2011
2 new changesets in ckanext-spatial:
http://bitbucket.org/okfn/ckanext-spatial/changeset/5dc53688d26c/
changeset: 5dc53688d26c
user: amercader
date: 2011-10-03 14:39:18
summary: Use datasets instead of packages
affected #: 3 files (-1 bytes)
--- a/README.rst Fri Sep 30 18:09:21 2011 +0100
+++ b/README.rst Mon Oct 03 13:39:18 2011 +0100
@@ -6,7 +6,7 @@
The following plugins are currently available:
* Automatic geo-indexing and spatial API call (`spatial_query`).
-* Map widget showing a package extent (`dataset_extent_map`).
+* Map widget showing a dataset extent (`dataset_extent_map`).
* A Web Map Service (WMS) previewer (`wms_preview`).
Dependencies
@@ -79,7 +79,7 @@
You can privide the SRID of the geometry column. Default is 4326.
extents
- - creates or updates the extent geometry column for packages with
+ - creates or updates the extent geometry column for datasets with
an extent defined in the 'spatial' extra.
The commands should be run from the ckanext-spatial directory and expect
@@ -96,9 +96,9 @@
ini file (See 'Configuration').
The extension adds the following call to the CKAN search API, which returns
-packages with an extent that intersects with the bounding box provided::
+datasets with an extent that intersects with the bounding box provided::
- /api/2/search/package/geo?bbox={minx,miny,maxx,maxy}[&crs={srid}]
+ /api/2/search/dataset/geo?bbox={minx,miny,maxx,maxy}[&crs={srid}]
If the bounding box coordinates are not in the same projection as the one
defined in the database, a CRS must be provided, in one of the following
@@ -108,10 +108,10 @@
- EPSG:4326
- 4326
-Geo-Indexing your packages
+Geo-Indexing your datasets
--------------------------
-In order to make a package queryable by location, an special extra must
+In order to make a dataset queryable by location, an special extra must
be defined, with its key named 'spatial'. The value must be a valid GeoJSON_
geometry, for example::
@@ -123,7 +123,7 @@
.. _GeoJSON: http://geojson.org
-Every time a package is created, updated or deleted, the extension will synchronize
+Every time a dataset is created, updated or deleted, the extension will synchronize
the information stored in the extra with the geometry table.
--- a/ckanext/spatial/commands/spatial.py Fri Sep 30 18:09:21 2011 +0100
+++ b/ckanext/spatial/commands/spatial.py Mon Oct 03 13:39:18 2011 +0100
@@ -18,7 +18,7 @@
You can provide the SRID of the geometry column. Default is 4326.
spatial extents
- Creates or updates the extent geometry column for packages with
+ Creates or updates the extent geometry column for datasets with
an extent defined in the 'spatial' extra.
The commands should be run from the ckanext-spatial directory and expect
--- a/ckanext/spatial/plugin.py Fri Sep 30 18:09:21 2011 +0100
+++ b/ckanext/spatial/plugin.py Mon Oct 03 13:39:18 2011 +0100
@@ -29,10 +29,9 @@
def before_map(self, map):
- map.connect('api_spatial_query', '/api/2/search/package/geo',
+ map.connect('api_spatial_query', '/api/2/search/{register:dataset|package}/geo',
controller='ckanext.spatial.controllers.api:ApiController',
action='spatial_query')
-
return map
def create(self, package):
@@ -154,10 +153,6 @@
controller='ckanext.spatial.controllers.view:ViewController',
action='proxy')
- map.connect('api_spatial_query', '/api/2/search/package/geo',
- controller='ckanext.spatial.controllers.api:ApiController',
- action='spatial_query')
-
return map
def update_config(self, config):
http://bitbucket.org/okfn/ckanext-spatial/changeset/0c12873ec441/
changeset: 0c12873ec441
user: amercader
date: 2011-10-17 00:40:19
summary: [tests] Add tests
affected #: 9 files (-1 bytes)
--- a/README.rst Mon Oct 03 13:39:18 2011 +0100
+++ b/README.rst Sun Oct 16 23:40:19 2011 +0100
@@ -65,7 +65,14 @@
ckan.spatial.srid = 4326
+Tests
+=====
+Please note that the tests currently only work with Postgres. You must use the
+test-core.ini located in the extension directory to run them. Most of the time
+you should run something like::
+
+ nosetests --ckan --with-pylons=test-core.ini ckanext/spatial/tests
Command line interface
======================
--- a/ckanext/spatial/plugin.py Mon Oct 03 13:39:18 2011 +0100
+++ b/ckanext/spatial/plugin.py Sun Oct 16 23:40:19 2011 +0100
@@ -41,6 +41,11 @@
self.check_spatial_extra(package)
def check_spatial_extra(self,package):
+ if not package.id:
+ log.warning('Couldn\'t store spatial extent because no id was provided for the package')
+ return
+
+ # TODO: deleted extra
for extra in package.extras_list:
if extra.key == 'spatial':
if extra.state == 'active':
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ckanext/spatial/tests/__init__.py Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,50 @@
+import os
+import re
+from sqlalchemy import Table
+from ckan.model import Session,repo, meta
+from ckanext.spatial.model import setup as spatial_db_setup
+
+def setup_postgis_tables():
+
+ conn = Session.connection()
+ script_path = os.path.join(os.path.dirname(os.path.abspath( __file__ )), 'scripts', 'postgis.sql')
+ script = open(script_path,'r').read()
+ for cmd in script.split(';'):
+ cmd = re.sub(r'--(.*)|[\n\t]','',cmd)
+ if len(cmd):
+ conn.execute(cmd)
+
+ Session.commit()
+
+
+class SpatialTestBase:
+
+ db_srid = 4326
+
+ geojson_examples = {
+ 'point':'{"type":"Point","coordinates":[100.0,0.0]}',
+ 'line':'{"type":"LineString","coordinates":[[100.0,0.0],[101.0,1.0]]}',
+ 'polygon':'{"type":"Polygon","coordinates":[[[100.0,0.0],[101.0,0.0],[101.0,1.0],[100.0,1.0],[100.0,0.0]]]}',
+ 'polygon_holes':'{"type":"Polygon","coordinates":[[[100.0,0.0],[101.0,0.0],[101.0,1.0],[100.0,1.0],[100.0,0.0]],[[100.2,0.2],[100.8,0.2],[100.8,0.8],[100.2,0.8],[100.2,0.2]]]}',
+ 'multipoint':'{"type":"MultiPoint","coordinates":[[100.0,0.0],[101.0,1.0]]}',
+ 'multiline':'{"type":"MultiLineString","coordinates":[[[100.0,0.0],[101.0,1.0]],[[102.0,2.0],[103.0,3.0]]]}',
+ 'multipolygon':'{"type":"MultiPolygon","coordinates":[[[[102.0,2.0],[103.0,2.0],[103.0,3.0],[102.0,3.0],[102.0,2.0]]],[[[100.0,0.0],[101.0,0.0],[101.0,1.0],[100.0,1.0],[100.0,0.0]],[[100.2,0.2],[100.8,0.2],[100.8,0.8],[100.2,0.8],[100.2,0.2]]]]}'}
+
+ @classmethod
+ def setup_class(cls):
+
+ # This will create the PostGIS tables (geometry_columns and
+ # spatial_ref_sys) which were deleted when rebuilding the database
+ table = Table('geometry_columns', meta.metadata)
+ if not table.exists():
+ setup_postgis_tables()
+
+ # Create the package_extent table
+ table = Table('package_extent', meta.metadata)
+ if not table.exists():
+ spatial_db_setup()
+
+ @classmethod
+ def teardown_class(cls):
+ repo.rebuild_db()
+
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ckanext/spatial/tests/scripts/postgis.sql Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,41 @@
+-------------------------------------------------------------------
+-- WARNING: This is probably the file you are looking for.
+-- This file is intended to be used only during tests, you won't
+-- get a functional PostGIS database executing it. Please install
+-- PostGIS as described in the README.
+-------------------------------------------------------------------
+
+-------------------------------------------------------------------
+-- SPATIAL_REF_SYS
+-------------------------------------------------------------------
+CREATE TABLE spatial_ref_sys (
+ srid integer not null primary key,
+ auth_name varchar(256),
+ auth_srid integer,
+ srtext varchar(2048),
+ proj4text varchar(2048)
+);
+
+-------------------------------------------------------------------
+-- GEOMETRY_COLUMNS
+-------------------------------------------------------------------
+CREATE TABLE geometry_columns (
+ f_table_catalog varchar(256) not null,
+ f_table_schema varchar(256) not null,
+ f_table_name varchar(256) not null,
+ f_geometry_column varchar(256) not null,
+ coord_dimension integer not null,
+ srid integer not null,
+ type varchar(30) not null,
+ CONSTRAINT geometry_columns_pk primary key (
+ f_table_catalog,
+ f_table_schema,
+ f_table_name,
+ f_geometry_column )
+) WITH OIDS;
+
+---
+--- EPSG 4326 : WGS 84
+---
+INSERT INTO "spatial_ref_sys" ("srid","auth_name","auth_srid","srtext","proj4text") VALUES (4326,'EPSG',4326,'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]','+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs ');
+
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ckanext/spatial/tests/test_api.py Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,74 @@
+import logging
+from pprint import pprint
+
+from ckan.logic.action.create import package_create
+from ckan.logic.action.delete import package_delete
+from ckan import model
+
+from ckan.model import Package, Session
+import ckan.lib.search as search
+from ckan.tests import CreateTestData, setup_test_search_index
+from ckan.tests.functional.api.base import ApiTestCase
+from ckan.tests import TestController as ControllerTestCase
+from ckanext.spatial.tests import SpatialTestBase
+
+log = logging.getLogger(__name__)
+
+
+
+class TestSpatialApi(ApiTestCase,SpatialTestBase,ControllerTestCase):
+
+ api_version = '2'
+
+ @classmethod
+ def setup_class(self):
+ super(TestSpatialApi,self).setup_class()
+ setup_test_search_index()
+ CreateTestData.create_test_user()
+ self.package_fixture_data = {
+ 'name' : u'test-spatial-dataset-search-point',
+ 'title': 'Some Title',
+ 'extras': [{'key':'spatial','value':self.geojson_examples['point']}]
+ }
+ self.base_url = self.offset('/search/dataset/geo')
+
+ def _offset_with_bbox(self,minx=-180,miny=-90,maxx=180,maxy=90,crs=None):
+ offset = self.base_url + '?bbox=%s,%s,%s,%s' % (minx,miny,maxx,maxy)
+ if crs:
+ offset = offset + '&crs=%s' % crs
+ return offset
+
+ def test_basic_query(self):
+ context = {'model':model,'session':Session,'user':'tester','extras_as_string':True}
+ package_dict = package_create(context,self.package_fixture_data)
+
+ # Point inside bbox
+ offset = self._offset_with_bbox()
+
+ res = self.app.get(offset, status=200)
+ res_dict = self.data_from_res(res)
+
+ assert res_dict['count'] == 1
+ assert res_dict['results'][0] == package_dict['id']
+
+ # Point outside bbox
+ offset = self._offset_with_bbox(-10,10,-20,20)
+
+ res = self.app.get(offset, status=200)
+ res_dict = self.data_from_res(res)
+
+ assert res_dict['count'] == 0
+ assert res_dict['results'] == []
+
+ # Delete the package and ensure it does not come up on
+ # search results
+ package_delete(context,{'id':package_dict['id']})
+
+ offset = self._offset_with_bbox()
+
+ res = self.app.get(offset, status=200)
+ res_dict = self.data_from_res(res)
+
+ assert res_dict['count'] == 0
+ assert res_dict['results'] == []
+
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ckanext/spatial/tests/test_dataset_map.py Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,39 @@
+import logging
+from pprint import pprint
+
+from ckan.model import Package, Session
+from ckan.lib.helpers import url_for,json
+
+from ckan.tests import CreateTestData
+from ckan.tests.functional.base import FunctionalTestCase
+
+from ckanext.spatial.tests import SpatialTestBase
+
+log = logging.getLogger(__name__)
+
+
+class TestDatasetMap(FunctionalTestCase,SpatialTestBase):
+
+ def test_map_shown(self):
+ CreateTestData.create()
+
+ name = 'annakarenina'
+
+ offset = url_for(controller='package', action='edit',id=name)
+ res = self.app.get(offset)
+ assert 'Edit - Datasets' in res
+ fv = res.forms['dataset-edit']
+ prefix = ''
+ fv[prefix+'extras__1__key'] = u'spatial'
+ fv[prefix+'extras__1__value'] = self.geojson_examples['point']
+
+ res = fv.submit('save')
+ assert not 'Error' in res, res
+
+ # Load the dataset page and check if the libraries have been loaded
+ offset = url_for(controller='package', action='read',id=name)
+ res = self.app.get(offset)
+
+ assert '<div class="dataset-map subsection">' in res, res
+ assert '<script type="text/javascript" src="/ckanext/spatial/js/dataset_map.js"></script>' in res
+ assert self.geojson_examples['point'] in res
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ckanext/spatial/tests/test_functional.py Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,135 @@
+import logging
+from pprint import pprint
+
+from ckan.model import Package, Session
+from ckan.lib.helpers import url_for,json
+
+from ckan.tests import CreateTestData
+from ckan.tests.functional.base import FunctionalTestCase
+from ckanext.spatial.model import PackageExtent
+
+from ckanext.spatial.tests import SpatialTestBase
+
+log = logging.getLogger(__name__)
+
+
+class TestPackageController(FunctionalTestCase,SpatialTestBase):
+ def setup(self):
+ CreateTestData.create()
+
+ def teardown(self):
+ CreateTestData.delete()
+
+ def test_new(self):
+ name = 'test-spatial-dataset-1'
+
+ offset = url_for(controller='package', action='new')
+ res = self.app.get(offset)
+ assert 'Add - Datasets' in res
+ fv = res.forms['dataset-edit']
+ prefix = ''
+ fv[prefix + 'name'] = name
+ fv[prefix+'extras__0__key'] = u'spatial'
+ fv[prefix+'extras__0__value'] = self.geojson_examples['point']
+
+ res = fv.submit('save')
+ assert not 'Error' in res, res
+
+ package = Package.get(name)
+
+ # Check that a PackageExtent object has been created
+ package_extent = Session.query(PackageExtent).filter(PackageExtent.package_id==package.id).first()
+
+ geojson = json.loads(self.geojson_examples['point'])
+
+ assert package_extent
+ assert package_extent.package_id == package.id
+ assert Session.scalar(package_extent.the_geom.x) == geojson['coordinates'][0]
+ assert Session.scalar(package_extent.the_geom.y) == geojson['coordinates'][1]
+ assert Session.scalar(package_extent.the_geom.srid) == self.db_srid
+
+ def test_new_bad_json(self):
+ name = 'test-spatial-dataset-2'
+
+ offset = url_for(controller='package', action='new')
+ res = self.app.get(offset)
+ assert 'Add - Datasets' in res
+ fv = res.forms['dataset-edit']
+ prefix = ''
+ fv[prefix + 'name'] = name
+ fv[prefix+'extras__0__key'] = u'spatial'
+ fv[prefix+'extras__0__value'] = u'{"Type":Bad Json]'
+
+ res = fv.submit('save')
+ assert 'Error' in res, res
+ assert 'Spatial' in res
+ assert 'Error decoding JSON object' in res
+
+ # Check that package was not created
+ assert not Package.get(name)
+
+ def test_new_bad_geojson(self):
+ name = 'test-spatial-dataset-3'
+
+ offset = url_for(controller='package', action='new')
+ res = self.app.get(offset)
+ assert 'Add - Datasets' in res
+ fv = res.forms['dataset-edit']
+ prefix = ''
+ fv[prefix + 'name'] = name
+ fv[prefix+'extras__0__key'] = u'spatial'
+ fv[prefix+'extras__0__value'] = u'{"Type":"Bad_GeoJSON","a":2}'
+
+ res = fv.submit('save')
+ assert 'Error' in res, res
+ assert 'Spatial' in res
+ assert 'Error creating geometry' in res
+
+ # Check that package was not created
+ assert not Package.get(name)
+
+ def test_edit(self):
+
+ name = 'annakarenina'
+
+ offset = url_for(controller='package', action='edit',id=name)
+ res = self.app.get(offset)
+ assert 'Edit - Datasets' in res
+ fv = res.forms['dataset-edit']
+ prefix = ''
+ fv[prefix+'extras__1__key'] = u'spatial'
+ fv[prefix+'extras__1__value'] = self.geojson_examples['point']
+
+ res = fv.submit('save')
+ assert not 'Error' in res, res
+
+ package = Package.get(name)
+
+ # Check that a PackageExtent object has been created
+ package_extent = Session.query(PackageExtent).filter(PackageExtent.package_id==package.id).first()
+ geojson = json.loads(self.geojson_examples['point'])
+
+ assert package_extent
+ assert package_extent.package_id == package.id
+ assert Session.scalar(package_extent.the_geom.x) == geojson['coordinates'][0]
+ assert Session.scalar(package_extent.the_geom.y) == geojson['coordinates'][1]
+ assert Session.scalar(package_extent.the_geom.srid) == self.db_srid
+
+ # Update the spatial extra
+ offset = url_for(controller='package', action='edit',id=name)
+ res = self.app.get(offset)
+ assert 'Edit - Datasets' in res
+ fv = res.forms['dataset-edit']
+ prefix = ''
+ fv[prefix+'extras__1__value'] = self.geojson_examples['polygon']
+
+ res = fv.submit('save')
+ assert not 'Error' in res, res
+
+ # Check that the PackageExtent object has been updated
+ package_extent = Session.query(PackageExtent).filter(PackageExtent.package_id==package.id).first()
+ assert package_extent
+ assert package_extent.package_id == package.id
+ assert Session.scalar(package_extent.the_geom.geometry_type) == 'ST_Polygon'
+ assert Session.scalar(package_extent.the_geom.srid) == self.db_srid
+
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/ckanext/spatial/tests/test_model.py Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,60 @@
+import logging
+from pprint import pprint
+
+from geoalchemy import WKTSpatialElement
+
+from shapely.geometry import asShape
+from ckan.model import Session, Package
+from ckan.lib.helpers import json
+from ckan.tests import CreateTestData
+from ckanext.spatial.model import PackageExtent
+
+from ckanext.spatial.tests import SpatialTestBase
+
+log = logging.getLogger(__name__)
+
+
+
+class TestPackageExtent(SpatialTestBase):
+ def setup(self):
+ CreateTestData.create()
+
+ def teardown(self):
+ CreateTestData.delete()
+
+ def test_create_extent(self):
+ package = Package.get('annakarenina')
+ assert package
+
+ geojson = json.loads(self.geojson_examples['point'])
+
+ shape = asShape(geojson)
+ package_extent = PackageExtent(package_id=package.id,the_geom=WKTSpatialElement(shape.wkt, self.db_srid))
+ package_extent.save()
+
+ assert package_extent.package_id == package.id
+ assert Session.scalar(package_extent.the_geom.x) == geojson['coordinates'][0]
+ assert Session.scalar(package_extent.the_geom.y) == geojson['coordinates'][1]
+ assert Session.scalar(package_extent.the_geom.srid) == self.db_srid
+
+ def test_update_extent(self):
+
+ package = Package.get('annakarenina')
+
+ geojson = json.loads(self.geojson_examples['point'])
+
+ shape = asShape(geojson)
+ package_extent = PackageExtent(package_id=package.id,the_geom=WKTSpatialElement(shape.wkt, self.db_srid))
+ package_extent.save()
+ assert Session.scalar(package_extent.the_geom.geometry_type) == 'ST_Point'
+
+ # Update the geometry (Point -> Polygon)
+ geojson = json.loads(self.geojson_examples['polygon'])
+
+ shape = asShape(geojson)
+ package_extent.the_geom=WKTSpatialElement(shape.wkt, self.db_srid)
+ package_extent.save()
+
+ assert package_extent.package_id == package.id
+ assert Session.scalar(package_extent.the_geom.geometry_type) == 'ST_Polygon'
+ assert Session.scalar(package_extent.the_geom.srid) == self.db_srid
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/test-core.ini Sun Oct 16 23:40:19 2011 +0100
@@ -0,0 +1,54 @@
+[DEFAULT]
+debug = true
+# Uncomment and replace with the address which should receive any error reports
+#email_to = you at yourdomain.com
+smtp_server = localhost
+error_email_from = paste at localhost
+
+[server:main]
+use = egg:Paste#http
+host = 0.0.0.0
+port = 5000
+
+
+[app:main]
+use = config:../ckan/test-core.ini
+# Here we hard-code the database and a flag to make default tests
+# run fast.
+ckan.plugins = spatial_query dataset_extent_map wms_preview
+# NB: other test configuration should go in test-core.ini, which is
+# what the postgres tests use.
+
+
+# Logging configuration
+[loggers]
+keys = root, ckan, sqlalchemy
+
+[handlers]
+keys = console
+
+[formatters]
+keys = generic
+
+[logger_root]
+level = WARN
+handlers = console
+
+[logger_ckan]
+qualname = ckan
+handlers =
+level = INFO
+
+[logger_sqlalchemy]
+handlers =
+qualname = sqlalchemy.engine
+level = WARN
+
+[handler_console]
+class = StreamHandler
+args = (sys.stdout,)
+level = NOTSET
+formatter = generic
+
+[formatter_generic]
+format = %(asctime)s %(levelname)-5.5s [%(name)s] %(message)s
Repository URL: https://bitbucket.org/okfn/ckanext-spatial/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the ckan-changes
mailing list