[ckan-changes] commit/ckan: 2 new changesets
Bitbucket
commits-noreply at bitbucket.org
Fri Jun 17 14:18:33 UTC 2011
2 new changesets in ckan:
http://bitbucket.org/okfn/ckan/changeset/ec8e74719a63/
changeset: ec8e74719a63
user: rgrp
date: 2011-06-17 16:17:26
summary: [doc/cep][s]: remove doc/cep/* because all C(R)EPs now in separate repo or on trac.
affected #: 3 files (0 bytes)
--- a/doc/cep/cep-0001.txt Thu Jun 16 09:37:07 2011 +0100
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,102 +0,0 @@
-CEP: 1
-Title: Package Identifiers, Renaming and Persistence
-Version: 2010-05-12
-Last-Modified: 2010-05-12
-Author: Rufus Pollock <rufus.pollock at okfn.org>
-Contributors: Sean Burlington, David Read, Evan King
-Status: Active
-Created: 2010-03-20
-
-
-Package Identifiers, Renaming and Persistence
-=============================================
-
-Packages have a name and an id attribute. The name attribute at the
-present aims to be:
-
- * human readable and usable
- * uri-usable (so restricted in characters it can contain)
- * stable but not unchangeable
- * not unique across instances
-
-Id attribute:
-
- * globally unique (uuid like)
- * uri-usable
- * stable and permanent
- * not that *human* (or search-engine) friendly
-
-Currently package names are widely used, and, furthermore are primary
-way of addressing packages in the API and WUI (web user interface).
-
-Because package names can change, the issue has arisen of how this can
-be handled in client applications, especially those using the package
-name as the entity reference or in some other prominent manner.
-
-Persistent Identifiers
-======================
-
-In addition there is the question of generating persistent identifiers
-(and urls), for example, referring to the package in RDF (and creating
-linked data). Here, it seems there is a clear requirment to create
-identifiers that are *permanent*, i.e. stable in perpetuity.
-
-Here there are two options:
-
- * Use names in generation of these persistent identifiers. In this
- case there is a requirement that names never change (or, more
- weakly, that a) names never get re-used b) there is a suitable
- redirect mechanism)
- * Use ids to generate permanent urls
-
-Going for the first option would move package names away from
-debian-like or or wikipedia-like set up where "usurpation" [1]_, though
-rare, is permitted -- i.e. names are generally stable but not guaranteed
-forever.
-
-.. [1]_: http://en.wikipedia.org/wiki/Wikipedia:USURPTITLE#Usurping_a_page_title
-
-Our preference is for the second option as we believe some flexibility
-in naming is useful and that names and ids serve different purposes (see
-layout above).
-
-NB: since uuid urls are rather opaque they are not that attractive in
-the web user interface. Thus, there (and even REST API) it may be better
-to use "names" by default but provide ids for those who want absolute
-stability.
-
-
-Proposed Coure of Action
-========================
-
-Propose the following:
-
- 1. Continuing use of package *names* as *primary* method for addressing
- packages in the web user interface, and API. Thus, for example in
- locations where only a single identifier is returned (e.g. list of
- packages, search) the package name will be provided. However, this does
- not the address the requirements for permanent identifiers (and uris)
- especially for linked data.
-
- 2. Provision of id package information in data returned by API in
- relevant "GET" locations including:
-
- * /api/rest/package/{xxx}
- * /api/search/package/{xxx}
- * /api/rest/revision
-
- Note that provision of package id information in revision and package
- api allow a client to know when package renames have occurred and
- adapt/update accordingly.
-
- 3. Support for *accepting* package ids in API including GET/POST
- locations
-
-
-Additional changes that could be made but propose not to do at the
-present time:
-
- 1. [optional] Provision of a simple method of obtaining name changes
- (specifically). This could be via a special tag on revisions or a
- dedicated API.
-
--- a/doc/cep/cep-0002.txt Thu Jun 16 09:37:07 2011 +0100
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,39 +0,0 @@
-CEP: 2
-Title: API Versioning
-Version: 1.0b
-Last-Modified: 2010-05-27
-Author: Rufus Pollock <rufus.pollock at okfn.org>
-Status: Active
-Created: 2010-05-04
-
-API versioning
-==============
-
-In making the API available for public consumption we have the challenge
-of handling changes (include backwards-breaking changes) in as graceful
-manner as possile.
-
-To do this it is proposed that the API be "versioned" so that users of
-an existing API can be supported through an upgrade to a new (and
-possibly incompatible) API.
-
-We propose to have the following basic URI scheme::
-
- .../api/{version}/*
-
-For example::
-
- /api/{version}/1.0/
-
-Deprecation and removal of APIs:
-
- * As new versions of the API become available it may be necessary to
- deprecate, and then remove, a particular version of the API.
- * Current supported versions of the API will be clearly marked in the
- documentation. API versions scheduled for removal will first be
- deprecated for a reasonable period prior to removal allowing clients
- time to transition.
- * In addition components of an API scheduled for change or removal
- will be clearly marked as such -- following the standard approach
- for software API documentation.
-
--- a/doc/cep/cep-0003.txt Thu Jun 16 09:37:07 2011 +0100
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,96 +0,0 @@
-CEP: 3
-Title: Plugins
-Version: 0.1
-Last-Modified: 2010-06-15
-Author: Rufus Pollock <rufus.pollock at okfn.org>
-Status: Active
-Created: 2010-05-04
-
-Plugins
-=======
-
-Summary:
-
-Plugin entry points:
-
- * [ckan.plugin]
- * model
- * model_create(session)
- * model_upgrade(session, version) method taking a session argument
- * templates.
- * template path (use pkgutil)
- * templates directroy as plugin/{plugin-name}
- * event hooks, integration points
- * ckan has to expose a set of interfaces
- * settings access
-
-
-Authentication
-==============
-
-Wants
------
-
-Have different methods of authentication e.g. openid, form based, external cookie
-
-Current status
---------------
-
-repoze.who + wsgi middleware provides significant flexibility at least for external methods of authentiation. For form-based question of integration with existing user model.
-
-configure repoze.who / install new middleware.
-
-Interface API
--------------
-
- * remote_user
- * what we pull out from repoze.who (repoze.who.identity)
-
-
-Authorization
-=============
-
-Wants
------
-
-1. post-object creation authorization setting
- * e.g. make this person or group an owner/admin of this package
-
-2. acceptance/rejection of object updaes (e.g. for spam)
- * Similar to authorization but separate from pure access control ...
-
-
-Package Editing
-===============
-
-Wants
------
-
-1. Custom Forms
-2. Custom from sections and fields
-3. Bulk updating
-4. Merge support
-
-Package View
-============
-
-1. Wikipedia like flags on pages based on tags
-2. Sidebar customization (widgets)
-
-
-Storage Integration
-===================
-
-Wants
------
-
-1. Integrated Uploader -- provide way for uploading and registering package resources at the same time
-2. Preview of data in package pages
-
-
-Validation etc
-==============
-
-1. If needed to be done pre-package save this has to be directly pluggable
-2. If this can be post-hoc (does this 404) then can be done via event notification and rest interface (given sufficient structure)
-
http://bitbucket.org/okfn/ckan/changeset/ae86da996402/
changeset: ae86da996402
user: rgrp
date: 2011-06-17 16:18:23
summary: [doc/distributed.rst][s]: remove file as 'abstract', unrelated to current ckan codebase and now moved to wiki at http://wiki.ckan.net/Distributed_Revision_Control_for_Data.
affected #: 1 file (0 bytes)
--- a/doc/distributed.rst Fri Jun 17 15:17:26 2011 +0100
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,199 +0,0 @@
-==========================================================
-Distributed Revision Control for Data: Format and Protocol
-==========================================================
-
-See also: http://ckan.org/wiki/DistributingChanges
-
-Aim
-===
-
-Facilitate collaboration and information sharing via the use of data
-revisioning and the transmission of associated changesets between multiple peer
-nodes. It is the P2P nature of this model (as opposed to classic server-client
-approach) that leads to it being termed: "Distributed Revision Control".
-
-We are particularly interested in transmitting "structured" data (i.e. not
-BLOBs), especially data coming from a specified domain model.
-
-
-Existing Work
-=============
-
-Distributed Revision/Version Control Systems (DVCS) such as Mercurial_ and Git_
-perform a very similar function in revisioning of files and filesytem trees to
-what we seek to do with data.
-
-Given this similarity, the protocol and format specified here directly reuse
-many of the concepts and approach of these existing solutions.
-
-More on the analogies can be found on the wiki page which details the specific
-application of this protocol to CKAN:
-http://ckan.org/wiki/DistributingChanges
-
-Further details of the distributed revision control systems for code can be
-found in the appendix below.
-
-.. _Mercurial: http://mercurial.selenic.com/
-.. _Git: http://git-scm.com/
-
-
-Use cases
-=========
-
-1. data.gov.uk and ckan.net
----------------------------
-
-One specific use case is pulling and pushing metadata between different CKAN
-registries.
-
-In particular we Want to make data on data.gov.uk (hmg.ckan.net) available in a
-public CKAN instance. We will therefore end up with:
-
- 1. Package on data.gov.uk
- 2. Package on ckan.net
-
-Need to keep these two representations of the package in "sync".
-
-Remarks: This is easy if we only edit in one place. But what if we want to let
-community edit on ckan.net? Two options:
-
- 1. have 2 copies on ckan.net one community owned and one locked down
-
- * Pro: easy to keep stuff separate
-
- * Con: terrible user experience and still have issue that two items can
- diverge
-
- 2. Have one copy that is world editable into which gets *merged back* into
- the official data every so often
-
-
-Abstract Problem Description
-============================
-
-Let us consider three different nodes: A,B,C
-
-Setup 0:
---------
-
-Simple 1-way::
-
- A ----> B
-
-Setup 1:
---------
-
-Simple bi-directional::
-
- A <---> B
-
-Setup 3:
---------
-
-3-node single way::
-
- A ----> B
- \ /
- \ /
- V
- C
-
-In words:
-
- 1. Changes go directly from A to C
- 2. Changes go directly from A to B. Then changes from B are pulled to C
-
-One must avoid duplicating changes pushed directly to C from A again when pushing changes from B to C.
-
-Setup 4:
---------
-
-Full bidirectional 3-node::
-
- A <-----> B
- A A
- \ /
- \ /
- V
- C
-
-Terminology
-===========
-
- * Revision: metadata about a change(set)
- * Patch: payload of a change(set) - description of the changes to the
- data/domain model
- * Changeset: the combination of a Revision and its associated Patch
- * Repository: a standalone instance containing data/domain objects.
- * Working copy: a "view" representing the application of a set of (perhaps filtered) Changesets. Classic examples are "HEAD": i.e. applying all changesets
- * Normally these changesets must be coherent -- i.e. if changeset X follows on from changeset Y to apply X 'coherently' one must first apply Y
- * Filtering: in applying a changeset to a working copy the changeset may be 'filtered', for example by only applying the changes in that changeset that apply to given set of entities or by skipping the application of changesets from a given source.
-
-Remarks:
-
- * Changeset's Revision records the ID of its Parents.
- * The set of changesets in a given Repository fomr a directed acyclic graph
- * The "leaves" of this graph are termed Heads
-
-Formats
-=======
-
-Revision Format
----------------
-
- * id: uuid OR sha1 hash of patch + parent rev ids (like mercurial)
- * timestamp
- * parent ids
- * author - free text field
- * message
- * ddvc format no (e.g. 1.0)
- * (extras: arbitrary addtional attributes - like X-headers)
-
-Patch Format
-------------
-
- * Patch format identifier (e.g. text diff plus manifest diff for normal
- source revision control)
- * Patch format version
- * Payload: patch in format specified by patch format
-
-For CKAN patch format is as follows:
- * List of object ids
- * For each object id diff of all fields presented as JSON-encoded strings
-
-
-Protocol
-========
-
-The most complex part of this specification is the definition of the protocol
-especially the patch application protocol and the merge process.
-
-Merge algorithm
----------------
-
-Merge algorithm used will vary from application to application.
-
-TODO: details of example merge algorithm (e.g. that from CKAN)
-
-Issues
-------
-
- * Enforcement of constraints (e.g. uniqueness) during merge
- * User intervention in merges and pending merges
-
-
-Appendix: Distributed Revision Control for Source Code
-======================================================
-
-Mercurial
----------
-
-Basic overview of the Mercurial model: http://mercurial.selenic.com/wiki/UnderstandingMercurial
-
-Git
----
-
-Glossary: http://www.kernel.org/pub/software/scm/git/docs/gitglossary.html
-
-Technical Docs: http://repo.or.cz/w/git.git?a=tree;f=Documentation/technical;hb=HEAD
-
Repository URL: https://bitbucket.org/okfn/ckan/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the ckan-changes
mailing list