[okfn-help] openbiblio update from 2010-09-22 catch up

[Benjamin O'Steen]

Hi Friedrich,

I've made some changes to repod (my repo is at
http://bitbucket.org/beno/repod)

Specifically, I've altered the label show controller to deal with
ofs.get_url functioning and returning either a file:// or http:// url -
serving the file via pylons FileApp or 302'ing respectively.

(also, how do you feel about files served up directly or proxied having
their metadata added as headers? eg as X-OFS-.... for example?)

I've also altered app_globals.py to adjust the backend store based on
config.ini:

ofs_store_type = local|GS

- for local you need to set 'ofs_local_storage_dir' in app:main to be an
app writable directory
- for GS you need to set the gs_... etc as you had before

I also made a cosmetic change to the label meta template, making all '_'
preceded keys appear bold (and without the _) and other user-set keys to
be normal weight.

As I said before, I'm unsure how to get a safe-to-distribute http:// url
from boto, so the label controller will work as it did before. I don't
know if 'return ofs.get_stream...' works, but that's what it'll do if it
cannot find a url for the file.

Ben


On Thu, 2010-09-23 at 22:34 +0200, Friedrich Lindenberg wrote:
> Hi all, 
> 
> On Sep 23, 2010, at 7:18 PM, Rufus Pollock wrote:
> >> 
> >> One thing that I had to add to pairtree when I was using it directly for
> >> web apps was the ability to get the direct file path to hand to pylon's
> >> download convenience class.
> >> 
> >> Would it make sense for OFS to include a method that returns a direct
> >> URL for a given bucket/label? eg:
> >> 
> >> ofs.get_url(bucket, label) --->
> > 
> > I think that would be useful, yes ...
> 
> Agreed, get_url would be fantastic. And we should consider designing a similar process for write access as well (ofs.put_stream_alternatives to return S3/GS POST params). 
> 
> Another thing that we should (for now) find a low-tech consensus to is naming things, esp. regarding versioning. For example, if we're going to store CKAN resources in OFS, my first approach would be to consider everything in OFS immutable and use a naming scheme like this: 
> 
> bucket=ckan_site_id, label=<pkg_id>-<res_id>-<res-revision-id>.<representation-name>
> 
> This assumes that we do not do any versioning or storing of deltas at the moment, but since storage at the levels we'll require in the near future is relatively cheap, I think we should just keep everything and sort this out later? 
> 
> >> PS I've been musing whether or not a pairtree-via-FTP OFS backend would
> >> be a useful. I think it might be, but would like to hear otherwise :)
> > 
> > Couldn't this go through repod (the rest store webapp friedrich has built ...)?
> 
> The required effort would essentially be the same: repod is talking to an OFS backend, which can be S3, Pairtree or FTP. 
> 
> Friedrich