[ckan-dev] Links of resources changed?

Mon Aug 17 11:59:00 UTC 2015

I have vague recollections of routes using the host-header to determine the qualified name so I went to have a look ....

Possible version mismatch, but ..

Looking at https://github.com/bbangert/routes/blob/6c0e9d607652a51d8afae13a86f791ed678628f5/routes/util.py#L257-L267 it seems like qualified grabs the host and port data from the config var, which is in fact a request_config() ... looking at https://github.com/bbangert/routes/blob/a9a95fce5c21a96c4941b611f98a6b4596dbadce/routes/__init__.py#L79 suggests to me this is populated through the request headers.

You *can* set the host/port vars in url_for though.  I suspect this is actually a bug and CKAN needs to implement something like the code at https://github.com/bbangert/routes/blob/a9a95fce5c21a96c4941b611f98a6b4596dbadce/routes/__init__.py#L112-L134 (assuming version matches etc) to inject the site_url settings.

Short term fix, when uploading from a local task (on the server), reference it using the FQDN that external tasks would use, rather than the local name (or 127).

Cheers

Ross.

> On 17 Aug 2015, at 12:42, Adrià Mercader <adria.mercader at okfn.org> wrote:
> 
> I think it is done here:
> 
> https://github.com/ckan/ckan/blob/master/ckan/lib/dictization/model_dictize.py#L118
> 
> This is basically calling the `url_for` function to generate the download URL. I would assume this to use ckan.site_url but I might be wrong. Is this happening on new resources?
> 
> 
> Adrià
> 
> On 17 August 2015 at 10:31, Stefan Oderbolz <stefan.oderbolz at liip.ch> wrote:
> Hi Adrià,
> 
> thanks for the clarification. Can you (or someone else) point me in the right direction as of where actually the download URL of a resource is generated? I found the uploader (https://github.com/ckan/ckan/blob/097e6ce7b840057bf6f22b1cf79dc8ee0b839580/ckan/lib/uploader.py), but the 'url' attribute seems not to be set there.
> 
> The problem we currently have is, that all the files that are manually uploaded get a download URL of the internal server domain name where the system is running on. This is different from the external address. We set the ckan.site_url to the external one, but still the uploads get the wrong domain name.
> 
> Any ideas?
> 
> Cheers, Stefan
> 
> On Fri, Aug 14, 2015 at 4:09 PM, Adrià Mercader <adria.mercader at okfn.org> wrote:
> Hi Stefan,
> 
> The FileStore was refactored on CKAN 2.2 and the links changed indeed to the second form you described. From the CHANGELOG for 2.2:
> 
> The FileStore and file uploads have been completely refactored and simplified to only support local storage backend. The links from previous versions of the FileStore to hosted files will still work, but there is a command available to migrate the files to new Filestore. See this page for more details: http://docs.ckan.org/en/latest/filestore.html#filestore-21-to-22-migration
> 
> If you are having trouble with corrupted files on the old OFS backend I recommend you migrate to the new one.
> 
> Hope this helps,
> 
> Adrià
> 
> 
> 
> On 14 August 2015 at 13:39, Stefan Oderbolz <stefan.oderbolz at liip.ch> wrote:
> Hi there,
> 
> has there been any change in how a resource link looks like? On an instance using OFS with "pairtree" we generated links to resources that looked like that:
> 
> https://ckan-instance.org/storage/f/dataset-name/filename.csv
> 
> But due to some unknown change, they now appear as
> 
> https://ckan-instance.org/dataset/dataset-name/resource/5024a0b4-246f-48b1-a447-4721a7d9b313/download/filename.csv
> 
> Is the second URL simply a redirect to the first one?
> 
> Unfortunately I currently have no direct access to the instance, so there is little more information at this point. Just wanted to ask you if this seems familiar to any of you. Or what change that might have caused.
> 
> By the way, we are using ofs-hero (https://github.com/ckan/ofs-hero) to regulary re-create the corrupted persisted_state.json.
> 
> 
> Cheers, Stefan
> 
> -- 
> Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
> Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> 
> 
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> 
> 
> 
> 
> -- 
> Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
> Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
> 
> 
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev