[ckan-dev] Cannot download files from File Storage even if the files are there

Tue Feb 4 13:07:23 UTC 2014

Hey Marco,

We've seen this happen before and I'm looking into it to see what can be
done. I'll update the list when I have a solution. Future versions of CKAN
use a new filestore should not run into this problem.

Nigel Babu

Developer  |  @nigelbabu <https://twitter.com/nigelbabu>

The Open Knowledge Foundation <http://okfn.org/>

Empowering through Open Knowledge

http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on
Facebook<https://www.facebook.com/OKFNetwork> |
Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>

 CKAN | http://ckan.org/ | @CKANproject
<http://twitter.com/CKANproject> |the world’s leading open-source data
portal platform

On 4 February 2014 14:09, Marco Combetto <marcomb at hotmail.com> wrote:

> Hi
>
> Yes. We have this file and it seems not containing all filenames.
> I have the feeling that at some point it was corrupted.
>
> Is there a way to rebuild it?
>
>
> Marco Combetto
> marcomb at hotmail.com
> Mobile: +39 3316329971
> www.*linkedin*.com/in/*marco*mb
>
>
>
>
> ------------------------------
> From: nigel.babu at okfn.org
> Date: Tue, 4 Feb 2014 09:42:08 +0530
> To: ckan-dev at lists.okfn.org
> Subject: Re: [ckan-dev] Cannot download files from File Storage even if
> the files are there
>
>
> Hi Marco,
>
> Can you look at your storage folder? There should be a JSON file called
> "persisted_state.json". Check if the files you're looking for are in there.
>
> Nigel Babu
>
> Developer  |  @nigelbabu <https://twitter.com/nigelbabu>
>
> The Open Knowledge Foundation <http://okfn.org/>
>
> Empowering through Open Knowledge
>
> http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on Facebook<https://www.facebook.com/OKFNetwork> |
> Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>
>
>  CKAN | http://ckan.org/ | @CKANproject <http://twitter.com/CKANproject> |the world’s leading open-source data portal platform
>
>
> On 3 February 2014 21:45, Marco Combetto <marcomb at hotmail.com> wrote:
>
> Hallo
>
> I am facing a critical problem on our CKAN production site (we are running
> a 2.2a version, release early December)
>
> http://dati.trentino.it
>
> From last Friday (30 Jan 2014)  our users are unable to download any  file
> stored on our the CKAN storage
> except the one uploaded after 30 January
>
> I absure you, we did't any deploy or any update on CKAN code/configuration
> from December 2013
> Of course, we have users (Group Admin, not SYS ADM) working on (using the
> Web Interface, they were updating some metadata)
>
>  The tipical error we see in the log is:
>
> [Mon Feb 03 12:02:08 2014] [error] [client 172.31.50.18] Error - <class
> 'pairtree.storage_exceptions.FileNotFoundException'>:
> [Mon Feb 03 12:02:12 2014] [error] 2014-02-03 12:02:12,510 INFO
>  [ckan.lib.base]  /dataset render time 0.149 seconds
> [Mon Feb 03 12:02:14 2014] [error] 2014-02-03 12:02:14,046 INFO
>  [ckan.lib.base]  /dataset render time 0.092 seconds
> [Mon Feb 03 12:02:14 2014] [error] 2014-02-03 12:02:14,115 INFO
>  [ckan.lib.base]  /storage/f/2013-06-20T093441/MITT---OpenData---v2.pdf
> render time 5.292 seconds
> [Mon Feb 03 12:02:14 2014] [error] 2014-02-03 12:02:14,278 INFO
>  [ckan.lib.base]  /api/i18n/it render time 0.002 seconds
> [Mon Feb 03 12:02:16 2014] [error] 2014-02-03 12:02:16,614 WARNI
> [ckan.lib.maintain] Use `c.search_facets` instead.
> [Mon Feb 03 12:02:16 2014] [error] 2014-02-03 12:02:16,741 INFO
>  [ckan.lib.base]  /group/e08070e1-f829-43cf-af0a-83ee6f1e4c3f render time
> 0.246 seconds
>
> Please note that "MITT---OpenData---v2.pdf" is the file that I am trying
> to download (example)
>
> We are sure that pairtree storage is configured well (we did the
> configuration 1 month ago and everything was secured)
>
> We rebuild SOLR indexes, we tried to step back with a POSTGRES backup
> dated 29 January, we delete the cache.. we triple checked the
> configuration... but no way to sort out the problem described before
>
> We have a file storage of 19GB of files.. (more than 1 year activity,
> moving from 1.8 to 2.2)
> The total number of files is 13159 files in 9055
>
> Any body faced similar problem?
> Any suggestion of what could be happened?
> Any suggestion to help us on troubleshooting this file storage management
>
>
>
> Marco Combetto
> marcomb at hotmail.com
> Mobile: +39 3316329971
> www.*linkedin*.com/in/*marco*mb
>
>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
>
> _______________________________________________ ckan-dev mailing list
> ckan-dev at lists.okfn.org https://lists.okfn.org/mailman/listinfo/ckan-devUnsubscribe:
> https://lists.okfn.org/mailman/options/ckan-dev
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140204/9a3af0ca/attachment-0003.html>