[ckan-dev] file storage setup in 1.7

• Previous message: [ckan-dev] file storage setup in 1.7
• Next message: [ckan-dev] file storage setup in 1.7
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

1) That did the trick -- After moving my storage config values up to
the app:main section, I was able to successfully upload via the
/storage/upload interface and by Adding a Resource to an existing
Dataset.  Thanks for the tip, Salman!

2) I'd be happy to help with fixing up documentation as I get familiar
with the software and the community.  Is there a formal documentation
team or is everything sort of ad hoc?  Also, I was able to
fork/pull-request documentation changes for the actual config files on
Github, but how is the "Read the Docs" content generated/edited?

Thanks again!
Jim

On Thu, May 24, 2012 at 9:57 AM, Rufus Pollock <rufus.pollock at okfn.org> wrote:
> Yes! As Salman correctly says, any "main" settings need to go above
> all the logging stuff in the [app:main] section (sections in the ini
> file are separated by [...] section headers)!
>
> Thinking about it now, this may be rather non-obvious and we need to
> perhaps think about making this clearer in the config file.
>
> Rufus
>
> On 24 May 2012 16:54, Haq, Salman <Salman.Haq at neustar.biz> wrote:
>> Move the following lines from the end of the file to end of the [app:main]
>> section:
>>
>>
>> #storage stuff
>> ckan.storage.bucket = /srv/data
>> ## OFS configuration
>> ofs.impl = pairtree
>> # directory on disk for data storage (should be empty)
>> ofs.storage_dir = /srv/data
>>
>>
>>
>> On 5/24/12 11:32 AM, "Jim Craner" <jim at codeforamerica.org> wrote:
>>
>>>Hi everyone,
>>>
>>>Thanks for the suggestions...  Here were the steps I took this morning
>>>based on feedback from Salman and Ross:
>>>
>>>1) Removed "storage" from .ini file's list of plugins  //no effect
>>>
>>>2) python-argparse was NOT installed, but installing it had  //no effect
>>>
>>>3) I created the following directory/file structure in my
>>>{ofs.storage_dir} (which is /srv/data):
>>>
>>>root at dataserver:/srv/data# ls -al
>>>drwxr-xr-x 3 www-data www-data 4096 2012-05-24 08:10 .
>>>drwxr-xr-x 3 root     root     4096 2012-05-23 21:55 ..
>>>drwxr-xr-x 2 www-data www-data 4096 2012-05-24 08:10 pairtree_root
>>>-rw-r--r-- 1 www-data www-data    0 2012-05-24 08:03 pairtree_version0_1
>>>
>>>//no effect
>>>
>>>I pasted my .ini file to pastebin -- storage settings are at the very
>>>bottom:
>>>
>>>http://pastebin.com/EgQV6vZE
>>>
>>>Thanks for any additional suggestions!
>>>-Jim
>>>
>>>
>>>On Thu, May 24, 2012 at 6:43 AM, Haq, Salman <Salman.Haq at neustar.biz>
>>>wrote:
>>>>
>>>> On 5/24/12 4:51 AM, "Ross Jones" <ross.jones at okfn.org> wrote:
>>>>
>>>>>Hi Jim
>>>>>
>>>>>If you're using 1.7 you shouldn't need ckanext-storage at all, it might
>>>>>just
>>>>>confuse the situation so probably best if you disable it (remove it from
>>>>>your .ini).
>>>>>
>>>>>Could you post your .ini to pastebin with any sensitive (database) info
>>>>>removed?
>>>>>Everytime this happens to me it is because I've forgotten to install
>>>>>pairtree
>>>>>or have set ofs.storage_dir to a folder where I don't have permissions.
>>>>>
>>>>>I'm assuming you are using local file storage, in which case it might be
>>>>>worth seeing
>>>>>if CKAN has created a pairtree_version0_1 file (empty) in the folder
>>>>>specified in
>>>>>ofs.storage_dir. If not create one and see if that makes a difference.
>>>>>
>>>>>Ross
>>>>
>>>>
>>>>
>>>> Jim,
>>>>
>>>> Building on Ross's suggestions, there may be several things that are
>>>> broken and need to be reported as bugs:
>>>>
>>>> 1. python-argparse debian package dependency may be missing.
>>>> 2. python-pairtree debian package dependency may be missing.
>>>> 3. {ofs.storage_dir}/pairtree_root/ directory has to be explicitly
>>>>created
>>>> with www-data:www-data ownership and mode 755.
>>>> 4. {ofs.storage_dir}/pairtree_version0_1 file has to be explicitly
>>>>touched
>>>> with www-data:www-data ownership and mode 644.
>>>>
>>>>
>>>> The mailing list has similar threads from the March/April time frame
>>>>where
>>>> the same problem was reported.
>>>>
>>>> Hope this helps.
>>>>
>>>> Salman
>>>>
>>>>
>>>>>
>>>>>
>>>>>On 24 May 2012, at 06:27, Jim Craner wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I'm trying to configure CKAN 1.7 (package install) on an Ubuntu 10.04
>>>>>> appliance and I'm having some problems with the file storage
>>>>>> functionality.  I'm trying to use local file storage based on these
>>>>>> instructions:
>>>>>>
>>>>>> http://docs.ckan.org/en/ckan-1.7/filestore.html
>>>>>>
>>>>>> In addition to the settings recommended in that doc, I also did a "pip
>>>>>> install ckanext-storage" and added "storage" to my .ini file's list of
>>>>>> extensions.
>>>>>>
>>>>>> 1) When I try to upload a file as a resource, I am able to select the
>>>>>> file but when I click the "upload" button I get the "Failed to get
>>>>>> credentials for storage upload. Upload cannot proceed" error message
>>>>>> (I am logged in as the admin user).
>>>>>>
>>>>>> 2) When I try to upload a file via the /storage/upload method, I am
>>>>>> again able to select the file but attempting to upload returns a
>>>>>> generic "Server Error / an internal error occurred" message.  In the
>>>>>> /var/log/apache2/INSTANCENAME.error.log, I can see dozens of error
>>>>>> messages related to "/storage/upload" and "/storage/upload_handle" but
>>>>>> I'm not really sure what I'm looking for.
>>>>>>
>>>>>> Any suggestions?
>>>>>>
>>>>>> Thanks!
>>>>>> Jim
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -----
>>>>>> Jim Craner, 2012 Fellow
>>>>>> Code for America
>>>>>> jim at codeforamerica.org
>>>>>> (+1) 773-809-4546
>>>>>> http://codeforamerica.org
>>>>>>
>>>>>> _______________________________________________
>>>>>> ckan-dev mailing list
>>>>>> ckan-dev at lists.okfn.org
>>>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>ckan-dev mailing list
>>>>>ckan-dev at lists.okfn.org
>>>>>http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>>>
>>>
>>>
>>>--
>>>-----
>>>Jim Craner, 2012 Fellow
>>>Code for America
>>>jim at codeforamerica.org
>>>(+1) 773-809-4546
>>>http://codeforamerica.org
>>>
>>>_______________________________________________
>>>ckan-dev mailing list
>>>ckan-dev at lists.okfn.org
>>>http://lists.okfn.org/mailman/listinfo/ckan-dev
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>
>
>
> --
> Co-Founder, Open Knowledge Foundation
> Promoting Open Knowledge in a Digital Age
> http://www.okfn.org/ - http://blog.okfn.org/
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev



-- 
-----
Jim Craner, 2012 Fellow
Code for America
jim at codeforamerica.org
(+1) 773-809-4546
http://codeforamerica.org

• Previous message: [ckan-dev] file storage setup in 1.7
• Next message: [ckan-dev] file storage setup in 1.7
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]