[ckan-dev] Filestore Migration ( 2.1 to 2.2 )

Wed Feb 12 15:28:40 UTC 2014

Thanks Nigel!

After running the migrate_filestore command, all files from remote S3 have
been created in the new local filestore ( I just verified the counts of
files on both locations as well as contents of some ). The function
migrate_filestore in cli.py is also doing the same, however, it is just not
updating the field url in resource table with the new url value. I was
wondering if that is the problem here.  ( not completely sure if I am right
here ).

Regards,
Sajan

On Wed, Feb 12, 2014 at 1:09 AM, Nigel Babu <nigel.babu at okfn.org> wrote:

> Hi Sajan,
>
> This file has not been migrated because this file was never in the
> filestore. The link to S3 was directly pasted into the url field instead of
> uploading via the resource upload mechanism in CKAN.
>
> Nigel Babu
>
> Developer  |  @nigelbabu <https://twitter.com/nigelbabu>
>
> The Open Knowledge Foundation <http://okfn.org/>
>
> Empowering through Open Knowledge
>
> http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on Facebook<https://www.facebook.com/OKFNetwork> |
> Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>
>
>  CKAN | http://ckan.org/ | @CKANproject <http://twitter.com/CKANproject> |the world's leading open-source data portal platform
>
>
> On 11 February 2014 20:23, Sajan Ravindran <sajanravindran at gmail.com>wrote:
>
>> Hi Nigel,
>>
>> Here is the pastebin of a resource_show for a resource:
>> http://pastebin.com/DxZbWJwM
>>
>> I have marked the URL ( which is still pointing to the old S3 url ) as
>> XXX.
>>
>> Regards,
>> Sajan
>>
>>
>> On Tue, Feb 11, 2014 at 6:03 AM, Nigel Babu <nigel.babu at okfn.org> wrote:
>>
>>> Can you send me a pastebin of the output of resource_show for a resource
>>> that's in the filestore?
>>>
>>> Nigel Babu
>>>
>>> Developer  |  @nigelbabu <https://twitter.com/nigelbabu>
>>>
>>> The Open Knowledge Foundation <http://okfn.org/>
>>>
>>> Empowering through Open Knowledge
>>>
>>> http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on Facebook<https://www.facebook.com/OKFNetwork> |
>>> Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>
>>>
>>>  CKAN | http://ckan.org/ | @CKANproject <http://twitter.com/CKANproject>|the world's leading open-source data portal platform
>>>
>>>
>>> On 10 February 2014 22:05, Sajan Ravindran <sajanravindran at gmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> After a fresh install of 2.2, I was doing a migration of CKAN 2.1
>>>> database into the new database using the command : paster db
>>>> migrate-filestore. Though new directories and files are getting created in
>>>> the local storage, the old S3 urls are showing up when viewed through the
>>>> web interface.
>>>>
>>>> I did check up the resource table in ckan_default and found that the
>>>> field url is still having old values. Should it not be updated with new
>>>> values during the migration?
>>>>
>>>> Regards,
>>>> Sajan
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140212/cf7f137d/attachment-0003.html>