[ckan-dev] How to maintain the database?

Mon Dec 4 21:48:06 UTC 2017

Actually this script was needed, because DataStore did not handle
"deletions" at all, so I hope it gets obsolete once we upgrade to 2.7 (see
https://github.com/ckan/ckan/pull/3425).

But yes it might be worth to put it into a separate extension, so that we
can all re-use it and make sure to get the latest updates (like your
improvements).

- Stefan

On Mon, Dec 4, 2017 at 9:47 AM, Cam Findlay <cam at camfindlay.com> wrote:

> Hi Martin,
>
> A big +1 for Stefan's datastore clean up script - we have reused on our
> data portal (we also added some improvements to avoid locking up database
> when clearing out really massive datastores).
>
> Ours is over at https://github.com/data-govt-nz/ckanext-dia/blob/
> master/ckanext/dia/commands.py#L44
>
> Makes me think we should turn this into an extension to help reuse and
> improvements?
>
> C.
>
>
> On 2 December 2017 at 07:14, Martin Szyszlican <MSzyszlican at jus.gob.ar>
> wrote:
>
>> We sometimes re-upload datasets after a failed or incomplete datapusher
>> run, and we would like to delete everything unused from the database. We
>> have reached a point where the DB is too big, and it´s slowing down the
>> server, which makes the datapusher crashes more frequent.
>>
>>
>>
>> I know I can call resource.purge or datastore_delete on a single
>> resource, but how do I detect everything that is deletable? Is there a
>> maintenance command already created for this?
>>
>>
>>
>> *Martín Szyszlican*
>> Programa Justicia Abierta | Secretaría de Planificación Estratégica
>> *mszyszlican at jus.gob.ar <mszyszlican at jus.gob.ar>*
>> (011) 5300-4000 int. 76067
>>
>> [image: logo_minjus]
>>
>>
>>
>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>

-- 
Liip AG  // Limmatstrasse 183 //  CH-8005 Zürich
Tel +41 43 500 39 80 // GnuPG 0x7B588C67 // www.liip.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20171204/571f0b0b/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 12218 bytes
Desc: not available
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20171204/571f0b0b/attachment-0003.png>