[ECODP-dev] EU ODP - 342

Mon Jan 27 21:29:00 UTC 2014

Hello,

The cronjob itself checks for files older than 7 days and all sessions
should have expired by then.  The age of the file is when the session
started.

So there is no need for any syncing and can be run every day as each
run will only delete old files.

Thanks

David

On 27 January 2014 19:23, Bert Van Nuffelen
<bert.van.nuffelen at tenforce.com> wrote:
> Hi David,
>
> thanks for the explaination and the pointer.
>
> How do we sync between the session timeout and the cronjob-cleanup?
>
> kind regards,
>
> Bert
>
>
>
>
> 2014-01-27 David Raznick <david.raznick at okfn.org>
>>
>> On 27 January 2014 14:59, Bert Van Nuffelen
>> <bert.van.nuffelen at tenforce.com> wrote:
>> > Hi David,
>> >
>> > so for this: what has to be be done to avoid the changes to happen?
>> >
>> > a) a cronjob to remove the sessions, on which timebasis?
>>
>> Daily is fine.  The cron only looks at session that are old.
>>
>> > b) removing a session was is the impact?
>>
>> None if the session is old.
>>
>> >
>> > kind regards,
>> >
>> > Bert
>>
>> The script we normally run is the following.
>>
>>
>> https://github.com/okfn/ckan-packaging/blob/master/etc/cron.daily/remove_old_sessions
>>
>> It is set to run daily but remove sessions that are older than 10 days
>> old.
>>
>> I am pretty sure ec are using memcached so this should not be an
>> issue.  Would it be possible to have a copy of the config file other
>> wise its hard to tell what is going on.
>>
>> David
>>
>>
>> >
>> >
>> >
>> > 2014-01-27 Bert Van Nuffelen <bert.van.nuffelen at tenforce.com>
>> >
>> >> Dear David,
>> >>
>> >> the system is the eu odp that OKF has delivered to us. Nothing is added
>> >> to
>> >> it, unless you have specified.
>> >>
>> >> kind regards,
>> >>
>> >> Bert
>> >>
>> >>
>> >>
>> >> 2014-01-27 David Raznick <david.raznick at okfn.org>
>> >>
>> >>> Hello
>> >>>
>> >>> There should be a cron set up on the server to remove the old
>> >>> sessions.  This was definitely a part of the server deployment scripts
>> >>> that we sent over.  I am wondering if this got lost.
>> >>>
>> >>> Also are they using memcached for their session data now?  If they are
>> >>> then this should not be an issue as they have a proper expiry.  It is
>> >>> hard to tell what is going on as we have no access to their system.
>> >>>
>> >>> David
>> >>>
>> >>> On 27 January 2014 11:32, Bert Van Nuffelen
>> >>> <bert.van.nuffelen at tenforce.com> wrote:
>> >>> > Dear Darwin, David and Adam,
>> >>> >
>> >>> > We have received a critical bug from the EU ODP project. Can you
>> >>> > take
>> >>> > care
>> >>> > of it. Can we have tomorrow a short call on how you will deal with
>> >>> > the
>> >>> > issue?
>> >>> >
>> >>> > kind regards,
>> >>> >
>> >>> > Bert
>> >>> >
>> >>> > The ticket content is the following:
>> >>> >
>> >>> > "After further investigations regarding the current problem on the
>> >>> > prod
>> >>> > frontend we have found that the problem is not linked to our
>> >>> > archive,
>> >>> > but to
>> >>> > a problem with CKAN which should be resolved by TenForce / AHRS.
>> >>> >
>> >>> > In a nutshell, CKAN is not closing its sessions at all and therefore
>> >>> > uses up
>> >>> > a huge amount of inodes which ultimately we now did run out of. Here
>> >>> > is
>> >>> > a
>> >>> > post on the official CKAN mailing list where someone else describes
>> >>> > the
>> >>> > same
>> >>> > problem:
>> >>> > https://lists.okfn.org/pipermail/ckan-dev/2011-June/000925.html
>> >>> >
>> >>> > This is a blocking issue, there is currently no ingestion by batch,
>> >>> > ingestion by web interface will probably fail and we are currently
>> >>> > not
>> >>> > receiving any new files on the production machine either. This has
>> >>> > to
>> >>> > be
>> >>> > resolved ASAP.
>> >>> > "
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Bert Van Nuffelen
>> >>> >
>> >>> > Semantic Technologies Software Architect at TenForce
>> >>> > www.tenforce.be
>> >>> >
>> >>> > Bert.Van.Nuffelen at tenforce.com
>> >>> > Office: +32 (0)16 31 48 60
>> >>> > Mobile:+32 479 06 24 26
>> >>> > skype: bert.van.nuffelen
>> >>> >
>> >>> > _______________________________________________
>> >>> > ecodp-dev mailing list
>> >>> > ecodp-dev at lists.okfn.org
>> >>> > https://lists.okfn.org/mailman/listinfo/ecodp-dev
>> >>> >
>> >>> _______________________________________________
>> >>> ecodp-dev mailing list
>> >>> ecodp-dev at lists.okfn.org
>> >>> https://lists.okfn.org/mailman/listinfo/ecodp-dev
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Bert Van Nuffelen
>> >>
>> >> Semantic Technologies Software Architect at TenForce
>> >> www.tenforce.be
>> >>
>> >> Bert.Van.Nuffelen at tenforce.com
>> >> Office: +32 (0)16 31 48 60
>> >> Mobile:+32 479 06 24 26
>> >> skype: bert.van.nuffelen
>> >
>> >
>> >
>> >
>> > --
>> > Bert Van Nuffelen
>> >
>> > Semantic Technologies Software Architect at TenForce
>> > www.tenforce.be
>> >
>> > Bert.Van.Nuffelen at tenforce.com
>> > Office: +32 (0)16 31 48 60
>> > Mobile:+32 479 06 24 26
>> > skype: bert.van.nuffelen
>> >
>> > _______________________________________________
>> > ecodp-dev mailing list
>> > ecodp-dev at lists.okfn.org
>> > https://lists.okfn.org/mailman/listinfo/ecodp-dev
>> >
>> _______________________________________________
>> ecodp-dev mailing list
>> ecodp-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ecodp-dev
>
>
>
>
> --
> Bert Van Nuffelen
>
> Semantic Technologies Software Architect at TenForce
> www.tenforce.be
>
> Bert.Van.Nuffelen at tenforce.com
> Office: +32 (0)16 31 48 60
> Mobile:+32 479 06 24 26
> skype: bert.van.nuffelen
>
> _______________________________________________
> ecodp-dev mailing list
> ecodp-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ecodp-dev
>