[ckan-dev] Problems uploading files in CKAN

Andrew Brooks arb at sat.dundee.ac.uk
Wed Dec 4 10:09:58 UTC 2019


There is a setting in the .ini file: ckan.max_resource_size
and nginx has a limit which I think is client_max_body_size
datapusher has a limit in datapusher_settings.py which is MAX_CONTENT_LENGTH

On Wed, 4 Dec 2019 at 09:46, Adil Hasan <paradox2005 at gmail.com> wrote:

> Hello,
>
> We are deploying CKAN in container in Kubernetes and we have mounted our
> storage in the container. We have /var/lib/ckan mounted. We also pointed
> the cache_dir to the mounted storage.
>
> We are able to upload small files, but when we try to upload large files
> (>500MB) through the web interface we notice that the upload just hangs
> after some period of time.
>
> Looking in the developer tools (in Firefox) it seems that the network
> monitoring suggests that XMB has been transferred and the transfer is
> complete. But, X is less than the size of the file.
>
> When I look at the ckan web page the page looks as if the transfer is
> still progressing (in fact it's stuck). I cannot click the 'Add' button.
>
> We have increased the timeout for the apache web server:
>
> RequestReadTimeout header=90,MinRate=500
> RequestReadTimeout body=90,MinRate=500
>
> and we also increased the nginx (our proxy server):
>
> nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
> nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
> nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
>
> We tried to see where ckan uploads the file to whilst the transfer is
> progressing, but we cannot find where it goes. Does anyone know?
>
> Has anyone experienced this before? Does anyone have a suggestion as to
> how to get around this issue?
>
> Many thanks,
> adil
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20191204/47d07cbe/attachment-0002.html>


More information about the ckan-dev mailing list