[ckan-dev] Datastorer: celeryd fails on invalid resource URL

• Previous message: [ckan-dev] Datastorer: celeryd fails on invalid resource URL
• Next message: [ckan-dev] Datastorer: celeryd fails on invalid resource URL
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hello,

Can you pull the latest ckanext-datastorer code and try again.  This was
subtle as the code that tries to generate the url relies on the url you are
currently in. I tested it without the /en in the url and it worked.

Sorry for the pain with this.  We have recently moved from having to
specify a site_url in your config

i.e ckan.site_url = http://vmubuntu.local:5000

to us trying to guess the site_url based on what the webserver sends and
this has thrown up many edge cases.

Hopefully this should be last of it.  If not I may decide to go with the
site_url config if it is there otherwise fall back to this method, but I
was hoping not to have to do this.

Thanks

David


On Fri, May 4, 2012 at 5:22 PM, BIHANIC Laurent <laurent.bihanic at atos.net>wrote:

> Hi,
>
> Le 03/05/12 00:18, David Raznick a écrit :
> > I have added a patch to ckanext-archiver which could fix the issue. It
> appends
> > the site url to give a fully qualified that in link checkable.
>
> Still not good: the URL I get has twice the language:
>
> link_checker - Target URL:
>
> http://vmubuntu.local:5000/en/en/storage/f/2012-05-04T162011/perf-evaluation-2.csv
>
> It now passes the URL scheme test but fails upon actual connection (stack
> trace below).
>
> Regards,
>
> Laurent
>
>
> [2012-05-04 18:20:21,255: ERROR/MainProcess] Task
> datastorer.upload[c50f7e0f-c2b2-488c-a07e-1eaf9bf5227f] raised exception:
> LinkCheckerError('URL unobtainable',)
> Traceback (most recent call last):
>   File
>
> "/home/lbihanic/work/ckan/test-latest/lib/python2.6/site-packages/celery/execute/trace.py",
> line 181, in trace_task
>     R = retval = fun(*args, **kwargs)
>   File
> "/home/lbihanic/work/ckan/ckanext-datastorer/ckanext/datastorer/tasks.py",
> line 73, in datastorer_upload
>     return _datastorer_upload(context, data)
>   File
> "/home/lbihanic/work/ckan/ckanext-datastorer/ckanext/datastorer/tasks.py",
> line 90, in _datastorer_upload
>     result = download(context, resource, data_formats=DATA_FORMATS)
>   File
>
> "/home/lbihanic/work/ckan/test-latest/src/ckanext-archiver/ckanext/archiver/tasks.py",
> line 85, in download
>      headers = json.loads(link_checker(link_context, link_data))
>   File
>
> "/home/lbihanic/work/ckan/test-latest/lib/python2.6/site-packages/celery/app/task/__init__.py",
> line 262, in __call__
>     return self.run(*args, **kwargs)
>   File
>
> "/home/lbihanic/work/ckan/test-latest/src/ckanext-archiver/ckanext/archiver/tasks.py",
> line 272, in link_checker
>     raise LinkCheckerError(error_message)
> LinkCheckerError: URL unobtainable
>
>
> > On Wed, May 2, 2012 at 5:18 PM, BIHANIC Laurent <
> laurent.bihanic at atos.net
> > <mailto:laurent.bihanic at atos.net>> wrote:
> >
> >     Me again, the HTTP proxy configuration removed...
> >
> >     I installed CKAN and Datastorer from the sources (latest).
> >     When adding a new resource to a dataset, celeryd fails with the
> following
> >     error:
> >
> >     [2012-05-02 17:37:20,909: ERROR/MainProcess] Task
> >     datastorer.upload[205b2be6-3188-4694-bac1-c01492046c72] raised
> exception:
> >     LinkCheckerError('Invalid url scheme',)
> >
> >     I added a debug trace and the URL being checked is
> >     "/en/storage/f/2012-05-02T153710/perf-evaluation-2.csv".
> >     But link_checker() expects an absolute URL with a scheme (http,
> https or ftp).
> >
> >     Any known fix ?
> >
> >     TIA,
> >
> >     Laurent
> >
> >
> >     Traceback (most recent call last):
> >     File
> >
> "/home/lbihanic/work/ckan/test-latest/lib/python2.6/site-packages/celery/execute/trace.py",
> >     line 181, in trace_task
> >     R = retval = fun(*args, **kwargs)
> >     File
> >
> "/home/lbihanic/work/ckan/ckanext-datastorer/ckanext/datastorer/tasks.py",
> >     line 73, in datastorer_upload
> >     return _datastorer_upload(context, data)
> >     File
> >
> "/home/lbihanic/work/ckan/ckanext-datastorer/ckanext/datastorer/tasks.py",
> >     line 90, in _datastorer_upload
> >     result = download(context, resource, data_formats=DATA_FORMATS)
> >     File
> >
> "/home/lbihanic/work/ckan/test-latest/src/ckanext-archiver/ckanext/archiver/tasks.py",
> >     line 78, in download
> >     headers = json.loads(link_checker(link_context, link_data))
> >     File
> >
> "/home/lbihanic/work/ckan/test-latest/lib/python2.6/site-packages/celery/app/task/__init__.py",
> >     line 262, in __call__
> >     return self.run(*args, **kwargs)
> >     File
> >
> "/home/lbihanic/work/ckan/test-latest/src/ckanext-archiver/ckanext/archiver/tasks.py",
> >     line 246, in link_checker
> >     raise LinkCheckerError("Invalid url scheme")
> >     LinkCheckerError: Invalid url scheme
> >     ________________________________
> >
> >
> >     Ce message et les pièces jointes sont confidentiels et réservés à
> l'usage
> >     exclusif de ses destinataires. Il peut également être protégé par le
> >     secret professionnel. Si vous recevez ce message par erreur, merci
> d'en
> >     avertir immédiatement l'expéditeur et de le détruire. L'intégrité du
> >     message ne pouvant être assurée sur Internet, la responsabilité du
> groupe
> >     Atos ne pourra être engagée quant au contenu de ce message. Bien que
> les
> >     meilleurs efforts soient faits pour maintenir cette transmission
> exempte
> >     de tout virus, l'expéditeur ne donne aucune garantie à cet égard et
> sa
> >     responsabilité ne saurait être engagée pour tout dommage résultant
> d'un
> >     virus transmis.
> >
> >     This e-mail and the documents attached are confidential and intended
> >     solely for the addressee; it may also be privileged. If you receive
> this
> >     e-mail in error, please notify the sender immediately and destroy
> it. As
> >     its integrity cannot be secured on the Internet, the Atos group
> liability
> >     cannot be triggered for the message content. Although the sender
> endeavors
> >     to maintain a computer virus-free network, the sender does not
> warrant
> >     that this transmission is virus-free and will not be liable for any
> >     damages resulting from any virus transmitted.
> >
> >     _______________________________________________
> >     ckan-dev mailing list
> >     ckan-dev at lists.okfn.org <mailto:ckan-dev at lists.okfn.org>
> >     http://lists.okfn.org/mailman/listinfo/ckan-dev
> >
> >
> >
> >
> > _______________________________________________
> > ckan-dev mailing list
> > ckan-dev at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/ckan-dev
>
> --
>                  wWw
>                 (o o)
> -------------ooO-(_)-Ooo-----------------------------------------------
> Laurent Bihanic              | Tel: +33 (0)1 73 26 12 92
> Atos Systems Integration     | GSM: +33 (0)6 25 73 03 74
> 80 quai Voltaire             |
> F-95870 Bezons               | mailto: laurent.bihanic at atos.net
> -----------------------------------------------------------------------
> Very funny, Scotty. Now beam down my clothes.
>
>
> Please consider your environmental responsibility before printing
> this e-mail.
>
> DISCLAIMER:
> The opinions expressed are entirely my own and may not necessarily be
> those of my employer.  Also, I am not now nor have I ever been a
> lawyer.  My opinions are provided as-is with absolutely no warranty of
> merchantability or fitness for any particular use.  Besides, you can't
> prove I typed this.  No body saw me type this.  Who says I typed this?
> ________________________________
>
>
> Ce message et les pièces jointes sont confidentiels et réservés à l'usage
> exclusif de ses destinataires. Il peut également être protégé par le secret
> professionnel. Si vous recevez ce message par erreur, merci d'en avertir
> immédiatement l'expéditeur et de le détruire. L'intégrité du message ne
> pouvant être assurée sur Internet, la responsabilité du groupe Atos ne
> pourra être engagée quant au contenu de ce message. Bien que les meilleurs
> efforts soient faits pour maintenir cette transmission exempte de tout
> virus, l'expéditeur ne donne aucune garantie à cet égard et sa
> responsabilité ne saurait être engagée pour tout dommage résultant d'un
> virus transmis.
>
> This e-mail and the documents attached are confidential and intended
> solely for the addressee; it may also be privileged. If you receive this
> e-mail in error, please notify the sender immediately and destroy it. As
> its integrity cannot be secured on the Internet, the Atos group liability
> cannot be triggered for the message content. Although the sender endeavors
> to maintain a computer virus-free network, the sender does not warrant that
> this transmission is virus-free and will not be liable for any damages
> resulting from any virus transmitted.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20120504/a842c194/attachment-0001.html>

• Previous message: [ckan-dev] Datastorer: celeryd fails on invalid resource URL
• Next message: [ckan-dev] Datastorer: celeryd fails on invalid resource URL
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]