[ckan-dev] Settings up FileStore/Datastore on CKAN 2.1

Wed Jan 29 06:12:23 UTC 2014

Can you hit the resource from the API? Is the DNS duplicated there too?

Nigel Babu

Developer  |  @nigelbabu <https://twitter.com/nigelbabu>

The Open Knowledge Foundation <http://okfn.org/>

Empowering through Open Knowledge

http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on
Facebook<https://www.facebook.com/OKFNetwork> |
Blog <http://blog.okfn.org/>  |  Newsletter<http://okfn.org/about/newsletter>

 CKAN | http://ckan.org/ | @CKANproject
<http://twitter.com/CKANproject> |the world’s leading open-source data
portal platform

On 29 January 2014 09:22, Michael Speth <spethm at landcareresearch.co.nz>wrote:

>  Hi,
>   Here is the output when I run the command against the database.
>
> {"help": "Search a DataStore resource.\n\n    The datastore_search action allows you to search data in a resource.\n    DataStore resources that belong to private CKAN resource can only be\n    read by you if you have access to the CKAN resource and send the appropriate\n    authorization.\n\n    :param resource_id: id or alias of the resource to be searched against\n    :type resource_id: string\n    :param filters: matching conditions to select, e.g {\"key1\": \"a\", \"key2\": \"b\"} (optional)\n    :type filters: dictionary\n    :param q: full text query (optional)\n    :type q: string\n    :param plain: treat as plain text query (optional, default: true)\n    :type plain: bool\n    :param language: language of the full text query (optional, default: english)\n    :type language: string\n    :param limit: maximum number of rows to return (optional, default: 100)\n    :type limit: int\n    :param offset: offset this number of rows (optional)\n    :type offset: int\n    :param fields: fields to return (optional, default: all fields in original order)\n    :type fields: list or comma separated string\n    :param sort: comma separated field names with ordering\n                 e.g.: \"fieldname1, fieldname2 desc\"\n    :type sort: string\n\n    Setting the ``plain`` flag to false enables the entire PostgreSQL `full text search query language`_.\n\n    A listing of all available resources can be found at the alias ``_table_metadata``.\n\n    .. _full text search query language: http://www.postgresql.org/docs/9.1/static/datatype-textsearch.html#DATATYPE-TSQUERY\n\n    If you need to download the full resource, read :ref:`dump`.\n\n    **Results:**\n\n    The result of this action is a dictionary with the following keys:\n\n    :rtype: A dictionary with the following keys\n    :param fields: fields/columns and their extra metadata\n    :type fields: list of dictionaries\n    :param offset: query offset value\n    :type offset: int\n    :param limit: query limit value\n    :type limit: int\n    :param filters: query filters\n    :type filters: list of dictionaries\n    :param total: number of total matching records\n    :type total: int\n    :param records: list of matching results\n    :type records: list of dictionaries\n\n    ", "success": true, "result": {"resource_id": "_table_metadata", "fields": [{"type": "text", "id": "_id"}, {"type": "name", "id": "name"}, {"type": "oid", "id": "oid"}, {"type": "name", "id": "alias_of"}], "records": [{"_id": "1fab8662e5772995", "alias_of": "pg_views", "name": "_table_metadata", "oid": 16391}, {"_id": "21b5fe766665b205", "alias_of": "pg_tables", "name": "_table_metadata", "oid": 16391}], "_links": {"start": "/api/3/action/datastore_search?resource_id=_table_metadata", "next": "/api/3/action/datastore_search?offset=100&resource_id=_table_metadata"}, "total": 2}}
>
>
> Does this help?
>
>
>
> On Tue, Jan 28, 2014 at 7:15 PM, Nigel Babu <nigel.babu at okfn.org> wrote:
>
>>
>>  On 28 January 2014 03:23, Michael Speth <spethm at landcareresearch.co.nz>wrote:
>>
>>>   * The DNS is duplicated. Can you take a look at your site_url in your
>>>> configuration.
>>>>
>>>
>>>  From the production.ini
>>> ckan.site_url = test.ckan.zen.landcareresearch.co.nz
>>>
>>>
>>>> * The resource is not accessible publically. To preview a resource
>>>> that's not in the datastore, CKAN uses dataproxy which is a public service
>>>> that will fetch a csv file and return JSON. If the dataproxy can't "see"
>>>> the file, you can't preview it.
>>>
>>>
>>>  curl -X GET "
>>> http://127.0.0.1:5000/api/3/action/datastore_search?resource_id=_table_metadata
>>> "
>>> output
>>> curl: (7) couldn't connect to host
>>>
>>>  Would this indicate a permissions problem?
>>>
>>
>>  No. You should run that action against your CKAN instance, not against
>> 127.0.0.1 on port 5000.
>>
>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 24, 2014 at 8:04 PM, Nigel Babu <nigel.babu at okfn.org> wrote:
>>>
>>>>  There's two problems here:
>>>>
>>>>  * The DNS is duplicated. Can you take a look at your site_url in your
>>>> configuration.
>>>>  * The resource is not accessible publically. To preview a resource
>>>> that's not in the datastore, CKAN uses dataproxy which is a public service
>>>> that will fetch a csv file and return JSON. If the dataproxy can't "see"
>>>> the file, you can't preview it.
>>>>
>>>>  Nigel Babu
>>>>
>>>> Developer  |  @nigelbabu <https://twitter.com/nigelbabu>
>>>>
>>>> The Open Knowledge Foundation <http://okfn.org/>
>>>>
>>>> Empowering through Open Knowledge
>>>>
>>>> http://okfn.org/  |  @okfn <http://twitter.com/OKFN>  |  OKF on
>>>> Facebook <https://www.facebook.com/OKFNetwork>  |  Blog<http://blog.okfn.org/> |
>>>> Newsletter <http://okfn.org/about/newsletter>
>>>>
>>>>  CKAN | http://ckan.org/ | @CKANproject<http://twitter.com/CKANproject>|the world’s leading open-source data portal platform
>>>>
>>>>
>>>>  On 22 January 2014 08:16, Michael Speth <spethm at landcareresearch.co.nz
>>>> > wrote:
>>>>
>>>>>   Greetings,
>>>>>   I am trying to setup <https://github.com/Conzar/ckan-puppet/> both
>>>>> FileStore and DataStore on the deb CKAN 2.1 package.  After uploading a csv
>>>>> and clicking on the preview button, the following error is displayed:
>>>>>
>>>>>   This resource can not be previewed at the moment. Click here for
>>>>> more information.<http://test.ckan.zen.landcareresearch.co.nz/dataset/test2/resource/f7038c5b-307a-485d-8c5c-cbb202157037#>
>>>>>
>>>>> Could not load preview: DataProxy returned an error (Data
>>>>> transformation failed. error: An error occured while connecting to the
>>>>> server: DNS lookup failed for URL:
>>>>> http://test.ckan.zen.landcareresearch.co.nz/test.ckan.zen.landcareresearch.co.nz/storage/f/2014-01-22T02%3A37%3A27.715Z/test.csv
>>>>> )
>>>>>
>>>>> *Problem*
>>>>>
>>>>> The DNS for the host has been replicated twice (
>>>>> http://test.ckan.zen.landcareresearch.co.nz/test.ckan.zen.landcareresearch.co.nz).
>>>>>  If I use this URL, my browser can download the file without issue.
>>>>>
>>>>>
>>>>> http://test.ckan.zen.landcareresearch.co.nz/storage/f/2014-01-22T02%3A37%3A27.715Z/test.csv
>>>>> )
>>>>>
>>>>>
>>>>>  Anyone know why the DNS is duplicated?
>>>>>
>>>>> Thanks
>>>>>
>>>>>  --
>>>>> Michael Speth
>>>>> DevOps Computer Engineer
>>>>> Landcare Research Consultant
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Please consider the environment before printing this email
>>>>> Warning: This electronic message together with any attachments is
>>>>> confidential. If you receive it in error: (i) you must not read, use,
>>>>> disclose, copy or retain it; (ii) please contact the sender immediately by
>>>>> reply email and then delete the emails.
>>>>> The views expressed in this email may not be those of Landcare
>>>>> Research New Zealand Limited. http://www.landcareresearch.co.nz
>>>>>
>>>>> _______________________________________________
>>>>> ckan-dev mailing list
>>>>> ckan-dev at lists.okfn.org
>>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> ckan-dev mailing list
>>>> ckan-dev at lists.okfn.org
>>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>>
>>>>
>>>
>>>
>>>  --
>>> Michael Speth
>>> DevOps Computer Engineer
>>> Landcare Research Consultant
>>>
>>> ------------------------------
>>>
>>> Please consider the environment before printing this email
>>> Warning: This electronic message together with any attachments is
>>> confidential. If you receive it in error: (i) you must not read, use,
>>> disclose, copy or retain it; (ii) please contact the sender immediately by
>>> reply email and then delete the emails.
>>> The views expressed in this email may not be those of Landcare Research
>>> New Zealand Limited. http://www.landcareresearch.co.nz
>>>
>>> _______________________________________________
>>> ckan-dev mailing list
>>> ckan-dev at lists.okfn.org
>>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>>
>>>
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>>
>>
>
>
>  --
> Michael Speth
> DevOps Computer Engineer
> Landcare Research Consultant
>
> ------------------------------
>
> Please consider the environment before printing this email
> Warning: This electronic message together with any attachments is
> confidential. If you receive it in error: (i) you must not read, use,
> disclose, copy or retain it; (ii) please contact the sender immediately by
> reply email and then delete the emails.
> The views expressed in this email may not be those of Landcare Research
> New Zealand Limited. http://www.landcareresearch.co.nz
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: https://lists.okfn.org/mailman/options/ckan-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/ckan-dev/attachments/20140129/52a26344/attachment-0003.html>