[okfn-labs] About elasticsearch

Sat Dec 21 20:00:04 UTC 2013

Have you considered changing your code so that whenever you save to
Postgres, it also saves to Elasticsearch?

On Sat, Dec 21, 2013 at 5:29 AM, Emanuil Tolev <emanuil at cottagelabs.com>wrote:

> Hi Alioune,
>
> I haven't done polling from a relational database, but by the sound of it
> you've gotten it work - just not as often as you like.
>
> Two things:
> 1/ It may actually be polling every 10 seconds. That doesn't mean the
> results will be available for querying though. The index will have to be
> refreshed before that happens. However, by default this happens every 1
> second, so you should be seeing the results pretty quickly.
>
> Nonetheless, try issuing a refresh to your index ( see
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-refresh.html) before you try to query for new data from your Postgres database.
>
> 2/ Are you sure the operation can complete in 10 seconds? This seems like
> a high frequency for a river update. The query has to run on Postgres,
> return results and transmit them (over network?) to elasticsearch.
>
> At least 2 things to try here:
>   a. try using a 1 minute ( "1m" ) poll interval
>   b. check the elasticsearch logs for errors and information
>
> Greetings,
> Emanuil
>
>
> On 20 December 2013 18:49, Alioune Dia <dia.aliounes at gmail.com> wrote:
>
>> Hello
>>
>> I don't Know if It is the best place To ask question about elasticsearch,
>> but I found many interresting link into the labs , that allow me to
>> progres
>> with elasticsearch installation and Query DSL
>>
>> http://okfnlabs.org/blog/2013/07/01/elasticsearch-query-tutorial.html
>>
>> Thank for putting This Up . So my question is  : Is it possible to tell
>> elasticseach
>> to poll new insertion from postgres db .  In my case I found that
>> elastichsearch
>> take too time to poll new insertion. I changed My River to add poll
>> attribute and set it to 10s ,
>> but it take too time to index , is it normale ?
>>
>> curl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{
>>     "type" : "jdbc",
>>     "jdbc" : {
>>         "driver" : "com.mysql.jdbc.Driver",
>>         "url" : "jdbc:mysql://localhost:3306/test",
>>         "user" : "",
>>         "password" : "",
>>         "sql" : "select * from orders",
>>         "index" : "my_jdbc_index",
>>         "type" : "my_jdbc_type",
>>         "poll" :''10s"
>>     }
>> }'
>>
>> Any help will be appreciate
>> --Ad
>>
>> _______________________________________________
>> okfn-labs mailing list
>> okfn-labs at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/okfn-labs
>> Unsubscribe: https://lists.okfn.org/mailman/options/okfn-labs
>>
>>
>
> _______________________________________________
> okfn-labs mailing list
> okfn-labs at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/okfn-labs
> Unsubscribe: https://lists.okfn.org/mailman/options/okfn-labs
>
>

-- 
Developer | sunlightfoundation.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20131221/3b2bda44/attachment-0004.html>