[openbiblio-dev] bibserver dao

Mon Aug 22 09:25:09 UTC 2011

hi Rufus,

Yep, the agreed functionality is fine. Have seen your changes to the
repo. I note you have changed the top of solreyes as well. I am
altering that to ES and to call the dao, so will write changes into
there.

However I disagree with removing the local store - the functionality
to store files that were retrieved or uploaded was already there in
the manager - sure, it did not schedule anything yet and that would
not be a major requirement until we expected large files, but having a
cache seems obvious to me - why would we want to grab copies of things
that have not changed? It just checked the last-modified date, so all
we had to add was a comparison to the created date of the local file.
Also, the old bibserver that Jim uses had caching, and he sees it as a
requirement. Can you add that back in to the refactored version
please?

Mark

On Mon, Aug 22, 2011 at 9:56 AM, Rufus Pollock <rufus.pollock at okfn.org> wrote:
> I think this was resolved by our chat on skype :-)
>
> As you saw the DAO layer *is* ultra-simple (though I think it will get
> somewhat more complex over time).
>
> Rufus
>
> On 20 August 2011 16:01, Mark MacGillivray <mark at cottagelabs.com> wrote:
>> hi rufus,
>>
>> thinking about the dao:
>>
>> If we are only building this for ES, I dont see the point of building
>> an overly complex dao layer - we can easily pass stuff back and fore
>> to ES via the http, and the only thing that really misses is having to
>> pass data objects from the front end.
>>
>> So, why dont we just add an extra param to queries - e.g. our front
>> end can build up search queries in ES format, and where we cannot do
>> that directly, such as passing facet info, we can add a param called
>> "data" and its value can be the JSON string that has to be passed.
>>
>> solreyes already uses a JSON string like this as the "a" parameter, so
>> I can just convert the way it reads and writes those. Then, all I want
>> to do is set up a dao object, and do something like
>> dao.query(param_string) - and it should unpack the "data" param if
>> there is one, or receive the data object if it has been contacted
>> directly, say via curl.
>>
>> This way, we can easily rely on ES to provide the JSON output, and
>> just explain in our API that the output is ES - therefore, to find
>> your actual result objects, just look in data.hits.hits and they will
>> by listed under _source, and to find or set various params, do so in
>> the normal ES way but with the added value of the "data" param on the
>> GET, if for some reason not sending in a way that data can be passed
>> (e.g. from the browser).
>>
>> All else our dao needs then is things like "delete" or "record"
>> functions where perhaps we do useful things like delete a record via
>> its ID, or get a specific record as a JSON object without the
>> surrounding ES stuff. This is what I had in mind from what is already
>> in the dao.
>>
>> The only real value of a dao that changes the way the results are
>> presented or accessed, that I can see, would be if we were wanting to
>> call on different back ends. If we are not going to do that though,
>> then what is the value of the extra effort?
>>
>