[openspending-dev] Table widget for Search and Aggregates

Mon Mar 5 21:29:02 UTC 2012

Nick, the reason I like SlickGrid's "show me everything" approach is pure
UX - when I look at an unfamiliar dataset:

* The first thing I want to do is scroll through it all + get a visual
sense of how large it is and what's in it.
* The second thing I want to do is use Ctrl+F to hunt for things quickly
(without having to figure out a custom search interface).

Pagination makes both more difficult.

However! I have just found that there is a DataTables plugin to load large
datasets in the background [1], so I guess you can always use that if other
users say the same. I'm sure you're making the right choice, anyway.

[1]
http://datatables.net/blog/Introducing_Scroller_-_Virtual_Scrolling_for_DataTables

On 5 March 2012 14:53, Nick Stenning <nick at whiteink.com> wrote:

> Hi Anna and others,
>
> Thanks for the contributions. For the time being, I've experimented enough
> with jQuery DataTables [http://datatables.net/] to establish that it's
> going to be fine for our usage in OpenSpending.
>
> Slickgrid does seem nice (and enormous), but the requirement for "infinite
> scroll" over millions of entries seems a bit artificial (what's wrong with
> pagination?) so I'm not too worried about this. DataTables has many nice
> features including multisorting, "lambda columns" (as Friedrich has coined
> them), and is easy to tie into a serverside search API. It's only major
> downside as far as I can see is an aesthetic one: the original author wrote
> it using a variant of Hungarian notation. But I've trained my nausea
> response and realised that it's actually a mature, well-tested piece of
> software.
>
> I'll update the list when we get this properly integrated into OS.
>
> Best,
> N
>
> On 1 Mar 2012, at 19:07, Anna Powell-Smith <annapowellsmith at gmail.com>
> wrote:
>
> I don't know all your requirements, but SlickGrid is worth another look if
> you want to be able to present and filter hundreds of thousands of rows,
> because as far as I know it's the only plugin that can currently do that:
> http://stackoverflow.com/questions/2402953/javascript-data-grid-for-millions-of-rows
>
> On the downside, it's feature-heavy, as you say, has its own idea of what
> a data model should be, and the documentation is minimal.
>
> On 1 March 2012 16:26, Friedrich Lindenberg <friedrich.lindenberg at okfn.org
> > wrote:
>
>> Just a quick update on this: it seems like Nick is reasonably happy with
>> the way datatables works and can be intercepted to hook in other backends,
>> so we'll do a spike on getting this to run on top of the aggregator API in
>> a clean way (i.e. unlike the current datatables plugin). Having gone
>> through some of the feature ideas with Nick, I think this is going to work
>> OK and will be the quickest way for us to provide a stable widget.
>>
>> I'll also try to do a strawman draft of the search v2 API soon so that we
>> can build a newer browser for this, too.
>>
>> - Friedrich
>>
>> On Wednesday, February 29, 2012 at 10:20 AM, Rufus Pollock wrote:
>>
>> On 20 February 2012 21:09, Friedrich Lindenberg
>> <friedrich.lindenberg at okfn.org> wrote:
>>
>> Hi all,
>>
>>
>> Sorry to not get to this earlier :-)
>>
>> I'm still undecided on which way to go with the new data tables in
>> OpenSpending. To recap the scope, they are first going to be used for
>> an embeddable search widget and then later also for displaying
>> aggregates (i.e. replace datatables). There are a couple of options
>> for this:
>>
>> 1) Use recline. Its nice because it already does a lot of what we need
>> and since its developed "in-house" we can influence its further
>> development to do more custom bits. But, to be honest, I'm also
>> feature-creeped out by it: going through the list of things that it is
>> supposed to do now or in the future, it's clear that we will have to
>> hack it very seriously to keep it down to match its task. (And, to be
>>
>>
>> Is that really a true statement. Right now you can just grab model.js
>> plus view-grid.js [1] and you are go.
>>
>> [1]: https://github.com/okfn/recline/blob/master/src/view-grid.js
>>
>> a bit brutal, that it will not be able to do all these things well
>> with the resources we have and that it therefore will remain an
>> experiment.) This is already becoming an issue in debates about it
>> including multiple views or a generalized query format.
>>
>>
>> I think it is our currently our best bet and more advanced than any
>> other option. It is also pretty easy to add to.
>>
>> 2) Use slickgrid, jqgrid or datatables. To be honest, I'm not too
>> excited about any of them - some (datatables) are trying to keep it
>> simple and then only accept APIs in a very specific format, while
>> others (slickgrid) are incredibly complex and try to boot what feels
>> like a few copies of excel in your browser. I haven't seen a nice
>> trade-off yet. On the other hand, you're likely to get mature code and
>> lots of configuration options.
>>
>>
>> We've used slickgrid before and various other options. We specifically
>> thought about using Slickgrid for the grid component in Recline and
>> decided against, the primary reason being that Backbone already
>> provided you with a lot of the proper 'data-oriented' structure that
>> Slickgrid provides in a cleaner way.
>>
>> 3) Roll our own, probably a simplified/feature-limited fork of recline
>> or the old spend-browser. Of course it's tempting but also probably a
>> huge waste of resources. The argument here is that we don't want that
>>
>>
>> :-) -- it probably is though I very much understand of having complete
>> control.
>>
>> much from it, but that those things are probably going to be fairly
>> specific (around query state with facets and embeddability).
>>
>>
>> The issue always is that the dev curves for rolling one's own versus
>> using other stuff always are a mirror of each other:
>>
>> a) Roll-your-own: easy at the start, and then progressively more costly
>> b) Re-use something else: more costly at the start but then relatively
>> cheaper.
>>
>> Where these intersect and b) becomes better than a) determines what
>> you should do. Given the innate tendency of coders to underestimate
>> the cost of a) and to prefer rolling their own my general rule is that
>> you should have *always* made one good faith attempt to build on an
>> existing library or solution before you roll your own.
>>
>> I honestly don't know where I stand on this, and I'd appreciate any
>> input. @Rufus: I know you feel strongly about recline, but please put
>> yourself in your OpenSpending shoes for this :)
>>
>>
>> Understood. I wrote the original table browser. Following my
>> injunction to reuse I used ajax-solr and the jquery tablesorter. I
>> think it worked OK though AJAX solr was probably overkill for what we
>> ended up doing.
>>
>> The core of my view on this is that you probably do want a proper
>> internal structure and you probably want to use Backbone because it
>> gives you good structure while remaining lightweight. Once you are
>> doing that you are going to at least re-invent the model part of
>> Recline. Copying and pasting that is no big deal so that's definitely
>> an option (remember its the conceptual components that take most of
>> the time ;-) ). Following my point above I'd still definitely
>> recommend trying one good faith effort to re-use Recline -- even if
>> you decide against you'll have avoided the trap of underestimating a)
>> and you'll have a better sense of what you are doing.
>>
>> Rufus
>>
>>
>>
>> _______________________________________________
>> openspending-dev mailing list
>> openspending-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/openspending-dev
>>
>>
>
>
> --
> 07812 357037
>
> _______________________________________________
> openspending-dev mailing list
> openspending-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/openspending-dev
>
>

-- 
07812 357037
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending-dev/attachments/20120305/5fa68844/attachment.html>