[ddj] SQL Vs Excel Vs Refine

Tue Apr 30 07:57:39 UTC 2013

Hey,

I think sqlite is great, but it really doesn't do joins all that well. So I
think its important that when your data grows beyond a gig and you still
need to do joins, you leave ship and use a proper DB server.

As for UIs: http://www.sequelpro.com/ is for MySQL on Mac, it's incredible.
phpMyAdmin against a remote server can also be good. I use Navicat for
Postgres at the moment, but its a usability nightmare.

- Friedrich

On Tue, Apr 30, 2013 at 9:21 AM, Michael Bauer <michael.bauer at okfn.org>wrote:

> SiFu,
>
> Exactly what I thought. Don't forget sqlite! However it's still a command
> line thing but it uses a simple file as a backend. Mac OS X comes with
> sqlite pre-installed - so there is little barrier of using it there.
>
> Just use sqlite3 <file> and the file will be used as a database - you can
> then handle it like any other file (and even send your database to friends,
> plug it into an application etc.)
>
> Michael
>
> On Tue, Apr 30, 2013 at 07:41:37AM +0200, Siegmund Führinger wrote:
> > hi joe!
> >
> > don't forget about http://www.sqlite.org/. no server you have to take
> care
> > of. it's just a simple command.
> > i use it quite a bit even though i have several postgres and couchdb
> > servers.
> >
> > SiFu
> >
> >
> > On Mon, Apr 29, 2013 at 5:21 PM, Joe Germuska <joe at germuska.com> wrote:
> >
> > > I like all of Sharon's answers quite a bit. Especially "it was designed
> > > for subsetting, slicing and dicing" and "easier to go back and check my
> > > work"
> > >
> > > But I sometimes wonder: when people ask "should I learn SQL?" aren't
> they
> > > usually asking "is SQL really worth all the arcana of installing MySQL
> or
> > > Postgres?" Unfortunately, it is pretty arcane, although the MAMP/WAMP
> > > package seems to be a pretty good way to get going.
> > >
> > > I'm sure people will chime in with other favorite installers, packages,
> > > GUI admin tools and the like, but I'm afraid that the plethora of
> responses
> > > is just going to reinforce for many journalists the basic problem—it
> can
> > > quickly become its own adventure. A great adventure, like learning to
> cook
> > > food from scratch instead of from kits and convenience packages, but an
> > > adventure nevertheless…
> > >
> > > Joe
> > >
> > >
> > >
> > > On Apr 29, 2013, at 7:46 AM, SMachlis at computerworld.com wrote:
> > >
> > > > What I like about SQL:
> > > >
> > > > = It was *designed* for subsetting, slicing and dicing data. Yes, I
> can
> > > do this to a large degree with Excel and Google Refine; but with a more
> > > complex project -- especially as others have pointed out, with data
> having
> > > one or more relationships between multiple tables -- there are times
> that I
> > > find that using a tool designed for the job to be less frustrating and
> > > considerably more robust.
> > > >
> > > > = If I am dealing with a large data set that is already in multiple
> > > tables, SQL makes more sense to be than trying to shoehorn that data
> into
> > > an Excel-friendly format.
> > > >
> > > > = It helps me think about data in a more structured way, which is
> very
> > > useful when I've got projects where I'm collecting and storing my own
> data.
> > > >
> > > > = It helps me understand what sorts of data I can and can't
> reasonably
> > > request from government agencies that store their data in structured
> > > databases.
> > > >
> > > > = If I am sharing data with colleagues, sometimes it's useful to be
> able
> > > to put up a simple PHP/MySQL app on our intranet (Rails or Jango might
> be a
> > > better choice for this, but the shared internal server I have access to
> > > does not include those platforms). Even if I'm creating a Web
> application
> > > with a third-party service such as Caspio, I find it helpful to be
> able to
> > > think about data in relational terms.
> > > >
> > > > = Having a series of SQL commands I can store in a file makes it
> easier
> > > for me or others to go back and check my work, versus a series of Excel
> > > point-and-click operations (or even multiple macros buried in Excel).
> > > >
> > > > Sharon Machlis
> > > >
> > > > ________________________________________
> > > > From: data-driven-journalism-bounces at lists.okfn.org [
> > > data-driven-journalism-bounces at lists.okfn.org] On Behalf Of Andrew
> Duffy [
> > > andrewjamesduffy at gmail.com]
> > > > Sent: Monday, April 29, 2013 12:37 AM
> > > > To: data-driven-journalism at lists.okfn.org
> > > > Subject: [ddj] SQL Vs Excel Vs Refine
> > > >
> > > > Question:
> > > >
> > > > Are there any data journalists/devs out there that can advise as to
> > > whether it's worth learning SQL? So far a combination of Excel/Google
> > > Refine has been more than enough for dumping, organising, and cleaning
> my
> > > data projects, but I have only worked with spreadsheets up to ~500
> rows.
> > > >
> > > > What can SQL do that refine/excel can't?
> > > >
> > > > --
> > > >
> > > > Andrew Duffy - Journalist
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > data-driven-journalism mailing list
> > > > data-driven-journalism at lists.okfn.org
> > > > http://lists.okfn.org/mailman/listinfo/data-driven-journalism
> > > > Unsubscribe:
> > > http://lists.okfn.org/mailman/options/data-driven-journalism
> > >
> > > --
> > > Joe Germuska
> > > Joe at Germuska.com * http://blog.germuska.com *
> > > http://twitter.com/JoeGermuska
> > >
> > > "Science's job is to map our ignorance." --David Byrne
> > >
> > >
> > > _______________________________________________
> > > data-driven-journalism mailing list
> > > data-driven-journalism at lists.okfn.org
> > > http://lists.okfn.org/mailman/listinfo/data-driven-journalism
> > > Unsubscribe:
> http://lists.okfn.org/mailman/options/data-driven-journalism
> > >
>
> > _______________________________________________
> > data-driven-journalism mailing list
> > data-driven-journalism at lists.okfn.org
> > http://lists.okfn.org/mailman/listinfo/data-driven-journalism
> > Unsubscribe:
> http://lists.okfn.org/mailman/options/data-driven-journalism
>
>
> --
> Data Wrangler with the Open Knowledge Foundation (OKFN.org)
> GPG/PGP key: http://tentacleriot.eu/mihi.asc
> Twitter: @mihi_tr Skype: mihi_tr
>
> _______________________________________________
> data-driven-journalism mailing list
> data-driven-journalism at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/data-driven-journalism
> Unsubscribe: http://lists.okfn.org/mailman/options/data-driven-journalism
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/data-driven-journalism/attachments/20130430/4a5e9d30/attachment-0001.html>