[ddj] SQL Vs Excel Vs Refine

Mon Apr 29 12:46:45 UTC 2013

What I like about SQL:

= It was *designed* for subsetting, slicing and dicing data. Yes, I can do this to a large degree with Excel and Google Refine; but with a more complex project -- especially as others have pointed out, with data having one or more relationships between multiple tables -- there are times that I find that using a tool designed for the job to be less frustrating and considerably more robust.

= If I am dealing with a large data set that is already in multiple tables, SQL makes more sense to be than trying to shoehorn that data into an Excel-friendly format.

= It helps me think about data in a more structured way, which is very useful when I've got projects where I'm collecting and storing my own data.

= It helps me understand what sorts of data I can and can't reasonably request from government agencies that store their data in structured databases.

= If I am sharing data with colleagues, sometimes it's useful to be able to put up a simple PHP/MySQL app on our intranet (Rails or Jango might be a better choice for this, but the shared internal server I have access to does not include those platforms). Even if I'm creating a Web application with a third-party service such as Caspio, I find it helpful to be able to think about data in relational terms.

= Having a series of SQL commands I can store in a file makes it easier for me or others to go back and check my work, versus a series of Excel point-and-click operations (or even multiple macros buried in Excel).

Sharon Machlis

________________________________________
From: data-driven-journalism-bounces at lists.okfn.org [data-driven-journalism-bounces at lists.okfn.org] On Behalf Of Andrew Duffy [andrewjamesduffy at gmail.com]
Sent: Monday, April 29, 2013 12:37 AM
To: data-driven-journalism at lists.okfn.org
Subject: [ddj] SQL Vs Excel Vs Refine

Question:

Are there any data journalists/devs out there that can advise as to whether it's worth learning SQL? So far a combination of Excel/Google Refine has been more than enough for dumping, organising, and cleaning my data projects, but I have only worked with spreadsheets up to ~500 rows.

What can SQL do that refine/excel can't?

--

Andrew Duffy - Journalist