[ddj] SQL Vs Excel Vs Refine

Mon Apr 29 08:24:39 UTC 2013

Hi all,
I think refine is good for reformatting data, but Excel and SQL has it too.
I would say, If i have more than 50 000 lines and 50 or more columns, and a
lot of cleaning to do, i would use SQL. Especially the relationship for SQL
is a very strong argument. You can do it with excel, but you have to go
into macro and it gets very unclear pretty fast. For exemple, in SQL you
can keep a view or a SQL sentence and label it, and try different "angle"
for you data. Which would be very difficult/heavy for Excel. Also, I found
google spreadsheets pretty limited compared to excel.
Finally, it really depends how you're going to use the data : if you're
putting them into Illustator, Excel is good enough. If you plan to do some
interactive work, You would gain some time on using SQL. I think SQL is a
good investment in time, but you won't have a lot of SQL capabilities if
you work in different offices. Excel is in a lot of computers but is not
the most powerfull tool (I think powerfullness comes with readability). I
was not convinced (yet) by Refine.
hope it helps.

Mehdi Guiraud
Journaliste multimédia, EMI-CFD
t. @mguiraud
m. 06 95 92 51 33
Tèl. : 09 53 14 98 49

2013/4/29 Nicolas Kayser-Bril <hi at nkb.fr>

> I haven't used Excel in a long time, using a mix of Google Spreadsheet,
> Refine and custom scripts to manage and analyze data.
>
> Although you can survive without SQL, it remains the standard in
> relational data, as Steve said. You'll find quite a lot of APIs rely on a
> SQL-like structure (like the Fusion Table, Scraperwiki or CartoDB API). SQL
> will also make it easier to grasp other query languages such as SPARQL or
> Cypher when you move into triple stores and the like.
>
> nicolas.
>
>
> On Mon, Apr 29, 2013 at 8:09 AM, Stephen Doig <steve.doig at asu.edu> wrote:
>
>>  A main reason to learn SQL is to handle relational data — datasets that
>> come with several tables that may need to be linked. A simple example would
>> be a large main table that uses codes for categorical variables like race,
>> type of crime, hospital diagnosis, etc., then separate lookup tables that
>> translate the codes into something readable. SQL also will handle large
>> datasets beyond the one million or so records that is the current Excel
>> limit. SQL also will do complicated queries that involve conditional
>> statements.
>>
>>  With that said, Excel will handle the majority of data problems most
>> likely to be faced by most data journalists. It's a good idea to start
>> learning SQL so as to be ready when that big project comes up, but skills
>> with Excel (or Fusion Tables) will take you a long way.
>>
>>  Steve Doig
>>
>>
>>
>>   From: Andrew Duffy <andrewjamesduffy at gmail.com>
>> Reply-To: "List about Data Driven Journalism and Open Data in
>> Journalism." <data-driven-journalism at lists.okfn.org>
>> Date: Mon, 29 Apr 2013 14:37:51 +1000
>> To: <data-driven-journalism at lists.okfn.org>
>> Subject: [ddj] SQL Vs Excel Vs Refine
>>
>>  Question:
>>
>>  Are there any data journalists/devs out there that can advise as to
>> whether it's worth learning SQL? So far a combination of Excel/Google
>> Refine has been more than enough for dumping, organising, and cleaning my
>> data projects, but I have only worked with spreadsheets up to ~500 rows.
>>
>>  What can SQL do that refine/excel can't?
>>
>>  --
>>
>> *Andrew Duffy - Journalist*
>>
>>
>>    _______________________________________________
>> data-driven-journalism mailing list data-driven-journalism at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/data-driven-journalismUnsubscribe:
>> http://lists.okfn.org/mailman/options/data-driven-journalism
>>
>> _______________________________________________
>> data-driven-journalism mailing list
>> data-driven-journalism at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/data-driven-journalism
>> Unsubscribe: http://lists.okfn.org/mailman/options/data-driven-journalism
>>
>>
>
> _______________________________________________
> data-driven-journalism mailing list
> data-driven-journalism at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/data-driven-journalism
> Unsubscribe: http://lists.okfn.org/mailman/options/data-driven-journalism
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/data-driven-journalism/attachments/20130429/afd46cf6/attachment-0002.html>