[School-of-data] BIG data... What is BIG?

Thu Jul 10 13:29:12 UTC 2014

On Wed, Jul 9, 2014 at 10:53 AM, Simon Cropper <
simoncropper at fossworkflowguides.com> wrote:

> I have been exploring various projects that claim to handle BIG data but
> to be honest most do not qualify what BIG actually means.
>
> I remember the days when programs specified the maximum number of records,
> maximum number of fields and maximum number of tables in a database that
> could be manipulated at any one time. Why don't these types of specs get
> provided for languages and libraries anymore?
>

Because in most cases there are no hard limits or they are so high as to be
insignificant.  It's easy to say 16-bit Excel can have a maximum of 32767
rows, but your poor little 1980s PC probably couldn't hold enough memory to
deal with that many reasonable anyway.  Today the answer to most of these
things is "it depends."  What it depends on is complex enough that people
bother trying to describe it.

> What are people's impression of what BIG actually means when used to
> describe large datasets?
>

The classic definition involves one or more of Volume, Velocity, & Variety:

http://radar.oreilly.com/2012/01/what-is-big-data.html

There's also a Wikipedia page devoted to the topic:

http://en.wikipedia.org/wiki/Big_data

As well as a number of overview articles:

http://lmgtfy.com/?q=What+is+big+data%3F

Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/school-of-data/attachments/20140710/19381aa3/attachment-0002.html>