[okfn-labs] Python Iterator over table (csv) *columns*

[Friedrich Lindenberg]

So you're looking for something that has better performance than reading
the whole file N (number of cols) times? That seems hard. The only thing
you might be able to do is cache the line lengths, so you can read without
seeking newlines and end-quotes after the first run...

- Friedrich

On Wed, Dec 17, 2014 at 10:17 AM, Paul Walsh <paulywalsh at gmail.com> wrote:
>
> Yes I know, I’m looking for some magic, or at least some possible
> approaches that anyone may have used in some context.
>
> On 17 Dec 2014, at 10:45, Tarek Amr <tarekamr at gmail.com> wrote:
>
> I do not think this is possible, there is no way to tell beforehand the
> indices for each new line without reading the whole file in memory to
> search for '\n's, also cells in the case of CSV are not of a fixed size, so
> no way build indices for cells without reading the whole line in memory
> looking for separators.
>
> Nevertheless, may be there is some magical solution out there that I don't
> know.
>
>
> On Wed, Dec 17, 2014 at 8:23 AM, Paul Walsh <paulywalsh at gmail.com> wrote:
>>
>> Hi,
>>
>> Does anyone have or know of a nice (existing) solution for iterative
>> reading of CSV/table data by *column*? It needs to be an iterator - I don’t
>> want everything in memory.
>> _______________________________________________
>> okfn-labs mailing list
>> okfn-labs at lists.okfn.org
>> https://lists.okfn.org/mailman/listinfo/okfn-labs
>> Unsubscribe: https://lists.okfn.org/mailman/options/okfn-labs
>>
>
>
> --
> Best Regards
> Tarek Amr
>
> http://tarekamr.appspot.com/
>
>
>
> _______________________________________________
> okfn-labs mailing list
> okfn-labs at lists.okfn.org
> https://lists.okfn.org/mailman/listinfo/okfn-labs
> Unsubscribe: https://lists.okfn.org/mailman/options/okfn-labs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/okfn-labs/attachments/20141217/2dd61bf6/attachment-0004.html>