[openspending-dev] OpenSpending for Brazilian data

Thu May 31 13:45:02 UTC 2012

Hey Thiago,

On Wed, May 30, 2012 at 8:44 PM, Thiago Rondon <thiago.rondon at gmail.com> wrote:
> On Wed, May 30, 2012 at 2:57 PM, Velichka Dimitrova
> <velichka.dimitrova at okfn.org> wrote:
>> Hi Thiago,
>
> Hi Velichka!
>
>> It was really nice meeting you in Sao Paolo last month and knowing that
>> there is like-minded person interested in open data and data visualisations
>> in Brazil. We were having a discussion about bringing some of the open
>> source tools developed by the OKF to Brazilian data.
>
> It's nice to meet you guys too!
>
>> I know you have spoken with Friedrich about having the data from the
>> Brazilian Transparency Portal loaded into OpenSpending. Presumably this
>> would be the data from this page
>> - http://www.transparencia.gov.br/planilhas/ - or maybe data from the links
>> given on the Transparencia site, which has to be scraped from the original
>> websites.
>>
>> I wanted to ask you if you are still considering this and what kind of
>> support you need. Do you need to learn more about how OpenSpending works or
>> simply there wasn't enough time in the past weeks to get around that? Will
>> be happy to help in any way.
>
> Yes, I don't have enoght time in the last days, but I try to work with another
> thing in the website of paraondefoiomeudinheiro.com.br, I try to make a new
> way to see how the governement make the plan and how execute that.
>
> For example:
>
> http://www.paraondefoiomeudinheiro.com.br/eficiencia/sp/camaramunicipal/2012
>
> But...
>
> I like too much to input the data that I work in the OpenSpending,
> but it's too much easy if we have an API to send the data, because I have
> scripts that sync the data with the governements website every day, to make
> the data up to date.
>
> How can I make a script to send this data for you ? If we can, do you an
> example ?

Sure. As I've mentioned before we don't have a proper API for this
yet, but its still possible to script this quite easily. Lets assume
you have a scraper that gives you a CSV file, then the operations
would be:

1) Upload the CSV file to some location on the web. We can of course
provide you with file storage if you need.
2) Inform OS about that location and make it load the data.

I've drafted a small sample to do 2) under
https://gist.github.com/2843491 (it's Python, sorry ;). This will
basically queue a load of the source URL passed in and a few minutes
later that data should be available on the web site.

Does that make sense?

I'd be really keen to get the Portal Data in, please let me know what
we can do to assist you guys!

- Friedrich