[OpenSpending] Portuguese database of contracts

[Rufus Pollock]

On 23 January 2014 16:31, Jorge Cardoso Leitão <jorgecarleitao at gmail.com>wrote:

> Hi Rufus
>
> - Portuguese database:
>
> From the db point of view, the data is the same.
>
> The db structure is different since one contract in mine has two
> many-to-many with entities
>  (a contract between several entities is possible), while in the
> OpenSpending is a Many-To-One, as pointed out in this blog entry<http://community.openspending.org/2013/12/processing-data-for-open-spending/>in openspending by Daniel and Olaf.
>

I think this raises an important point about "split transactions" - which
we've actually been debating since before OpenSpending even officially
began. How should we handle cases where multiple entities are involved (on
one side) of a transaction. Should we allocate it to all of them (but which
leads to double-counting when you sum), or should we do a "split"
transaction and allocate it among those parties.

>From an accounting point of view I'd prefer the latter but it leads to
complexities (e.g. you need to allow for a "transaction" which has many
separate payments. We actually had this in the very early version of
OpenSpending but it added a *lot* of complexity and performance challenges
and was rarely used so we moved to have a more simple OLAP style setup.

Maybe there is another difference, which I believe is less relevant: the
> categories in the public procurements in EU create a tree (CPVS).
>

Right the taxonomy has a hierarchy. That can be supported by OpenSpending
DB structure in a reasonable way.


> In my db this is created using a nested set model<https://en.wikipedia.org/wiki/Nested_set_model>(very expensive to add, very easy to read because it is static), which I
> don't know how is implemented in OpenSpending.
>

This sounds interesting (though if we get technical we should probably move
to https://lists.okfn.org/mailman/listinfo/openspending-dev :-) ...)

I talked with Olaf and I believe the issue of the dataset in OpenSpending
> will be solved soon or is already solved :)
>

:-)

- automate analysis:
>
> I think I don't know enough about database design and database technology
> to help you at this point.
>

This is not complex - its about a tool for automatically loading from the
CSV files in an openspending dataset into e.g. Google BigQuery.


> - OpenSpending
>
> I already took a look at the code, and It seems quite good,
> congratulations. I will definitely contribute when and where I can.
> From what I understood, OpenSpending uses Pylons. I tried SQLAlchemy and
> Jinga2 before, but Pylons I've never used before; I think it is no problem
> thought for small to medium issues. :P
>

:-)

Rufus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.okfn.org/pipermail/openspending/attachments/20140124/eceb98bf/attachment-0002.html>