[openspending-dev] How to update data from specific source?

Fri Nov 20 17:47:11 UTC 2015

Even dropping and reloading the data didn't work, for I got errors like this
for each line:

	(IntegrityError) insert or update on table "orcamento_federal__entry"
	violates foreign key constraint "fk_orcamento_federal_orgao" DETAIL: Key
	(orgao_id)=(3) is not present in table "orcamento_federal__orgao". 'INSERT
	INTO orcamento_federal__entry (id, orgao_id, funcao_id, uo_id, gnd_id,
	time_id, subfuncao_id, pago, amount, rppago) VALUES (%(id)s, %(orgao_id)s,
	%(funcao_id)s, %(uo_id)s, %(gnd_id)s, %(time_id)s, %(subfuncao_id)s,
	%%(pago)s, %(amount)s, %(rppago)s)' {'subfuncao_id': 3, 'orgao_id': 3,
	%'time_id': 1, 'uo_id': 3, 'amount': 5310158.0, 'funcao_id': 3, 'pago':
	%95113.48, 'gnd_id': 1, 'id': '3f4c569fa77019f2c1b9e7847f0894f9e25b86f4',
	%'rppago': 2717187.94}

Even using the same data and the same model.json that was working before.

So I created another dataset, added the data, deleted the entire dataset (to
test), added the data again and it seems to work.
So... the only way to update data is to delete the entire dataset? Am I
missing something?

[2015-11-19 15:52] :

> 
> Hello! In a dataset, is there a way to update data from a specific source?
> 
> We have a dataset with many sources (one source for each year), and we want to
> update the current year data weekly.
> 
> After a source is marked as "Successfully loaded" there is no button in the
> website interface to unload it, and the "/sources/{id}/delete" endpoint seems
> not to work neither.
> The "Delete loaded data" button unloads all the sources, what is bad in this
> case, because then we have to reload ALL the sources (years) again and we just
> need to update the current year, not all of them.
> So I tried to just add a new source for the current year (duplicating it), the
> interface returned errors about duplicated PK (what is expected), but I hoped
> it to replace old lines with new lines where PKs collided, updating the data.
> After some months doing that, it seems I was wrong and instead of updating the
> data OpenSpending is keeping old lines when PKs collide, inserting only new
> lines (with new PKs).
> 
> So, is our only option to drop and reload ALL the sources?
> 
> Is there a way to delete data based on a field value?
> e.g.: delete lines where year==2015
> 
> 
> Thanks for the attention