[wdmmg-dev] dfig loader and dataset (DFIDv3.xls)

Fri Mar 11 17:47:42 UTC 2011

Hi all,

first a hello to everybody. I started to work on wdmmg yesterday and dig
through the different parts of it atm.

dfig data (DFIDv3.xls)
My current task is to update the dfig loader. wdmmgext.load.dfig tries to
get the data from DFIDv3.xls. There is a dfid package in ckan
(<http://ckan.net/package/ukgov-dfid-project-db>), but that contains
different data. dataDFIDv3.xls does not seem to be available online, e.g.
from the dfig site. Friedrich mentioned that Nick (?) is the brain behind
it and working on a bigger dataset atm. Is that right?
If that's the case should I update the loader to import dataDFIDv3.xls or
is it better to wait for the a new version of the data. If dataDFIDv3.xls
will stay relevant should I turn it into a ckan package. Friedrich fetched
a copy from one of our servers and it can be downloaded now from
http://rawdatanow.com/DFIDv3.xls, but is nowhere published.

wdmmgext.load.utils
This package contains helper functions that still contain sqlalchemy
related code:
* get_or_create_dataset
* get_or_create_key
* get_or_create_value

These functions are used in the gla and ordnance loaders which fail. Should
they be updated too.

..Carsten