[ckan-discuss] Linked Data and the DataHub (and CKAN generally) DataStore(s)

[Rufus Pollock]

[Forwarding more thread parts]

On 28 April 2012 10:37, Pablo Mendes <pablomendes at gmail.com> wrote:
>
> Hi all,
>
>> This is *not* about metadata. It's about storing linked *data* in the CKAN
>> / DataHub DataStore: <http://docs.ckan.org/en/latest/datastore.html>
>
> Interesting...
>
> Does this mean that TheDataHub is now entering the "data market" space?
> Would it be a free alternative to Kasabi, InfoChimps, and the likes? Would
> it withstand something like DBpedia being loaded into and queried from it?

I don't particularly like the term data market or data market place.
However, in the sense you mean that CKAN / DataHub would be a place to
store data as well as catalog it then the definite answer is yes --
and has been for some time: CKAN (and hence the DataHub) has had
integrated (file) storage for over a year and the first version of the
DataStore (i.e. structured data storage with data API) was out last
summer and the major upgraded and revised version being announced last
month [1].

[1]: http://ckan.org/2012/03/27/ckan-datastore-and-data-api/

Regarding the capacity question: yes, the intention would be to be
able to store large datasets as needed though. That said, we'd
probably want to make special preparations if people were suddenly to
regularly store multi-GB files in the DataStore e.g. in making sure we
spun up more servers, put appropriate caching in place etc (I note
that you can, of course, happily store such multi-GB files as 'blobs'
in the FileStore).

I should also emphasize that if really want a full on triple-store
with SPARQL etc the DataStore in CKAN is definitely *not* appropriate
since it ain't a triple-store! Though I note one of our other
interests (discussed for some time) would be to have integration of
CKAN with a proper TripleStore -- in the same way we currently
integrate with ElasticSearch for the default CKAN DataStore -- see
this slightly old diagram
<http://wiki.ckan.org/Storage#Overview_.5BVersion_1.5D>

Rufus