[ckan-dev] created resources in dataset - can't access the dataset anymore?

Mon Jan 28 13:13:43 UTC 2013

This is something I would like some clarification on as well.

My gut tells me that if you have 1000 resources you've gone wrong
somewhere, but there might not be any guidance in the documentation
about what the expectation for resources is (though I might be
wrong!).  What I've been able to gather is that resources are often
different formats of the same information, or related information for
interpreting the data.

If you find you have too many resources, it might mean you should be
using separate datasets. But, how do you relate these datasets?  There
are a number of options:
- keywords
- groups
- relationships (parent, depends, links)
- organizations
- an extra field or some other external grouping

Of course if there is no extra information for each image it would be
difficult to create a dataset for each one, because at a minimum you
need to be able to name them.  Maybe that sort of data doesn't "fit"
in CKAN?

On Mon, Jan 28, 2013 at 6:18 AM, Mark Wainwright
<mark.wainwright at okfn.org> wrote:
> I can see this is quite a natural thing to want to do depending on
> what type of data you're working with. In this case, it's a single
> large collection of related images with no compelling reason to put
> each in its own dataset (or other clear logical division into small
> sets). I suspect as people use CKAN for more diverse kinds of data -
> which is great - we'll see more of this.
>
> If datasets with many resources will break the interface we should
> probably put a health warning somewhere. What is the recommended
> alternative? There are groups, but they seem to be moving more towards
> aligning with authorisation stuff.
>
> Mark
>
>
> On 25/01/2013, Sean Hammond <sean.hammond at okfn.org> wrote:
>>> For future development of CKAN, it might be a good investment to look at
>>> solutions where datasets could hold more than a few thousand resources:
>>> that does not seem like an unusual scenario.
>>
>> It's an interesting idea. I think that is actually an unusual scenario,
>> I don't think I've ever seen a dataset with that many resources. Why
>> exactly do you need so many resources in one dataset?
>>
>> To handle datasets with thousands of resources, as well as making sure
>> that requests for the dataset don't timeout I think CKAN would also need
>> pagination of the resource list and probably search for resources within
>> a dataset. It may not all be worthwhile, since I think almost all
>> datasets have only a few resources.
>>
>> _______________________________________________
>> ckan-dev mailing list
>> ckan-dev at lists.okfn.org
>> http://lists.okfn.org/mailman/listinfo/ckan-dev
>> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev
>>
>
> _______________________________________________
> ckan-dev mailing list
> ckan-dev at lists.okfn.org
> http://lists.okfn.org/mailman/listinfo/ckan-dev
> Unsubscribe: http://lists.okfn.org/mailman/options/ckan-dev