[open-linguistics] [OPEN CALL] OPEN CALL for BENCHMARKS (challenges-maillists)

Tue Aug 22 10:14:13 UTC 2017

—————————————————————————————————————————————————————
Open Call for Benchmarks
—————————————————————————————————————————————————————
http://project-hobbit.us12.list-manage1.com/track/click?u=00d040a1bedee64c86d8911b0&id=5bb30a1736&e=ba2124fabf

This is an Open Call for adding benchmarks to HOBBIT project platform. HOBBIT aims at abolishing the barriers in the adoption and deployment of Big Linked Data by European companies, by means of open benchmarking reports that allow them to assess the fitness of existing solutions for their purposes. These benchmarks are based on data that reflects reality and measures industry-relevant Key Performance Indicators (KPIs) with comparable results using standardized hardware.

Inside the project, we have already developed several benchmarks  for Question Answering, Data Storage, Sensor Data etc.[1] The benchmarks have been used in open challenges on major Semantic Web conferences, for example: DEBS [2] , MOCHA [3]  and OKE [4] .

To drive the Semantic Web and Big Linked Data community towards transparent performance evaluation using standardized hardware (i.e. by running on HOBBIT platform), we ask you to add your benchmarks to the HOBBIT CKAN instance. Just follow the steps below:

1. Register on HOBBIT benchmark CKAN portal: http://project-hobbit.us12.list-manage.com/track/click?u=00d040a1bedee64c86d8911b0&id=0d05a3d138&e=ba2124fabf
2. Go to Benchmarks → Add Benchmark and fill out the necessary information:
    a. Description of the benchmark
    b. Tags
    c. License (if not listed, select Other (Open))
    d. Version: 1.0
    e. Author and email address [your email]
    f. Maintainer and email address (fill out even if it is the same as author)
3. Click Next: Add Data
    a. Add link to benchmark repository (necessary)
    b. Format HTML
4. Click Save & add another
    a. Add links to gold standard dataset, tasks, publications etc. (optional)
    b. Format CSV, RDF, HTML etc.
5. Click finish
6. Thanks! :)

Adding your benchmarks to HOBBIT CKAN will help with their adoption and dissemination

---------------------
[1] All the benchmarks are available on our CKAN: http://project-hobbit.us12.list-manage1.com/track/click?u=00d040a1bedee64c86d8911b0&id=ea50e6ea7b&e=ba2124fabf
[2] http://project-hobbit.us12.list-manage.com/track/click?u=00d040a1bedee64c86d8911b0&id=c1cd18946c&e=ba2124fabf
[3] http://project-hobbit.us12.list-manage.com/track/click?u=00d040a1bedee64c86d8911b0&id=2af1782f35&e=ba2124fabf
[4] http://project-hobbit.us12.list-manage.com/track/click?u=00d040a1bedee64c86d8911b0&id=a505152c6e&e=ba2124fabf

==============================================
Holistic Benchmarking of Big Linked Data - HOBBIT EU Project
https://project-hobbit.eu

Unsubscribe open-linguistics at lists.okfn.org from this list:
http://project-hobbit.us12.list-manage.com/unsubscribe?u=00d040a1bedee64c86d8911b0&id=1e33221579&e=ba2124fabf&c=88976eb468

Forward this email to a friend:
http://us12.forward-to-friend1.com/forward?u=00d040a1bedee64c86d8911b0&id=88976eb468&e=ba2124fabf

 Update your profile:
http://project-hobbit.us12.list-manage1.com/profile?u=00d040a1bedee64c86d8911b0&id=1e33221579&e=ba2124fabf