[open-linguistics] ANN: NLP Interchange Format (NIF) 1.0 Spec, Demo and Reference Implementation

Tue Nov 29 10:46:52 UTC 2011

Dear Emily,

(I'm answering for Sebastian, because we're working together on scaling up
NIF to a more generic formalism.)

The key difference is a difference of technologies employed: the standard
linearization of GrAF is by means of standoff XML (with known
disadvantages such as limited readability and the need to develop an
infrastructure [API, data bases, query language, etc.] from scratch),
whereas NIF uses RDF and OWL (not necessarily better readable, but with
infrastructures provided by the Semantic Web community). Also, both differ
in scope: The goal behing NIF was to develop NLP pipelines that make use
of RDF as interchange format, whereas GrAF is primarily concerned with
modeling linguistic corpora. Further, GrAF allows only to model a single
resource, RDF also allows to interlink it with other resources (e.g., for
PropBanking or parallel corpora). Both approaches augment each other, and
it is possible that they converge in the future to a certain extent -
maybe, this could be a topic to discuss at the LDL where we invited Nancy
Ide as keynote speaker.

On the more detailed level, NIF is a relatively pragmatic approach to
represent the output of several typical NLP tools, i.e., tokenizers,
lemmatizers, POS taggers, and constituent parsers, by means of RDF.
(Higher-level annotations, say, alignment, bridging, or discourse
structure, cannot be adequately represented by NIF at the moment.) As
compared to this, GrAF was developed as a means to represent any kind of
linguistic annotation, and in particular, annotated corpora. So, it comes
with a certain theoretical overhead. GrAF is based on a graph-theoretic
model and just uses labeled directed graphs as data model (without any
more specific constraints on the data model). Of course, RDF can also be
understood as being based on labeled directed graphs. In that sense, both
are building on similar conceptions of linguistic annotations as directed
(acyclic) (hyper-) graphs, as others did before (cf. Bird & Liberman 2001).

NIF comprises means to address strings (this roughly corresponds to the
segmentation in LAF, which is, however, not directly a part of GrAF [Ide &
Suderman 2007 describe the introduction of "dummy nodes" which do not seem
to be proper nodes in the GrAF graph]). Differently from GrAF, these
strings are addressed by URIs, so it allows references from other
resources directly into a corpus/document.

Further, NIF contains the String Ontology and the Structured Sentence
Ontology that provide data types for annotation, but from a very
surface-oriented perspective (Sentences, Phrases and Words as defined in
the Structured Sentence Ontology serve as units of annotation, all would
be specializations of a GrAF node; a String as defined in the String
Ontology, however, would be more general than a GrAF node, because a GrAF
node is - implicitly - required to be a meaningful unit of linguistic
annotation, whereas a string is just a sequence of characters.) One
important difference is that a GrAF node can be discontinuous, whereas a
NIF String is continuous.

An important practical difference is that NIF allows to handle structural
interoperability (comparable formats) for linguistic annotations, but also
conceptual interoperability (interpretable annotations), *within the same
formalism*. In GrAF, ISOcat can be used to define reference categories,
but at the moment, the integration of ISOcat in GrAF does not seem to be
solved. (Although there may have been recent developments that I am not
aware of.) In NIF, the OLiA ontologies can be used which provide an OWL
wrapper for ISOcat (and other terminology repositories). Same for metadata
(using lexvo/lingvoj).

So far, NIF covers only selected levels of annotation, for higher levels,
additional data types would have to be introduced, and these will be based
on the LAF, either directly on GrAF, or on its sibling format PAULA
(originating from early LAF sketches, Ide & Romary 2004). At least two
people (Nancy Ide and myself) wrote earlier on the list that they could
provide RDF representations of such generic formalisms, and this
information will be the input to carry the development of NIF further.

In the longer perspective, we plan to develop means to represent
linguistic annotations such that GrAF annotations can be rendered
appropriately in RDF and/or OWL, with NIF for NLP pipelines and POWLA for
linguistic corpora.

Best,
Christian

On Mon, 28 Nov 2011 21:59:51 +0100, Emily M. Bender <ebender at uw.edu> wrote:

> Dear Sebastian,
>
> How does NIF relate to/compare to GrAF?
>
> Emily
>
> On Sun, Nov 27, 2011 at 11:34 PM, Sebastian Hellmann
> <hellmann at informatik.uni-leipzig.de> wrote:
>> The Natural Language Processing Interchange Format (NIF) is an  
>> RDF/OWL-based
>> format that aims to achieve interoperability between Natural Language
>> Processing (NLP) tools, language resources and annotations. The core of  
>> NIF
>> consists of a vocabulary, which can represent Strings as RDF resources.  
>> A
>> special URI Design is used to pinpoint annotations to a part of a  
>> document.
>> These URIs can then be used to attach arbitrary annotations to the
>> respective character sequence. Employing these URIs, annotations can be
>> published on the Web as Linked Data and interchanged between different  
>> NLP
>> tools and applications.
>>
>> In order to simplify the combination of tools, improve their
>> interoperability and facilitating the use of Linked Data we developed  
>> the
>> NLP Interchange Format (NIF). NIF addresses the interoperability  
>> problem on
>> three layers: the structural, conceptual and access layer. NIF is based  
>> on a

>> Linked Data enabled URI scheme for identifying elements in (hyper-)  
>> texts
>> (structural layer) and a comprehensive ontology for describing common  
>> NLP
>> terms and concepts (conceptual layer). NIF-aware applications will  
>> produce
>> output (and possibly also consume input) adhering to the NIF ontology as
>> REST services (access layer). Other than more centralized solutions  
>> such as
>> UIMA and GATE, NIF enables the creation of heterogeneous, distributed  
>> and
>> loosely coupled NLP applications, which use the Web as an integration
>> platform. Another benefit is, that a NIF wrapper has to be only created  
>> once
>> for a particular tool, but enables the tool to interoperate with a
>> potentially large number of other
>> tools without additional adaptations. Ultimately, we envision an  
>> ecosystem
>> of NLP tools and services to emerge using NIF for exchanging and  
>> integrating
>> rich annotations.
>>
>> We designed NIF to be very light-weight and to reduce the amount of  
>> triples
>> to achieve better scalability. The following triples in N3 Syntax  
>> express
>> that the string “W3C” on http://www.w3.org/DesignIssues/LinkedData.html
>> (index 22849 to 22852) is linked to the DBpedia resource of
>> “World_Wide_Web_Consortium”:
>>
>> @prefix ld: <http://www.w3.org/DesignIssues/LinkedData.html#> .
>> @prefix str: <http://nlp2rdf.lod2.eu/schema/string/> .
>> @prefix dbo: <http://dbpedia.org/ontology/> .
>> @prefix scms: <http://ns.aksw.org/scms/> .
>> @prefix nerd: <http://nerd.eurecom.fr/ontology#> .
>> ld:offset_22849_22852_W3C str:anchorOf "W3C" .
>> ld:offset_22849_22852_W3C scms:means dbpedia:World_Wide_Web_Consortium .
>> ld:offset_22849_22852_W3C a dbo:Organisation , nerd:Organization .
>>
>> NIF already incorporates the Ontologies of Linguistic Annotation (OLiA,
>> http://nachhalt.sfb632.uni-potsdam.de/owl/) and the Named Entity  
>> Recognition
>> and Disambiguation (NERD, http://nerd.eurecom.fr/ontology/) ontology.  
>> Please
>> get in contact, if you know of further NLP ontologies, which we can  
>> reuse
>> and integrate in NIF.
>>
>> This release consists of the following items:
>> 1. The specification of NIF 1.0 ( http://nlp2rdf.org/nif-1-0 ) This  
>> document
>> will guide the further implementation of NIF-enabled services. An  
>> average
>> wrapper requires around 200-500 lines of code. The spec integrates  
>> several
>> domain ontologies (OLiA, NERD) and will be extended in the future to  
>> cover
>> more domains.
>> 2. A community portal ( http://nlp2rdf.org )
>> -- mailing list (nlp2rdf at lists.informatik.uni-leipzig.de ) -
>> http://lists.informatik.uni-leipzig.de/mailman/listinfo/nlp2rdf
>> -- Read how to get involved (http://nlp2rdf.org/get-involved )
>> 3. A reference implementations of NIF 1.0 in Java
>> -- Release 1.2 (
>> http://code.google.com/p/nlp2rdf/downloads/detail?name=nlp2rdf-1.2.tar.gz  
>> )
>> -- Source code ( http://code.google.com/p/nlp2rdf/ )
>> 4. Wrapper implementations for Stanford CoreNLP, SnowballStemmer,  
>> OpenNLP,
>> MontyLingua, DBpedia Spotlight, UIMA, Gate (for ANNIE and also generic
>> output), Mallet (alpha)
>> -- Demo GUI (with links to implementations):  
>> http://nlp2rdf.lod2.eu/demo.php
>> -- List of implementations: http://nlp2rdf.org/implementations
>> 5. Tutorials and Tutorial Challenges (
>> http://nlp2rdf.org/tutorials-challenge )
>> -- Tutorial: How to call a NIF web service with your favorite SemWeb  
>> library
>> -
>> http://nlp2rdf.org/tutorials/tutorial-how-to-call-a-nif-webservice-with-your-favorite-semweb-library
>> -- Tutorial Challenge: Semantic Search -
>> http://nlp2rdf.org/tutorial-challenges/tutorial-challenge-semantic-search/
>> -- Tutorial Challenge: Multilingual Part-Of-Speech Tagger -
>> http://nlp2rdf.org/tutorial-challenges/tutorial-challenge-multilingual-part-of-speech-tagger
>> -- Tutorial Challenge: Semantic Yellow Pages -
>> http://nlp2rdf.org/tutorial-challenges/tutorial-challenge-semantic-yellow-pages
>> 6. Slides - http://www.slideshare.net/kurzum/nif-version-10
>> 7. A technical report http://svn.aksw.org/papers/2012/WWW_NIF/public.pdf
>> including some evaluation.
>>
>> We would like to thank our colleagues from AKSW (http://aksw.org)  
>> research
>> group and the LOD2 (http://lod2.eu) project for their helpful comments  
>> and
>> inspiring discussions during the development of NIF. Especially, we  
>> would
>> like to thank Christian Chiarcos
>> (http://www.sfb632.uni-potsdam.de/~chiarcos/) for his support while  
>> using
>> OLiA, the members of the Working Group on Open Data in Linguistics
>> (http://linguistics.okfn.org/) and the students that participated in  
>> the NIF
>> field study: Markus Ackermann, Martin Brümmer, Didier Cherix, Marcus
>> Nitzschke, Robert Schulze.
>>
>> Regards,
>> Sebastian Hellmann, Jens Lehmann and Sören Auer