Repository navigation
Expand file tree
/
Copy pathapi.tex
More file actions
2745 lines (2395 loc) · 123 KB
/
Copy pathapi.tex
File metadata and controls
2745 lines (2395 loc) · 123 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% api.tex
%
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapt[chap:api]{GATE Embedded}
\markboth{GATE Embedded}{GATE Embedded}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\nnormalsize
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:embed]{Quick Start with GATE Embedded}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\mbox{ }
Embedding GATE-based language processing in other applications using GATE
Embedded (the GATE API) is straightforward:
\begin{flushleft}
\sloppy
\begin{itemize}
\item add the GATE libraries to your application's classpath.
\begin{itemize}
\item if you use a build tool with dependency management, such
as Maven or Gradle, add a dependency on the right version of
\texttt{uk.ac.gate:gate-core} -- this is the recommended way
to build against the GATE APIs.
\item if you can't use a dependency manager, you can instead add
all the JAR files from the \texttt{lib} directory of a GATE
installation to your compile classpath in your build tool.
\end{itemize}
\item initialise GATE with \texttt{gate.Gate.init();}
\item program to the framework API.
\end{itemize}
\fussy
\end{flushleft}
%
For example, this code will create the default ANNIE extraction system,
the same as the ``load ANNIE'' button in GATE Developer:
%
\begin{lstlisting}
// initialise the GATE library
Gate.init();
// load the ANNIE plugin
Plugin anniePlugin = new Plugin.Maven(
"uk.ac.gate.plugins", "annie", gate.Main.version);
Gate.getCreoleRegister().registerPlugin(anniePlugin);
// load ANNIE application from inside the plugin
SerialAnalyserController controller = (SerialAnalyserController)
PersistenceManager.loadObjectFromUrl(new ResourceReference(
anniePlugin, "resources/" + ANNIEConstants.DEFAULT_FILE)
.toURL());
\end{lstlisting}
%
If you want to use resources from any plugins, you need to
load the plugins before calling \verb|createResource|:
\begin{lstlisting}
Gate.init();
// need Tools plugin for the Morphological analyser
Gate.getCreoleRegister().registerPlugin(new Plugin.Maven(
"uk.ac.gate.plugins", "tools", gate.Main.version));
...
ProcessingResource morpher = (ProcessingResource)
Factory.createResource("gate.creole.morph.Morph");
\end{lstlisting}
%
Instead of creating your processing resources individually using the
\texttt{Factory}, you can create your application in GATE Developer, save it using the
`save application state' option (see
Section~\ref{sec:developer:savestate}), and then load the saved state from
your code. This will automatically reload any plugins that were
loaded when the state was saved, you do not need to load them
manually.
\begin{lstlisting}
Gate.init();
CorpusController controller = (CorpusController)
PersistenceManager.loadObjectFromFile(new File("savedState.xgapp"));
\end{lstlisting}
There are many examples of using GATE Embedded available at:\\
\htlinkplain{http://gate.ac.uk/wiki/code-repository/}.
See Section~\ref{sec:gettingstarted:sysprop} for details of the system
properties GATE uses to find its configuration files.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:factory]{Resource Management in GATE Embedded}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% $ID$
As outlined earlier, GATE defines three different types of resources:
\begin{description}
\item[Language Resources]{: (\textbf{LRs}) entities that hold linguistic data.
}
\item[Processing Resources]{: (\textbf{PRs}) entities that process data.
}
\item[Visual Resources]{: (\textbf{VRs}) components used for building
graphical interfaces. }
\end{description}
These resources are collectively named \textbf{CREOLE}\footnote{CREOLE
stands for Collection of REusable Objects for Language Engineering} resources.
All CREOLE resources have some associated meta-data in the form of annotations
on the resource class and some of its methods. The most important role of that
meta-data is to specify the set of parameters that a resource understands, which
of them are required and which not, if they have default values and what those
are. See Section \ref{sec:creole-model:config} for full details of the
configuration mechanism.
All resource types have creation-time parameters that are used during the
initialisation phase. Processing Resources also have run-time parameters
that get used during execution (see Section \ref{sec:api:pr} for more
details).
\textbf{Controllers} are used to define GATE applications and have the role of
controlling the execution flow (see Section \ref{sec:api:controllers} for
more details).
This section describes how to create and delete CREOLE resources as
objects in a running Java virtual machine. This process involves using
GATE's Factory class\footnote{Fully qualified
name: \tt{gate.Factory}}, and, in the case of LRs, may also involve
using a DataStore.
CREOLE resources are Java Beans; creation of a resource object
involves using a default constructor, then setting parameters on the
bean, then calling an {\tt init()} method. The Factory takes care of
all this, makes sure that the GATE Developer GUI is told about what is
happening (when GUI components exist at runtime), and also takes care
of restoring LRs from DataStores. \textbf{A programmer using GATE
Embedded should never call the constructor of a resource: always use
the Factory!}
Creating a resource involves providing the following information:
\begin{itemize}
\item \textbf{fully qualified class name} for the resource. This is the only
{\bf required} value. For all the rest, defaults will be used if actual values
are not provided.
\item values for the \textbf{creation time parameters}.$^\dagger\ $
\item initial values for \textbf{resource features}.$^\dagger\ $ For an
explanation on features see Section \ref{sec:api:features}.
\item a \textbf{name} for the new resource;
\end{itemize}
$^\dagger\ $ Parameters and features need to be provided in the form of a
GATE Feature Map which is essentially a java Map ({\tt java.util.Map})
implementation, see Section \ref{sec:api:features} for more details
on Feature Maps.
%\begin{minipage}{\textwidth}
%Here is an example that creates a GATE document:
%
%
%\begin{small}\begin{verbatim}
%URL u = new URL("http://gate.ac.uk/");
%FeatureMap params = Factory.newFeatureMap();
%params.put("sourceUrl", u);
%FeatureMap features = Factory.newFeatureMap();
%
%Document doc = (Document)
% Factory.createResource("gate.corpora.DocumentImpl",
% params, features, "GATE Homepage");
%\end{verbatim}\end{small}
%
%\end{minipage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\subsect[sec:howto:instant]{Instantiating CREOLE Resources}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%This section describes how to create CREOLE resources as objects in a running
%Java virtual machine. This process involves using GATE's {\tt Factory} class,
%and, in the case of LRs, may also involve using a {\tt DataStore}.
%CREOLE resources are Java Beans; creation of a resource object
%involves using a default constructor, then setting parameters on the
%bean, then calling an {\tt init()} method\footnote{This method is not
%part of the beans spec.}. The Factory takes care of all this, makes
%sure that GATE Developer is told about what is happening (when GUI
%components exist at runtime), and also takes care of restoring LRs
%from DataStores. So a programmer using GATE Embedded should {\bf never
%call the constructor} of a resource: always use the Factory.
%The valid parameters for a resource are described in the resource's section
%of its {\tt creole.xml} file or in Java annotations on the resource class --
%see section \ref{sec:creole-model:config}.
Creating a resource via the Factory involves passing values for any
create-time parameters that require setting to the Factory's {\tt
createResource} method. If no parameters are passed, the defaults are used.
So, for example, the following code creates a default ANNIE part-of-speech
tagger:
%
\begin{lstlisting}
Gate.getCreoleRegister().registerPlugin(new Plugin.Maven(
"uk.ac.gate.plugins", "annie", gate.Main.version));
FeatureMap params = Factory.newFeatureMap(); //empty map:default params
ProcessingResource tagger = (ProcessingResource)
Factory.createResource("gate.creole.POSTagger", params);
\end{lstlisting}
%
Note that if the resource created here had any parameters that were both
mandatory and had no default value, the {\tt createResource} call would throw
an exception. In the case of the POS tagger, all the required parameters have
default values so no \texttt{params} need to be passed in.
When creating a Document, however, the URL of the source for the document must be
provided\footnote{Alternatively a string giving the document source may be
provided.}. For example:
\begin{lstlisting}
URL u = new URL("https://gate.ac.uk/");
FeatureMap params = Factory.newFeatureMap();
params.put("sourceUrl", u);
Document doc = (Document)
Factory.createResource("gate.corpora.DocumentImpl", params);
\end{lstlisting}
%
Note that the document created here is transient: when you quit the JVM the
document will no longer exist. If you want the document to be persistent, you need to
store it in a {\tt DataStore} (see Section~\ref{sec:api:corpora}).
Apart from {\tt createResource()} methods with different signatures, {\tt
Factory} also provides some shortcuts for common operations, listed in table
\ref{table:factory-op}.
\begin{table}[htbp]
\begin{small}
\begin{center}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt {\bf newFeatureMap}()} & Creates a new Feature Map (as used in the
example above). \\
\hline
{\tt {\bf newDocument}(String content)} & Creates a new GATE Document starting
from a String value that will be used to generate the document content.\\
\hline
{\tt {\bf newDocument}(URL sourceUrl)} & Creates a new GATE Document
using the text pointed by an URL to generate the document content.\\
\hline
{\tt {\bf newDocument}(URL sourceUrl, String encoding)} & Same as
above but allows the specification of an encoding to be used while
downloading the document content.\\
\hline
{\tt {\bf newCorpus}(String name)} & creates a new GATE Corpus with a specified
name.\\
\hline
\end{tabular}
\caption{Factory Operations}
\label{table:factory-op}
\end{center}
\end{small}
\end{table}
GATE maintains various data structures that allow the retrieval of loaded
resources. When a resource is no longer required, it needs to be removed from
those structures in order to remove all references to it, thus making it a
candidate for garbage collection. This is achieved using the {\tt
deleteResource(Resource res)} method on {\tt Factory}.
Simply removing all references to a resource from the user code will
{\bf NOT} be enough to make the resource collect-able. Not calling
{\tt Factory.deleteResource()} {\bf will} lead to memory leaks!
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:plugins]{Using CREOLE Plugins}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%When using GATE Embedded, the following API calls are relevant to working
%with plugins:
As shown in the examples above, in order to use a CREOLE resource the relevant
CREOLE plugin must be loaded. Processing Resources, Visual Resources and
Language Resources other than Document, Corpus and DataStore all require that the
appropriate plugin is first loaded. When using Document, Corpus or DataStore, you
do not need to first load a plugin. The following API calls listed in table
\ref{table:creole-calls} are relevant to working with CREOLE plugins.
\begin{table}[htbp]
\begin{small}
\begin{center}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\multicolumn{2}{|c|}{\textbf{Class \underline{gate.Gate}}}\\
\hline
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt public static void {\bf addKnownPlugin}(Plugin plugin)} & adds the plugin
to the list of known plugins.\\
\hline
{\tt public static void {\bf removeKnownPlugin}(Plugin plugin)} & tells the
system to `forget' about one previously known directory. If the specified plugin
was loaded, it will be unloaded as well - i.e. all the metadata relating to
resources defined by this plugin will be removed from memory.\\
\hline
{\tt public static void {\bf addAutoloadPlugin}(Plugin plugin)} & adds a new
plugin to the list of plugins that are loaded automatically at start-up.\\
\hline
{\tt public static void {\bf removeAutoloadPlugin}(Plugin plugin)} & tells the
system to remove a plugin from the list of plugins that are loaded
automatically at system start-up. This will be reflected in the user's
configuration data file.\\
\hline
\multicolumn{2}{|c|}{\textbf{Class \underline{gate.CreoleRegister}}}\\
\hline
\hline
{\tt public void {\bf registerPlugin}(Plugin plugin)} & loads a new
CREOLE plugin. The new plugin is added to the list of known plugins if not already
there.\\
\hline
{\tt public void {\bf unregisterPlugin}(Plugin plugin)} & unloads a loaded CREOLE
plugin.\\
\hline
\end{tabular}
\caption{Calls Relevant to CREOLE Plugins}
\label{table:creole-calls}
\end{center}
\end{small}
\end{table}
There are several different subclasses of \texttt{Plugin} that can be passed
to these methods. The most common one is \texttt{Plugin.Maven}, as seen in the
examples above, which is a plugin that is a single JAR file specified via its
\texttt{group:artifact:version} ``coordinates'', and which is downloaded from a
Maven repository at runtime by GATE the first time the plugin is loaded. The
vast majority of standard GATE plugins are of this type. To load version 8.5
of the ANNIE plugin, for example, you would use:
\begin{lstlisting}
Gate.getCreoleRegister().registerPlugin(new Plugin.Maven(
"uk.ac.gate.plugins", "annie", "8.5"));
\end{lstlisting}
By default GATE looks in the Central Repository and in the GATE repository
(\verb!http://repo.gate.ac.uk/content/groups/public/!, where we deploy snapshot
builds of the standard plugins), plus any repositories declared in active
profiles in the normal Maven \verb!settings.xml! file. Mirror and proxy
settings from this file are also respected.
In addition to Maven plugins, GATE still supports the style of plugins used in
GATE version 8.4.1 and earlier where the plugin is a directory on disk
which contains a \texttt{creole.xml} configuration file and optionally one or
more JAR files containing the compiled classes of the plugin's CREOLE
resources. These plugins are represented by the class
\texttt{Plugin.Directory}, with a \texttt{URL} pointing to the directory that
contains the \texttt{creole.xml} file:
\begin{lstlisting}
Gate.getCreoleRegister().registerPlugin(new Plugin.Directory(
new URL("file:/home/example/my-plugins/FishCounter/"));
\end{lstlisting}
Finally, if you are writing a GATE Embedded application and have a single
resource class that will only be used from your embedded code (and so does not
need to be distributed as a complete plugin), and all the configuration for
that resource is provided as Java annotations on the class, then it is possible
to register the class as a special type of \texttt{Plugin} called a
``component'':
\begin{lstlisting}
Gate.getCreoleRegister().registerPlugin(new Plugin.Component(
MySpecialPurposePR.class));
\end{lstlisting}
Note that components cannot be registered this way in the developer GUI, and
cannot be included in saved application states (see
section~\ref{sec:api:persistent} below).
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:lr]{Language Resources}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
This section describes the
implementation of documents and corpora in GATE.
\subsect{GATE Documents}
Documents are modelled as content plus annotations (see Section
\ref{sec:api:annotations}) plus features (see Section
\ref{sec:api:features}).
The content of a document can be any implementation of the \linebreak\ {\tt
gate.DocumentContent} interface; the features are \verb!<!attribute,
value\verb!>! pairs stored a Feature Map. Attributes are String values while
the values can be any Java object.
The annotations are grouped in sets (see section \ref{sec:api:ann-sets}). A
document has a default (anonymous) annotations set and any number of named
annotations sets.
Documents are defined by the {\tt gate.Document} interface and there is also a
provided implementation:
\begin{description}
\item[{\tt \javadoclink{gate/corpora/DocumentImpl.html}{gate.corpora.DocumentImpl}}]: transient document. Can be stored
persistently through Java serialisation.
%
\end{description}
Main Document functions are presented in table \ref{table:document}.
\begin{table}[htbp]
\begin{small}
\begin{center}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\multicolumn{2}{|c|}{\textbf{Content Manipulation}}\\
\hline
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt DocumentContent {\bf getContent}()} & Gets the Document content. \\
\hline
{\tt void {\bf edit}(Long start, Long end, DocumentContent replacement)} &
Modifies the Document content.\\
\hline
{\tt void {\bf setContent}(DocumentContent newContent)} & Replaces the entire
content.\\
\hline
\multicolumn{2}{|c|}{\textbf{Annotations Manipulation}}\\
\hline
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt public AnnotationSet {\bf getAnnotations}()} & Returns the default annotation
set.\\
\hline
{\tt public AnnotationSet {\bf getAnnotations}(String name)} & Returns a {\em
named} annotation set.\\
\hline
{\tt public Map {\bf getNamedAnnotationSets}()} & Returns {\em all} the named
annotation sets.\\
\hline
{\tt void {\bf removeAnnotationSet}(String name)} & Removes a named
annotation set.\\
\hline
\multicolumn{2}{|c|}{\textbf{Input Output}}\\
\hline
\hline
{\tt String {\bf toXml}()} & Serialises the Document in XML format.\\
\hline
{\tt String {\bf toXml}(Set aSourceAnnotationSet, boolean includeFeatures)}
& Generates XML from a set of annotations only, trying to preserve the
original format of the file used to create the document.\\
\hline
\end{tabular}
\caption{{\tt gate.Document} methods.}
\label{table:document}
\end{center}
\end{small}
\end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsect[sec:api:features]{Feature Maps}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
All CREOLE resources as well as the {\em Controllers} and the annotations
can have attached meta-data in the form of {\em Feature Maps}.
A Feature Map is a Java Map (i.e. it implements the {\tt java.util.Map}
interface) and holds \verb!<!attribute-name, attribute-value\verb!>! pairs.
The attribute names are Strings while the values can be any Java Objects.
The use of non-{\em Serialisable} objects as values is strongly discouraged.
Feature Maps are created using the {\tt gate.Factory.newFeatureMap()}
method.
The actual implementation for FeatureMaps is provided by the \linebreak\ {\tt
gate.util.SimpleFeatureMapImpl} class.
Objects that have features in GATE implement the {\tt
gate.util.FeatureBearer} interface which has only the two accessor methods
for the object features: {\tt FeatureMap getFeatures()} and {\tt void
setFeatures(FeatureMap features)}.
\begin{minipage}{\textwidth}
{\b Getting a particular feature from an object}
\begin{lstlisting}
Object obj;
String featureName = "length";
if(obj instanceof FeatureBearer){
FeatureMap features = ((FeatureBearer)obj).getFeatures();
Object value = (features == null) ? null :
features.get(featureName);
}
\end{lstlisting}
\end{minipage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsect[sec:api:ann-sets]{Annotation Sets}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
A GATE document can have one or more annotation layers --- an anonymous one,
(also called {\em default}), and as many {\em named} ones as necessary.
An annotation layer is organised as a \emph{Directed Acyclic Graph (DAG)} on
which the nodes are particular locations ---\emph{anchors}--- in the document content
and the arcs are made out of annotations reaching from the location indicated
by the start node to the one pointed by the end node (see Figure
\ref{fig:ann-graph} for an illustration). Because of the \emph{graph} metaphor,
the annotation layers are also called \emph{annotation graphs}.
In terms of Java objects, the annotation layers are represented using
the \emph{Set} paradigm as defined by the collections library and they are hence
named \emph{annotation sets}. The terms of annotation \emph{layer}, \emph{graph}
and \emph{set} are interchangeable and refer to the same concept when used in
this \thing{}.
\begin{figure}[htbp]
\begin{center}
\includegraphics[scale=0.5]{annotationGraph.png}
\caption{The Annotation Graph model.}
\label{fig:ann-graph}
\end{center}
\end{figure}
An annotation set holds a number of annotations and maintains a series of
indices in order to provide fast access to the contained annotations.
The GATE Annotation Sets are defined by the {\tt gate.AnnotationSet}
interface and there is a default implementation provided:
\begin{description}
%
\item[{\tt \javadoclink{gate/annotation/AnnotationSetImpl.html}{gate.annotation.AnnotationSetImpl}}]
annotation set implementation used by transient documents.
\end{description}
The annotation sets are created by the document as required. The first time
a particular annotation set is requested from a document it will be
transparently created if it doesn't exist.
Tables \ref{table:annSet1} and \ref{table:annSet2} list the most used
Annotation Set functions.
\begin{table}[htbp]
\begin{small}
\begin{center}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\multicolumn{2}{|c|}{\textbf{Annotations Manipulation}}\\
\hline
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt Integer {\bf add}(Long start, Long end, String type, FeatureMap features)}
& Creates a new annotation between two offsets, adds it to this set and returns its id.\\
\hline
{\tt Integer {\bf add}(Node start, Node end, String type, FeatureMap
features)} & Creates a new annotation between two nodes, adds it to this
set and returns its id.\\
\hline
{\tt boolean {\bf remove}(Object o)} & Removes an annotation from this
set.\\
\hline
\multicolumn{2}{|c|}{\textbf{Nodes}}\\
\hline
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt Node {\bf firstNode}()} & Gets the node with the smallest offset.\\
\hline
{\tt Node {\bf lastNode}()} & Gets the node with the largest offset.\\
\hline
{\tt Node {\bf nextNode}(Node node)} & Get the first node that is relevant
for this annotation set and which has the offset larger than the one of the
node provided.\\
\hline
\multicolumn{2}{|c|}{\textbf{Set implementation}}\\
\hline
\hline
{\tt Iterator {\bf iterator}() } & \\
\hline
{\tt int {\bf size}()} & \\
\hline
\end{tabular}
\caption{{\tt gate.AnnotationSet} methods (general purpose).}
\label{table:annSet1}
\end{center}
\end{small}
\end{table}
\begin{table}[htbp]
\begin{center}
\begin{small}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\multicolumn{2}{|c|}{\textbf{Searching}}\\
\hline
\hline
{\tt AnnotationSet {\bf get}(Long offset)} & Select annotations by offset.
This returns the set of annotations whose start node is the least such that
it is greater than or equal to offset. If a positional index doesn't exist it
is created. If there are no nodes at or beyond the offset parameter then it
will return null.\\
\hline
{\tt AnnotationSet {\bf get}(Long startOffset, Long endOffset)} &
Select annotations by offset. This returns the set of annotations that
overlap totally or partially with the interval defined by the two provided
offsets. The result will include all the annotations that either:
\begin{itemize}
\item start before the start offset and end strictly after it
\item start at a position between the start and the end offsets
\end{itemize}\\
\hline
{\tt AnnotationSet {\bf get}(String type)} & Returns all annotations of the
specified type.\\
\hline
{\tt AnnotationSet {\bf get}(Set types)} & Returns all annotations of the
specified types.\\
\hline
{\tt AnnotationSet {\bf get}(String type, FeatureMap constraints)} & Selects
annotations by type and features.\\
\hline
{\tt Set {\bf getAllTypes}()} & Gets a set of java.lang.String objects
representing all the annotation types present in this annotation set.\\
\hline
{\tt AnnotationSet {\bf getContained}(Long startOffset, Long endOffset)} &
Select annotations contained within an interval, i.e.\\
\hline
{\tt AnnotationSet {\bf getCovering}(String neededType, Long startOffset,
Long endOffset)} &
Select annotations of the given type that completely span the range.\\
\hline
\end{tabular}
\caption{{\tt gate.AnnotationSet} methods (searching).}
\label{table:annSet2}
\end{small}
\end{center}
\end{table}
\begin{minipage}{\textwidth}
{\b Iterating from left to right over all annotations of a given
type}
\
\begin{lstlisting}
AnnotationSet annSet = ...;
String type = "Person";
//Get all person annotations
AnnotationSet persSet = annSet.get(type);
//Sort the annotations
List persList = new ArrayList(persSet);
Collections.sort(persList, new gate.util.OffsetComparator());
//Iterate
Iterator persIter = persList.iterator();
while(persIter.hasNext()){
...
}
\end{lstlisting}
\end{minipage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsect[sec:api:annotations]{Annotations}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
An \textbf{annotation} is a form of meta-data attached to a particular
section of document content. The connection between the annotation and the
content it refers to is made by means of two pointers that represent the
start and end locations of the covered content. An annotation must also have
a type (or a name) which is used to create classes of similar annotations,
usually linked together by their semantics.
An Annotation is defined by:
\begin{description}
\item[start node] a location in the document content defined by an offset.
\item[end node] a location in the document content defined by an offset.
\item[type] a String value.
\item[features] (see Section~\ref{sec:api:features}).
\item[ID] an Integer value. All annotations IDs are unique inside an
annotation set.
\end{description}
In GATE Embedded, annotations are defined by the {\tt gate.Annotation}
interface and implemented by the {\tt gate.annotation.AnnotationImpl}
class. Annotations exist only as members of annotation sets (see
Section~\ref{sec:api:ann-sets}) and they should not be directly created
by means of a constructor. Their creation should always be delegated
to the containing annotation set.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsect[sec:api:corpora]{GATE Corpora}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
A corpus in GATE is a Java List (i.e. an implementation of {\tt
java.util.List}) of documents. GATE corpora are defined by the {\tt
gate.Corpus} interface and the following implementations are available:
\begin{description}
\item[{\tt gate.corpora.CorpusImpl}] used for transient corpora.
\item[{\tt gate.corpora.SerialCorpusImpl}] used for persistent corpora that
are stored in a serial datastore (i.e. as a directory in a file system).
\end{description}
Apart from implementation for the standard List methods, a Corpus also
implements the methods in table \ref{table:corpus}.
\begin{table}[htbp]
\begin{small}
\begin{center}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt String {\bf getDocumentName}(int index)} & Gets the name of a document
in this corpus.\\
\hline
{\tt List {\bf getDocumentNames}()} & Gets the names of all the documents
in this corpus.\\
\hline
{\tt void {\bf populate}(URL directory, FileFilter filter, String encoding,
boolean recurseDirectories)} & Fills this corpus with documents created on
the fly from selected files in a directory. Uses a {\tt FileFilter} to
select which files will be used and which will be ignored. A simple file
filter based on extensions is provided in the Gate distribution ({\tt
gate.util.ExtensionFileFilter}).\\
{\tt void {\bf populate}(URL singleConcatenatedFile, String documentRootElement,
String encoding, int numberOfDocumentsToExtract, String documentNamePrefix,
DocType documentType)} & Fills the provided corpus with documents extracted from
the provided single concatenated file. Uses the content between the start and
end of the element as specified by {\tt documentRootElement} for each document.
The parameter {\tt documentType} specifies if the resulting files are html, xml
or of any other type. User can also restrict the number of documents to extract
by providing the relevant value for {\tt numberOfDocumentsToExtract} parameter.\\
\hline
\end{tabular}
\caption{{\tt gate.Corpus} methods.}
\label{table:corpus}
\end{center}
\end{small}
\end{table}
\begin{minipage}{\textwidth}
{\bf Creating a corpus from all XML files in a directory}
\begin{lstlisting}
Corpus corpus = Factory.newCorpus("My XML Files");
File directory = ...;
ExtensionFileFilter filter = new ExtensionFileFilter("XML files", "xml");
URL url = directory.toURL();
corpus.populate(url, filter, null, false);
\end{lstlisting}
\end{minipage}
{\bf Using a DataStore}
Assuming that you have a {\tt DataStore} already open called {\tt myDataStore},
this code will ask the datastore to take over persistence of your document, and
to synchronise the memory representation of the document with the disk storage:
\begin{small}\begin{verbatim}
Document persistentDoc = myDataStore.adopt(doc, mySecurity);
myDataStore.sync(persistentDoc);
\end{verbatim}\end{small}
%{\bf Security:}\\
%User access to the LRs is provided by a security mechanism of
%users and groups, similar to those on an operating system. When
%users create/save LRs into Oracle, they specify reading and
%writing access rights for users from their group and other users.
%For example, LRs created by one user/group can be made read-only
%to others, so they can use the data, but not modify it.
%The access modes are:
%\begin{itemize}
%\item
%others: read/none;
%\item
%group: modify/read/none;
%\item
%owner: modify/read.
%\end{itemize}
%If needed, ownership can be transferred from one user to another. Users,
%groups and LR permissions are
%administered in a special administration tool, by a privileged
%user. For more details see \Chapthing\ \ref{chap:security}.
When you want to restore a document (or other LR) from a datastore, you make
the same {\tt createResource} call to the Factory as for the creation of a
transient resource, but this time you tell it the datastore the resource
came from, and the ID of the resource in that datastore:
\begin{lstlisting}
URL u = ....; // URL of a serial datastore directory
SerialDataStore sds = new SerialDataStore(u.toString());
sds.open();
// getLrIds returns a list of LR Ids, so we get the first one
Object lrId = sds.getLrIds("gate.corpora.DocumentImpl").get(0);
// we need to tell the factory about the LR's ID in the data
// store, and about which datastore it is in - we do this
// via a feature map:
FeatureMap features = Factory.newFeatureMap();
features.put(DataStore.LR_ID_FEATURE_NAME, lrId);
features.put(DataStore.DATASTORE_FEATURE_NAME, sds);
// read the document back
Document doc = (Document)
Factory.createResource("gate.corpora.DocumentImpl", features);
\end{lstlisting}
%See the example code at \htlinkplain{http://gate.ac.uk/wiki/code-repository/}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:pr]{Processing Resources}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Processing Resources ({\bf PRs}) represent entities that are primarily
algorithmic, such as parsers, generators or ngram modellers.
They are created using the GATE Factory in manner similar the
Language Resources. Besides the creation-time parameters they also
have a set of run-time parameters that are set by the system just
before executing them.
Analysers are a particular type of processing resources in the sense that
they always have a {\bf {\tt document}} and a {\bf {\tt corpus}} among their
run-time parameters.
The most used methods for Processing Resources are presented in table
\ref{table:pr}
\begin{table}[htbp]
\begin{small}
\begin{center}
\begin{tabular}{|p{.4\textwidth}|p{.4\textwidth}|}
\hline
\textbf{Method} & \textbf{Purpose}\\
\hline
{\tt void {\bf setParameterValue}(String paramaterName, Object
parameterValue)} & Sets the value for a specified parameter. {\small method
inherited from {\tt gate.Resource}}\\
\hline
{\tt void {\bf setParameterValues}(FeatureMap parameters)} & Sets the
values for more parameters in one step. {\small method inherited from {\tt
gate.Resource}}\\
\hline
{\tt Object {\bf getParameterValue}(String paramaterName)} & Gets the value
of a named parameter of this resource. {\small method inherited from {\tt
gate.Resource}}\\
\hline
{\tt Resource {\bf init}()} & Initialise this resource, and return it.
{\small method inherited from {\tt gate.Resource}}\\
\hline
{\tt void {\bf reInit}()} & Reinitialises the processing resource. After
calling this method the resource should be in the state it is after calling
init. If the resource depends on external resources (such as rules files)
then the resource will re-read those resources. If the data used to create
the resource has changed since the resource has been created then the
resource will change too after calling reInit().\\
\hline
{\tt void {\bf execute}()} & Starts the execution of this Processing
Resource.\\
\hline
{\tt void {\bf interrupt}()} & Notifies this PR that it should stop its
execution as soon as possible.\\
\hline
{\tt boolean {\bf isInterrupted}()} & Checks whether this PR has been
interrupted since the last time its Executable.execute() method was
called.\\
\hline
\end{tabular}
\caption{{\tt gate.ProcessingResource} methods.}
\label{table:pr}
\end{center}
\end{small}
\end{table}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:controllers]{Controllers}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Controllers are used to create GATE applications. A Controller handles a set of
Processing Resources and can execute them following a particular strategy. GATE
provides a series of serial controllers (i.e. controllers that run their PRs in
sequence):
\begin{description}
\item[{\bf {\tt gate.creole.SerialController}:}] a serial controller that takes any
kind of PRs.
\item[{\bf {\tt gate.creole.SerialAnalyserController}:}] a serial controller that
only accepts Language Analysers as member PRs.
\item[{\bf {\tt gate.creole.ConditionalSerialController}:}] a serial controller
that accepts all types of PRs and that allows the inclusion or exclusion of
member PRs from the execution chain according to certain run-time conditions
(currently features on the document being processed are used).
\item[{\bf {\tt gate.creole.ConditionalSerialAnalyserController}:}] a serial
controller that only accepts Language Analysers and that allows the conditional
run of member PRs.
\item[{\bf {\tt gate.creole.RealtimeCorpusController}:}] a
{\tt SerialAnalyserController} that allows you to specify {\em graceful} and {\em timeout}
parameters (times in milliseconds). If processing for a document takes longer than the
amount of time specified for {\em graceful}, then the
controller will attempt to gracefully end it by sending an interrupt
request to it. If the {\em graceful} parameter is `-1' then no attempt to
gracefully end it is made. If processing takes longer than the amount of time
specified for the {\em timeout} parameter, it
will be forcibly terminated and the controller will move on to the next
document.
The parameter {\em suppressExceptions} controls if time-outs and other
exceptions will be suppressed or passed on to the caller: if this parameter
is set to `true', then any exception or a timeout will
simply cause the controller to move on to the next document rather than failing
the entire corpus processing. If the parameter is set to `false' both
time-outs and exceptions will be passed on as exceptions to the caller.
\end{description}
Additionally there is a {\em scriptable controller} provided by the Groovy
plugin. See section~\ref{sec:api:groovy:controller} for details.
%\begin{minipage}{\textwidth}
{\bf Creating an ANNIE application and running it over a corpus}
\begin{lstlisting}
// load the ANNIE plugin
Plugin anniePlugin = new Plugin.Maven(
"uk.ac.gate.plugins", "annie", gate.Main.version);
Gate.getCreoleRegister().registerPlugin(anniePlugin);
// create a serial analyser controller to run ANNIE with
SerialAnalyserController annieController =
(SerialAnalyserController) Factory.createResource(
"gate.creole.SerialAnalyserController",
Factory.newFeatureMap(),
Factory.newFeatureMap(), "ANNIE");
// load each PR as defined in ANNIEConstants
// Note this code is for demonstration purposes only,
// in practice if you want to load the ANNIE app you
// should use the PersistenceManager as shown at the
// start of this chapter
for(int i = 0; i < ANNIEConstants.PR_NAMES.length; i++) {
// use default parameters
FeatureMap params = Factory.newFeatureMap();
ProcessingResource pr = (ProcessingResource)
Factory.createResource(ANNIEConstants.PR_NAMES[i],
params);
// add the PR to the pipeline controller
annieController.add(pr);
} // for each ANNIE PR
// Tell ANNIE's controller about the corpus you want to run on
Corpus corpus = ...;
annieController.setCorpus(corpus);
// Run ANNIE
annieController.execute();
\end{lstlisting}
%\end{minipage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\sect[sec:api:relations]{Modelling Relations between Annotations}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Most text processing tasks in GATE model metadata associated with text snippets
as annotations. In some cases, however, it is useful to to have another layer of
metadata, associated with the annotations themselves. One such case is the
modelling of relations between annotations. One typical example of relations
between annotation is that of co-reference. Two annotations of type {\tt Person}
may be referring to the same actual person; in this case the two annotations are
said to be co-referring.
Starting with version $7.1$, GATE Embedded supports the representation of
relations between annotations. A relation set is associated with, and accssed via,
an annotation set. All members of a relation must be either annotations from the
associated annotation set or other relations within the same set.
The classes supporting relations can be found in the \lstinline!gate.relations!
package.
A relation, as described by the \lstinline!gate.relations.Relation! interface,
is defined by the following values:
\begin{description}