Chapter 8
CASE STUDY AND EVALUATION
Case studies give the concrete proof of realizing the hypothesis which had initiated
the research work. They help to illustrate the application of the solutions that have
been proposed for realizing the various aspects of the hypothesis.
This chapter describes two case studies illustrating the application of the models that
have been progressively developed for reallzing agents with language ability. The first
case study exactly maps the proposed models onto a MULtilingual natural Language
Agent Interface for mail service (MULLAI), where the language ability is attributed
for a single agent. The second case study describes how the proposed architecture has
been extended to adapt the same to realize the language ability of a multi-agent
system (Multil~ngual Multi-agent System (MMAS). Though, the language ability of
multi-agent system IS not within the scope o f t h ~ s work, yet it has been described here
in order to illustrate that the model applies for an MAS also with small extensions. In
addition, evaluation of an agent with two-dimensional autonomy and the existing
multilingual d~alogue agents has also been carried out.
The objectives of this chapter are to
Illustrate the application of the proposed solutions in realizing the language
faculty for a
- Single Agent - MULLAI
- Multi-Agent System - MMAS.
s Evaluate the agent with two-dimensional language autonomy and the existing
multilingual dialoguing agents.
8.1 MULLAI -Multilingual Natural Language &gent Lnterface for Mail Service
Providing a natural language interface to the mail service operations would help to
bridge the language barrier that arises when mail services are to be used by people
who are familiar in their natural language only. A conventional form-based GUI
means of interaction may not be convenient for these users as they would prefer
interacting with the system in the same way as they interact with people. Hence, a
collaborative form of natural language interface that interacts with the user in a
comprehensive manner so as to infer his intentions from his abstract task specification
and carry out the task is required. This interface may have to support many languages,
especially when it has to be used in language diversified country like India. The
prospect of having to augment the language ability to new languages is also high.
These requirements instigate the need for a multilingual natural language agent
interface for mail service (MULLAI). These requirements have also been the subject
ofthe thesis for which suitable solutions have been provided in the earlier chapters.
Hence, these solutions have been used to realize MULLAI. Thereby, it contributes as
a test bed for analyzing the appropriateness of the proposed solutions towards
realizing MULLAI. The natural language interface has been provided for mail service
operations like
List
Read
Delete
Compose
Reply
Save.
Multilingualism has been realized using two languages - English and Tamil. Of these
languages, the system was initially built with support for English language only. The
Tamil language has been acquired and implemented in the system. The Surgemail has
been used as the back end mail server. In the subsequent discussion, how the various
solutions described in the previous chapters have been used in realizing the agent is
described. Especially, the case-study would explicit how the management functions
are accomplished with respect to their manager and behavior roles. In MULLAI, the
manager role gains predominance in certain management functions and in others the
behavior roles gain predominance. For example, the manager role is of significant
importance in the case of the knowledge management and the behavior role is of
importance in new behavior acquisition management. Dynamic role binding to the
required language behavior is of importance to the Interface and Function
Management. Hence, the description of the case-study is done by considering the
management functions with respect to their predominant role.
8.1.1 Manager Role - Knowledge Behavior Management
Here, the various responsibilities of the Manager role are considered and how these
are fulfilled in the above management functions are described. The sample screen
shots help to illustrate the accomplishment of the same in MULLAI.
The varlous responsibilities of the manager role are.
Communicating with other management functlons
Maintaming Task Context
Assume the requ~red behav~or role
Ma~ntaining Behavior References
Perform Task Level Inference
Implement newly acqu~red behaviors.
Communicat~ng wlth other management functions happens using send() and receive()
functlons as illustrated in the design and implementation model in the previous
chapter.
The task context of Knowledge Behav~or Management is constituted of the following
beliefs as described in the BTB paradigm:
(Bel TCk, language knowledge (Ik, DO)) +(Be1 TCk, language resources(lk ))
A (Bel TCA, , vocabulary(l~. DO))
"(Bel TCi, grammar rules(ii, DO))
A (Bel TCk,, knowledgeserv~ces(Ik))
The Language resources correspond to those required to facilitate input and output.
For example, in order to facilitate input output in Tamil, resource files corresponding
to font file (Bamini Font), coding scheme mapping file between Ascii to Tascii for
perception and vice-versa for response are required. Tascii is a character-based coding
scheme which is used to represent the characters of the languages supported,
internally. The Tascii codes used for Tamil language are given in Appendix A. The
keyboard layout is air0 given in the appendix A. The following example illustrates the
need for the mapping.
Typed keys
Ascii value in English Fonts
Corresponding character in Tamil : 61csn
Tascii Code for character Q 6 n , 2 9 5
Ascii value for the glyphs '61' 's' 'n' in Bamini Font : 78 70 72
That is, when the user types the characters 'k' .o' in the keyboard, it corresponds to
the ascii values 75 and 79. This should be interpreted internally as character '61en'
with Tascii code as '295'. In Bamini font which is used as the font file for Tamil, the
glyphs '61' '16' .r' are mapped to characters 'n' 'f 'h'. Hence. in order to display the
glyphs corresponding to '61csn' using the Bam~ni font, the ascii values 78 70 and 72
are required. Therefore, during input 75 and 79 are mapped to 295 (Ascii to Tascii).
To display the same character, 295 IS mapped to glyph codes 78, 70 and 72 (Tascii to
ASCII)
In MULLAI, a relational knowledge representation is used. The vocabulary is given
in terms of tables for each of the word types - noun, verbs, adjectives, pronouns etc.
These tables contain the word and the suffix code. The suffix code ~ndicates what
~nflect~on has been carried out on the word and what that ~nflectlon is. For example,
for nouns, the inflection may be with respect to gender, number, etc. In order to keep
track of the tables that contaln the various word types, another metadata table as
shown below in Table 8. i is maintained.
Table 8.1: Metadata of language details
tam11 ,-- I%- - sentence structure~
The grammar indicates the rules to be followed in the formation of words, inflection
of words, formation of phrases, formation of sentences etc. For example in Tamil,
noun phrases can be formed as follows:
np: noun ( pronoun
np: adj ; np
The following patterns are used in the formation of a sentence consisting of subject S.
verb V, object 0 1 and object 0 2 , adjective A
S v S O V
O S V
S 0 1 0 2 V
0 1 0 2 S V
0 2 0 1 S V
S O A V
O S A V
The metadata about the files containing the lexicon and grammar rules are maintained
separately so that they can be referenced and updated whenever required. The
follow~ng table in Table 8.2 glves the path details about where the knowledge files
corresponding to the varlous languages are stored.
Table 8.2: The path details of languages supported
The knowledge services that are to be rendered by the knowledge manager role are.
Servicing knowledge requests
Update of knowledge
Organlzlng and storing knowledge
The tirct servlce 15 e x p l ~ c ~ t l y not visible as 11 1s belng used b? the naturdl 1angudi.r
pr(1ces\lng hehav1or5 lor proceas~ng a naturdl language request The screen shot In
I;~gure X I shows k ~ i o ~ l e d g e update The update IS belng iarrled out lor noun
c'ircgor! uf'\\ord In the table with name Noun I The other command buttons pro\ idc
opl~onb l i)r modif)lng character?. phrases and sentences
I - L - u s . -.. .. . .. . . dlb B
L - W I I I I M ICI
i - ~~-
I I
WIlCtl U , . L I Y !
. I- .. .LJ 1 1 " ' - 1 a.rlj;. ---- ...-- ... Y P l l , , -I--
I !
m-rra . ; , irra.npa
-,--
Figure: 8.1: Interface for update of language konalcdge
I LII orgdnl/lng and \tclrlng k~io\\ledge. thc knowlcdpc rilaliagrr hclps to crcnte
\cp.ir,itc J ~ r e ~ l c ~ r ~ c a lor \toilng all files pertdlnlng to thdt laiigu.ige I: uI\L) help, to
ire;ltc thc rcqu~rcd tnbic structures fc>r stor~ng thc hriin\lcdge In relationdl
replc~cnt,ltluli du r l~ ig Idnpii.igr azqulsltlon This 15 deplited 111 I'lgure X ?
I-he w e e n sliot dep~cts I i < ~ u the hnouledgr englnccl createa .I inhie Sor htorlng \crb
dctail5 I'hc table IS irearcd ~1111 fields word, sul'li\ code dnd root I hc t?pe o f tliche
lields can also be specllied I he iirlnmand buttons. ddd dr rcino\e ficldr hclp ro
pc l fo r~ i i the iorrc>pund~ng operauonr In the table U hen the t'lhlc ~ ~ c a r l o n I\ r , \ c ~ tlir
b u ~ l d idblc hclps l u create a tablc In the bdch-end database \\ 1111 the \pec~tied liclds
and t lpea Thus. the knowledge manager helps to orgdnlze '111d \tore hrio\\ledge
Ftgurc: 8.2: Interface for creating tables for storlng language knowledge
I lie t.i\h LL>I~IL\I 0 1 111c \en Behd\i<)r . ~ C ~ U I \ I ~ I L > I ~ ~ I d ~ i ~ ~ g e i ~ i c ~ ~ ~ lid, i ~ i t e i n ~ i l bcl~ci'\
pzrl.ilriilig r i , t l ~ c td\h orltvlu_c and lrlngudgc ontolog! 'I he IL~ i lg i i r l g~ ~oiiti~log! I 0
< I \ c \ deta~l \ , rh i ) i~~ the rc\oiircc,. hnmrledye m d io i~ lpe tc~ lc ia hi acqiiiic 111 ordc~ to
t c . ~ l i / ~ J i ~ c \ \ I.~nyiiayc iii tlic dycnt ;ind thc i i inct~vnal dt,iiiaiii i>ill\>log! DO L O I ~ ~ I , I I ~ %
t l i ~ di,in.~iti o l Ilir 1.1iiguagc i1ch11l\ LO he r l ~ q i ~ i r c d Thia 15 ~cprc\e i~ ied JS.
( 1 3 ~ 1 IC , , , L O D O )
ll tlic lrilig~l.iye JL~~II~I~IIIII tdbh perII>rmb ~ C L I U I \ I ~ I O I ~ ~ > f rl inti\ IJI~~LI.I~C -I,,,,\' rhcil 11
ai~gnicnla the hellel set o f rlic language tocult! \ \ ~ t l i bs l ic t i ill the nc\\ Ia~igudpu I Ilia
I\ rcp~caentcd ah lo l lo \ rJ
o(Uel I lu r igu~gr (I,,,,, 110))
1 hcrc I,,,,, 15 thc nr\\ language dcyu~red
In order to ach~eve this. thc path deta~la of the nc\\ I! ncqulred Idngiinge I S appended in
the 1,111guogc dccrlils t.~bic. and tlic table details rablc IS upda~cd n ~ t h tiic n.irnr o l the
I'ingudgc and the ct~ric\pundlng tables name:. thdt colitdln I!> I,inguogr. hno\\ledge
resuulcca 'l'h~a la dwne \r it11 thc help ol'ttlc. lrliigi~rlgc h i l ~ l ~ ledye 11li1ilrlg~1
I he acrcen \hot In Flgure 8 3 lndlcates the Inltlatlon of the acqulrltlon hehavlur ~ o l e
~1 iTd1 i i l l language
i
I
I I 1 I I I
F l g ~ l r c 8.3: Acquiring new language ('Tamil)
I lic ,ILL]LII~I~RII~ p1~11ci~ h ~ g l n b \%ith the acqulsitron i i t ih,ii,iitcri h! ~iht.11n111; tile
IIIII~I~ICI (>I L l i G ~ ~ c ~ L t c i t!pe> 1Iie11 lia1nca slid I~IC ~ i u ~ i i b e r o I ' ~ l i ' ~ ~ ~ i ~ t e r \ 111 edcll 1)pe
l l icn. 11 \ I d r r ohtnlnlng the Ion1 f i le, thc '1aac11 code dnd tlic ASCII code till the
.I~~r, lcter\ I hen. 11 ohtdln, the I y p h code\ ~orrespondlng ta thc fbnt file that ale ti)
hc used tor d ~ ~ p l a ! 11ig 111c clidractrr glbpha Attcr th~s. the lo< 5 l i i c that I\ requircd to
I ~ ~ i i l ~ l a t e 11ipu1 and i ~ u t p u l b! convening hr thcen dscl! ILI t ax i1 and 1d\c11 tu a a ~ l i 15
~ ~ q u l r c d Th15 IOCS la a i laas l i le that la d!narnlcall! ~ n c l ~ l d c d b! nslnp d!llamli
i l a s loading Onl! 11 this IOCS class t i lc ia Included d > n a i n ~ ~ a l l ? . thc suhsrqucnt
\rurd acqulaltlun and aenIenct. acqulsltlon IS pi~sslhlc hrcdux! nnl! 1h1$ ICKY cdn
1 ~ ~ 1 l l t s t c Input and uutput 111 the correspondlny Idnguape tu cnter tl>c \rord and
icntcricc detalla l'heae are dep~cted In F ~ g u r c 8 4
' i l l l , " .n l i , , t . a,,",",
r d , 1
Ulsr l l l l l " . l~ ,nu 2
UI "l '$6 1
M1" n,, vwx, 1
.I..l.lu rr.I,,,IIIIIIII, 258
'"I illbIIa11 DwIDI.tN.cmIIo..ol...
MI1, I I I I ' I l l , . lSY<I< . WIO/.tN..IY.mlll.~,,~m.'!nfnfr,...
F ~ g u r e 8.4: Acquiring basrc language details
I c\cr! t!pr <)I clidrd~ler and for all the characlera In ever! hpe . tlic aacli code u icd
lur rcprc\enllllg l!ic ii1.1rdcter on the Lrybodrd. the t d \ i i i cridr v,!i~cli 15 u,cd lo
rcprc\c!ll the ihnr,tiler ~ntrrnall) a n d thr glypll code\ wed tor d~ydaying itir
L ~ ~ . I I J L I C I I !pI>\ Jrc : l i i l ~ ~ ~ r e d l h ~ \ 15 ileplcted in rigilrc X 5
Figure 8.5: Character Coding Details Acquiclhoo
1 his IS followed by word detail\ acquisltlon Here. the number of typcs o f words dnd
t h e ~ r name\ are ohtamed ~ n l t ~ a l l ? Then for each word type. a table u ~ t h the r rq i i~red
Wucture 15 crcated 'Then the table d c t a ~ l s are entered This 15 c d r r ~ e d out for all the
k o r d typcs r i g u r e 8 6 shows an cvample of 'word detdll acqulaltlon
F,gurc 8.0: Word Dctails acquisit~on
l l ic \dr luus plird>,il struziurer and scntence structure\ a l c also ohtdincd Tlicsc
~ i r r r c s p ~ r n d to tile \ d i ~ o i i a pdrreriis In \ \hlili tlirhe htructurci i o ~ i ?\)at iii tile
i o r r c \ p o n d ~ n g Iangudge 1 hi> 15 depicied in t lgures 8 7 and X t;
I , I j 1 I I
I I F ~ g u r e 8.7: Phrase details acqulsltloll
Figure 8.8: Sentence details acquisition
Subsequent to thls. the statlc sentences that correspond to the standard rrsponrc
Inessages lo he ionbeyed to the user are acqu~red This 1s because, for standard
rerpon\es. thc language generation process need not be carrled out unnecessarll) and
the acquired nle\sdge\ could he readll? con\,eyed
8.1.3 Dynamic Role Binding - Function Behavior Management a n d Interface Behavior %lanagenlent
In thc case o l funitlon hrhe\lar and imerhce heha\lor nianageinent. the manager role
d)nd~nliall! bind\ ~ t \ c l f to thc requlred language beha\lor role b! \%ea\'lng the
requ~red language aspect T h ~ s I S ~llustratcd In the ld lou iny screenshot5 In Figures
8 Y and 8 10 \%here the ~ntert'ace and competence funcllons \ \ caw the Tamll language
n\pect to pro\. ~ d e Interaction In Tam11 language
The s~recnahotb In the tigures ~llustrate how the reaearch contrlbutlons have been
practically real17ed In MlJLLAl
8.2. Multilingual Multi-Agent System
For applylng the language faculty models for a multi-agent system, the architecture
needs to be extended The need for thls extension, the extended 117odel and thc
appllcatlon of the model for a mult~l~ngual multi-agent system are explained in the
following d~scusslons
Figurc 8.9: Interface Behawor Management weaves w ~ t h 'I an111 language aspect
F ~ g u r e 8.10: F u r ~ c t ~ o r ~ Behavior Managenlent weakes Tan111 language processing Aspect
8.2.1 Extending the Behav~or hlanagcment Architecture for hlulh-agent Systenl
A Multi-agcnt a!stclll (MAS) 15 one in ~ h ~ c h tuurc than LIIIC dycnt e\ists dnd e\L.t!
agem possesses d 5pecilic tash e\penl&e The) arc l~aed ti>r \ul\ 1111. i.r>mplc\ problcin~
which rcqulrc the conlr~hut~on of mult~ple tash c\prrtlrr I'hc dyentr 111 M/\S
cooperate or negotlatc ~ ~ t h each other to solve the problem
The language ability of MAS should help to cater to the interaction requirements of
all the agents present in the MAS. That is, the language ability should suppon the
domain ontology of every agent. Mapping this requirement to the defined architecture
components, the function of every management function is as given in Table 8.3.
The table shows that the responsibility of each of the above management functions
become more as they have to manage additional behaviors and their complexity also
increases. Hence, each of the language behavior management components could be
realized as a separate agent with the corresponding behaviors under each of them.
Thus. language handling in MAS becomes a dtstributed problem solv~ng activity,
which is achieved by the cooperation ofthe behavior management agents.
Table 8.3: Functions of the Management components in the context of MAS
/ Behavior Management Components
Interface Behavior Management
The architecture which is depicted in Figure 8.1 1 consists of two layers - the
functional agent layer and language faculty agent layer. The language faculty agent
layer consists of the agents corresponding to the various management functions of the
language faculty. Except being divided into individual agents, the functions of each of
these management agents are similar to the functions defined for these components in
the architecture described for the language faculty of a single agent.
Functions
This component is not affected by the presence of multiple agents in the MAS and
Function Behavior Management
Knowledge Behavior Management
New Behavior Acquisition Management
The functional agent layer consists of the various functional agents of the MAS. There
is a facilitator agent which takes care of registering and deregistering of the funct~onal
remains the same. Help to comprehend a given request which is expressed in any of the supported languages and map to the appropriate domain ontology. . Manage the language knowledge and resources corresponding to all the task agents in all the supported languages. 8 Acquire the language resources in all the
supported languages corresponding to the task agents which are registered. Acquire new language knowledge.
agents in the MAS. Whenever a new agent registers with the facilitator, it registers its
domain ontology also with the facilitator. The facilitator immediately initiates new
behavior acquisition management agent to acquire the language resources pertaining
to this domain ontology in all the languages supported. When a functional agent
deregisters from the functional agent layer, the facilitator informs the behavior
knowledge and the behavior competence management agents to dismount the
language knowledge pertaining to this agent in all the languages.
1
Knowledge Competence New Behavlor Behav~or Behav~or
Management Management Acquls~t~on Agent Agent Management
Agent
Intellace Language Behavlor Faculty
Management Agent Agent Layer
Percept I Response
Figure. 8.11: Extended Behavior Management Architecture Model for the language Ability of Multi-Agent System
The request for the task to be performed by the MAS is received through the Interface
Behavior Management Agent. It transfers it to the Function Behavior Management
Agent which processes it and converts it to an intermediate structure. This
intermediate structure is compared with the domain ontologies of the various
functional agents that are registered with the facilitator to determine to which of the
agent the task is delegated. This conversion to intermediate structure and determining
the appropriate functional agent are the additional functions required to be managed
by the behavior competence management. After this determination, the request is sent
to the corresponding functional agent through the facilitator. This request may then be
executed by the functional agent either independently or as a coordinated activity
among the various agents of the MAS.
When a new language has to be acquired, the New Behavior Acquisition Management
agent configures to the language acquisit~on role and uses the registered domain
ontologies of the functional agents and the generic language ontology to acquire the
language knowledge and resources corresponding to all the domains.
The main advantage of this model is that it is able to provide the language ability of a
language faculty to a multi-agent system also. But, the limitation is that this model in
its current state is most suitable for closed multi-agent systems or open multi-agent
systems with minimum number of agents.
8.2.2 MMAS - Multillnpd Multi-agent System
This case study describes how the above model is applied for developing a
Multilingual MAS. The multi-agent system was developed with two functional agents
where one of the agents is an e-mail service agent and the other agent is a file service
agent. The e-mail service agent makes use of the file service agent to store or retrieve
mails. If any file is to be e-mailed, the file service agent contacts the e-mail service
agent. The MAS supports two languages Tamil and English.
The screen shots below indicate the operations of the MMAS. The screen shots in
Figures 8.12 and 8.13 indicate the registration of the functional agents (Email service
agent and File service agent) along with their functional domain ontology with the
facilitator.
I'rgurc. 8 I?. Rep~stratton of >]ail Agent wt t l~ tlie fi~cilrtator
Figure 8.13: Regrstrstrori of File Ser\rcc Agent arth tlir facrl~tt~tor
Figure. 8.1 4: Ernail Service Operation ru Tamil
,,,= ",,. *,,, ,N*, . >.n,-. **
","I*.>< .A",
Figure 8.15: File Ser~rce Operatiou ~n English
8.3 Eva1u;ltion
I h e case studies dsscr~bcd a b o ~ c help to proke the vai~d~t! ol the miidels dc.\elc~ped
In order 10 ' A ~ I I C I I the advdlltagcs 01 the agent k\~th t\\o-d~mens~un;rl languagc
autonoln?, an evaluatli~n o f t h c dgent \ \ ~ t h the ehlstlng ~nult i l~ngual dlnlogulng dgent,
(MDAa) 1s carried out Among the varlour tbpea ofdgcni, ldent~tied In thc ld\oiiom!
prusrnled In the Ilterature revlelr, onl) the MDAs are churrn becaurc 0111) t11e.c
agents fulfill the b a r ~ c language d b ~ l ~ t ! rcqulrcllienl~ of dl1 dgent \+h~cli linvu ~ C C I I
ldentrfied aa CNLl and DM II I t h~h thearr ,411 tlir MDAs are ionaldrrrd .I\ a s ~ n g l r
category irrespective of the technique used for achieving multilingualism. This is to
provide a clear representation of the advantages of the existing MDAs with respect to
an agent with twodimensional language autonomy.
In order to perform the evaluation, the criteria for evaluation have to be identified
first. Also, the mode of assessment has to be identified. The following are the
evaluation criteria considered:
Language functionality properties for CNLI
Language functionality properties for DM
Internal State Language Properties
Language Management Properties
Language Management Architectural Properties
Multiple Language Handling Complexity Properties
The above set of criteria is a consolidation of the requirements or properties against
which the conceptual, internal state, architecture, design and implementation models
have been developed in this thesis. The assessment is represented quantitatively on a
scale of 0 to 1. The values are awarded against the properties based on the extent to
which the corresponding property is fulfilled.
Table 8.4 shows the evaluation of the agents. The awarded values indicate whether
the corresponding property is fulfilled completely (I), not considered at all in the
agent (0) or is partially fulfilled (0.1 to 0.9). The values of the existing MDAs are
found to be less except for the properties pertaining to CNLI. The reasons for the
reduction of values are given in the table as comments. The values for the agent with
twodimensional autonomy are given based on the results of analysis discussed at the
end of each of the chapters. In these analysis it has already been proved that the
properties arc fulfilled and hence the value of 1 is given. Finally, using these values, a
graph as shown in figure 8.16 is drawn to show the overall evaluation results. From
the evaluation it is clear that an agent with two-dimensional language autonomy not
only fulfills the properties / requirements, but also excels the existing MDAs in many
ways.
- The dynamic configuration to the required language should have minimum latency.
-- -
languages and the task performed by the agent.
Dy..mieily Should help to support multiple languages and dynamically configure to the required language. Perlorrmna - Should pmvide for a performance
which is comparable to that of an agent supporting a single language.
I
I
1
(There is no Language Management Architecture
available)
Scalability Should enable to acquire more languages. M.inWnabUity Should enable to update and maintain the language knowledge and competence. Reuubility The architecture should be suitable not only for language faculty, hut also for any other task faculty of the agent.
I
1
I
8.4 Summary
The descrited case studies and the screen shots provide illustrations of the application
of the solutions proposed for realizing the language faculty of an agent as well as a
multi-agent s9stem. The extended architecture model shows how the same behav~or
management architecture could be adapted for MAS to &Ifill its language
requirements.
The contributions of t h~s chapter are:
Proof of hypothesis proposed in problem statement.
Proof that the proposed solutions are suitable even when hypothesis is
extended to accommodate the language requirements of MAS.
Evaluation of an agent with two-dimensional language autonomy with existing
multilingual dialoguing agents.