digital concept builder
Post on 04-Apr-2018
217 Views
Preview:
TRANSCRIPT
-
7/30/2019 Digital Concept Builder
1/74
DigitalConceptBuilder
True Natural Language Understanding
through a Conceptual LanguageUnderstanding Engine
ByRoy, Philippe| 11 Jan 2010
Win2KWinXPVS2008C++JavascriptWin32STLArchitectDevIntermediate
An article introducing an Artificial Intelligence technique that understands and manipulates
concepts from text.
See Also
More like this More by this author
Print Article
Twitter
Digg
Facebook
Del.icio.us
Reddit
Stumbleupon
Newsvine
Technorati
Mr. Wong
Yahoo!
Google
Windows Live
Send as Email
Add to your CodeProject bookmarks
Discuss this article
13
Article Browse Code StatsRevisions (5)
4.78 (43 votes)
1 23 4 5
Sponsored Links
Is your email address OK? You are signed up for our newsletters but your emailaddress is either unconfirmed, or has not been reconfirmed in a long time. Please clickhereto
http://www.codeproject.com/script/Membership/View.aspx?mid=1078443http://www.codeproject.com/script/Membership/View.aspx?mid=1078443http://www.codeproject.com/script/Membership/View.aspx?mid=1078443http://www.codeproject.com/info/search.aspx?aidlst=23http://www.codeproject.com/info/search.aspx?aidlst=77http://www.codeproject.com/info/search.aspx?aidlst=87http://www.codeproject.com/info/search.aspx?aidlst=110http://www.codeproject.com/info/search.aspx?aidlst=118http://www.codeproject.com/info/search.aspx?aidlst=118http://www.codeproject.com/info/search.aspx?artkw=True+Natural+Language+Understanding+through+a+Conceptual+Language+Understanding+Enginehttp://www.codeproject.com/info/search.aspx?artkw=True+Natural+Language+Understanding+through+a+Conceptual+Language+Understanding+Enginehttp://www.codeproject.com/script/Articles/MemberArticles.aspx?amid=1078443http://www.codeproject.com/script/Articles/MemberArticles.aspx?amid=1078443http://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#_commentshttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#_commentshttp://www.codeproject.com/script/Articles/ViewDownloads.aspx?aid=51026http://www.codeproject.com/script/Articles/ViewDownloads.aspx?aid=51026http://www.codeproject.com/script/Articles/Statistics.aspx?aid=51026http://www.codeproject.com/script/Articles/ListVersions.aspx?aid=51026http://www.codeproject.com/script/Membership/SendConfirmRequest.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Membership/SendConfirmRequest.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Membership/SendConfirmRequest.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Bookmarks/Add.aspx?obid=51026&obtid=2&action=AddBookmark&bio=truehttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/script/Membership/SendConfirmRequest.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Articles/ListVersions.aspx?aid=51026http://www.codeproject.com/script/Articles/Statistics.aspx?aid=51026http://www.codeproject.com/script/Articles/ViewDownloads.aspx?aid=51026http://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#_commentshttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#_commentshttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx?display=Printhttp://www.codeproject.com/script/Articles/MemberArticles.aspx?amid=1078443http://www.codeproject.com/info/search.aspx?artkw=True+Natural+Language+Understanding+through+a+Conceptual+Language+Understanding+Enginehttp://www.codeproject.com/info/search.aspx?aidlst=118http://www.codeproject.com/info/search.aspx?aidlst=118http://www.codeproject.com/info/search.aspx?aidlst=110http://www.codeproject.com/info/search.aspx?aidlst=110http://www.codeproject.com/info/search.aspx?aidlst=87http://www.codeproject.com/info/search.aspx?aidlst=87http://www.codeproject.com/info/search.aspx?aidlst=77http://www.codeproject.com/info/search.aspx?aidlst=77http://www.codeproject.com/info/search.aspx?aidlst=23http://www.codeproject.com/info/search.aspx?aidlst=23http://www.codeproject.com/script/Membership/View.aspx?mid=1078443 -
7/30/2019 Digital Concept Builder
2/74
have a confirmation email sent so we can confirm your email address and start sending you
newsletters again. Alternatively, you canupdate your subscriptions.
/w EPDwUKMTAy
/w EWCALr89jXA
Download source - 831 KB
Download executable (Win32) - 894 KB
Download non-abridged dictionary - 2.2 MB
Introduction
http://www.codeproject.com/script/Membership/Subscribe.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Membership/Subscribe.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/script/Membership/Subscribe.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspxhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderSources.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderSources.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderSources.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderExecuteable.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderExecuteable.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderExecuteable.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/DigitalConceptBuilderSources.ziphttp://www.codeproject.com/script/Membership/Subscribe.aspx?rp=%2fKB%2frecipes%2fDigitalConceptBuilder.aspx -
7/30/2019 Digital Concept Builder
3/74
People spend a significant amount of time and energy at putting their thoughts into
writing. So, it is time for a technology to concentrate on automating the retrieval and
manipulation of these thoughts in such a way that the original meaning is not lost in theprocess. To that end, a means by which concepts are reconstructed and then manipulated
from natural language is required. That goal cannot be achieved by a diminutive form of
natural language understanding that limits itself to word-spotting or a superficial associationof semantic blocks, but rather by a process that mimics the functions of the human brain in its
processing and outcome. This article exposes a novel means by which such processing is
achieved through a Conceptual Language Understanding Engine (CLUE).
The dominant part of the word "recognition" is "cognition", which is defined in the Oxford
dictionary as "the mental acquisition of knowledge through thought, experience, and the
senses". The present approach uses techniques that encapsulate most aspects associated with
a cognitive approach to recognition.
Communication is a procedural sequence that involves the following processes:
a "de-cognition" process - producing a syntactic stream representing the cognitive
aspect to communicate.
a transporting process - putting such syntactic stream on a medium.
a perceptive process - senses acquiring the syntactic stream.
a "re-cognition" process - rebuilds the original cognitive substance that hopefully has
not been lost throughout the syntactic, transporting, and perceptive processing.
Because a full cycle language analysis involving natural language understanding requires
putting the concept back in its original form, without incurring any loss of the conceptual
manipulations that can be achieved following a "de-cognition" and "re-cognition" processes,
the conceptual aspect of language cannot and must not be overlooked. Only when a
conceptual dimension to speech analysis becomes available will the syntactic aspect of
processing language becomes unconstrained. That is, syntactic analysis will limit to a
required transient step for communications to fit on a medium. Only then the limitations that
we have endured until today related to natural language understanding and processing will
start to fade.
The reward with a CLUE is the ability to abstract the written syntactic forms from conveyed
concepts - the words used to communicate become irrelevant as soon as the concept is
calculated - while maintaining the ability to intelligently react to recognized content. It
further shifts the final responsibility of disambiguation to the conceptual analysis layer,instead of the phonetic and syntactic layers as it has typically been done to this day. That
frees the person communicating from the obligation of using predetermined sequences of
words or commands in order to be successfully understood by an automated device. In an era
where automated devices have become the norm, the bottleneck has shifted to the inability of
technology to deal effectively with the human element. People are not comfortable with a set
of syntactic related rules that appear to them as counter-natural in relation to their conceptual
natural abilities.
Background
-
7/30/2019 Digital Concept Builder
4/74
This article is the third of a sequence. In the first article, "Implementing a std::map
Replacement that Never Runs Out of Memory and Instructions on Producing an ARPA
Compliant Language Model to Test the Implementation", the memory management technique
used within the code-base provided in the present article is exposed. As a result of that, you
shall observe there is only one delete call throughout the entire code-base, and you are
certain there is no memory leak within the said C++ code. In the second article, "TheBuilding of a Knowledge Base Using C++ and an Introduction to the Power of Predicate
Calculus," predicate calculus is introduced, but falls short of processing natural language
input. Although it also exposes techniques in order to infer from a knowledge base - not used
in this article, but could easily be adapted to do so - the article covers basic predicate calculus
techniques that are widely used in this article. A reading of these two articles will help you
gain a better understanding of the basis upon which this code is built. Although Conceptual
Speech Recognition is used to interpret speech (sound), for the sake of simplicity and
demonstration, this article limits itself to textual content input. A software implementation of
this technology can be referred to as a Conceptual Language Understanding Engine, or
"CLUE" for short. The approach to process speech recognition is comparable to what is
exposed here, but with a couple of software engineering twists in order to include furtherbiases from speech through the integration of a Hidden-Markov-Model that is not discussed
in this article.
Using the Code
The current project builds under Visual Studio 2008. It is composed of about 10,000 lines of
C++ source code, and about 500 lines of JavaScript code that is processed byGoogle V8.
The main components of the code are divided as follows:
The dictionary: A 195,443 words, 222,048 parts-of-speech dictionary, held in a 2.1
MB file, which can return information on spelling almost instantly. [IndexStructure.h,
StoredPOSNode.h, StoredPOSNode.cpp, DigitalConceptBuilder::BuildDictionary]
Tokenization: Transforming a stream of natural language input into tokens based on
content included into a dictionary. [IndexStructure.h, POSList.h, POSList.cpp,
POSNode.h, POSNode.cpp, IndexInStream.h, DigitalConceptBuilder::Tokenize]
Syntactic Analysis: The algorithms to extrapolate complex nodes, such as SENTENCE,
from atomic nodes, such as NOUN, ADJECTIVE, and VERB obtained from the dictionary
following the tokenization phase. [SyntaxTransform.txt, POSTransformScript.h,
POSTransformScript.cpp, Permutation.h, Permutation.cpp, POSList.h, POSList.cpp,
POSNode.h, POSNode.cpp, Parser.h, Parser.cpp, BottomUpParser.h,BottomUpParser.cpp, MultiDimEarleyParser.h, MultiDimEarleyParser.cpp]
Conceptual Analysis: The building of concepts based on syntactic organizations, and
howGoogle V8, the JavaScript engine, is integrated into the project. [Conceptual
Analysis/*.js, Conceptual Analysis/Permutation scripts/*.js, Conceptual Analysis/POS
scripts/*.js, POSList.h, POSList.cpp, POSNode.h, POSNode.cpp, Predicate.h,
Predicate.cpp, JSObjectSupport.h, JSObjectSupport.cpp, POSNode::BuildPredicates]
Execution of the Code
http://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://code.google.com/p/v8/http://code.google.com/p/v8/http://code.google.com/p/v8/http://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#dictionaryhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#dictionaryhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#tokenizationhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#tokenizationhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#syntactic_analysishttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#syntactic_analysishttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#conceptual_analysishttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#conceptual_analysishttp://code.google.com/p/v8/http://code.google.com/p/v8/http://code.google.com/p/v8/http://code.google.com/p/v8/http://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#conceptual_analysishttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#syntactic_analysishttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#tokenizationhttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder.aspx#dictionaryhttp://code.google.com/p/v8/http://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://www.codeproject.com/KB/cpp/conceptual_dependency.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspx -
7/30/2019 Digital Concept Builder
5/74
Execution of the code as provided with this article runs the test cases specified in
SimpleTestCases.txt. Each block within curly brackets defines a scope to run, where CONTENTis the natural language stream to analyze.
Each variable definition prior to the test case sentences can also be inserted within curly
bracket scopes to override its value. For example, to enable only the first test case, thefollowing change is possible:
Collapse...#-------------------------------------------------------------------
# ENABLED# _______## Possible values: TRUE, FALSE#
# TRUE: enable the test cases within that scope.## FALSE: disable the test cases within that scope.
ENABLED = FALSE
#-------------------------------------------------------------------
{ENABLED = TRUECONTENT = Is a red car a car?ID = CAR1
}
...
Executing the test cases, as available from within the zip files attached, results in the
following output. The remainder of this article exposes the approach, philosophies, and
techniques used in order to transition from the Natural Language input from these test-cases
to concepts and responses as exposed here.
CollapseEvaluating: "Is a red car a car?" (ID:CAR1)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 250 msSyntactic: 63 ms
Conceptual: 187 ms.
http://www.codeproject.com/KB/recipes/DigitalConceptBuilder/SimpleTestCases.txthttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/SimpleTestCases.txthttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/SimpleTestCases.txt -
7/30/2019 Digital Concept Builder
6/74
Evaluating: "Is a red car the car?" (ID:CAR2)
MAYBE:
DO[ACTION:RECEIVEINPUT
MOOD:INTEROGATIVEOBJECT:DO[OPERATION:ISVALUE1:PP[CLASS:CAR
COLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARDETERMINED:TRUEQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 2 sec (2235 ms)Syntactic: 31 msConceptual: 2 sec (2204 ms).
Evaluating: "Is the red car a car?" (ID:CAR3)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDDETERMINED:TRUEQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 203 msSyntactic: 31 ms
Conceptual: 172 ms.
Evaluating: "Is the red car red?" (ID:CAR4)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDDETERMINED:TRUEQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[COLOR:RED]]]
Total time: 219 ms
-
7/30/2019 Digital Concept Builder
7/74
Syntactic: 31 msConceptual: 188 ms.
Evaluating: "The car is red" (ID:CAR5)
No inquiry to analyze here:
DO[ACTION:RECEIVEINPUTMOOD:AFFIRMATIONOBJECT:PP[CLASS:CAR
COLOR:REDDETERMINED:TRUEQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]
Total time: 547 msSyntactic: 15 ms
Conceptual: 532 ms.
Evaluating: "Is a red car blue?" (ID:CAR6)
NO:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[COLOR:BLUE]]]
Total time: 2 sec (2453 ms)Syntactic: 31 ms
Conceptual: 2 sec (2422 ms).
Evaluating: "Is a red car red?" (ID:CAR7)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[COLOR:RED]]]
Total time: 235 msSyntactic: 32 ms
Conceptual: 203 ms.
Evaluating: "Is a car or a boat a car?" (ID:CAR8)
YES:
DO[ACTION:RECEIVEINPUT
-
7/30/2019 Digital Concept Builder
8/74
MOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:OR[VALUE1:BOATVALUE2:CAR]
QUANTITY:1TYPE:VEHICLE
WHEELCOUNT:4]VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 390 msSyntactic: 187 ms
Conceptual: 203 ms.
Evaluating: "Is a car an object that is not a car?" (ID:CAR9)
NO:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:AND[VALUE1:PP[QUANTITY:1TYPE:{DEFINED}]
VALUE2:NOT[VALUE:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]]]
Total time: 3 sec (3297 ms)Syntactic: 141 ms
Conceptual: 3 sec (3156 ms).
Evaluating: "Is a boat an object that is not a car?" (ID:CAR10)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:BOATQUANTITY:1TYPE:VEHICLE]
VALUE2:AND[VALUE1:PP[QUANTITY:1TYPE:{DEFINED}]
VALUE2:NOT[VALUE:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]]]
Total time: 328 msSyntactic: 78 ms
Conceptual: 250 ms.
Evaluating: "Is an object that is not a car a boat?" (ID:CAR11)
-
7/30/2019 Digital Concept Builder
9/74
MAYBE:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:AND[VALUE1:PP[QUANTITY:1
TYPE:{DEFINED}]VALUE2:NOT[VALUE:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
VALUE2:PP[CLASS:BOATQUANTITY:1TYPE:VEHICLE]]]
Total time: 3 sec (3500 ms)Syntactic: 63 ms
Conceptual: 3 sec (3437 ms).
Evaluating: "Is a car that is not red a car?" (ID:CAR12)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:!REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 313 msSyntactic: 63 ms
Conceptual: 250 ms.
Evaluating: "Is a car an object that is a car or a boat?" (ID:CAR13)
MAYBE:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:OR[VALUE1:PP[CLASS:BOATQUANTITY:1TYPE:VEHICLE]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]]
Total time: 3 sec (3875 ms)Syntactic: 125 ms
-
7/30/2019 Digital Concept Builder
10/74
Conceptual: 3 sec (3750 ms).
Evaluating: "Is a red car an object that is a car or a boat?" (ID:CAR14)
MAYBE:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:OR[VALUE1:PP[CLASS:BOATQUANTITY:1TYPE:VEHICLE]
VALUE2:PP[CLASS:CARQUANTITY:1
TYPE:VEHICLEWHEELCOUNT:4]]]]
Total time: 4 sec (4563 ms)Syntactic: 344 ms
Conceptual: 4 sec (4219 ms).
Evaluating: "Is a car that is not red a car?" (ID:CAR15)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:!REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 360 msSyntactic: 63 ms
Conceptual: 297 ms.
Evaluating: "Is a red car not red?" (ID:CAR16)
NO:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[COLOR:!RED]]]
-
7/30/2019 Digital Concept Builder
11/74
Total time: 1 sec (1844 ms)Syntactic: 32 ms
Conceptual: 1 sec (1812 ms).
Evaluating: "Is a car a car that is not red?" (ID:CAR17)
MAYBE:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARCOLOR:!REDQUANTITY:1TYPE:VEHICLE
WHEELCOUNT:4]]]
Total time: 3 sec (3125 ms)Syntactic: 78 ms
Conceptual: 3 sec (3047 ms).
Evaluating: "Is a car that is not red a blue car?" (ID:CAR18)
MAYBE:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:!REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARCOLOR:BLUEQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 4 sec (4250 ms)Syntactic: 250 ms
Conceptual: 4 sec (4000 ms).
Evaluating: "Is a red car a car that is not red?" (ID:CAR19)
NO:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARCOLOR:!RED
-
7/30/2019 Digital Concept Builder
12/74
QUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 3 sec (3547 ms)Syntactic: 234 ms
Conceptual: 3 sec (3313 ms).
Evaluating: "Is an object that is not a car a car?" (ID:CAR20)
NO:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:AND[VALUE1:PP[QUANTITY:1TYPE:{DEFINED}]
VALUE2:NOT[VALUE:PP[CLASS:CARQUANTITY:1
TYPE:VEHICLEWHEELCOUNT:4]]]VALUE2:PP[CLASS:CAR
QUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 3 sec (3891 ms)Syntactic: 47 ms
Conceptual: 3 sec (3844 ms).
Evaluating: "Is an object that is a car or a boat a car?" (ID:CAR21)
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:OR[VALUE1:BOATVALUE2:CAR]
QUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 781 msSyntactic: 141 ms
Conceptual: 640 ms.
Evaluating: "Is an object that is a car or a boat a red car?" (ID:CAR22)
MAYBE:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:OR[VALUE1:PP[CLASS:BOAT
QUANTITY:1TYPE:VEHICLE]
-
7/30/2019 Digital Concept Builder
13/74
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]
VALUE2:PP[CLASS:CARCOLOR:RED
QUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 5 sec (5984 ms)Syntactic: 344 ms
Conceptual: 5 sec (5640 ms).
Evaluating: "Is an object that is a car and a boat a red car?" (ID:CAR23)
MAYBE:
DO[ACTION:RECEIVEINPUT
MOOD:INTEROGATIVEOBJECT:DO[OPERATION:ISVALUE1:PP[CLASS:AND[VALUE1:BOAT
VALUE2:CAR]QUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARCOLOR:REDQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 6 sec (6437 ms)Syntactic: 734 ms
Conceptual: 5 sec (5703 ms).
Evaluating: "Is a car an object that is a car and a boat?" (ID:CAR24)
NO:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:AND[VALUE1:PP[CLASS:BOATQUANTITY:1TYPE:VEHICLE]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]]
Total time: 4 sec (4313 ms)Syntactic: 125 ms
Conceptual: 4 sec (4188 ms).
Evaluating: "Is an object that is a car and a boat a car?" (ID:CAR25)
-
7/30/2019 Digital Concept Builder
14/74
YES:
DO[ACTION:RECEIVEINPUTMOOD:INTEROGATIVEOBJECT:DO[OPERATION:IS
VALUE1:PP[CLASS:AND[VALUE1:BOAT
VALUE2:CAR]QUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]
VALUE2:PP[CLASS:CARQUANTITY:1TYPE:VEHICLEWHEELCOUNT:4]]]
Total time: 797 msSyntactic: 156 ms
Conceptual: 641 ms.
Done.
The Dictionary
The dictionary that is made available through the zip files is a file,
article_testcases_streams.txt, with partial content cumulated for the sole purpose to provide
the words and parts-of-speech necessary to run the test-cases successfully. That special step
was done in order to save readers of this article from a significant download in size since the
non-abridged dictionary is more than 8 MB in size.
The non-abridged dictionary is available for download byclicking here(2.2 MB). Unzip
the downloaded file and place streams.txtinto theDigitalConceptBuilderdirectory, at thesame level as article_testcases_streams.txt, to have it loaded the next time the program will
be launched.
http://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.ziphttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/FullDictionary.zip -
7/30/2019 Digital Concept Builder
15/74
The text files streams.txtand article_testcases_streams.txtare editable. The content in
streams.txtwas obtained over the years from a variety of sources, some of which are under
copyright as expressed in the "Licensing information.txt" file from the non-abridged
dictionary zip file.
The format used is as follows:
Collapse::
()
Example: December:dusembar:DATE(m_12).
The pronunciation is mandatory, yet is not used for the purpose of the current article. When
the topic of speech recognition is covered in a later article, its use will be necessary.
The text file holding the spellings and parts-of-speech is loaded into a three-way decisiontree, whose code can be found in the IndexStructure template, as described in the article:"Implementing a std::map Replacement that Never Runs Out of Memory and Instructions on
Producing an ARPA Compliant Language Model to Test the Implementation". A
representation of the three-way decision tree can be seen here holding the primary keys
"node", "do", "did", "nesting", "null", and "void", as illustrated below:
Holding the dictionary in such a structure provides the best possible performance whentokenizing, while ensuring that the tokens generated from the process have a corresponding
http://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspxhttp://www.codeproject.com/KB/audio-video/ARPALM.aspx -
7/30/2019 Digital Concept Builder
16/74
entry in the dictionary. Furthermore, the IndexStructure template used can hold data
partially on disk and partially in memory, making it a suitable medium for that purpose.
Tokenization
The goal of tokenization is to create a POSList object that holds spelling and correspondingparts-of-speech for the provided syntactic stream.
Following is the partial output from the first test-case executed with DEFINITIONNEEDED =
FALSE, OUTPUTTOKENS = TRUE with the non-abridged dictionary loaded.
CollapseEvaluating: "Is a red car a car?" (ID:CAR1)
Tokens before syntactic analysis:
[NOUN & "IS"], from 0, to: 1, bridge: 3, index: 0[VERB & "IS"], from 0, to: 1, bridge: 3, index: 0[AUX & "IS"], from 0, to: 1, bridge: 3, index: 0[NOUN & "A"], from 3, to: 3, bridge: 5, index: 1[PREPOSITION & "A"], from 3, to: 3, bridge: 5, index: 1[VERB & "A"], from 3, to: 3, bridge: 5, index: 1[DEFINITE_ARTICLE & "A"], from 3, to: 3, bridge: 5, index: 1[ADJECTIVE & "RED"], from 5, to: 7, bridge: 9, index: 2[NOUN & "RED"], from 5, to: 7, bridge: 9, index: 2[VERB & "RED"], from 5, to: 7, bridge: 9, index: 2[PROPER_NOUN & "RED"], from 5, to: 7, bridge: 9, index: 2[NOUN & "CAR"], from 9, to: 11, bridge: 13, index: 3[PROPER_NOUN & "CAR"], from 9, to: 11, bridge: 13, index: 3[NOUN & "A"], from 13, to: 13, bridge: 15, index: 4[PREPOSITION & "A"], from 13, to: 13, bridge: 15, index: 4[VERB & "A"], from 13, to: 13, bridge: 15, index: 4[DEFINITE_ARTICLE & "A"], from 13, to: 13, bridge: 15, index: 4[NOUN & "CAR"], from 15, to: 17, bridge: 18, index: 5[PROPER_NOUN & "CAR"], from 15, to: 17, bridge: 18, index: 5[PUNCTUATION & "?"], from 18, to: 18, bridge: 0, index: 6
Tokenization can appear as an easy task, but it has hidden difficulties of its own.
Tokenization must allow words to be included in other words (for example, 'all' and
'all-in-one').
While tokenizing, special provisions for numbers handling, which are not part of thedictionary, must be covered.
Allowed punctuation must be taken into account.
Punctuation that is not allowed must be ignored, unless it is part of a dictionary entry.
Any word from the resulting list of words in POSList must be readily accessible
based on its position in the stream and its part of speech to ensure that later processing
will not be negatively affected in regards to performance.
Collapse#ifndef __POSLIST_HH__#define __POSLIST_HH__
#include
-
7/30/2019 Digital Concept Builder
17/74
#include
#include
using std::vector;
using std::map;using std::string;
#include "POSNode.h"
#include "shared_auto_ptr.h"
#include "IndexInStream.h"
class POSList;
class POSList
{ public:
// Constructor, pass duplicateDefense// to true if the POSList should filter out// apparent duplicates.POSList(bool duplicateDefense = false);
// AddToPOSList, call to add a POSNode to the list.virtual POSNode *AddToPOSList(shared_auto_ptr dNode);
// BrigdgeableNodes, call to acquire all POSNode// in the POSList object that respect the conditions// stipulated in dPOSNode at stream index positionvector
BrigdgeableNodes(int position, shared_auto_ptrdPOSNode);
// GetLowestStartPosition, returns the lowest// start position from all POSNode in the POSList.int GetLowestStartPosition();
// GetHighestEndPosition, returns the highest// end position from all POSNode in the POSList.int GetHighestEndPosition();
// AccumulateAll, accumulates all POSNode respecting// the condition stipulated in dPOSNode and sorting// the resulting vector based on sort,// all accumulated nodes need to respect the start position fromPos// (or -1 to ignore) and end position toPos (or -1 to ignore).vector AccumulateAll(
shared_auto_ptr dPOSNode,POSNode::ESortType sort = POSNode::eNoSort,int fromPos = -1, int toPos = -1);
// Output, outputs the POSNode accumulated into// the POSList at position pos (or -1 if the position is// irrelevant).void Output(int pos = -1);
-
7/30/2019 Digital Concept Builder
18/74
// Count, returns how many POSNode are accumulated into thePOSList.
unsigned long Count();
// Clear, removes all POSNode from the POSList.virtual void Clear();
// ResetPOSNodeIteration, resets the tracer// to the first POSNode in the POSList.void ResetPOSNodeIteration();
// GetPositionInList, returns a copy// of the tracer for the current position in stream.virtual IndexInStream *GetPositionInList();
// GetNext, gets the next POSNode from the// tracer's position, returns true if one is available,// false otherwise.virtual bool GetNext(shared_auto_ptr &dPOSNode);
protected:virtual string GetLineOutput(shared_auto_ptr dNode);IndexInStream m_position;int m_lowestStart;int m_highestEnd;bool m_duplicateDefense;map m_uniqueEntries;int m_count;
};
#endif
In order to have a quick retrieval mechanism, the POSList class uses the m_position
member that is an IndexInStream instance. The IndexInStream template is usedin some cases when quick retrieval of an object, in our case, a POSNode, is needed based on a
position in the stream and a POSEntry type. The implementation of the IndexInStream
template follows:
Collapse#ifndef __INDEXSINSTREAM_H__#define __INDEXSINSTREAM_H__
#include
#include
using std::map;using std::vector;
#include "POSEntry.h"
template class IndexInStreamPosition{
public:IndexInStreamPosition();void Add(shared_auto_ptr dContent, POSEntry dPosEntry);
typedef typename mapcontainer_map_vector_type;
-
7/30/2019 Digital Concept Builder
19/74
typedef typenamecontainer_map_vector_type::iterator iterator_map_vector_type;
container_map_vector_type m_content;};
template IndexInStreamPosition::IndexInStreamPosition() {}
template void IndexInStreamPosition::Add(shared_auto_ptr dContent, POSEntry dPosEntry)
{m_content[dPosEntry.GetValue()].push_back(dContent);
}
template class IndexInStream{
public:IndexInStream();void Reset();void Clear();
void Add(shared_auto_ptr dObject, int position, POSEntry dPos);bool GetNext(shared_auto_ptr &dPOSNode);vector ObjectsAtPosition(int position,
POSEntry dPOSEntry, int *wildcardPosition = NULL);protected:
typedef typename mapcontainer_map_index_type;
typedef typenamecontainer_map_index_type::iterator iterator_map_index_type;
typedef typename map container_map_vector_type;
typedef typename container_map_vector_type::iteratoriterator_map_vector_type;
container_map_index_type m_allSameEventPOSList;iterator_map_index_type m_iterator1;iterator_map_vector_type m_iterator2;int m_posVectorEntryIndex;
};
template IndexInStream::IndexInStream():m_posVectorEntryIndex(-1){
Reset();m_allSameEventPOSList.clear();
}
template void IndexInStream::Clear(){
Reset();}
template vectorIndexInStream::ObjectsAtPosition(int position,POSEntry dPOSEntry, int *wildcardPosition)
{vector dReturn;if (m_allSameEventPOSList.find(position) !=
m_allSameEventPOSList.end())
{
-
7/30/2019 Digital Concept Builder
20/74
if (m_allSameEventPOSList[position]->m_content.find(dPOSEntry.GetValue()) !=
m_allSameEventPOSList[position]->m_content.end()){
dReturn =m_allSameEventPOSList[position]-
>m_content[dPOSEntry.GetValue()];}}if (wildcardPosition != NULL){
vector temp =ObjectsAtPosition(*wildcardPosition, dPOSEntry);
for (unsigned int i = 0; i < temp.size(); i++){
dReturn.push_back(temp[i]);}
}return dReturn;
}
template bool IndexInStream::GetNext(shared_auto_ptr&dObject){
while (m_iterator1 != m_allSameEventPOSList.end()){
while ((m_posVectorEntryIndex != -1) &&(m_iterator2 != m_iterator1->second.get()->m_content.end()))
{if (m_posVectorEntryIndex < (int)m_iterator2->second.size()){
dObject = m_iterator2->second[m_posVectorEntryIndex];m_posVectorEntryIndex++;return true;
}else{
m_iterator2++;m_posVectorEntryIndex = 0;
}}m_iterator1++;m_posVectorEntryIndex = 0;if (m_iterator1 != m_allSameEventPOSList.end()){
m_iterator2 = m_iterator1->second.get()->m_content.begin();}else{
m_posVectorEntryIndex = -1;}
}Reset();return false;
}
template void IndexInStream::Add(shared_auto_ptr dObject, int position, POSEntry dPos)
{
if (m_allSameEventPOSList.find(position) ==m_allSameEventPOSList.end())
-
7/30/2019 Digital Concept Builder
21/74
{m_allSameEventPOSList[position] =
shared_auto_ptr(new IndexInStreamPosition());
}m_allSameEventPOSList[position]->Add(dObject, dPos);
}
template void IndexInStream::Reset(){
m_iterator1 = m_allSameEventPOSList.begin();m_posVectorEntryIndex = 0;if (m_iterator1 != m_allSameEventPOSList.end()){
m_iterator2 = m_iterator1->second.get()->m_content.begin();}else{
m_posVectorEntryIndex = -1;
}}
#endif
With the POSList class implemented, the tokenization is implemented as follows:
Collapseshared_auto_ptr DigitalConceptBuilder::Tokenize(string dContent,
string posNumbers,string posPunctuation,string punctuationAllowed,bool definitionNeeded)
{struct TokenizationPath{
TokenizationPath(shared_auto_ptr
dPosition,unsigned int dStartIndex): m_position(dPosition),
m_startIndex(dStartIndex) {}shared_auto_ptr
m_position;unsigned int m_startIndex;
};unsigned int dWordIndex = 0;shared_auto_ptr dReturn = shared_auto_ptr(new
POSList(true));string dNumberBuffer;vector activePaths;vector floatingBridges;int latestBridge = -1;for (unsigned int i = 0; i
-
7/30/2019 Digital Concept Builder
22/74
{isAllowedPunctuation = true;latestBridge = i;
}}activePaths.push_back(TokenizationPath(
shared_auto_ptr(new IndexStructureNodePosition(m_POS_Dictionary.GetTopNode())), i));
for (unsigned int j = 0; j < activePaths.size(); j++){
if ((activePaths[j].m_position.get() != NULL) &&(activePaths[j].m_position->get() != NULL) &&(activePaths[j].m_position->get()->m_data != NULL))
{if ((i == dContent.length()) ||
(IsDelimitor(dContent.c_str()[i]))){
string dKey = activePaths[j].m_position->GetNode()-
>GetKey(); if ((i < dKey.length()) ||(IsDelimitor(dContent.c_str()[i - dKey.length() -
1]))){
StoredPOSNode *dPOS = activePaths[j].m_position->get();
for (int k = 0; k < kMAXPOSALLOWED; k++){
if (dPOS->m_pos[k] != -1){
shared_auto_ptr candidate =
POSNode::Construct("["+POSEntry::StatGetDescriptor(dPOS->m_pos[k]) + " & \"" + dKey + "\"]",
NULL,activePaths[j].m_startIndex, i-1, 0,(dPOS->m_data[k] != -1)?m_data[dPOS-
>m_data[k]]:"");if (PassedDefinitionRequirement(candidate,
definitionNeeded)){
POSNode *dNewNode = dReturn->AddToPOSList(candidate);
if (dNewNode != NULL){
floatingBridges.push_back(dNewNode);}latestBridge =
activePaths[j].m_startIndex;}
}}
}}
}if ((latestBridge != -1) && (((i < dContent.length()) &&
(!IsDelimitor(dContent.c_str()[i]))) ||(i == dContent.length()) || (isAllowedPunctuation)))
{bool atLeastOneAdded = false;
-
7/30/2019 Digital Concept Builder
23/74
for (int l = (floatingBridges.size() - 1); l >= 0; l--){
if ((floatingBridges[l]->GetBridgePosition() == 0) &&(floatingBridges[l]->GetStartPosition() !=
latestBridge)){
atLeastOneAdded = true;floatingBridges[l]->SetWordIndex(dWordIndex);floatingBridges[l]-
>SetBridgePosition(latestBridge);floatingBridges.erase(floatingBridges.begin() + l);
}}if (atLeastOneAdded){
dWordIndex++;}if (isAllowedPunctuation){
shared_auto_ptr candidate =POSNode::Construct("[" + posPunctuation + " & \"" +dCharStr + "\"]", NULL, i, i, 0);
if (PassedDefinitionRequirement(candidate,definitionNeeded))
{POSNode *dNewNode = dReturn-
>AddToPOSList(candidate);if (dNewNode != NULL){
floatingBridges.push_back(dNewNode);}latestBridge = i;
}}else{
latestBridge = -1;}
}if (i == dContent.length()){
break;}shared_auto_ptrdNewPosition = m_POS_Dictionary.ForwardNodeOneChar(activePaths[j].m_position, toupper(dContent.c_str()[i]));
if (dNewPosition.get() != NULL){
activePaths[j].m_position = dNewPosition;}else{
activePaths[j].m_position->Clear();}
}if ((posNumbers.length() > 0) && ((dContent.c_str()[i] >= '0') &&
(dContent.c_str()[i] 0))))
{
if ((i == 0) || (dNumberBuffer.length() > 0) ||(IsDelimitor(dContent.c_str()[i-1])))
-
7/30/2019 Digital Concept Builder
24/74
{dNumberBuffer += dContent.c_str()[i];
}}else if (dNumberBuffer.length() > 0){
shared_auto_ptr candidate =POSNode::Construct("["+posNumbers + " & \"" +dNumberBuffer + "\"]", NULL, i - dNumberBuffer.length(), i -
1, 0);if (PassedDefinitionRequirement(candidate, definitionNeeded)){
POSNode *dNewNode = dReturn->AddToPOSList(candidate);if (dNewNode != NULL){
floatingBridges.push_back(dNewNode);}latestBridge = i - dNumberBuffer.length();dNumberBuffer = "";
}}for (int j = (activePaths.size() - 1); j >= 0; j--){
if ((activePaths[j].m_position.get() == NULL) ||(activePaths[j].m_position->GetNode().get() == NULL))
{activePaths.erase(activePaths.begin() + j);
}}
}for (int l = (floatingBridges.size() - 1); l >= 0; l--){
floatingBridges[l]->SetWordIndex(dWordIndex);}return dReturn;
}
Syntactic Analysis
The purpose of Syntactic Analysis is to produce complex nodes from the atomic nodes passed
in the POSList and to identify the targeted complex nodes to provide to the Conceptual
Analysis process. In a CLUE, Syntactic Analysis is not the final disambiguator; rather, theConceptual Analysis process working in conjunction with the Syntactic Analysis process
shall determine which concept prevails over the others based on meaning and syntactic
integrity. Consequently, there is no requirement to fully disambiguate during the Syntactic
Analysis process, meaning that Syntactic Analysis produces a multitude of syntactic
organizations that later need to be disambiguated by Conceptual Analysis. Prior to Syntactic
Analysis, there is a lot of ambiguity because the process only holds a list ofPOSNodes that
each have associated parts-of-speech; following Syntactic Analysis, there is less ambiguity
because targeted parts-of-speech have been identified and associated with their corresponding
sequences of words and parts-of-speech required to build them. Syntactic Analysis is also
useful in providing Conceptual Analysis with syntactic information to rely upon in predicate
calculation. As is later exposed in this article, a Predicate Builder Script is composed of code
that mostly relates to syntax, and making the transition from a syntactic stream to concepts
relies heavily on syntactic information produced during Syntactic Analysis.
-
7/30/2019 Digital Concept Builder
25/74
The Syntactic Transform Script stored in SyntaxTransform.txtis central to Syntactic
Analysis. The content of that file holds sequencing decisions used to build complex nodes
from a configuration of complex nodes and atomic nodes found in the dictionary. The
Syntactic Transform Script is composed of about 50 lines of code of a language that is
created for the sole purpose of permuting nodes.
A closer look into the first three lines of code from SyntaxTransform.txthelps in
understanding that language.
CollapseADJECTIVE PHRASE CONSTRUCTION 1: ([ADVERB])[ADJECTIVE] -> ADJECTIVE_PHRASEMAXRECURSIVITY:2ADJECTIVE PHRASE ENUMERATION: [ADJECTIVE_PHRASE]([CONJUNCTION])
[ADJECTIVE_PHRASE] -> ADJECTIVE_PHRASE
# Verbs
MAXRECURSIVITY:1COMPLEX VERB CONSTRUCTION: [VERB & "is" | "was" | "will" | "have" |
"has" | "to" | "will be" | "have been" |"has been" | "to be" | "will have been" |"be" | "would" | "could" |
"should"]([ADVERB])[VERB] -> VERB
Lines have been wrapped in the above snippet to avoid scrolling.
The first line permutes all possibilities from ([ADVERB])[ADJECTIVE] and creates a resulting
node that is a part-of-speech, ADJECTIVE_PHRASE. For example, tokens such as "more red"
result in an ADJECTIVE_PHRASE since "more" is an ADVERB and "red" is an ADJECTIVE. But
since the ADVERB node is between parentheses, it is identified as being optional.Consequently, the ADJECTIVE token "blue" also results in an ADJECTIVE_PHRASE node.
The following line, [ADJECTIVE_PHRASE]([CONJUNCTION])[ADJECTIVE_PHRASE] , takes
sequences ofADJECTIVE_PHRASE nodes, optionally separated by a CONJUNCTION node, and
creates a new ADJECTIVE_PHRASE node with them. Such a script line is recursive since it
transforms into a part-of-speech that is part of its sequencing. To that effect, in order to limit
computing to a reasonable level of parsing, we may want to limit recursion as it is done on
the preceding line: MAXRECURSIVITY:2 . That basically states that only two successful passesat this transformation line are allowed. That means that tokens such as "more blue and less
green" are transformed successfully, while tokens such as "red, some green and grey" are nottransformed successfully since a recursion level of at least 3 is required for that transform to
happen. Note that recursion limitations are only relevant while performing bottom-up parsing
and not multi-dimensional Earley parsing. More on that later...
The next line, [VERB & "is" | "was" | "will" | "have" | "has" | "to" | "willbe" | "have been" | "has been" | "to be" | "will have been" | "be" |
"would" | "could" | "should"]([ADVERB])[VERB], has comparable rules, but also states
some conditions in regards to spellings for the first node. The components between double-
quotes are spelling conditions, where at least one of which must succeed for the node match
to occur. From that line of code, a successful transform happens for the token sequence:
"could never see", but fails for the token sequence: "see always ear".
-
7/30/2019 Digital Concept Builder
26/74
Here is the complete script. It encapsulates most of the English language, although slight
adaptations may be required if more complex test cases are not transformed as expected.
Collapse# NOTES ON THE SYNTAX OF THE SCRIPTING LANGUAGE
# - BEFORE THE ':' CHARACTER ON A LINE IS THE LINE NAME# - A NODE BETWEEN PARENTHESIS IS INTERPREATED AS BEING OPTIONAL# - CONTENT BETWEEN QUOTES RELATES TO SPELLING# - SPELLINGS THAT BEGIN WITH A '*' CHARACTER ARE INTERPREATED#- AS A 'START WITH' STRING MATCH# - ON THE RIGHT SIDE OF THE CHARACTERS '->'# - IS THE DEFINITION OF THE NEW ENTITY (AFFECTATION)
# SCRIPT
# Adjective phrase construction
ADJECTIVE PHRASE CONSTRUCTION 1: ([ADVERB])[ADJECTIVE] -> ADJECTIVE_PHRASEMAXRECURSIVITY:2ADJECTIVE PHRASE ENUMERATION: [ADJECTIVE_PHRASE]([CONJUNCTION])
[ADJECTIVE_PHRASE] -> ADJECTIVE_PHRASE
# Verbs
MAXRECURSIVITY:1COMPLEX VERB CONSTRUCTION: [VERB & "is" | "was" | "will" | "have" |
"has" | "to" | "will be" | "have been" |"has been" | "to be" | "will have been"
|"be" | "would" | "could" |"should"]([ADVERB])[VERB] -> VERB
# Noun phrase construction
GERUNDIVE ING: [VERB & "*ing"] -> GERUNDIVE_VERBGERUNDIVE ED: [VERB & "*ed"] -> GERUNDIVE_VERBPLAIN NOUN PHRASE CONSTRUCTION: ([DEFINITE_ARTICLE |INDEFINITE_ARTICLE])
([ORDINAL_NUMBER])([CARDINAL_NUMBER])([ADJECTIVE_PHRASE])[NOUN | PLURAL |
PROPER_NOUN | TIME |DATE | PRONOUN] -> NOUN_PHRASE
MAXRECURSIVITY:2
NOUN PHRASE ENUMERATION: [NOUN_PHRASE]([CONJUNCTION])[NOUN_PHRASE] ->NOUN_PHRASEMAXRECURSIVITY:1
# Preposition phrase construction
PREPOSITION PHRASE CONSTRUCTION: [PREPOSITION][NOUN_PHRASE] ->PREPOSITION_PHRASEMAXRECURSIVITY:2PREPOSITION PHRASE ENUMERATION: [PREPOSITION_PHRASE]([CONJUNCTION])
[PREPOSITION_PHRASE] ->PREPOSITION_PHRASE
# Verb phrase construction
-
7/30/2019 Digital Concept Builder
27/74
VERB PHRASE CONSTRUCTION 1: [VERB]([ADVERB])[NOUN_PHRASE]([PREPOSITION_PHRASE]) -> VERB_PHRASE
VERB PHRASE CONSTRUCTION 2: [VERB][PREPOSITION_PHRASE] ->VERB_PHRASEVERB PHRASE CONSTRUCTION 3: [ADJECTIVE_PHRASE][PREPOSITION][VERB] ->VERB_PHRASE
# Noun phrase construction while considering gerundive
GERUNDIVE PHRASE CONSTRUCTION: [GERUNDIVE_VERB]([NOUN_PHRASE])([VERB_PHRASE])([ADVERB]) ->
GERUNDIVE_PHRASEMAXRECURSIVITY:2NOUN PHRASE CONST WITH GERUNDIVE: [NOUN_PHRASE][GERUNDIVE_PHRASE]
([GERUNDIVE_PHRASE])([GERUNDIVE_PHRASE]) -> NOUN_PHRASEPREPOSITION PHRASE CONSTRUCTION 3: [PREPOSITION][GERUNDIVE_PHRASE] ->PREPOSITION_PHRASE
# Noun phrase construction while considering restrictive relative clauses
RESTRICTIVE RELATIVE CLAUSE: [WH_PRONOUN & "who" | "where" | "when"|
"which"][VERB_PHRASE] -> REL_CLAUSERESTRICTIVE RELATIVE CLAUSE 2: [PRONOUN & "that"][VERB_PHRASE] ->REL_CLAUSEMAXRECURSIVITY:2NOUN PHRASE WITH REL_CLAUSE: [NOUN_PHRASE][REL_CLAUSE] ->NOUN_PHRASE
# Make sure the restrictive relative clauses built are part of the verbphrases
VERB PHRASE WITH REL_CLAUSE: [VERB_PHRASE][REL_CLAUSE] -> VERB_PHRASEVERB PHRASE CONSTRUCTION 4: [VERB][NOUN_PHRASE][REL_CLAUSE]
([PREPOSITION_PHRASE]) -> VERB_PHRASEMAXRECURSIVITY:2WH_PRONOUN CONSTRUCTION ENUMERATION: [WH_PRONOUN][CONJUNCTION][WH_PRONOUN]-> WH_PRONOUN
# Make sure the gerundive built are part of the verb phrases
VERB PHRASE CONSTRUCTION 5: [VERB][NOUN_PHRASE][GERUNDIVE_PHRASE]([GERUNDIVE_PHRASE])([GERUNDIVE_PHRASE])([PREPOSITION_PHRASE]) -> VERB_PHRASE
VERB PHRASE CONSTRUCTION 6: [VERB][NOUN_PHRASE][ADJECTIVE_PHRASE] ->VERB_PHRASEVERB PHRASE CONSTRUCTION 7: ([VERB])[NOUN_PHRASE][VERB] ->VERB_PHRASEMAXRECURSIVITY:2VERB PHRASE CONSTRUCTION 8:[VERB_PHRASE][NOUN_PHRASE][GERUNDIVE_PHRASE]
([GERUNDIVE_PHRASE])([GERUNDIVE_PHRASE])([PREPOSITION_PHRASE]) -> VERB_PHRASE
MAXRECURSIVITY:2VERB PHRASE CONSTRUCTION 9: [VERB_PHRASE]([NOUN_PHRASE])
[ADJECTIVE_PHRASE] -> VERB_PHRASEMAXRECURSIVITY:2VERB PHRASE CONSTRUCTION 10: [WH_PRONOUN][VERB_PHRASE] ->
VERB_PHRASEMAXRECURSIVITY:2
-
7/30/2019 Digital Concept Builder
28/74
VERB PHRASE CONSTRUCTION 11: [VERB_PHRASE][NOUN_PHRASE]([PREPOSITION_PHRASE |GERUNDIVE_PHRASE]) -> VERB_PHRASE
MAXRECURSIVITY:2VERB PHRASE CONSTRUCTION 12: [VERB_PHRASE][PREPOSITION_PHRASE] ->VERB_PHRASE
# WH-Phrases construction
WH_NP CONSTRUCTION 1: [WH_PRONOUN][NOUN_PHRASE] -> WH_NPWH_NP CONSTRUCTION 2: [WH_PRONOUN][ADJECTIVE]([ADVERB]) -> WH_NPWH_NP CONSTRUCTION 3: [WH_PRONOUN][ADVERB][ADJECTIVE] -> WH_NPMAXRECURSIVITY:2WH_NP CONSTRUCTION 4: [WH_NP][CONJUNCTION][WH_NP | WH_PRONOUN] -> WH_NP
# Sentence construction
SENTENCE CONSTRUCTION QUESTION 1: [VERB & "is" | "was" | "were"]
[NOUN_PHRASE][NOUN_PHRASE]([PUNCTUATION & "?"]) -> SENTENCESENTENCE CONSTRUCTION QUESTION 2: [VERB & "is" | "was" | "were"]
[VERB_PHRASE][VERB_PHRASE]([PUNCTUATION & "?"]) -> SENTENCE
SENTENCE CONSTRUCTION 1: [VERB_PHRASE]([PREPOSITION & "at" | "in" |"of" |
"on" | "for" | "into" | "from"])([PUNCTUATION & "?"]) -> SENTENCE
SENTENCE CONSTRUCTION 2: ([AUX])[NOUN_PHRASE][VERB_PHRASE | VERB]([PREPOSITION & "at" | "in" | "of" |"on" | "for"])([ADVERB])([PUNCTUATION &
"?"]) -> SENTENCEWH_NP SENTENCE CONSTRUCTION 1: [WH_NP][VERB_PHRASE]([PREPOSITION &"at" |
"in" | "of" | "on" | "for" | "into"|
"from"])([PUNCTUATION & "?"]) ->SENTENCEWH_NP SENTENCE CONSTRUCTION 2:[WH_NP]([AUX])[NOUN_PHRASE][VERB_PHRASE | VERB](
[PREPOSITION & "at" | "in" | "of" |"on" | "for"])
([ADVERB])([PUNCTUATION & "?"]) ->SENTENCEWH_NP SENTENCE CONSTRUCTION 3: [NOUN_PHRASE |VERB_PHRASE]([PREPOSITION &
"at" | "in" | "of" | "on" | "for" |"into" |
"from"])[WH_NP]([PUNCTUATION & "?"])-> SENTENCEMAXRECURSIVITY:2SENTENCE CONSTRUCTION 4: [SENTENCE]([CONJUNCTION])[SENTENCE] ->SENTENCE
Lines have been wrapped in the above snippet to avoid scrolling.
One goal for the Syntactic Transform Script is to create [SENTENCE] complex nodes.
Sentences are special as they are expected to encapsulate a complete thread of thoughts that
can later be represented by a predicate. Although predicates can also be calculated for other
-
7/30/2019 Digital Concept Builder
29/74
complex nodes, only [SENTENCE] parts-of-speech can reliably fully be encapsulated into a
predicate. That does not mean the [SENTENCE] part-of-speech is necessarily self-contained,
though. For example, think about the two following [SENTENCE]s: "I saw Edith yesterday.
She is feeling great." The object of knowledge "she" in the second [SENTENCE] refers to the
object of knowledge "Edith" from the first [SENTENCE]. For the purpose of this sample
project, although it is possible to do so, we will not keep a context between different[SENTENCE] nodes.
Permuting the Syntactic Transform Script
One of the first things done by the syntactic analyzer is to create a decision-free version of
the Syntactic Transform Script. That is, it creates a version of the Syntactic Transform Script
into POSDictionary_flat.txtthat is functionally equivalent to SyntaxTransform.txtthat does
not hold any decision nodes (nodes between parenthesis in SyntaxTransform.txt, or different
spellings conditions that are delimited by the pipe character within a node). The content of
POSDictionary_flat.txt, as generated by the downloadable executable attached to this article,
can be referred to byclicking here. In order to do that, it performs two distinct steps intoPOSTransformScript::ManageSyntacticDecisions , where the parameter decisionFile
is the string holding the name of the Syntax Transform Script.
Collapsevoid POSTransformScript::ManageSyntacticDecisions(string decisionFile,
string dFlatFileName){
FlattenDecisions(decisionFile, "temp.txt");FlattenDecisionNodes("temp.txt", dFlatFileName);
}
During the first step (into POSTransformScript::FlattenDecisions ), each line of the fileis read, and each element between parenthesis is removed from parenthesis in one line and
ignored into another. Since an element between parenthesis is optional, there are indeed only
two possibilities - it is included or not.
Collapsevoid POSTransformScript::FlattenDecisions(string dDecisionFileName,
string dFlattenDecisionFileName){
fstream dOutputStream;dOutputStream.open(dFlattenDecisionFileName.c_str(),
ios::out | ios::binary | ios::trunc);ifstream dInputStream(dDecisionFileName.c_str(), ios::in |ios::binary);
dInputStream.unsetf(ios::skipws);string dBuffer;char dChar;while (dInputStream >> dChar){
switch (dChar){
case '\n':case '\r':
if (dBuffer.find("#") != string::npos)
{ dBuffer = SubstringUpTo(dBuffer, "#");}
http://www.codeproject.com/KB/recipes/DigitalConceptBuilder/POSDictionary_flat.txthttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/POSDictionary_flat.txthttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/POSDictionary_flat.txthttp://www.codeproject.com/KB/recipes/DigitalConceptBuilder/POSDictionary_flat.txt -
7/30/2019 Digital Concept Builder
30/74
if (dBuffer.length() > 0){
unsigned int i;vector *matches;Permutation perms(dBuffer);matches = perms.GetResult();
for (i = 0; i < matches->size(); i++){dOutputStream.write(
matches->at(i).c_str(), matches->at(i).length());dOutputStream.write("\r\n", strlen("\r\n"));
}}dBuffer = "";break;
default:dBuffer += dChar;
}}
dInputStream.close();dOutputStream.close();}
To help the calculation of all possible permutations of a single line, the Permutation class isused. Since there may be more than one permutation on a line, the use of the recursive
Permutation::FillResult method is the best approach to perform these calculations.
Collapsevoid Permutation::FillResult(){
m_result.clear();
vector matches;if (PatternMatch(m_value, "(*)", &matches, true)){
string tmpString = m_value;size_t pos = tmpString.find("(" + matches[0] + ")");if (SearchAndReplace(tmpString, "(" + matches[0] +
")", "", 1, true) == 1){
Permutation otherPerms(tmpString);vector *otherMatches = otherPerms.GetResult();for (unsigned int i = 0; i < otherMatches->size(); i++){
// We push the value with
// the potential value out and then in...m_result.push_back(otherMatches->at(i));m_result.push_back(m_value.substr(0, pos) +
matches[0] + otherMatches->at(i).substr(pos));}
}}else{
m_result.push_back(m_value);}
}
Once this first step of taking care of content between parenthesis is done, decisions withinatomic nodes need to be handled. An atomic node with a decision is like
-
7/30/2019 Digital Concept Builder
31/74
[DEFINITE_ARTICLE | INDEFINITE_ARTICLE], where the part-of-speech is a choice, or
[WH_PRONOUN & "who" | "where" | "when" | "which"], where the spelling constraint is
also a choice. Most of the logic is in POSTransformScript::FlattenOneDecisionNodes ,
which itself relies heavily on POSNode::Construct for parsing.
Collapsevector POSTransformScript::FlattenOneDecisionNodes(string dLine){
vector dReturn;vector matches;if (PatternMatch(dLine, "[*]", &matches, true) > 0){
for (unsigned int i = 0; i < matches.size(); i++){
vector possibilities;POSNode::Construct("[" + matches[i] + "]", &possibilities);if (possibilities.size() > 1){
for (unsigned int j = 0; j < possibilities.size(); j++){
string dLineCopy = dLine;SearchAndReplace(dLineCopy, "[" + matches[i] + "]",
possibilities[j].get()->GetNodeDesc(),1);
vector dNewLineCopies =FlattenOneDecisionNodes(dLineCopy);
for (unsigned int k = 0; k < dNewLineCopies.size();k++)
{dReturn.push_back(dNewLineCopies[k]);
}
}break;
}}
}if (dReturn.size() == 0){
dReturn.push_back(dLine);}return dReturn;
}
shared_auto_ptr POSNode::Construct(string dNodeDesc,
vector *allPOSNodes,unsigned int dStartPosition,unsigned int dEndPosition,unsigned int dBridgePosition,string data)
{vector tmpVector;if (allPOSNodes == NULL){
allPOSNodes = &tmpVector;}vector matches;vector spellings;
vector dPOS;string dLeftPart = "";if (PatternMatch(dNodeDesc, "[* & *]", &matches, false) == 2)
-
7/30/2019 Digital Concept Builder
32/74
{dLeftPart = matches[0];string dRightPart = matches[1];if (PatternMatch(dRightPart, "\"*\"", &matches, true, "|") > 0){
for (unsigned int i = 0; i < matches.size(); i++)
{ spellings.push_back(matches[i]);}
top related