{"id":1502,"date":"2016-07-28T12:31:15","date_gmt":"2016-07-28T12:31:15","guid":{"rendered":"http:\/\/silke.warmstal.nl\/?p=1502"},"modified":"2020-01-17T15:55:56","modified_gmt":"2020-01-17T15:55:56","slug":"creating-a-sophisticated-qa-system","status":"publish","type":"post","link":"https:\/\/silketerstal.nl\/wp\/2016\/07\/28\/creating-a-sophisticated-qa-system\/","title":{"rendered":"An Intelligent Q&#038;A System"},"content":{"rendered":"\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-7i89cc-ee53c9a159bd807e0aa42d768675847b\">\n.avia-section.av-7i89cc-ee53c9a159bd807e0aa42d768675847b{\nbackground-color:#f8f8f8;\n}\n.avia-section.av-7i89cc-ee53c9a159bd807e0aa42d768675847b .av-extra-border-element .av-extra-border-inner{\nbackground-color:#ffffff;\n}\n<\/style>\n<div id='av_section_1'  class='avia-section av-7i89cc-ee53c9a159bd807e0aa42d768675847b main_color avia-section-huge avia-no-border-styling  avia-builder-el-0  el_before_av_section  avia-builder-el-first  avia-bg-style-scroll container_wrap fullsize'  ><div class='container av-section-cont-open' ><main  role=\"main\" itemprop=\"mainContentOfPage\"  class='template-page content  av-content-full alpha units'><div class='post-entry post-entry-type-page post-entry-1502'><div class='entry-content-wrapper clearfix'>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-5kz498-4a7c4280a5c39a61766ad0604bca5944\">\n.flex_column.av-5kz498-4a7c4280a5c39a61766ad0604bca5944{\nborder-radius:0px 0px 0px 0px;\n-webkit-border-radius:0px 0px 0px 0px;\n-moz-border-radius:0px 0px 0px 0px;\npadding:0px 5% 5% 0px;\n}\n<\/style>\n<div class='flex_column av-5kz498-4a7c4280a5c39a61766ad0604bca5944 av_two_third  avia-builder-el-1  avia-builder-el-no-sibling  first flex_column_div '   ><section  class='av_textblock_section av-k5864p0a-3cd90e3fc8a8952b8196fc7195f8b3b8'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><div class='avia_textblock'  itemprop=\"text\" ><h1>An Intelligent Q&#038;A System<\/h1>\n<\/div><\/section><br \/>\n<section  class='av_textblock_section av-k55gx2oe-e67bcecb2b6ca0c5479c0c2f0577b836'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><div class='avia_textblock'  itemprop=\"text\" ><p>The goal of this project was to improve the performance of a question-answer (QA) system using the VH Toolkit. This project was made in a group. \u00a0The VH Toolkit was used to create a QA system in the first part of this course, resulting in very poor performance. First of all, we decided to focus on a closed-domain system to narrow down the types of questions asked. The topic chosen was the Eiffeltower, since the highlight is known by many people. Three different QA systems were compared. To gather input, some people were asked to write down 15 questions they would like to ask a tourist guide about the Eiffeltower, the answers on these questions were found online. \u00a0The first system used was a basic QA system created by adding questions, answers and some variations to the VH toolkit.<\/p>\n<p>The second and third system were both self-made, for these systems the database consisted of a tree structure of main topics, sub topics and a corresponding answer. The user could ask his questions by typing in a GUI. The basic self-made QA system made use of normalisation techniques for the input string, such as removing random symbols and punctuation, converting all words to lower case, deleting abbreviations and converting numbers to words. It also included a spellchecker, which used the Levenshtein distance and an English dictionary. The advanced self-made QA system added lexical, syntactic and sematic analysis. First, a part of speech tagger (POS) tagged all words. Afterwards, less important words, such as \u201cthe\u201d, were removed, to remain only important words. \u201cW-words\u201d, such as \u201cwhy\u201d and \u201cwhere\u201d were directly mapped to the keywords \u201creason\u201d and \u201cplace\u201d of the database. A similarity matcher, using WordNet, matched the remaining words to the keywords in the database. As an example, \u201chigh\u201d was matched to \u201csize\u201d. Finally, the answering unit used the keywords of the question as input and returned the answer if the keywords of the input string were found in the database.<\/p>\n<p>The program was written in openFrameworks with C++. The WordNet functionality only ran in Python, so the system combined Python with C++. Evalution of the three systems showed that both self-made systems outperformed the VH Toolkit. Especially normalisation improved performance, but also the similarity matching had influence.<\/p>\n<\/div><\/section><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-4c13ny-449accaafc019512fc4be70a30686133\">\n#top .av-special-heading.av-4c13ny-449accaafc019512fc4be70a30686133{\npadding-bottom:10px;\n}\nbody .av-special-heading.av-4c13ny-449accaafc019512fc4be70a30686133 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-4c13ny-449accaafc019512fc4be70a30686133 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-4c13ny-449accaafc019512fc4be70a30686133 av-special-heading-h3  avia-builder-el-4  el_after_av_textblock  el_before_av_textblock '><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Tools used<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n<section  class='av_textblock_section av-k584p6tl-743feba043773adf044c475250ec6f77'  itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/BlogPosting\" itemprop=\"blogPost\" ><div class='avia_textblock'  itemprop=\"text\" ><p class=\"skills\">C++ with openFrameworks<\/p>\n<p class=\"skills\">Python with WordNet<\/p>\n<\/div><\/section><\/p><\/div>\n\n<\/div><\/div><\/main><!-- close content main element --><\/div><div class='av-extra-border-element border-extra-diagonal  diagonal-box-shadow'><div class='av-extra-border-outer'><div class='av-extra-border-inner'><\/div><\/div><\/div><\/div>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-k583tzyz-31c5b7a5e8a324805adb34613a298eab\">\n.avia-section.av-k583tzyz-31c5b7a5e8a324805adb34613a298eab{\nbackground-color:#ffffff;\n}\n.avia-section.av-k583tzyz-31c5b7a5e8a324805adb34613a298eab .av-extra-border-element .av-extra-border-inner{\nbackground-color:#333333;\n}\n<\/style>\n<div id='av_section_2'  class='avia-section av-k583tzyz-31c5b7a5e8a324805adb34613a298eab main_color avia-section-default avia-no-border-styling  avia-builder-el-6  el_after_av_section  avia-builder-el-last  avia-bg-style-scroll container_wrap fullsize'  ><div class='container av-section-cont-open' ><div class='template-page content  av-content-full alpha units'><div class='post-entry post-entry-type-page post-entry-1502'><div class='entry-content-wrapper clearfix'>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-4ylfmk-db1164092d3be1253ac5c7f2b3c89dd7\">\n.flex_column.av-4ylfmk-db1164092d3be1253ac5c7f2b3c89dd7{\nborder-radius:0px 0px 0px 0px;\n-webkit-border-radius:0px 0px 0px 0px;\n-moz-border-radius:0px 0px 0px 0px;\n}\n<\/style>\n<div class='flex_column av-4ylfmk-db1164092d3be1253ac5c7f2b3c89dd7 av_one_third  avia-builder-el-7  avia-builder-el-no-sibling  first flex_column_div av-zero-column-padding '   ><p>\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-av_heading-fa9de369834c9ef30ec7beaa6790c434\">\n#top .av-special-heading.av-av_heading-fa9de369834c9ef30ec7beaa6790c434{\npadding-bottom:10px;\n}\nbody .av-special-heading.av-av_heading-fa9de369834c9ef30ec7beaa6790c434 .av-special-heading-tag .heading-char{\nfont-size:25px;\n}\n.av-special-heading.av-av_heading-fa9de369834c9ef30ec7beaa6790c434 .av-subheading{\nfont-size:15px;\n}\n<\/style>\n<div  class='av-special-heading av-av_heading-fa9de369834c9ef30ec7beaa6790c434 av-special-heading-h3  avia-builder-el-8  el_before_av_gallery  avia-builder-el-first '><h3 class='av-special-heading-tag'  itemprop=\"headline\"  >Photos<\/h3><div class=\"special-heading-border\"><div class=\"special-heading-inner-border\"><\/div><\/div><\/div><br \/>\n\n<style type=\"text\/css\" data-created_by=\"avia_inline_auto\" id=\"style-css-av-k55hbep5-d284a47c2ecf556f2a5111a22b1ec8af\">\n#top .avia-gallery.av-k55hbep5-d284a47c2ecf556f2a5111a22b1ec8af .avia-gallery-thumb a{\nwidth:20%;\n}\n<\/style>\n<div  class='avia-gallery av-k55hbep5-d284a47c2ecf556f2a5111a22b1ec8af avia_animate_when_visible  avia-builder-el-9  el_after_av_heading  avia-builder-el-last  avia_lazyload avia-gallery-animate avia-gallery-1'  itemprop=\"image\" itemscope=\"itemscope\" itemtype=\"https:\/\/schema.org\/ImageObject\" ><meta itemprop=\"contentURL\" content=\"https:\/\/silketerstal.nl\/wp\/2016\/07\/28\/creating-a-sophisticated-qa-system\/\"><div class='avia-gallery-thumb'><a href=\"https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching.jpg\" data-srcset=\"https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching.jpg 432w, https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching-289x300.jpg 289w, https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching-36x36.jpg 36w\" data-sizes=\"(max-width: 432px) 100vw, 432px\" data-rel='gallery-1' data-prev-img='https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching-432x400.jpg' class='first_thumb lightbox ' data-onclick='1' title='GUI Of Self Made Sophisticated QA Stem'  itemprop=\"thumbnailUrl\"  ><img decoding=\"async\" class=\"wp-image-1508 avia-img-lazy-loading-not-1508\"   src=\"https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching-432x400.jpg\" width=\"432\" height=\"400\"  title='GUI Of Self Made Sophisticated QA Stem' alt='GUI of self-made sophisticated QA sytem' \/><div class='big-prev-fake'><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-1508 avia-img-lazy-loading-1508\"  width=\"432\" height=\"400\" src=\"https:\/\/silketerstal.nl\/wp\/wp-content\/uploads\/2016\/09\/CA-Paper-Sophisticated-QA-Matching-432x400.jpg\" title='GUI Of Self Made Sophisticated QA Stem' alt='GUI of self-made sophisticated QA sytem' \/><\/div><\/a><\/div><\/div><\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>This project, part of the course Conversational Agents, focused on creating a sophisticated closed-domain question-answer (QA) system.<\/p>\n","protected":false},"author":2,"featured_media":1508,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[57,56,54,40,32,30],"tags":[],"class_list":["post-1502","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-fourth-year-block-2b","category-human-machine-interaction","category-master-hmi","category-portfolio","category-progamming","category-ut"],"_links":{"self":[{"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/posts\/1502","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/comments?post=1502"}],"version-history":[{"count":4,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/posts\/1502\/revisions"}],"predecessor-version":[{"id":2095,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/posts\/1502\/revisions\/2095"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/media\/1508"}],"wp:attachment":[{"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/media?parent=1502"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/categories?post=1502"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/silketerstal.nl\/wp\/wp-json\/wp\/v2\/tags?post=1502"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}