A Visual Question Answering System for Graphical Plots and Figures

  • Sunny Katiyar, M. S. Wakode


Visual  Question  Answering  (VQA)  systems  are  one of  the  fascinating  examples  of  application  of  Artificial  Intelligence.  A  lot  of  VQA  models  are  proposed  over  time  which  are focused  on  datasets  such  as  VQA,  MS-COCO,  Flicker30k  etc. which  mostly  contain  randomly  taken  abstract  natural  images. But these models dont perform well if the images are graphical plots  or  say  generated  from  numeric  data.  Figure-QA  dataset is  a  recently  available  dataset  which  contain  such  statistical images like pie charts, line plots, histograms etc. But the models available  are  not  efficient  enough.  And  there  is  still  lot  of  work needed to be done to create an efficient enough model which can provide  satisfactory  results  on  these  statistical  images.  Taking a  step  towards  that  goal,  we  are  proposing  a  model  based  on baseline model [3] and embeds attention mechanism to the model and modifying layers accordingly to improve the model. We are committed  to  develop  a  Visual  Question  Answering  System  for synthetic  images  representing  numerical  data  that  can  perform better  than  the  state  of  art  models  available.

 Keywords : Artificial  Intelligence,  Machine  Learning,  Ma-chine  Vision,  Object  detection,  Context  awareness,  Prediction Methods.