Multimodal user interaction framework for context aware e-commerce

Abstract

E-commerce has grown up to be a major use of e-services and online purchases through the e-commerce are largely preferred over the traditional brick and mortar purchasing. Yet it is challenging for the consumers to fully experience the products or services with limited senses, lack of tangibility and sense of presence. Therefore a vital research question can be identified; how multimodal interactions can be used in e-commerce with context awareness, to improve the consumer experience. To address that question, this research aimed to study multimodal interactions, contextual factors and their effects on consumers. A set of multimodal interactions including 3D visualization and hand gestures and related contextual factors such as user, access device were identified in this research. They have been used to develop a multimodal interactions enabled prototype e-commerce framework. Several experiments and user studies have been conducted using the developed e-commerce framework and interesting effects on consumers have been discovered including positive user experience, improved value perceptions, and positive product opinions. Most importantly it has been shown that consumers perceive about 50% increased product value, and they are more likely to purchase when interacted multimodally. Usability Evaluations on the framework showed that users are mostly successful and comfortable in using multimodal interactions. Some technical, social and cultural barriers and challenges for enabling multimodal interactions were also revealed in those evaluations. From the findings of this research, it is suggested that further research focus should be on overcoming the identified technical, social and cultural barriers and bringing multimodal interactions to mass usage in electronic commerce platforms. Also the multimodal interactive e-commerce framework developed in this research can be used as platform to further study consumer dynamics by changing various variables.