Virtual Reality (VR) has been characterized by several researchers based on three fundamental properties, including telepresence, interactivity, and immersion. Delineating precise boundaries between presence, interactivity, and immersion has proven challenging due to varying interpretations. Consequently, the definitions of VR remain diverse.
Image credit: Generated using DALL.E.3
While some define VR as technologies enabling immersive experiences beyond reality, others describe it as computer-generated environments indistinguishable from reality. As per research in VR, there is an ongoing pursuit of a unified definition of VR, considering its foundations in presence, interactivity, and immersion. In this context, VR harnesses immersive technologies to recreate interactive virtual realms that elicit subjective involvement and a profound sense of physical presence among users.
Artificial Intelligence and VR
Both artificial intelligence (AI) and VR technologies, often perceived as contemporary innovations, have a longer history dating back over 70 years. AI, particularly software-based AI, made its debut in 1951 at the University of Manchester in the UK, giving birth to programs for checkers and chess. A broader perspective reveals "automata" from history, mechanical figures capable of basic decision-making, as early examples of AI.
Similarly, the roots of virtual reality can be traced back almost 200 years, or possibly even further. Charles Wheatstone's 1838 invention of the stereoscopic viewer created the illusion of 3D scenes from paired photographs. Edward Link's 1929 flight simulator prepared pilots for World War II.
In the 1950s, Morton Heilig's Sensorama offered an immersive experience with stereoscopic displays, sound, wind simulation, and vibrating chairs. Ivan Sutherland's 1965 description marked the first modern VR system using computers to generate imagery. Expanding the definition uncovers historical instances that meet the fundamental criteria of creating artificial immersion.
Over the past decade, AI, machine learning, VR, and augmented reality (AR) have experienced rapid progress. Breakthroughs in computer science, such as deep learning and extensive training data, have yielded impressive results. Advances in optics, display technology, microprocessors, and algorithms have further improved VR and AR experiences. Consumers can engage with various virtual environments, whether through VR headsets or without, spanning video games, virtual tours, social interactions, and educational platforms.
The question remains: can AI and VR effectively complement each other, and what role might AI play in enhancing VR experiences?
Intergrating AI in VR
A successful VR experience seamlessly merges hardware, software, and immersive user engagement. Incorporating AI into VR offers the potential to enhance the immersive and personalized aspects of VR technology. However, their adoption and usage patterns diverge considerably based on application areas. VR, particularly headset-based VR, enjoys more maturity thanks to the widespread availability of cost-effective hardware.
When examining the shared and distinct applications of AI in VR, it is insightful to explore examples beyond the VR realm first. The video game industry, known for pioneering practical AI, initially used AI algorithms to tackle the ubiquitous issue of providing challenging opponents to players. AI evolved from rudimentary expert systems to sophisticated implementations, ultimately enhancing various game facets such as world-building, pathfinding, data analysis, and player experience modeling.
A captivating AI application is a content generation, extending from environmental elements and music to whole game levels. This connection between AI in games and VR is noticeable as more games become available in VR. Another cross-industry AI application is natural language processing (NLP), which multiple sectors, including virtual shopping, explore. Voice recognition in both VR and AR relies heavily on AI and machine learning.
A prevalent AI application in both VR and augmented reality (AR) is computer vision. AI-driven camera inputs enable motion tracking, aligning the virtual or augmented reality experience with the user's perspective. AI research also delves into creating 3D scenes from single images, a nascent technology with the potential to significantly reduce immersive experience costs. Augmented intelligence, using AI to amplify human intelligence and productivity, holds promise when paired with VR or AR. Consider the possibility of designing a product in VR while receiving input from AI endowed with extensive knowledge of production methods, thus streamlining the design process.
Nonetheless, limitations exist regarding AI's role in VR and AR. VR and AR headsets must strike a balance between factors like power consumption, processing capacity, size, weight, heat generation, and user comfort. AI often demands substantial processing power, consuming more electrical energy and adding weight and heat to the headset.
Ongoing efforts focus on developing energy-efficient AI chipsets and implementing AI in low-power alternatives. Nevertheless, the perpetual quest to balance power and functionality remains a challenge as increasingly advanced AI applications continue to emerge.
VR technology finds applications across diverse industries, including medical training, engineering, and entertainment, with significant potential in education. Within the educational sphere, VR offers a transformative learning experience, immersing learners in scenarios that would otherwise be challenging to access, such as historical landmarks, outer space, or the human body. This immersive quality enhances comprehension and engagement among students, providing unique perspectives.
A notable advantage of VR in education is its cost-effectiveness, enabling educational institutions to create virtual environments accessible to multiple students concurrently. This approach ensures a regulated and secure learning environment, particularly when dealing with complex machinery or hazardous materials.
Through deep learning, a robot is pivotal in aiding class teachers to tailor personalized additional topics for students based on their proficiency levels. For instance, in an English class, at the commencement of the academic year, students can partake in a placement test. Subsequently, the robot can furnish supplementary weekly material, accommodating both those seeking to enhance their language skills and those requiring more challenging content due to already possessing proficiency matching or exceeding the curriculum.
Furthermore, English as a Second Language teachers, notably in regions such as India, China, and Egypt, where native accents and pronunciation may present challenges, can benefit from robotic assistance in reading and conversation instruction. These robotic assistants, accessible through applications or websites, can be finely tuned to cater to students' proficiency levels and structure courses aimed at specific English language proficiency objectives. These robots employ various technologies, including speech recognition, speech synthesis, natural language processing, recommendation systems, and reinforcement learning, to foster proficiency in reading, writing, listening, and speaking skills.
Enhancing VR with AI
Numerous avenues exist for enhancing VR through the integration of AI. A prominent approach involves employing AI and machine learning to craft intelligent, interactive avatars. These virtual assistants can comprehend natural language conversations and offer detailed, highly tailored assistance.
Many VR experiences necessitate introductions and user training. Instead of delivering these outside of the VR application, an immersive VR training experience can be created. This approach mirrors how many video games operate, immersing players directly into the gameplay and explaining the mechanics through the interactive experience.
Beyond application-specific training, the same methodology can be applied to a wide range of training scenarios. In VR applications, training materials can be personified as interactive characters. Coupled with AI functionalities like natural language processing, sentiment analysis, and pathfinding, it is evident how immersive technology could revolutionize training.
VR holds significant potential across various domains, with training being particularly crucial. Training remains a persistent challenge in numerous industries, often characterized by sluggishness, high costs, and inefficiency. Sectors like manufacturing confront the issue of an aging workforce, leading to the loss of valuable knowledge and experience as older employees retire. Digital transformation and automation render many mundane jobs obsolete, giving rise to roles that demand more advanced skills, further emphasizing the need for improved training.
Pros and Cons of Incorporating AI in VR
Utilizing AI in virtual and extended reality offers evident advantages, enhancing user interactions and experiences, augmenting human intelligence and senses, and reducing the cost of crafting immersive virtual environments. AI technology can elevate almost any VR simulation. Extended reality, when paired with AI, forms compelling learning environments. Training, particularly in VR, stands out as a clear AI opportunity, aiding workforce transitions. Nevertheless, AI faces challenges, including processing power, power consumption, and non-technical concerns like bias. Addressing unintended bias is vital to ensuring AI systems serve all users, regardless of their regional accents or backgrounds.
Future Developments
In recent times, the term "metaverse" has gained significant popularity. To many, the metaverse conjures images from science fiction films such as "Ready Player One," where individuals use immersive technology to enter an expansive virtual realm brimming with new and innovative experiences. Despite the advancements in VR, AR, and immersive technology, achieving a fully realized metaverse remains a formidable challenge. Nonetheless, rapid digital transformation, an expanding array of AI applications, and swift progress in immersive technology characterize the landscape.
Major corporations, numerous startups, and various others are at the forefront of innovation in AI, machine learning, AR, mixed reality, VR, and related fields. While considerable progress has been made since the early days of basic VR headsets and rudimentary AI algorithms, there is still substantial work ahead. The future appears promising for immersive technology, AI, and intelligent reality.
References and Further Readings
Brzezinski, M., and Krzeminska, I. (2023). The strategies for innovating with virtual reality and artificial intelligence: a literature review. Technium: Romanian Journal of Applied Sciences and Technology, 8, 72–83. DOI: https://doi.org/10.47577/technium.v8i.8671
Liu, H., and Cheng, X. (2023). Application of virtual reality technology based on artificial intelligence in fashion design style. International Journal of System Assurance Engineering and Management. DOI: https://doi.org/10.1007/s13198-023-02148-z
von Ende, E.; Ryan, S.; Crain, M.A.; Makary, M.S. (2023), Artificial Intelligence, Augmented Reality, and Virtual Reality Advances and Applications in Interventional Radiology. Diagnostics, 13, 892. DOI: https://doi.org/10.3390/diagnostics13050892
Wohlgenannt, I., Simons, A. and Stieglitz, S. (2020). Virtual Reality. Business and Information Systems Engineering 62, 455–461. DOI: https://doi.org/10.1007/s12599-020-00658-9
Sindu, I. G. P., Hartati, R. S., Sudarma, M., and Gunantara, N. (2023). Systematic Literature Review of Machine Learning in Virtual Reality and Augmented Reality. Jurnal Nasional Pendidikan Teknik Informatika: JANAPATI, 12(1), 108–118. DOI: https://doi.org/10.23887/janapati.v12i1.60126