Storytelling is evolving beyond visuals and sound, now incorporating touch, smell, taste, and to build narrative structures for immersive, sensory driven storytelling. With the infusion of AI, VR, AR, storytelling has entered a new era—one where the lines between creator and audience blur, as both the production and consumption of narrative becomes a personalized, immersive journey. This shift is not merely technological but cultural, transforming the very nature of how we experience and therefore make meaning as both the “author” and the “audience.”
The Rise of Multisensory Engagement
In a world where immersive experiences dominate, narratives are beginning to incorporate a full range of sensory elements: touch, smell, and yes, even taste. Virtual and augmented realities are pushing the boundaries of traditional storytelling, allowing users to not only view and listen to a story but to participate in it, interact with characters, and influence the plot’s direction. This “multisensory” approach immerses the audience in environments that feel tangible, where emotions, actions, and environments are directly experienced rather than observed..
Dr. Kathleen Sitter, a researcher at the University of Calgary and Canadian Research Chair in Multi-sensory Storytelling in Research and Knowledge Translation, integrates multisensory techniques in order to make storytelling more accessible. Sitter’s research focuses on using soundscapes and "smellscapes" to map historical narratives from marginalized communities, including people with disabilities. Her project, “Disability Histories, Soundscapes, and Smellwalks,” explores how multisensory methods can enhance inclusivity by enabling disabled individuals to share their experiences through unconventional sensory channels. This work aims to break down barriers to participation and empower marginalized voices in public discourse.
Similarly, Emilie Baltz explores the construction of narrative through taste, smell, and touch, using food as a central storytelling medium. Her work challenges conventional storytelling by transforming food into an interactive experience, inviting audiences to consume, taste, and even participate in her narratives. In her AI powered project EAT | TECH | KITCHEN, done in collaboration with Klasien van de Zandschulp reimagined stories of the Futurist Cookbook into 21st Century interactive experiences. Baltz’s installations and performances make storytelling multisensory, creating vivid, memorable interactions that leave lasting impressions on participants. Baltz routinely uses food not only as sustenance but as a means to evoke emotions, memories, and sensory engagement, inviting audiences into a more immersive and personal connection with the story.
AI tools now allow creators to develop interactive narratives that adapt in real-time based on user input. These systems go beyond pre-programmed responses, learning user preferences, and adjusting the story to fit, making every experience unique. Platforms such as OpenAI’s GPT models and AI-generated content (AIGC) tools are integral to the process, allowing creators to generate personalized characters, dialogue, and environments on the fly. These technologies enable characters to respond dynamically, to grow, and even to change based on the audience’s emotional engagement.
Known for his "live simulations" and AI-driven art installations, Ian Cheng creates artworks that evolve independently over time, driven by AI and machine learning models. His notable work, "BOB (Bag of Beliefs)," features an AI-driven creature that interacts with viewers, learns, and changes based on its "beliefs." This creates an evolving narrative experience as BOB's behavior and personality adapt in real-time, influenced by its environment and audience interactions. Cheng’s work challenges traditional storytelling by creating living, autonomous characters that build their own narratives over time.
From Passive Consumption to Active Participation
Another notable shift in narrative construction and storytelling using multisensory inputs and outputs are the roles of the creators or authors and the audience. Multisensory storytelling enables participation wherein audiences can co-create and influence the story and storytelling process itself. This shift is evident in video games, VR experiences, and AI-driven narratives where users have the power to shape the storyline.
Gaming, as one of the most significant sectors of digital storytelling, exemplifies this transition. Games like “The Last of Us” or “Detroit: Become Human” emphasize player choice, with narratives that evolve depending on the decisions made throughout the game. The result is a deeply personalized experience, one that acknowledges the agency of the player and integrates their actions into the fabric of the story. Moreover, these games use advanced AI and machine learning to adapt the storyline based on how players interact with characters, adjusting plot lines, emotional stakes, and even dialogue.
VR storytelling experiences are also exploring new frontiers of multisensory engagement. These projects move beyond passive viewing, inviting audiences to step into the shoes of others, engage with their surroundings, and make decisions that shape the outcome of the story. Platforms like Oculus, with experiences such as “The Invisible Hours” or “Wolves in the Walls,” are integrating visual storytelling with haptic feedback, dynamic soundscapes, and even physical interaction to create immersive experiences. These experiences not only present a narrative but invite the audience to live it, offering new ways to experience empathy and understanding.
The Future: A Fully Immersive, Fully Personalized Digital Experience
As we look ahead, the future of digital storytelling promises to be more immersive and interactive, leading to more personalized experiences. It is reasonable to speculate that in the not-to-distant future, we will experience stories that not only respond to our actions but anticipate our emotional needs, adapt in real-time, and create entirely new experiences. In doing so, the lines between creator, audience, and narrative will continue to blur—creating new forms of personal, emotional, and participatory experiences that will redefine the way we tell and consume stories.
