Human-Machine collaboration

My RPO initially focused on hand gestures as a natural and intuitive form of communication between humans and machines, exploring how gesture recognition technologies could facilitate co-creation in interactive and media art contexts. The core idea was to use gestures as a seamless interface for enhancing human-machine collaboration, particularly in creative processes. However, as my research progressed, I found that the scope needed to expand beyond just the mechanics of gesture recognition. I began exploring the ethical and philosophical implications of human-machine collaboration, including issues of authorship, agency, and the evolving role of machines in artistic creation. These considerations led me to frame my dissertation around not only how gestures enable new forms of interaction but also how they challenge traditional notions of creativity and authorship in collaborative systems.

In addition to the initial pillars of gesture-based communication and multimodal interaction, my dissertation introduces new concepts that extend the framework of human-machine collaboration. A significant shift is the incorporation of machine learning and computer vision technologies to enhance the real-time responsiveness of the system to human gestures. This has transformed the project from focusing primarily on user inputs to examining how machines themselves can participate actively in the creative process, offering more dynamic and unpredictable responses. The dissertation also introduces the concept of embodied experiences, where the body’s movements interact with digital and physical spaces to create immersive, co-creative environments. These developments have added a layer of complexity to my work, transforming the project from a simple exploration of gesture recognition to an in-depth examination of how embodied interactions with machines can redefine creativity, agency, and the boundaries between human and machine roles in co-creation.

Phygital Experiences

for my practical work, ive been looking at sites like creativeapplications to find some inspiration. I recently came across this project called narratron,by Aria Xiying Bao and Yubo Zhao which offers a fascinating example of how AI and human interaction can merge to create rich, immersive co-creative experiences. By using hand shadows as the primary medium, the system recognizes these gestures as characters and shapes the narrative around them, generating visual settings and narrating the story. This dynamic interaction encourages users to become active participants, where the narrative is not a fixed story, but one that can evolve and change based on the shadows created by the user. This aligns with the growing trend of phygital experiences—blending physical and digital elements—to create immersive, interactive environments that feel both real and digital simultaneously. In this case, the physical act of shadow puppetry is enhanced by digital algorithms, allowing users to engage with the AI in a more intuitive and collaborative way.

The concept of phygital experiences extends beyond the blend of physical and digital environments; it’s about creating new modes of interaction where both the human and machine are active participants. In Narratron, the human is not just controlling the machine but is co-creating with it, shifting the traditional roles of passive user and active machine. This mirrors the kind of human-machine collaborations I am exploring in my dissertation, where gestures are not just input signals but contribute to the creation of something new and meaningful. By introducing AI into the process of storytelling through gesture-based inputs, this project highlights the potential for phygital interactions to transcend passive entertainment, becoming a collaborative space for creativity. Such projects inspire me to think about how gestures, both physical and digital, can act as a bridge between human creativity and AI systems, enabling new forms of artistic expression that are both interactive and co-creative.