Tomato

"Tomato" is a media system designed for children to learn about tomatoes through interactive physical and digital materials, developed in the course "Devising Experiential Media Systems."
The project explores embodied interaction in virtual environments and how physical and digital content can work together in an experiential system to enhance collaborative learning. I built a physical globe as an input device. Participants can touch specific locations on the globe, triggering different virtual environments that feature local geographical and climatic characteristics. They can then interact with natural elements in these virtual environments using their bodies. This project includes two cycles of development and testing. Below are the documentations for this project:
In this project, I explored how physical and digital materials can work together to enhance sense-making, and how individual and collaborative interactions can motivate learning. The chosen theme was tomatoes. In the default scene, participants see a tomato garden with birds and insects, which they can wave away with their hands. Background sounds of nature, including wind, insects, and chimes, create an immersive atmosphere. The controllers in this project are a green tomato, a red tomato, and two leaves (one fresh and one dry), aimed at creating the sensation of being in a tomato garden holding tomatoes. When a participant holds the green tomato and the fresh leaf, they see the growing process of tomatoes. Holding the red tomato and the dry leaf, there shows the rotting process. When two participants hold different tomatoes together, they see the microscopic structures of tomatoes.

During classroom testing, players were surprised and intrigued by the different video clips triggered by their actions. They found the clips interesting and well-chosen, and one participant noted that the rounded style of the videos made the experience feel intimate. However, players had to keep touching the tomatoes while watching the videos, which did not effectively stimulate connections between the tactile sensations and the digital content. The feedback indicated that the connections were weak, with players concentrating more on the videos and less on the touch sense. This highlighted the need for further exploration on how to design integral interactions with both physical and virtual materials.

I also tested the system with a ten-year-old at home. However, I struggled to act as a good guide and companion, often telling him what to do to trigger events and explaining the system's mechanisms. During the experience, the boy mentioned he had seen a similar video at school about a seed growing into a big tree. When we held the two tomatoes and watched the video together, he asked questions about which part of the tomato was observed under the microscope. This aspect of the system successfully established conversations between children and adults, indicating its potential to foster educational dialogue.
If the video doesn’t load, you can watch them on Bilibili instead: https://www.bilibili.com/video/BV1nfxfeZEvH/