Theory and Method of Affective Computing in Human Habitats
Xinyi Fu and Prof. Yingqing Xu,
Tsinghua University Future Lab, Beijing
Paper "Gesture-based fear recognition using non-performance dataset from VR horror games" accepted by International Conference on Affective Computing and Intelligent Interaction (ACII'21)
September 2020 - August 2021
Team member (multi-camera system design and setup, sensing floor setup, paper drafting, project presentation)
With the development and growing ubiquity of smart home technologies that enable creative interactions between humans and devices in the habitats, we envision the possibility of smart and sympathetic homes in the future. The FutureHome project is a long term project that seeks to build such smart human habitats which can fully understand human behavior and the living environment, process information through affective computing (passive input) and interaction intension analysis (active input), and then formulate a central control hub and information feedback system that would commend household devices and help humans to be more context-aware.
Human behavior detection
Center control hub for human activity recognition, affective computing, and coordinating smart home devices.
Creating a context-aware FutureHome
We started with "prototyping" the FutureHome. Sensors, smart home appliances, and computers are embedded in a household. We are planning on long term human habitant data collection in this house to construct a natural, multimodal, and comprehensive human behavior dataset in the future.
For a mid-phase exploration, we investigated gesture-based fear recognition. We set up a VR horror game environment to illicit fear and collected participants' skeleton points data for model training.
Xinyi Fu, Cheng Xue, Qiuyi Yin, Yu Jiang, Ye Li, Yichen Cai, and Weilin Sun. 2021. Gesture based fear recognition using nonperformance dataset from VR horror games. In 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII). 1–8. doi: 10.1109/ACII52823.2021.9597450.