Gaming for Arm-Hand Rehabilitation

For my three-months during traineeship at Roessingh Research and Development (RRD), a physical rehabilitation centre, I analysed the technical possibilities for the development of a low-cost, hybrid gaming system for arm-hand rehabilitation in a 3D space. RRD already created a hybrid gaming system to train arm-hand movements in a 2D space. The system, HandsOn, uses a beamer to project a virtual environment and tracks tangible objects with a Microsoft Kinect. However, the current setup results in translation difficulties when it is required to track tangible objects in a 3D space and results in projection problems when projection onto multiple planes is desirable. Although many other arm-hand gaming systems exist, they either limit training movements or are expensive.

During my traineeship I created an overview of the technical possibilities. Based on the findings, I proposed a promising system design. The design consists of two regular monitors connected in a L-shape. On top of the monitors infrared touch overlays are placed to be able to detect the position of tangible objects. The gaming environment is displayed onto the monitors. In addition, rotational sensors can be integrated inside the tangible objects, to be able to track the rotation of the objects. The main advantage of the proposed system is its flexibility. This flexibility is created by combining positional tracking and visual displaying into one device, a monitor. Therefore, no tracking and displaying problems occur when shifting around the setup. The first development phase should consists of adapting the existing 2D game to the proposed system design. In a next phase, the setup can be extended with more displays or physical objects. Physical objects, such as shelves or physical stands, can be added to increase the possibilities for placement of the tangible objects. Later, the visual appearance of the tangible objects can be adapted to fit better with the game and new game concepts, specially designed for the new setup, can be implemented.

In addition to the report explaining the technical possibilities, I created a proof-of-concept prototype to demonstrate the main characteristics of the system design proposed. I designed my own game; the Farmer. In the game, the user takes the role of the farmer. Animals on the farm randomly show their needs in thinking clouds. The user should fulfill these needs by rotating a tangible cube in such a way the icon on the face of the cube matches the icon of the animal’s need. The game was implemented with a setup consisting of two regular monitors in a L-shape. To be able to demonstrate touch actions, two pressure sensors where added to the screens. In the tangible cube an IMU, a rotational sensor, and an Arduino were placed to be able to sense the rotation of the cube and to transmit the rotational data. The game receives the data from the Arduino wirelessly and combines the data with the data of the touch sensors to determine if the user performs the correct action.