Design of a game-based rehabilitation system using Kinect sensor


Manna, S. and Dubuy, V. N. 2019. Design of a game-based rehabilitation system using Kinect sensor. Minneapolis, MN, USA ASME.
AuthorsManna, S. and Dubuy, V. N.

As technological innovation is fused into the rehabilitation process, it gives conventional therapy a new direction with the products of interactive nature and easy to measure techniques. In the recent years, virtual reality based game therapy has turned out to be a promising option for post-stroke patients since it engages patients with fun based exercises during rehabilitation process. It also triggers their neuro-motor functions and accelerates the recovery process. Nevertheless it is necessary to extract some valuable information from the joint movements to measure the recovery condition of patients. Most of the designed games have introduced features to make them interesting as well as challenging for patients, however, only a few measure the joint parameters. We have designed a Kinect based game in Unity3D platform where patients can play game by moving their joints which results in different orthopaedic lessons required for rehabilitation therapy. In contrast to many Kinect based games where only joint movements are considered for playing the game, we have also introduced voice control through speech recognition and feedback provided in terms of audio-visual command to enhance patient’s engagement. Different joint parameters such as trajectory, range of motion, joint velocity, acceleration, reaching time and joint torque are also measured to help quantify the heath condition.

Keywords Rehabilitation; Joint parameters; Unity; Kinect
Output statusPublished
Publication dates
Print19 Jul 2019
Publication process dates
Deposited11 Jan 2021
Place of publicationMinneapolis, MN, USA
EditionProceedings of the 2019 Design of Medical Devices Conference
Digital Object Identifier (DOI)
Official URL

[1] Rand, D., Weingarden, H., Weiss, R., Yacoby, A., Reif, S., Malka, R., Shiller, D. A., and Zeilig, G., 2017, "Self-training to improve UE function at the chronic stage post-stroke: a pilot randomized controlled trial," Disability and rehabilitation, 39(15), pp. 1541-1548.
[2] Kurillo, G., Chen, A., Bajcsy, R., and Han, J. J., 2013,
"Evaluation of upper extremity reachable workspace using
Kinect camera," Technology and Health Care, 21(6), pp. 641-
[3] Klopčar, N., and Lenarčič, J., 2005, "Kinematic model for
determination of human arm reachable workspace," Meccanica,
40(2), pp. 203-219.
[4] Bhadane, M., Liu, J., Rymer, W. Z., Zhou, P., and Li, S.,
2016, "Re-evaluation of EMG-torque relation in chronic stroke
using linear electrode array EMG recordings," Scientific
Reports, 6, p. 28957.
[5] Lim, C. K., Luo, Z., Chen, I.-M., and Yeo, S. H., 2011,
"Wearable wireless sensing system for capturing human arm
motion," Sensors and Actuators A: Physical, 166(1), pp. 125-
[6] Huang, M.-C., Chen, E., Xu, W., and Sarrafzadeh, M.,
"Gaming for upper extremities rehabilitation," Proceedings of
the 2nd Conference on Wireless Health, ACM, p. 27.
[7] Roy, A. K., Soni, Y., and Dubey, S., "Enhancing
effectiveness of motor rehabilitation using kinect motion
sensing technology," Proc. Global Humanitarian Technology
Conference: South Asia Satellite, 2013, IEEE, pp. 298-304.
[8] Anderson, K. R., Woodbury, M. L., Phillips, K., and
Gauthier, L. V., 2015, "Virtual reality video games to promote movement recovery in stroke rehabilitation: a guide for clinicians," Archives of physical medicine and rehabilitation, 96(5), pp. 973-976.
[9] Da Gama, A., Fallavollita, P., Teichrieb, V., and Navab, N., 2015, "Motor rehabilitation using Kinect: a systematic review," Games for health journal, 4(2), pp. 123-135.
[10] Windolf, M., Götzen, N., and Morlock, M., 2008,
"Systematic accuracy and precision analysis of video motion
capturing systems—exemplified on the Vicon-460 system,"
Journal of biomechanics, 41(12), pp. 2776-2780.
[11] Liao, W.-w., McCombe Waller, S., and Whitall, J., 2018,
"Kinect-based individualized upper extremity rehabilitation is effective and feasible for individuals with stroke using a
transition from clinic to home protocol," Cogent Medicine,
5(1), p. 1428038.
[12] Lin, C.-H., Sun, P.-Y., and Yu, F., "Space connection: a
new 3D tele-immersion platform for web-based gesturecollaborative games and services," Proceedings of the Fourth International Workshop on Games and Software Engineering, IEEE Press, pp. 22-28.
[13] Sookhanaphibarn, K., Phukongchai, W., Santad, T., and
Choensawat, W., "Towards Bilateral Upper-Limb Rehabilitation
after Stroke using Kinect Game," 2018 7th Global Conference
on Consumer Electronics (GCCE), IEEE, pp. 818-819.
[14] Obdrzalek, S., Kurillo, G., Ofli, F., Bajcsy, R., Seto, E., Jimison, H., and Pavel, M., "Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly
population," Proc. Engineering in medicine and biology society (EMBC), 2012, IEEE, pp. 1188-1193.
[15] Esfahlani, S. S., Muresan, B., Sanaei, A., and Wilson, G., 2018, "Validity of the Kinect and Myo armband in a serious game for assessing upper limb movement," Entertainment Computing, 27, pp. 150-156.
[16] Viegas, V., Postolache, O., Pereira, J., and Girão, P., "NUI therapeutic serious games with metrics validation based on wearable devices," Proc. Instrumentation and Measurement
Technology Conference, 2016, IEEE, pp. 1-6.
[17] Antón, D., Goñi, A., Illarramendi, A., Torres-Unda, J. J., and Seco, J., "KiReS: A Kinect-based telerehabilitation
system," 2013 15th International Conference on e-Health
Networking, Applications & Services, IEEE, pp. 444-448.
[18] Lugrin, J.-L., Latt, J., and Latoschik, M. E., "Avatar
anthropomorphism and illusion of body ownership in VR,"
Proc. Virtual Reality (VR), 2015, IEEE, pp. 229-230.
[19] Jaume-i-Capó, A., Martínez-Bueso, P., Moyà-Alcover, B.,
and Varona, J., 2014, "Improving vision-based motor
rehabilitation interactive systems for users with disabilities using mirror feedback," The Scientific World Journal, 2014, pp. 1-9.
[20] Postolache, O., "Remote sensing technologies for
physiotherapy assessment," 2017 10th International
Symposium on Advanced Topics in Electrical Engineering,
IEEE, pp. 305-312.
[21] Guneysu, A., Siyli, R. D., and Salah, A. A., "Autoevaluation of motion imitation in a child-robot imitation game
for upper arm rehabilitation," The 23rd IEEE International
Symposium on Robot and Human Interactive Communication,
2014, IEEE, pp. 199-204.
[22] Chang, C.-Y., Lange, B., Zhang, M., Koenig, S., Requejo,
P., Somboon, N., Sawchuk, A. A., and Rizzo, A. A., "Towards
pervasive physical rehabilitation using Microsoft Kinect," Proc.
PervasiveHealth, 2012, pp. 159-162.
[23] Ma, M., Proffitt, R., and Skubic, M., "Quantitative
assessment and validation of a stroke rehabilitation game,"
2017 IEEE/ACM International Conference on Connected
Health: Applications, Systems and Engineering Technologies,
IEEE, pp. 255-257.
[24] Winter, D. A., 1990, "Biomechanics and motor control of
human motion," New York: Wiley-Interscience.

Permalink -

  • 2
    total views
  • 0
    total downloads
  • 2
    views this month
  • 0
    downloads this month

Export as

Related outputs

Developing engineering growth mindset through CDIO outreach activites
Manna, S., Nortcliffe, A. and Sheikholeslami, G. 2020. Developing engineering growth mindset through CDIO outreach activites. Gothenburg, Sweden
Assessment of joint parameters in a Kinect sensor based rehabilitation game
Manna, S. and Dubey, V. N. 2019. Assessment of joint parameters in a Kinect sensor based rehabilitation game. Anaheim, California, USA ASME.
Rehabilitation strategy for post-stroke recovery using an innovative elbow exoskeleton
Manna, S. and Dubey, V. N. 2019. Rehabilitation strategy for post-stroke recovery using an innovative elbow exoskeleton. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine. 233 (6), pp. 668-680.
A portable elbow exoskeleton for three stages of rehabilitation
Manna, S. and Dubey, V. N. 2019. A portable elbow exoskeleton for three stages of rehabilitation. Journal of Mechanisms and Robotics. 11 (6), p. 065002.