Automated Vision-Based Wellness Analysis of Elderly Care Center Citizens

Elderly care is important yet very expensive to perform. The demand for quality care is increasing but the manpower is not sufficient. Recent Covid-19 pandemic further highlights the gap, where there has been several viral outbreaks in the elderly care-center because workers need to do their shift across care centers.


Collaboratively working with haven-of-hope Christian Service in Hong Kong, we propose an automatic video-based analysis system to assist caretakers in monitoring the wellbeing of senior citizens with dementia over a long period of time (around a month). At the moment, we focus on analyzing the personalized facial and activity analysis of each senior citizen and alert the caretaker when there is a sudden anomaly or changes on their behavior, which may imply that the elder are in need of assistance.


Some features we used to analyze the behaviors and wellbeing of elderlies are, but not limited to, the amount of sleep a senior citizen experiences during the day and how active an elder is during a group physical exercise. We believe both these information conveys important semantic meaning towards how engage a demented elder is during the day, which is always useful to design a more appropriate activity for them to follow. Both information are presented in the graph below.


During the day, a senior citizen usually spends most of his / her time sitting and napping. By tracking the amount of nap, we may have a better understanding on their sleeping quality the night before and provide a more attentive care for them by for instance, allowing them to rest in a different area to provide them with enough sleep.


In order to keep an elder physically active, there are multiple physical exercises performed, during a day. For instance, we can see that the senior citizen on the bottom, in the graph above, tends to have a lower activity score. Maybe s/he has some difficulty in moving his / her limbs and a softer physical exercise or activity might be more suitable.

This particular line of study is distinctive because, while most expression detection research focuses on building a model to predict emotions within large populations, Prof Cheng’s work seeks to monitor a particular subject over a long period of time and analyze the correlation between physical expressions and certain diseases. In future studies, this same set of valuable visual data