Memory Lane

The Memory Lane project address a new research line related to capture and recognition of human activities for well-being of people with special needs. The project is done by to research groups with large experience in previous works in the application field and complementary scientific/technical expertise; CVLab in activity recognition by computer vision and HOWLab in ambient intelligence environments, sensor networks and wireless communications. This proposal addresses the problem of labelling the scenario and the activities by fusioning data from images and audio as well as the information provided by other electronics sensor spreaded along the environment and on the user. On the other hand, the focus on this approach is not in the user, as carried out in previous projects, but in the environment around the subject, since he/she is wearing a portable camera.

Objectives

Clinical context Dementia, a leading cause of disability in the elderly, currently affects nearly 10 million people in Europe and over 35 million worldwide. Rising at unprecedented rates, these figures are projected to increase to 14 and 65.7 million respectively by 2030. The socioeconomic repercussions are equally staggering. In Europe alone, the total costs of dementia amount to over €180 billion in 2010 and are estimated to exceed €250 billion by 2030. The aforementioned inflict a significant burden on healthcare systems, society and the economy, necessitating effective treatment means, while preserving quality of life for the people affected and for their carers.

The goal of this project is about providing a tool to automatically and unobtrusively create a contextualized life-blog for people with special needs and make it available for later context-dependent retrieval. Such life-blog will contain images and sounds as perceived by the person, chronologically ordered and automatically tagged by the system providing them with contextual meaning. Thus, it will be possible to make searches of events or applying different algorithms to create different applications such as exercising the memory by revising emotional bindings with the past, serving as task tutorial when memory worsens, and working as alarm detection, evaluating person’s quality of life or just for entertainment. We also propose to validate the system with some users.

Project overview

Memory lane overview

Memory lane overview

 

Maria example

It is obvious that the population in Europe is becoming older. This proposal is related to Ambient Assisted Living in the aim to extend the time during which elderly people can live independently in their preferred environment with the support of Information and Communication Technologies. Following scenario illustrate our objectives:

Maria is a 70 years old person who lives autonomously making all daily living tasks at home; she observes some slight decline of her cognitive capacities and thus she decides to test the new memory lane system. The system is installed in Maria’s home and she is trained in its use. In the initial phase, training of the system is necessary, so Maria must supervise the learning process. This supervision process is settled as natural as possible: when realizing new activities, Maria pronounces the key words to label them; when reviewing the life-blog of the day, the system proceeds to classify the different activities reviewed and asks to Maria for the corresponding verbal approval or correction.

As training process increases the recognition performance, fewer activities are delivered to the Maria supervision. After few weeks using it, Maria is used to the system and very happy with it. The electric sensor determines which appliance (oven, TV, washing machine, etc.) is being used, wearable sensor tracks and registers Maria’s movement at home, microphone recognizes Maria’s words, camera saves snapshots, image & sound processed identifies Maria’s activities…all together constitutes Maria’s life-blog. As all the information is stored in a computer she has at home, she (and her son Eduardo) doesn’t have any privacy concerns. From time to time and thanks to its intuitive search interface, she revises the contents and show parts of the life-blog to her visits (e.g. when her grandsons visited her, the procedure to cook her special tiramisu, etc.).

Few years after (and many Terabytes of data), Maria is still autonomous but she finds difficult to remember some of her recipes or how to program the washing machine. In order to help her, a screen has been installed in the kitchen and Maria can use the system as a personalized tutorial tool to see how she did things. Eduardo sees how his mother is losing some of her capacities and fears she can have an accident while she is alone. As Maria wants to remain at home, the system activates its realtime monitoring and alarm capacity. This feature also respects Maria’s private life as no sensible data is sent outside the house.

The system now locally analyses Maria activities and compares them with previous data stored in order to find different events. One day the system sends a message to Eduardo indicating that there might be a problem at Maria’s as she didn’t leave home (there is a sensor for that in the door) and there has been no morning activity in the kitchen (something unusual as Maria haves breakfast between 9 and 11 every morning). Eduardo goes to his mother’s and find her fallen in the bedroom’s floor. She has been there for two hours (falls are primary etiology of accidental deaths in elderly and early detection exponentially reduces mortality). Thanks to the quick attention and after a recovery time in hospital, Maria gets back to home.

Now a social worker periodically visits her to asses her quality of life through personal interviews. The information gathered this way is of great importance, nevertheless it is somehow limited: Personal interviews are influenced by known factors such as empathy between social worker and elderly (that may modify the person’s mood and consequently produce a bias) and also variability on the person’s mood throughout the day, week, etc. (observation in an interview is an isolated event in time which may produce a bias). In order to complement the said information, the Quality of Life Evaluation module is activated. This feature periodically analyses main features of activities detected and compares them with all previous

Proyecto financiado por el

Ministerio de Economía y Competitividad – TIN2013-45312-R

Logo MEC

Logo MEC