Context

The situation in hospitals, nursing homes and homes for mentally challenged people is becoming more and more difficult. The medical staff and special educators are often responsible for an increasingly large number of residents and can no longer spend as much time with each resident as before. The risk of not noticing or not being able to respond to problems increases proportionately to the number of residents per medical staff member.

Moreover, it is sometimes difficult for an elderly person to leave an independent life when living in a nursing home. Overnight, the person finds herself / himself in a structure where activities, meals, and care are fixed in time and where freedom is relatively restricted. But the reality is that these people need regular medical care and assistance because they are no longer able to manage by themselves for physical or mental (e.g. Alzheimer) reasons. The risk of injuries, falls, loss of consciousness or simply not being able to manage their health (e.g. take medication) leads to the decision to place the person in a socio-medical environment.

To be able to monitor residents in a none-intrusive manner, we must provide a certain degree of independence, safety and well-being for the residents and relieve some of the pressure on nurses and educators. The ideal monitoring system should in fact be an ecosystem which includes several mechanisms: adapted sensors that can localise and detect resident’s activities and collect physiological data, a way to send regular updates about the situation of the residents, a complete understanding of the medical staff situation, a centralized system for both residents and medical staff and finally a logic to decide the most appropriate available person to intervene in case of problems with a resident.

Such systems can also help elderly people who still live at their own home, but who do not feel as secure as they would like. A loss of confidence in one’s physical or mental abilities is one of the major reasons for elderly people moving into a nursing home. In order to keep those people to live at home if they want to, Hestia can bring a guarantee of response if any problem occurs. These people would therefore stay at home and feel safe and secure.

Focus

Hestia aims to explore solutions that blend new technologies with a respect for human relationships in the context of nursing homes or apartments by installing an intelligent environment that monitors, in a none-intrusive way, the situation of the resident in a broad sense. By situation, we mean his / her physiological state and his / her general state.

This system brings technological resources within nursing homes in order to improve the life comfort of their residents and the working conditions of their medical staff. Hestia can also help elderly people at their personal home to feel safe during their daily activities. Therefore, various monitoring techniques are integrated through mobile applications for both medical staff and elderly. Data are thus collected in order to fulfill its role.

Architecture

Two mobile applications (RMMA and HMMA ) communicate through a central server, SEMS, which logs all exchanges between the two applications. This server contains several algorithms that manage the smart environment. All alerts and instant messages are handled by the Messaging Server, which is responsible of sending push notifications to clients.

This architecture is really easy to adopt, as much for the hardware, devices and the ecosystem installation. All the physical devices, sensors and the overall architecture do not require any specific material nor proprietary technologies. Moreover, its development has been conducted with openness in mind. We settup up the system, the only needed arrangement is to place beacons in strategic parts of the building(s) rooms. Those beacons permit a precise indoor location tracking for both residents and medical staff members.

RMMA – Resident Monitoring Mobile App

The goal of the resident mobile application is to monitor the resident’s location, activity, and heart rate in real time. Those metrics are then sent to the medical personnel (nurse, doctor, educator) through the central server SEMS. The location, activity and cardiac frequency are especially used to raise alarms. If a resident is out of a given perimeter, shows an abnormal heart rate or exhibits an abnormal or suspicious activity, an alarm is immediately sent to the server. A basic algorithm aggregating the 3 parameters has been developed for alarm handling. This application has been especially developed for people who are not computer, tablet or smartphone oriented. The interface is intended to be simple and efficient.

External devices with sensors are used for two metrics. The cardiac frequency is monitored using a thoracic belt and the blood pressure is measured through a connected wristband. Those two devices communicate with the smartphone via Bluetooth.

This application has a feature that reminds the resident when he or she must take medication. It also proposes a messaging system that allows to send free-form or predefined instant messages.

SRMMA – Smartwatch Resident Monitoring Mobile App

An Android smartwatch version of RMMA has been developed in order to offer a lighter solution for the monitoring of residents. The same features as RMMA are implemented. Resident might feel more comfortable wearing a smartwatch than having to carry a smartphone all the time.

HMMA – Health Monitoring Mobile App

This Android and iOS mobile application has been developed for the medical staff. It gives them access to real-time information about residents, including the identity of the resident, his/her location, activity, heart rate and the list of medications that the resident must take.

If a resident has a problem, the application receives the alarm raised by the resident’s application (RMMA) and the nurse or the doctor whether accepts the alarm or declines it if they cannot take care of the resident at that time. The medical member can indicate his / her availability to the central server SEMS through the application, allowing for a more efficient handling of where alarms should be sent. The application also allows the medical staff to communicate with the residents through an instant messaging system similar to WhatsApp. With this option, the medical staff can receive short messages from the residents or send messages, such as medication reminders.

SEMS – Smart Environment Monitoring System

In order to link all the client applications, we use a server through which all information transits. The resident applications (RMMA or SRMMA) send all information to the server, as well as the alarms, in JSON format. The server logs the information and if an alert is raised, it contextually search for the best medical staff member that can be responsible for the resident. Then, it sends the information about the alarm generated by the resident’s smartphone to the medical staff member responsible and waits for a feedback (intervention accepted or denied). If the medical staff member declines the intervention, the server sends the alarm to the next medical staff member in the list. All events are kept in its storage. It also handles the user’s connection for both medical staff and residents and handle the whole environment.

SEMS-UI – Monitoring Web Application

The SEMS server also runs a monitoring Web Application on its web server. It allows the head nurse or other managers of the institution to access to a complete overview of the entire environment.

Through this application, the user can access to several information. First, two maps display the location of both the residents and the medical staff. Two views of the building(s) are available: indoor and outdoor. All records on the main server SEMS can also be consulted: the resident personal details, the medical member personal details and the list of all raised alerts including their context. Finally, the user can discuss with residents through the instant messaging feature of Hestia.

On the alert side, SEMS-UI can receive alerts from residents if no staff member has accepted a help request in the HMMA application on their device.

MAAS – Messaging And Alert Server

The last part of this ecosystem is the communication server, which handles all notification deliveries generated by the central server. In order to build a fully independent ecosystem, we had to develop our own push notification system.

It uses the WebSocket technology though a Node.js application to communicate with client applications.

Partnership

Currently developed within the iCoSys institute at the School of Engineering and Architecture of Fribourg (HEIA-fr), the Hestia prototype is already in an advanced state. A partnership is established with the University of Winchester, which brings their expertise regarding social impacts, new techniques or more academic fields into the project. We also have contacts with medical and social specialists.

Scientific objectives

New techniques have been developed for the detection of falls. We studied and integrated Artificial Intelligence mechanisms for this end and we aim to improve the Hestia ecosystem (response time, problem detection, performance) by various other elaborations. We also focus on the privacy of the final users, which must be protected during such usage. We also place the system security at the hearth of our approach.

Opportunities

In the future, we are aiming to discover, develop and implement new techniques that fulfill our goal to bring a complete, efficient and reliable tool for elderly people. If so, papers, articles or publications will be published in order to explain our approach.

Actors

The project is developed by the department of Computer science in the School of Engineering and Architecture of Fribourg and realised by the team of students, assistants and professors:

• Dr. Pascal Bruegger – prof. and project coordinator
• Adriana Wilde – Lecturer and academical coordinator
• Aurelien Etienne – Bsc in computer science
• Wiliam Greppi – Bsc in computer science
• Rafic Galli – Master in computer science
• Nicolas Zurbuchen – Master in computer science
• Armando Koljaj – Bsc Student
• Sylvain Renaud – Master in computer science
• Loïc Guibert – Bsc in computer science