Virtual/Mixed Reality

The POSEIDON system aims at automating the learning process for people with Down syndrome through a Mixed Reality (MR) solution – a mix of reality and virtual reality. 

One of the main factors we considered in designing the proposed Mixed Environments is the sensitivity of people with Down syndrome to small changes in the environment and in their routine. Thus, we found it important to investigate the level of abstraction that allows for a skill transfer between worlds. We developed a set of solutions that target wayfinding skills inside and outside. These solutions support fully customisable steps by the secondary user, setting specific waypoints, adding personalised messages and custom imagery. The learning process is divided in two tasks: training and assessment. The assessment mode reinforces whether the primary user requires more training or not.

The POSEIDON-project has investigated several opportunities of interaction between people with Down syndrome and modern technologies.

We have considered Virtual Reality and Mixed Reality, assessments were run in workshops, which are periodically organized by DSA in London. Prototypes were also presented at workshops, which our project organized with families according to the DOW and even in extra events in Germany, which attract large number of families related to the DS German Association. Out of all these experiences, we learnt the benefit of using systems which are not completely virtual but which includes direct references to the real world.

One of the main factors we considered in designing the proposed Mixed Environments is the sensitivity of people with Down syndrome to small changes in the environment and in their routine. Thus, we found it important to investigate the level of abstraction that allows for a skill transfer between worlds. We developed a set of solutions that target wayfinding skills inside and outside. These solutions support fully customisable steps by the secondary user, setting specific waypoints, adding personalised messages and custom imagery. The learning process is divided in two tasks: training and assessment. The assessment mode reinforces whether the primary user requires more training or not.

This is how the home training system is currently organized. It has received positive comments. Our contribution is organized as we agreed in the DOW. For training, education and preparation of the primary users to face the challenges of a complex world which is not designed to help them, it is developed as a game like experience, which can be guided and shared with supportive family members and/or other friend primary users which can be playing the game together.

The Ambient Intelligence layer of the system follows the individual where s/he is and through different applications, some of which, e.g., the calendar, we did not foresee when writing the proposal but were selected after feedback from stakeholders through questionnaires and workshops. During all this interaction with stakeholders it became more evident that contexts related to the user are paramount, that these are better gathered through the mobile platform, which is the one the user has when the most critical contexts may happen. Some of these relates to safety contexts. Contexts perceived by different platforms in the system can be combined in different ways providing smart digital environments in different physical environments (e.g., home and school).

Our project detected early on through workshops (this was an especially important topic discussed the workshop hosted in Mainz) that some of the most ambitious ideas in terms of adaptive interfaces had a high risk of being confusing to the primary users. For example, initially we considered the idea that an interface will rearrange icons of apps depending on the context to make more visible the one which the system judged more potentially useful. The families were against this stating it will be more confusing than helpful. There was also an analysis of using different technologies, which can sense emotional status, for example smart watches and other gadgets. Again, the families were against these options stating children wanted not to call the attention and also that cost would become prohibitive. As a result for this feedback from our user-centred activities, we refocused our interface-related work into clarifying the most suitable content (e.g., content type and level of complexity) and on personalization of different aspects of the system.

Scroll Up