by Christina Harrich -

WebAR: Anwendungsmöglichkeiten

Webbasierte Augmented Reality gestaltet virtuelle Erlebnisse zugänglicher und niederschwelliger.
Notwendig ist dafür nur ein Smartphone mit Handykamera.

Es gab bereits einige Versuche, um Printprodukte mittels Augmented Reality interaktiv zu gestalten. Es blieb dabei aber meist bei einzelnen Tests, da sich User dafür eine App downloaden mussten. Die dadurch entstehende Schwelle schreckte viele ab und es blieb beim klassischen Magazin. 

Allerdings gibt es seit einiger Zeit die Möglichkeit, via QR Code auf webbasierte AR Anwendungen zuzugreifen. User müssen also keine App mehr downloaden, sondern können die Inhalte direkt über ihren Browser konsumieren. 

Eine Studie des Content Marketing Institute zeigt, dass Botschaften mit interaktiven Inhalten deutlich höheren Anklang finden. User beschäftigen sich intensiver damit, deshalb behalten sie die Inhalte auch stärker im Gedächtnis. 

Augmented Reality wird oft mit Apps wie Pokemon GO in Verbindung gebracht, hat aber eigentlich viel mehr zu bieten als ein fesselndes Spielerlebnis. Von interaktiven Produktetiketten bis hin zu Filmplakaten, die sprechen können – mittels AR stehen einem plötzlich viele zusätzliche Gestaltungsmöglichkeiten zur Verfügung. Einige Beispiele für die gelungene Verknüpfung von herkömmlichen Produkten mit Augmented Reality sind die folgenden:

1. Interaktive Produktetiketten: “Mission Tiger” von Frosted Flakes

Kellog’s hat kürzlich eine Kampagne gestartet, um das Bewusstsein für seine Arbeit zur Unterstützung des Schulsports zu schärfen. Mit Gesichtsfiltern, Minispielen und einer speziellen Botschaft von Tony dem Tiger trug das WebAR-Erlebnis dazu bei, das Bewusstsein für die Initiative des Unternehmens zu schärfen und Spenden zu sammeln.

2. Exklusive Markeninhalte: Sam Smiths Album-Veröffentlichung 

Die Fans des Künstlers Sam Smith erhielten exklusiven Zugang zu einem besonderen Erlebnis, das auf seine neueste Single-Veröffentlichung abgestimmt war, um den Download des Albums und die Verbreitung in sozialen Netzwerken zu fördern. 

3. Printmaterialien: Einführung des Toyota Supra 

Im Rahmen einer kürzlich durchgeführten Werbekampagne für den neuen Supra gab Toyota seinen Kunden die Möglichkeit, den neuen Sportwagen in ihrer Garage zu sehen, eine kurze Probefahrt zu machen und verschiedene Lackierungen zu testen – mit Hilfe von WebAR-gestützten Zeitschriftenanzeigen und Postern.  

4. Persönliche Botschaften: Rede des Präsidenten zum Unabhängigkeitstag 

Mit Hilfe von holografischer Technologie, die über WebAR verteilt wurde, überbrachte der israelische Präsident seinen Bürgern während des Höhepunkts der COVID-19-Pandemie eine besondere Botschaft zum Unabhängigkeitstag und blieb damit seiner Tradition treu, die Bürger an diesem besonderen Tag persönlich zu treffen.

5. WebAR-Visitenkarten 

Der Werkzeughersteller Stanley Black & Decker hat seine Visitenkarten mit Hilfe von WebAR auf die nächste Stufe gehoben, indem er ein maßstabsgetreues Maßband und Click-Through-Links zur Unternehmenswebsite und zu sozialen Kanälen hinzugefügt hat. 

by Marton Szabo-Kass -

User-Centered Perspectives for Automotive AR

| a short summary of a paper on human aspects related to automotive AR application design

A research paper, titled like this blog post, by experts from the Honda Research Institute (USA), the Stanford University (USA) and the Max-Planck-Institut für Informatik (Germany) [1] discusses benefits, challenges, their design approach and open questions regarding Augmented Reality in automotive context with a focus on the users.

Augmented Reality can help drivers in pointig out important and potentially dangerous objects in the driver’s view and increase the driver’s situation awareness. Though if the information is presented incorrectly, the distraction and confusion of the driver can lead to dangerous situations.

The authors of the paper put up a design process with focusing on the appropriate form of solution to a driver’s problem (rather than just describing ideas technically).

To understand the drivers’ problems in the first place, they conducted in-car user interviews with different demographic groups to gather information about driving habits, concerns and the integration of driving into daily life.

After the interviews they ideated prototype solutions and tested concepts in a driving simulator with a HUD. One realization was that at a left turn, drivers needed more help in timing the turn according to oncoming traffic rather than an arrow or graphical aids for the turning path – which even distracted them from the oncoming traffic. The design solutions of the authors therefor focus on giving the driver additional cues to enhance awareness rather than giving only navigation commands. After researching different graphical styles of turning path indication, results showed less distraction with solid red path projection – that is visible in the peripheral vision while focusing on traffic – than simple chevron style lines.

Human visual perception

Regarding human perception, the authors of the paper analized influences of visual depth perception and the field of view. The human eye is built to focus on one distance at a time, so AR displays / Head Up Displays can cause a problem due to their see-through design. The driver’s focus has to remain on the road ahead and not change to the windshield’s distance, blurring out the farther imagery.

The eye’s foveal focus with the highest acuity is only at a ca. 2° center area of the vision field. This determins the so called “Useful Field Of View” (UFOV), the limited area from which information can be gained without head movement. These restrictions imply the use of augmented systems only in the driver’s main field of view, and not throughout the whole windshield. Objects in the peripheries should be therefor signalized either inside of the UFOV or through other methods.

Distraction

The National Highway Traffic Safety Administration (NHTSA) of the USA states three types of driver distraction:

  • visual distraction (eyes off road)
  • cognitive distraction (mind off driving)
  • manual distraction (hands off the wheel)

Each of these types can be aided but also caused by Augmented Reality applications in vehicles.

The authors discussed the human attention system and cognitive dissonance problems further.

  • Attention system
    Regarding the human attention system, the so called “selective visual attention” and the “inattentional blindness” can be problems in driving conditions. Important visual cues can be suppressed when the driver is focusing on secondary tasks or if they are outside of the focus of attention. Warning signs on a HUD can either help by attracting attention, but also distract from other objects that are outside of the augmented field of view. The study states the need of further research on the balance between increasing attantion and avoiding unvanted distraction.
  • Cognitive Dissonance
    Cognitive dissonance, the perception of contradictory information, could occur e.g. with bad overlapping of 2D graphics on the 3D vision of the surroundings, causing confusion or misinterpretation of the visual clues.

Human behaviour

As a third category, the study discusses the effects of AR technology on human behaviour.

Situation awareness – maintaining state and future state information from the surroundings – is detailed by a source in three steps:

  1. Perception of elements in the environment
  2. Comprehension of their meaning
  3. Projection of future system states

Augmented Reality can help drivers not only in perception but also in the further steps. State-of-the-art computers, AI technology and connected car data from surroundings can be especially of help in cases where additional computational power can predict traffic dynamics. [comment of MSK]

One aspect is the behavioural change of drivers after longer use of assitance systems. A study implies that the reduced mental workload could lead to the retention of the drivers’ native skills. Further, the phenomena called “risk compensation” can occur after getting used to the aids. This means a riskier behaviour of the driver than normal, due to higher confidence in the surroundings. These behavioural changes can have dangerous consequences, why the authors suggest the use of driver aids only when needed.

According to one source, the user’s trust in a technology is can be increased with more realistic visual displays, like AR rather than simple map displays. Further, AR can also help to build trust in autonomous cars, communicating the system’s perception, plans and reasons for decision making.

Some open questions were stated at the end of the paper, to be considered further on.
Such were for example how multiple aiding systems can interact at the same time? Or how will the use of AR over longer time effect the drivers’ behaviour and skills when they have to switch back and drive a non-AR vehicle? Will the drivers’ skills deteriorate over and will they become dependent on these aiding systems?

My conclusion

This paper was published in 2013, since when the technology was significantly developed further. Nevertheless the basic principles and human factors are still the same, which have to be considered when designing safety critical automotive applications.

Reliability and understanding the behaviour of autonomous vehicles will be an essential aspect in creating acceptance by the driver / passengers. Augmented Reality can be of much help not only for extra driving assistance systems, but also for the complete user experience at different automation levels.

The mentioned topics of human factors in this paper were only focusing on visual augmentation and assistance. These could be expanded to other modalities like sound and haptic augmentation, and analyse the perception of a combined driver assistance as well.

Source

[1] Ng-Thow-Hing, Victor & Bark, Karlin & Beckwith, Lee & Tran, Cuong & Bhandari, Rishabh & Sridhar, Srinath. (2013). User-centered perspectives for automotive augmented reality. 2013 IEEE International Symposium on Mixed and Augmented Reality – Arts, Media, and Humanities, ISMAR-AMH 2013. 13-22. 10.1109/ISMAR-AMH.2013.6671262.
Retrieved on 30.01.2022. https://www.researchgate.net/publication/261447349_User-centered_perspectives_for_automotive_augmented_reality

by Laura Varhegyi -

Difference between VR, AR, MR, XR

Let’s start with the basics. We first must know the difference between these technologies, to be able to adapt future projects to the right environment.

Reality as a construct

What we perceive with our senses seems to be reality, whether what we perceive comes from the digital or the physical world. Take for instance watching a movie, we know it is not real, but it feels real to us. It triggers emotions, we feel empathetic to characters and we create connections with them.

It’s really important to understand we’re not seeing reality. We’re seeing a story that’s being created for us.

– Patrick Cavanagh, Research Professor

The virtuality continuum is a scale that goes from reality to virtuality. In it, technologies can be categorised by how immersive they are. The virtuality continuum is a theoretical framework introduced in 1994 by Paul Milgram and Fumio Kishino. It helps us visualize and understand the differences between the various technologies that exist today.

XR

Extended Reality is the real-world environment with technology overlapping, it includes AR, MR and VR. It is blurring the line between the physical and the digital world. The technologies AR and MR overlap with reality and thus also create different impressions and impact towards the environment. We could see that when Pokémon Go came out.

There are no mental models in how to interact in XR, it’s a new area and a lot has to be designed, tested and standardized.

AR

Augmented reality allows us to overlay digital elements into the real world. Using a screen that display real surroundings with digital elements, but they don’t interact in any way. It has its limitations but is still extremely powerful, not for immersive environments but can be used as a tool for solving problems.

MR

Mixed Reality goes a bit further because the digital overlay can interact with the physical world. MR gets input from the environment and will change according to it. It removes the boundaries between real and virtual interaction via occlusion. Your physical surroundings become your boundaries. The lines here became blurry what really exists and what seems to exist in the real worlds.

VR

Like the name suggests Virtual Reality is an immersive digital environment and the physical world has no part in it. VR takes advantage of the visual and auditory systems, this world seems real to us.

The perception of our environment has a huge effect on us. We should keep that in mind. VR should never be too intense for us to handle, like standing on a plank at the top of a skyscraper and looking down. We need to feel save when entering a new world. Participants should always know that the extensions aren’t real but can still enjoy the journey. Like watching a movie.

spectrum of immersion

References:

Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR? https://www.interaction-design.org/literature/article/beyond-ar-vs-vr-what-is-the-difference-between-ar-vs-mr-vs-vr-vs-xr

XR: VR, AR, MR—What’s the Difference? https://www.viget.com/articles/xr-vr-ar-mr-whats-the-difference/

by Victoria Reichinger -

#7 Sustainability x Augmented Reality

In order to wrap up the topic, I would like to engage a collaboration of sustainability and augmented reality by showing how AR could improve sustainability in e-commerce. Additionally to the 6 criteria of sustainable fashion, AR could push sustainable practices even further.
AR could support responsible fashion brands to:
• Help/nudge customers to practice responsible consumption by ordering what they actually like/ what them actually fits
• Reduce waste and emissions caused by returns (transport, labels, packaging, …)
• Display brand transparency
• Level up customer satisfaction to get more customers and a bigger budget for producing clothes ecologically and fair

Augmented Reality is a powerfully engaging medium that can support and promote sustainable practices. It can educate consumers, improve brand transparency and loyalty, and spread the messages most important to a company’s sustainable endeavours. 

by Hannah Nopp -

AR in education #6: Things to consider when designing educational AR products

This blog entry will be a growing collection of questions that educators, designers and practitioners in general need to consider when designing/developing educational AR products.

Questions, questions and more questions

  • Who is the target group? What’s their educational level?
  • What is the learning environment? —> Classroom? Distance learning? Workplace? Indoors? Outdoors? …
  • What contents are to be conveyed?
  • Which part of the content to learn should be enhanced by AR?
  • What goal(s) should be achieved by using AR technology?
  • In what proportion will real and augmented content be combined?
  • How is the content prepared didactically?
  • Which AR device(s) will be used?
  • Which AR technology fits best? —> Trigger-based, View-based?
  • What are the advantages of AR in the learning context compared to traditional approaches? —> Which added value has AR in this case?
  • How can multiple senses be addressed?
  • How can cognitive overload be avoided?
  • How can teachers easily and quickly add/adapt content?

(to be continued)

by Hannah Nopp -

AR in education #5: Advantages and Disadvantages

Hello again! In the following blog entry I will be writing about the advantages and limitations of using AR technology in the educational sector, which many studies already have been conducted to establish.

Advantages & Benefits

Many studies indicate that the use of AR in the educational field brings many benefits. According to a meta-review by Garzón, Pavón and Baldiris (2019), which analyzed 61 scientific publications, a total of 100% mentioned some kind of advantage when using AR systems in education. The following factors are the main advantages mentioned in their paper:

  • Learning gain: When using AR systems, students can improve their academic performance or even obtain better scores than students using traditional approaches. This improvement was reported not only by data, but also for different teachers and the students themselves
  • Motivation: The use of AR can increase the motivation of students as well as their level of fun while learning, compared to other pedagogical tools
  • Sensory engagement: When AR activates multiple senses, knowledge retention can improve.
  • Abstract concepts: AR can ideal to explain unobservable phenomena (i.e. the movement of the sun)
  • Autonomy: AR technology can not only help retain knowledge, but also gives students the possibility of retaining it for longer periods of time compared to other pedagogical methodologies
  • Memory retention: The combination of real and virtual worlds can increase the autonomy of students taking into account their natural abilities and motivation for using technological devices
  • Collaboration: AR can create possibilities for collaborative learning around virtual content which can facilitate learning, since it allows learners to interact with their partners, as well as with the educational content
  • Accessibility (not further described in the study)
  • Creativity (not further described in the study)

In a blog (not scientific!) by Sinha (2021) I found some more advantages of AR in education, that were not listed in the aforementioned study: 

  • Easy access to learning materials anytime, anywhere: AR could replace textbooks, physical forms, posters, and printed brochures. This mode of mobile learning could also reduce the cost of learning materials and make it easy for everyone to access
  • Safer practice: In cases like practicing a heart surgery or operating a space shuttle can be done with AR without putting other people in danger or risking millions of dollars in damage if something goes wrong

Disadvantages & Limitations

According to the aforementioned meta review by Garzón, Pavón and Baldiris (2019) 15% of the reviewed publications reported some disadvantages or problems when using AR in educational settings. The following factors are the main disadvantages mentioned in their paper:  

  • Complexity: Complexity can be an issure especially when designing for children. AR being a novel technology, which involves multiple senses, can become a very complex tool especially for those who do not have technological abilities
  • Technical difficulties: Technical problems like latency of wireless networks or limited bandwidth can become a problem as well as lack of teachers’ experience with tech
  • Multitasking: AR applications can demand too much attention, which can be a distraction factor. This can cause students to ignore instructions or important stages of the experience
  • Resistance from teachers: Some teachers may prefer having total control over content, despite recognizing the benefits of using AR applications

In a blog (not scientific!) by Omelchenko (2021) and another blog by Aleksandrova (2021) I found some more advantages of AR in education, that were not listed in the aforementioned study: 

  • Need of proper hardware: The use of AR requires at least a mobile device like a smartphone or tablet (which has to be up-to-date in order to install AR apps), which not all students may have
  • Content portability issues: An AR app needs to work equally well on all platforms and devices

Conclusion

Many studies indicate that AR has the potential to make learning processes faster, more fun and more effective. But some also point out that there are also several problems that can occur when AR is used in educational setting. Some studies also state that the context in which this technology is more effective than other educational media is still not clear and needs further research (Hantono, Nugroho & Santosa, 2018). Some future work could focus on support for teachers in adding and updating content as well as the comparison of AR to traditional teaching methods based on empirical data. It would also be important to do further research on special needs of specific user groups and accessibility features (Garzón, Pavón & Baldiris, 2019).

_____

Sources:

Aleksandrova, M. (2021, 17. August). Augmented Reality in Education. Dzone.Com. https://dzone.com/articles/augmented-reality-in-education

Garzón, J., Pavón, J., & Baldiris, S. (2019). Systematic review and meta-analysis of augmented reality in educational settings. Virtual Reality, 23, 447-459.

Hantono, B., Nugroho, L.E., & Santosa, P.I. (2018). Meta-Review of Augmented Reality in Education. 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE), 312-315.

Omelchenko, S. (2021, 5. Dezember). Augmented Reality in Education: Use Cases, Benefits & Examples. Program-Ace. https://program-ace.com/blog/augmented-reality-in-education/

Sinha, S. (2021, 12. Mai). Augmented Reality In Education: A Staggering Insight Into The Future. eLearning Industry. https://elearningindustry.com/augmented-reality-in-education-staggering-insight-into-future

by Victoria Reichinger -

#4 AR in e-commerce – what’s already out there?

The fourth in the league of blog entries on the topic of UX in the fashion industry offers some examples of companies that are using augmented reality for their e-commerce businesses. This part of my research is dedicated to find out how companies include the technology and what goals they are pursuing with it.

According to IBM’s 2020 U.S. Retail Index report, the pandemic accelerated the shift to digital shopping by roughly five years. According to a Neilsen global survey from 2019, consumers listed Augmented and Virtual Reality as the top technologies they’re seeking to assist them in their daily lives. In fact, just over half said they were willing to use this technology to assess products. AR has proven that it can add enormous value for consumers in the shopping journey. Therefore, some brands are already re-imagining retail to provide a better shopping experience for their users.

Taking a look at existing products

A virtual “try-before-you-buy” experience is already implemented by IKEA, offering a free App called IKEA Place. The app lets users virtually place furnishing in their space.

The home improvement retailer Home Depot added augmented reality capabilities to its app and mobile web before the pandemic. The feature helps lift engagement and conversion as consumers spend more time shopping on their phones.

For consumers to virtually try on luxury fashion, AR became an essential tool for brands such as Louis Vuitton and Gucci. Among some innovative AR collaborations, Gucci launched a virtual shoe ‘try-on’ called Lenses through the Snapchat platform.

Gucci’s virtual shoe try-on
With AR, Louis Vuitton allows users to play around with their luxury products

Physical stores reopening again, requires a high level of hygiene and safety. In response beauty retailer Sephora offers customers to test out makeup products with AR. Another beauty retailer, Ulta, uses the app GLAMlab to increase customers engagement to find the right shade of foundation.

The jewellery brand Kendra Scott introduced an AR tool enabling customers to virtually try-on different earring styles from the comfort of their homes.

All AR examples have the same aim: to help consumers to find the right product by offering them a preview of the product. Ar in retail seems to work and boost conversion for products such as furniture, home appliances, accessories, jewellery, shoes and makeup. Nevertheless, I couldn’t find running AR applications that are used for trying on clothes virtually and helping customers to find the right fit and size. Since body types differ enormously, clothes look different on everybody and pose a great challenge for implementing AR in the online shopping experience for garments.

In order to make this topic more tangible, I will take a look at the approaches of size guiding of clothes in online shops. The next blog entry will show best practice examples of size guiding online.

REFERENCES

https://www.ibm.com/thought-leadership/institute-business-value/report/consumer-2020

https://hbr.org/2020/10/how-ar-is-redefining-retail-in-the-pandemic?registration=success

Augmented retail: The new consumer reality
‘AR’-commerce – a novel and functional shopping experience
by Hannah Nopp -

AR in education #4: Taking a look at existing products

Hello again! For this blog entry I had a look at several educational AR apps (there are a loooot of them) in order to get a picture of when AR has added value for educational purposes and when it doesn’t. So I picked out a few examples and categorized them in good and bad ones and summed up why I did (not) like them. It’s also to mention that I only looked at digital apps that use visual augmentation. But first I want to give a short overview on the wide range of educational fields and educational levels existing AR products on the market cover (this list provided by Garzón, Pavón and Baldiris [2019] might not be complete):

  • Educational fields: Natural sciences, Mathematics, Statistics, Abstract concepts, Arts, Social sciences, Engineering
  • Education levels: Early childhood education, Primary education, Lower secondary education, Upper secondary education, Post-secondary non-tertiary education, Short-cycle tertiary education, Bachelor’s or equivalent level, Non-schoolers (work related trainings) – It’s to mention that educational AR products for Master’s or equivalent level and Doctoral or equivalent level might exist, but weren’t conducted in the study

The good

Augmented Creativity 

Augmented Creativity includes a total of six prototypes that can be used with mobile devides: Coloring Book, Music Arrangement, Physical-Interaction Game, City-Wide Gaming, Authoring Interactive Narratives and Robot Programming – I had a look at the first two of them. 

The Coloring Book is an application available that brings colored drawings to life: It comes with several templates that can be printed out and colored. When the drawing is scanned with the app on a smartphone or tablet (iOS and Android), it detects and tracks the drawing and displays an augmented, animated 3D version of the character, which is textured according to the child’s coloring (See Fig. 1).

Advantages the authors mention: 

  • Creative Goal: Fosters imagination, allows character individualization, helps to express feelings about character
  • Educational Goal: Improves coloring skills, 3D perception, and challenges imagination
  • Potential Impact: User-painted characters and levels, scripting virtual worlds through coloring

Why I like it:

  • The augmentation doesn’t intervene the act of drawing and coloring by hand (which I think is an important way of creative expression in early ages), but adds additional value by digitalizing it afterwards
  • Stimulates several senses
  • Works really well and looks super cute (smooth animations; exact coloring; live updates)
Fig. 1: Augmented Creativity – Coloring Book

The Music Arrangement is a set of flashcards where each card represents a musical element like instruments and music styles. The user can then choose instruments and styles independently and rearrange the song as imagined. By placing a card on a physical board, the app detects the marker on it and displays an augmented version of the instrument and plays the corresponding audio, as depicted in Fig. 2. AR even allows the user to change the position and the volume of the instruments while the song is playing, allowing them to direct the virtual band.

Advantages the authors mention: 

  • Creative Goal: Experiment with different instruments and styles to rearrange a song
  • Educational Goal: Teaches concepts of arrangements, styles, and the disposition of the band components
  • Potential Impact: Collaborative music arrangement experience, learn about the disposition of an orchestra

Why I like it:

  • Combines physical and digital interaction 
  • It stimulates several senses
  • Works really well and looks super nice
Fig. 2: Augmented Creativity – Music Arrangement

Quiver Education

Quiver Education is similar to the Coloring Book mentioned above, but with a greater focus on educational content: The user can choose from a range of coloring packs, print them and color them by hand. When the coloring is scanned with the app on a smartphone or tablet (iOS and Android), a colored, animated 3D model is displayed and additional information and interaction options are provided (see Fig. 3). The content is designed around topics as diverse as biology, geometry, the solar system and more. 

Why I like it:

  • The augmentation doesn’t intervene the process of coloring by hand
  • Stimulates several senses
  • A wide range of topics
  • ~ I’m still a little sceptical if it’s necessary to color a scene first in order to learn about it (i.e. a volcano)
Fig. 3: Quiver

Merge EDU

Merge EDU engages students in STEM fields with 3D objects and simulations they can touch, hold and interact with. The special thing about Merge is that the user has to hold a special cube in their hands where the augmentation is placed on, so the user feels like actually holding the object in their hands and can then interact with it (See Fig. 4). Merge is available for iOS and Android and can be used with mobile devices – It also offers glasses where a user can put their phone in to have their hands free to interact with the cube. 

Advantages the authors mention: 

  • 3D tactile learning
  • Flexibility: Can be used at home and at school
  • Curriculum aligned
  • Multisensory Instruction
  • Spatial Development
  • Accelerate Understanding
  • Focused Engagement

Why I like it:

  • The potential of the cube: It could potentially replace physical teaching aids
  • Big library of topics to explore
  • Users can upload and share their own creations
Fig. 4: Merge EDU

Human Anatomy Atlas

With the Human Anatomy Atlas medical students can turn any room into an anatomy lab: They can view and dissect a virtual model of a human organ or complete human body by scanning a printed picture (see Fig. 5) or simply placing a model on a flat surface (see Fig. 6). It’s also possible to study human muscles in motion by scanning a person as shown in Fig. 7.

Why I like it:

  • Students can study from anywhere and don’t have to go to an actual lab
  • Doing a dissection virtually might be helpdul to prepare for doing a dissection in real life (As far as I know from several people who are currently studying medicine, preparation for dissections is mostly done with the help of books, pictures, videos and physical models, but not with interactive digital models)
Fig. 5: Human Anatomy Atlas – Image marker
Fig. 6: Human Anatomy Atlas – Placing an object in space
Fig. 7: Human Anatomy Atlas – Live tracking of muscles

The bad

Sketch AR

With Sketch AR users can learn how to draw by using their smartphone camera: They can choose a sketch from a library and display it on a sheet of paper in front of them. The user can then follow the virtual lines on the paper step-by-step (See Fig. 8). The app also offers more features like minigames and AI portraits, but I only had a look at the AR feature. In general the app is designed really well and is also personalizable, but all in all I did not see the added value that AR has in this case.

Why I don’t like it:

  • Drawing might be difficult when looking at the paper though a small screen
  • While drawing I personally like to fixate the paper with one hand, which is not possible, because you have to hold your mobile device
  • I don’t see the advantes of AR compared to common image tracing (by printing it out and using it as a template)

An app that does pretty much the same is “Tracing Projector”, where I also don’t see the added value.

Fig. 8: SketchAR

On a general note

There are a lot of apps on the market – especially in children’s education – that try to replace a physical game with a digital one (i.e. playing with dominos), which is in my opinion not what AR should be used for. AR is supposed to enhance the user’s physical world and not replace it. I believe that it’s important to experience the world with as many senses as possible – especially in early ages – and haptic experiences should not be limited to holding and controlling a smartphone. Furthermore there are a lot of apps where the user can just randomly place 3D objects in the real world, but can’t do anything with them, which might be fun and playful though, but doesn’t have many educational values in my opinion.

 

That’s it for today, bye and good night! 

_________

Sources:

https://studios.disneyresearch.com/augmented-creativity/
https://quivervision.com/products/apps/quiver-education
https://mergeedu.com/
https://www.visiblebody.com/ar
https://apkpure.com/sketchar-create-art-and-get-nft-instantly/ktech.sketchar

Garzón, J., Pavón, J., & Baldiris, S. (2019). Systematic review and meta-analysis of augmented reality in educational settings. Virtual Reality, 23, 447-459.

Zünd, F., Ryffel, M., Magnenat, S., Marra, A., Nitti, M., Kapadia, M., Noris, G., Mitchell, K., Gross, M.H., & Sumner, R.W. (2015). Augmented creativity: bridging the real and virtual worlds to enhance creative play. SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications.

by Magdalena Samonigg -

Augmented and Virtual Reality Exhibitions

Museums and exhibitions aim to bring their collections to live. Since the ongoing development of augmented and virtual reality technologies it seems obvious to integrate them in the classical exhibitions. Through the usage of AR and VR technologies, museums can add a virtual layer to their exhibitions and create immersive experiences. Some areas of application could, for example be, allowing users to explore Egyptian burial chambers, meet historical characters or find out more about an artist by virtually visiting their hometown.

As part of a study, the Research Centre of Excellence in Cyprus (RISE) has interviewed 15 global museums about their experience in including AR and VR technologies in their exhibitions. Around 50% of them stated, that they made use of these technologies in order to create an augmented spaces for visitors to experience the exhibition, for example in form of a virtual time travel. They integrated VR and AR experiences in their exhibitions as an extension to the classic exhibitions, instead of outclassing them.

Another possibility to create a virtual exhibition can be done by scan exhibitions and arrange them in a virtual space. In this way, exhibitions can be accessible from all around the world. It could also enable a larger audience, for example disabled people, to visit exhibitions they could not visit in the real life.

Examples

Mona Lisa: Beyond Glass

Source: https://www.viveport.com/18d91af1-9fa5-4ec2-959b-4f8161064796

The Virtual Reality experience “Mona Lisa: Beyond Glass” was part of the Leonardo da Vinci blockbuster exhibition taken place at the Louvre in Paris, in October 2019. Through the use of animated images, interactive design and sound, it allowed the users to explore it’s details, the wood panel texture and how it has changed over the time.

Source: https://www.gmw3.com/2018/02/national-museum-of-finland-offers-virtual-time-travel/

The National Museum of Finland enabled their visiters a virtual time travel back to the year 1863, by letting the users walking inside the painting “The Opening of he Diet 1863 by Alexander II” by R. W. Ekman. In this VR experience the visitors could speak with the emperor and representatives of the different social classes or visit historical places.

References

by Marton Szabo-Kass -

Haptics & driving safety

| A summary of an interesting research paper that fits well into the multimodal view of my research on in-car AR solutions.

All information summarised in this blog post was taken from the research survey cited at the end of the post, which contains the exact sources of the statements.

“The Use of Haptic and Tactile Information in the Car to Improve Driving Safety: A Review of Current Technologies” – by Y. Gaffary and A. Lécuyer, 2018

The paper summerizes results of experimental studies in the above mentioned topic, categorizes them and discusses findings, limits and open ends.

Several instruments and devices on a car’s dashboard require visual attention from the driver, who is already busy with the driving tasks. While the visual and auditory channels are highly engaged, the tactile and kinesthetic channels could be used for additional, parralel input.

Several sources of the paper state that the haptic feedback can be perceived despite of high cognitive load, more effectively than visual or auditory feedback.

Within the haptic modality there are two kinds of possible feedback:

  • tactile feedback: perception from the skin
  • kinesthetic feedback: perception through muscular effort (force feedback)

Haptic technologies in cars

For transfering haptic feedback, the actuators need to be fitted to specific positions in the car’s interface, to have a direct connection to the driver: steering wheel, pedals, seat, seat belt, clothes and the dashboard.

A source, Van Erp and van Veen classified the information that could be transferred through haptics in cars:

  • spatial information about surrounding objects
  • warning signals
  • silent communication only with the driver
  • coded information about statii
  • general information about settings

This paper focuses on two groups: haptic assistance systems (feedback triggered by voluntary action) and haptic warning systems.

Haptic assistance systems

Controlling the car’s functions

Several sourced of this paper analysed the influence of tactile feedback on the “eyes-off-road time” with rotary knobs and sliders on the dashboard, central console and steering wheel (the main sources of haptic feedback). The devices had clicking effects or could change their movement friction or vibration frequency. The results were the most effetive with visuo-haptic-feedback (combining visuals and haptics), reducing the glancing time by ca. 0.5 s and 39%. One study resulted in the preference of 230 Hz vibration on the steering wheel over lower frequencies. At this input method the vibrations of the road are a limiting factor.

Maneuver support

The paper states that the main source of haptic help for maneuvring is kinetic feedback on the steering wheel. Several studies were mentioned looking at difficult driving situations: parking, driving backwards with a trailer, low visibility. In all of these cases the results showed positive improvements (lower mental demand while same performance), when force feedback was helping the driver to steer in the right direction at the right time.

Navigation

For preventing additional visual or auditorial load and distraction, studies were described on using different actuator placements to give directional feedback to the driver. Such examples were besides the steering wheel the augmentation of waist belts or the driver’s seat with actuator matrices, indicating navigational directions. The results showed less distraction than with only auditory guidance, and even a 3.7 times less failure rate with haptic-auditory feedback.

Haptic warning systems

Awareness of surroundings

Similarly to the navigational purposes, current studies described in the paper propose the augmentation of waist belts and seats for giving directional information as warning signals about surrounding cars or other objects – most importantly in blind spots or behind the vehicle.

Collision prevention

Collision prevention needs fast driver reactions, once the danger is noticed. According to the paper, haptic feedback can significantly improve reaction times. As collision warnings are also based on spatial information, therefor same methods were analysed in studies as for helping navigation or awareness of surroundings – augmented belts, seats and pedal. One system with actuators in the seat showed improvements in spatial localization of threats by 52% compared to only audio warnings.

Lane departure

The main methods to warn about lane departure were tactile and kinesthetic feedback on the steering wheel. As the direction has to be corrected by turning the wheel, the drivers responded intuitively on the augmentation of the wheel with vibrators and motors. These solutions can be found widely spread in the automotive industry. Vibrotactile seats and pedals were also tested and found to work better, be less annoying and cause less interference than audio warnings.

Speed control

As the accelerator pedal is the device of controlling speed, this survey reports many studies to be found on its augmentation. They are looking at implementation of tactile feedback and also force feedback (resistance to pressure and controlled reaction force). Both methods lead to positive results in adjusting too high speeds and maintaining a given speed, and reported by users to be satisfying and useful.

Limits of existing experimental protocols

There are several limiting factors described, which should be considered for further analysis:

  • The age of users and the differences in perception of haptic feedback. Older people seem to be more affected by them.
  • Augmented seats: the thickness of clothing, the height and the weight of the users.
  • Different ways (habits) of holding and turning the steering wheel.
  • Static vs. dynamic signals can have different effects (dynamic signals were seen to be more effective).
  • Effects of multiple haptic feedback systems working parallel in the same car have to be analysed.

Almost all of the described researches were done with the help of driving simulators. They can deliver compareable results but do not fully represent the real driving environment. Realistic stress and also overconfidence in the feedback systems were not analysed either.

My Summary

During driving the driver is under high visual and auditorial cognitive loads from th basic tasks. In these cases haptic feedback can be a very effective solution to trigger reactions of the driver. The interfaces to be used are limited to the areas with which the driver is permanently in contact (steering wheel, seat, pedals, clothes), except the dashboard for changing car functions and settings.

It can be concluded that it makes sense to augment those interfaces with haptic feedback which are relevant for the specific tasks the feedback relates to. For example tactile or force feedback on the steering wheel for maneuvring support or lane departure warning and haptic feedback from the accelerator pedal for speed keeping warnings.

It is interesting to see that spatial information can be perceived well through the body via vibrator matrices in augmented seats. This method carries more limitations than interfaces touched by the hands though.

The most effective solutions seem to be combinations of modalities (visual-haptic, auditory-haptic feedbacks), but in all cases the situations and possible use cases have to be considered as well. E.g. a vibration of the seat can be percieved well while parking slowly, but not while driving fast on a bumpy road…

As the information gathered from this paper is based on simulated experiments, I will also try to find further studies or at least reports on currently implemented haptic systems in production cars.

Source

Gaffary, Y. and Lécuyer, A., on Frontiers in ICT 5:5: The Use of Haptic and Tactile Information in the Car to Improve Driving Safety: A Review of Current Technologies; 2018.
Retreived on 12.12.2021.
https://www.frontiersin.org/articles/10.3389/fict.2018.00005/full