Dangers
The “smart city”, where do these ideas come from? is the question raised by Adam Greenfield, in his talk with reSITE, a forum for city development. As smart cities are frequently mentioned in the context of discussing the future of cities I want to highlight some of the negatives associated with it.
One definition of smart cities is that they are, “imbedded networked informatics in every surface, object and relation of the city.” (Meaning everything should be smart). Sounds a bit dystopic doesn’t it?
By this definition smart cities lean towards providing a lifestyle of convenience, consumption and security (for a few). By saying the few, I’m talking about the ones that access the “smartness” of the city they live in. From a lot of cases, e.g. IBM’s smart city, it’s apparent that only top-positioned decision makers can access the data that the population of the city generates. Until now smart cities are largely controlled by large scale actors with their own capitalistic agenda.
Synchronized and analyzed efforts among sectors and agencies as they happen, giving decision makers consolidated information that helps them anticipate problems and manage growth and development
IBM – defining their smart city efforts
The goal of such a city is to optimally regulate and control resources by means of autonomous IT systems
Siemens – about smart cities
A complete picture of building state, usage and operations continually maintained, allowing constant optimization of energy, resources, enviroment, and occupant support and convenience systems.
Living PlanIT
These visions painted by some of the biggest actors behind our smart cities include words like “manage growth”, “optimally regulate and control resources”, “continually maintained, allowing constant optimization”. As Greenfield argues in his reSite talk, these words are purely focused towards the convenience of managing actors completely separated by the people generating the data. Said in an dramatic pictorially manner, “Reflecting lights of our faces to see what engagement we have with what’s around us. But preserving it from the ones that are generating it.”
Referring back to the utopian cities such as Brasilia these smart cities definitions have similarities. As the utopian cities had tendencies of shredding old cities into clean slates to rebuild perfect societies, these smart cities builds a rigid system of managing people of the cities. In the end using politics and strict goals for the cities as means for changing people’s behavior to reach those goals.
“Order is built over time by an infinity of small acts.”
Jane Jacobs
This quote by the famous urbanist and activist Jane Jacobs goes to say that we ought to be careful how we implement change as it’s slowly defining order over time.
A side-effect of such acts are “gentrification”, which means investing in a middle-class district in order to improve it’s quality, often leading to chasing away current inhabitants and replacing them with wealthier people, often in central areas.
The smart city industry is a Trojan horse for technology companies. They come in under the guise of environmentalism and improving the quality of life, but they’re here for money.
Bianca Wylie, The Washington Post
https://www.ft.com/content/f866dfc8-a4b3-11e9-974c-ad1c6ab5efd1
Dreams cause action
In a lecture on the topic of cities of tomorrow Winy Maas opens by saying, “Dreams will be there to be developed by students. How we paint our cities is how they will look like and be interpreted.” Those are words of the world famous architect which tells how he is visioning the cities of tomorrow, but they are not just words. MVRDV as his architecture company is names have numerous projects around the world backing these words.
In Rotterdam MVRDV constructed a bridge reaching towards the sky. As a symbol of an elevated future which is accessible for everyone. The staircase is a steel and glass construction, but as it was adopted by the inhabitants and by-passers of Rotterdam it became much more. Activating events such as fashion shows, weddings, tourism e.g. the bridge became a activator of people.
The skybridge of Rotterdam is a example of Winy Maas’s vision of changing cities. Sights, areas and infrastructure as activators of people creates sparks and identity of cities. “Courage in the cities can activate a chain of activities in an area.” he says. change by change, spark by spark he wished to transform cities by organic change.
It’s a city out of connections, not blocks and towers
Winy Maas
“Connections will stich the cities together. Don’t think in blocks and buildings, think in connections.” is how Winy Maas goes about doing architecture. He want to develop cities district by district, slowly sparking change. Using cheaper affordable accommodations for the middle class, transforming them into buildings interacting with their audience by e.g. skybridges is a vision they have in MVRDV. The believe that a mix of analog and technological interactions is where we are going.
Seoul, in South Korea, is another area where they prove their concepts in their doing. It previously was a grey city, until the mayor made a huge lawn in front of his house, accessible to everyone later leading to the project “sky garden” which was won by MVRDV in a competition. With small iterative steps they developed a system of buildings with varying plants surrounding and inhabiting them. Cafes, toilets, shops e.g. in a collection of building sized flower pots now creates a green sphere in districts of Seoul. Each “oasis” attracted heaps of locals and tourists further kickstarting the project by mere attention and interactivity. Now Seoul already is and envisions itself as a greener city making ripple effects of democratization.
“Could it be more dense, and more green”
Winy Maas
More dense, more green, that’s where Maas wants cities development to head. With clear similarities to the mindsets of new urbanists and visionaries previous to him. His approach is change in one neighborhood at a time, affecting all it’s connections each time. Dreaming about the future possibilities will create sparks in fellow architects, designers and politicians as well as inhabitants in his opinion. Meaning putting action behind dreams small increments at a time will push us where we want to be in an organically manner.
User-Centered Perspectives for Automotive AR
| a short summary of a paper on human aspects related to automotive AR application design
A research paper, titled like this blog post, by experts from the Honda Research Institute (USA), the Stanford University (USA) and the Max-Planck-Institut für Informatik (Germany) [1] discusses benefits, challenges, their design approach and open questions regarding Augmented Reality in automotive context with a focus on the users.
Augmented Reality can help drivers in pointig out important and potentially dangerous objects in the driver’s view and increase the driver’s situation awareness. Though if the information is presented incorrectly, the distraction and confusion of the driver can lead to dangerous situations.
The authors of the paper put up a design process with focusing on the appropriate form of solution to a driver’s problem (rather than just describing ideas technically).
To understand the drivers’ problems in the first place, they conducted in-car user interviews with different demographic groups to gather information about driving habits, concerns and the integration of driving into daily life.
After the interviews they ideated prototype solutions and tested concepts in a driving simulator with a HUD. One realization was that at a left turn, drivers needed more help in timing the turn according to oncoming traffic rather than an arrow or graphical aids for the turning path – which even distracted them from the oncoming traffic. The design solutions of the authors therefor focus on giving the driver additional cues to enhance awareness rather than giving only navigation commands. After researching different graphical styles of turning path indication, results showed less distraction with solid red path projection – that is visible in the peripheral vision while focusing on traffic – than simple chevron style lines.
Human visual perception
Regarding human perception, the authors of the paper analized influences of visual depth perception and the field of view. The human eye is built to focus on one distance at a time, so AR displays / Head Up Displays can cause a problem due to their see-through design. The driver’s focus has to remain on the road ahead and not change to the windshield’s distance, blurring out the farther imagery.
The eye’s foveal focus with the highest acuity is only at a ca. 2° center area of the vision field. This determins the so called “Useful Field Of View” (UFOV), the limited area from which information can be gained without head movement. These restrictions imply the use of augmented systems only in the driver’s main field of view, and not throughout the whole windshield. Objects in the peripheries should be therefor signalized either inside of the UFOV or through other methods.
Distraction
The National Highway Traffic Safety Administration (NHTSA) of the USA states three types of driver distraction:
- visual distraction (eyes off road)
- cognitive distraction (mind off driving)
- manual distraction (hands off the wheel)
Each of these types can be aided but also caused by Augmented Reality applications in vehicles.
The authors discussed the human attention system and cognitive dissonance problems further.
- Attention system
Regarding the human attention system, the so called “selective visual attention” and the “inattentional blindness” can be problems in driving conditions. Important visual cues can be suppressed when the driver is focusing on secondary tasks or if they are outside of the focus of attention. Warning signs on a HUD can either help by attracting attention, but also distract from other objects that are outside of the augmented field of view. The study states the need of further research on the balance between increasing attantion and avoiding unvanted distraction.
- Cognitive Dissonance
Cognitive dissonance, the perception of contradictory information, could occur e.g. with bad overlapping of 2D graphics on the 3D vision of the surroundings, causing confusion or misinterpretation of the visual clues.
Human behaviour
As a third category, the study discusses the effects of AR technology on human behaviour.
Situation awareness – maintaining state and future state information from the surroundings – is detailed by a source in three steps:
- Perception of elements in the environment
- Comprehension of their meaning
- Projection of future system states
Augmented Reality can help drivers not only in perception but also in the further steps. State-of-the-art computers, AI technology and connected car data from surroundings can be especially of help in cases where additional computational power can predict traffic dynamics. [comment of MSK]
One aspect is the behavioural change of drivers after longer use of assitance systems. A study implies that the reduced mental workload could lead to the retention of the drivers’ native skills. Further, the phenomena called “risk compensation” can occur after getting used to the aids. This means a riskier behaviour of the driver than normal, due to higher confidence in the surroundings. These behavioural changes can have dangerous consequences, why the authors suggest the use of driver aids only when needed.
According to one source, the user’s trust in a technology is can be increased with more realistic visual displays, like AR rather than simple map displays. Further, AR can also help to build trust in autonomous cars, communicating the system’s perception, plans and reasons for decision making.
Some open questions were stated at the end of the paper, to be considered further on.
Such were for example how multiple aiding systems can interact at the same time? Or how will the use of AR over longer time effect the drivers’ behaviour and skills when they have to switch back and drive a non-AR vehicle? Will the drivers’ skills deteriorate over and will they become dependent on these aiding systems?
My conclusion
This paper was published in 2013, since when the technology was significantly developed further. Nevertheless the basic principles and human factors are still the same, which have to be considered when designing safety critical automotive applications.
Reliability and understanding the behaviour of autonomous vehicles will be an essential aspect in creating acceptance by the driver / passengers. Augmented Reality can be of much help not only for extra driving assistance systems, but also for the complete user experience at different automation levels.
The mentioned topics of human factors in this paper were only focusing on visual augmentation and assistance. These could be expanded to other modalities like sound and haptic augmentation, and analyse the perception of a combined driver assistance as well.
Source
[1] Ng-Thow-Hing, Victor & Bark, Karlin & Beckwith, Lee & Tran, Cuong & Bhandari, Rishabh & Sridhar, Srinath. (2013). User-centered perspectives for automotive augmented reality. 2013 IEEE International Symposium on Mixed and Augmented Reality – Arts, Media, and Humanities, ISMAR-AMH 2013. 13-22. 10.1109/ISMAR-AMH.2013.6671262.
Retrieved on 30.01.2022. https://www.researchgate.net/publication/261447349_User-centered_perspectives_for_automotive_augmented_reality
Analog/Digital Environments
In this article we will look at other types of installations and projections to see what other settings we can use as designers for immersive experiences.
Projection Mapping
Projection mapping is a 3D video projection technique using physical environments and objects as the surface for a projection, instead of a screen. Light, colors and oftentimes sound is used to outdraw a story with the edges of a building being the only boundaries for the scenery. Architectural elements, such as facades, are filled with life through precise lighting of the projection surface. The structure dissolves and the illusions take over. Houses bend or fall apart, 3D objects move in the direction of the audience.
AV Installations
Worth mentioning are Audio-Video Installations using giant or more smaller screens, showing videos or animations. They are often used in exhibitions and public places and are most times not interactive. But in the lobby of the Terrell Place in Washington DC there is an installation with motion-activated screens responding to the people going by.
Immersive Sensory Rooms
CAVE (Cave Automatic Virtual Environment) is a virtual reality environment consisting of a cube-shaped VR room you can enter. The walls, floors and ceilings are projection screens and with a VR headset, that is synchronized with the projectors, the user can walk around an image to study it from all angles. Sensors within the room track the viewer’s position to align the perspective correctly.
References:
Projection mapping https://www.barco.com/en/solutions/projection-mapping
Videomapping Projection https://www.visualimpression.de/en/3d-visualisierung/videomapping-projection/
Amazing Screen Installation http://www.fubiz.net/en/2016/07/30/amazing-screen-installation/
CAVE (Cave Automatic Virtual Environment) https://whatis.techtarget.com/definition/CAVE-Cave-Automatic-Virtual-Environment
Werbung ist implizit, Wahrnehmung individuell
Man sagt sogar, Werbebotschaften sollen am besten am Piloten vorbei direkt zum Autopiloten gelangen. Eine direkte Aussage sei in den meisten Fällen unwirksam. Daher muss ganz sensibel an das Kernmotiv herangeführt werden.
Es ist nicht unbedingt nötig, für eine implizite Werbung das Produkt großflächig im Vordergrund zu halten. Die Marke wird durch anderes veemittelt. Dafür muss man zuerst verstehen, was für Motive das zu bewerbende Produkt eigentlich hat. Hier sind wieder einmal Zielgruppe und Produktrecherche enorm wichtig. Wie wirkt das Produkt ect. von sich aus und was kommuniziert es?
Bedeutung alleine reicht für den Erfolg von Markenkommunikation nicht aus. Erst durch den Anschluss an relevante Motive lösen wir beim Kunden gewünschtes Verhalten aus.
Das Motiv wird nun in einen visuellen Code gepackt. Für diesen gibt es vier relevante Ebenen: Sprache, Geschichte, Symbolik und Sensorik. Sprache ist der explizite Code. Text oder Apell zum Beispiel. Die Geschichte gibt den Kontext für das abgebildete Geschehen. Symbolik wird von einzelnen Elementen und eben ihren Bedeutungen im Kontext vermittelt. Die Sensorische Wirkung impliziert Emotion – zum Beispiel durch eine bestimmte Farbwahl. Dieses Codesystem kann bei verschiedenen Werbeplakaten beobachtet und auf eigene Beispiele angewendet werden.
Bilder sind wirkungsmächtig. Sie erfüllen ihren Zweck – informieren, emotionalisieren ect. Damit ist nicht gemeint, dass sie eine Wahrheit wiedergeben. Der Mensch reagiert aber trotzdem immer sofort auf sie und ist ihnen sozusagen ausgesetzt. Damit ist die Abbildung ein Werkzeug, dessen Bedienung erlernt werden kann und das immer eine Aufgabe hat und nach Benutzung eine Folge hat.
Brink, C. Bildeffekte: Uberlegungen zum Zusammenhang von Fotografie und Emotionen. In: Geschichte und Gesellschaft, 37. Jahrg., H. 1, Geschichte, Emotionen und visuelle Medien (Januar – März 2011), pp. 104-129.
Held, D. Scheier, C. Wie Werbung Wirkt: Erkenntnisse des Neuromarketings. Haufe Verlag, 2006.
The Emotional Space | #10 | Next Steps
Now that the semester is coming to an end, there is finally a little bit of time to reflect upon what has happened in the last four months. And talking specifically about this project, it has come quite some way. In the past weeks I have finalized my exposé (which you can view and download below), worked more on the vision I have, laid out some basic design context and presented it to various people. I will keep this blog entry rather short and just touch upon some of the next steps.
The Emotional Space | #9 | Wristbands
I have been talking about wristbands in most of my recent blog entries and even mentioned that there is an own entry to come, dedicated to the wristbands. Well, here it is- the wristband post.
The choice of wristbands might seem like a small choice, but it actually had quite an impact on the conceptual design progress. To recap – the wristbands are supposed to measure motion data, which is most easily done by accelerometers and gyro sensors. And they should send this data with low latency (so basically in real time) to any module that can be interfaced with a computer.
Immersive Experience
In this blog post we will look at the fundamentals of immersive experiences. What makes them immersive in the first place?
What is an Immersive Experience?
An immersive experience is pulling a person into a new or augmented world, giving them space to explore and engage on their own and enrich everyday life via technology. We must start with a purpose when designing an immersive interaction or installation. Understanding the human condition, how they behave, feel and think about the world around them, gives us information how we can build stories they can be part of. Great immersive experiences make the user discover the story themselves. That’s really the key to creating a compelling immersive experience.
If the experience doesn’t have a purpose, the technology becomes a gadget. Making a 360° Video is nice, but if you can’t engage, if it doesn’t feel reel to you, then why bother. You could just use a screen.
You’ve got to start with the customer experience and work backwards to the technology. You can’t start with the technology and try to figure out where you’re going to sell it.
– Steve Jobs, 1997
Strategies to Create an Immersive Experience
1. Start with the Big Idea
Always start with the concept, the big idea, not the implementation and what tools to use. The experience is in the foreground, when people are using the tools (headset, controllers), they should melt into the environment and become invisible. This is the goal.
Start asking these questions: What do you want people to feel? To experience? What role do your visitors play in the story? What is their purpose? What is the message you want to leave them with?
When you know what their journey will look like you can choose the elements that will bring that idea to life.
2. Nail the Details
The little, transformative things are what bring a setting to life. Details in the visualization but other sensations as well, can make the experience becomes more immersive. Things like sound, video quality and intuitive interaction make the immersion even greater.
Also providing a sense of place and what is beyond that place gives a greater impression of the story and that is part of something larger. This is as true for an immersive environment that never leaves a small room, or an epic feature film.
3. Incorporate Location-Based Interactions
A powerful marketing tool is geofencing, it utilizes location data to establish a virtual territory. Like walking into a store and having a discount code automatically pop up on your phone. Location-based interactions can be used in different scenarios. A story could be presented on your phone while you go through an exhibition, showing you hidden details and facts to the objects. Or at a conference or meeting you could be easily connected with other people who share the same value as you do and locate them.
4. Enable Multi-User Interaction
Creating connections between people or make them interact together in the environment extend their experience. Enabling socializing in an event makes it more immersive.
5. Use Positional-Tracking Hardware
Most mobile VR headsets like Oculus Go, Samsung Gear VR and Google Daydream View only have rotational tracking, you can look around and tilt your head, but your position is fixed to one spot, you are not part of the world. A great VR experience lets you move through the virtual world physically to gain a feeling for the space. Positional tracking hardware uses sensors to assess motion and position relative to its environment.
A successful immersive experience is the sum of its parts, and sometimes the simplest use of technology is the best solution. The goal has to be well defined, what the user should see and feel. Interactions should be simple and take low effort.
References:
Designing for Interactive Environments and Smart Spaces https://www.toptal.com/designers/interactive/designing-for-interactive-environments-and-smart-spaces
What is an Immersive Experience And How Do You Create One? https://clevertap.com/blog/immersive-experience/
The 6 Secrets To Creating A Truly Immersive Experience https://www.forbes.com/sites/sethporges/2016/06/09/the-6-secrets-to-creating-a-truly-immersive-experience/?sh=27aeeb223918
How virtual reality positional tracking works https://venturebeat.com/2019/05/05/how-virtual-reality-positional-tracking-works/
Difference between VR, AR, MR, XR
Let’s start with the basics. We first must know the difference between these technologies, to be able to adapt future projects to the right environment.
Reality as a construct
What we perceive with our senses seems to be reality, whether what we perceive comes from the digital or the physical world. Take for instance watching a movie, we know it is not real, but it feels real to us. It triggers emotions, we feel empathetic to characters and we create connections with them.
It’s really important to understand we’re not seeing reality. We’re seeing a story that’s being created for us.
– Patrick Cavanagh, Research Professor
The virtuality continuum is a scale that goes from reality to virtuality. In it, technologies can be categorised by how immersive they are. The virtuality continuum is a theoretical framework introduced in 1994 by Paul Milgram and Fumio Kishino. It helps us visualize and understand the differences between the various technologies that exist today.
XR
Extended Reality is the real-world environment with technology overlapping, it includes AR, MR and VR. It is blurring the line between the physical and the digital world. The technologies AR and MR overlap with reality and thus also create different impressions and impact towards the environment. We could see that when Pokémon Go came out.
There are no mental models in how to interact in XR, it’s a new area and a lot has to be designed, tested and standardized.
AR
Augmented reality allows us to overlay digital elements into the real world. Using a screen that display real surroundings with digital elements, but they don’t interact in any way. It has its limitations but is still extremely powerful, not for immersive environments but can be used as a tool for solving problems.
MR
Mixed Reality goes a bit further because the digital overlay can interact with the physical world. MR gets input from the environment and will change according to it. It removes the boundaries between real and virtual interaction via occlusion. Your physical surroundings become your boundaries. The lines here became blurry what really exists and what seems to exist in the real worlds.
VR
Like the name suggests Virtual Reality is an immersive digital environment and the physical world has no part in it. VR takes advantage of the visual and auditory systems, this world seems real to us.
The perception of our environment has a huge effect on us. We should keep that in mind. VR should never be too intense for us to handle, like standing on a plank at the top of a skyscraper and looking down. We need to feel save when entering a new world. Participants should always know that the extensions aren’t real but can still enjoy the journey. Like watching a movie.
References:
Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR? https://www.interaction-design.org/literature/article/beyond-ar-vs-vr-what-is-the-difference-between-ar-vs-mr-vs-vr-vs-xr
XR: VR, AR, MR—What’s the Difference? https://www.viget.com/articles/xr-vr-ar-mr-whats-the-difference/