Designing with, for and through the sensorial richness of interaction design

  • February 2018 – June 2019: How interaction design can help to envision a tangible communication device for users that experience a physical gap?

  • September 2020 – … : In which ways designing with, for and through the sensorial richness of interaction design might contribute to how empathy is produced, expressed and perceived in computer-mediated communications?

FOREWORD

“In which ways designing with, for and through the sensorial richness of interaction design might contribute to how empathy is produced, expressed and perceived in computer-mediated communications?” is the question that I ask myself as a designer and a doctoral student. This research question is dynamic, and as results of various experiments, theory extensions, and data collection, it is expected to evolve and strengthen.

In this text, I initiate a dialogue to locate the context of my research. For this purpose, I think of my research, from two distinct vantage points: background and outline of the study. In the first view, I compound the motivations to pursue this research topic and acknowledge the diversity of situated contexts. In the second view, I determine the methods I am using, why these methods, and how I use them, and why in that way. Furthermore, I represent the objects I intend to develop as an embodiment of my research.

BACKGROUND OF THE STUDY

Empathy Crisis for Digital Natives 

To my way of thinking, why I am tackling this topic has many answers constituted by my first-person, situated, embodied experience and includes my phenomenological experience of the research topic itself. Hence, I start this text with a personal introduction. First and foremost, I was born in Turkey and have been working, studying and living in Estonia since 2017. Thus, I have been physically separated from my family by distance for several years now. I work as a lecturer of interaction design at the Estonian Academy of Arts and also study as a first-year doctoral student at the Academy. Occasionally, if I am asked what I am doing for a living, I reply by stating: “I am a designer-researcher and an educator based in Tallinn”. Furthermore, I am a digital native, born in 1993, grown up in an electronics-filled and increasingly online and socially-networked world, and I am a “native speaker of the digital language of computers, video games and the Internet” (Prensky 2001). With reference to Prensky’s definition (2001) of digital natives, it’s no surprise to me that I have developed a frequent computer-mediated communication (CMC) behaviour to connect with my family back home. 

CMC is a term that involves various forms of human-human communication through networked computers, which can be synchronous or asynchronous and encompasses one-to-one, one-to-many, or many-to-many exchanges of text, audio, and/or video messages (Lee & Oh 2015). My main communication channels are, online instant (synchronous) messaging platforms that involve mostly a one-to-one exchange of text, audio and video messages. Although CMC allows me to reconnect with my loved ones despite the physical gap, it also caused me to experience a lack of emotional connection with them. 

With my frequent usage of CMC, a feeling of distanced view by the screen arose between me and my loved ones. I started yearning for eye-contact with them, observing their gestures, smelling their smell and so on. This distanced view grew with time when the COVID19 pandemic caused me to not be able to leave Tallinn and visit my family back in Istanbul for over a year. Getting together by the video calls started to feel more and more hollow to me, I couldn’t feel my loved ones’ joy, pain, nor fear through the screen. I even started to believe that I am no longer an empathic person.

Empathy in the digital era receives close attention within both practitioners and theorists. I am particularly inspired by the research of Sherry Turkle (2015), borrowing concepts from psychologists, neuroscientists, and the field of human-computer interaction, Turkle assembles data (mostly in the form of stories and anecdotes) about an empathy crisis and what’s the part technology has with it. She points out that regardless of its many advantages when it’s used with intention, technology, and CMC, has fundamental downsides, the extreme of which relates to a decrease in digital natives’ capacity for empathy: an empathy crisis. According to Turkle (2015), excessive usage of CMC has impaired our potential to relate to each other, particularly in face-to-face conversational settings. Since such conversations are crucial to our moral development; without them, humans have trouble establishing bonds of empathy with different people.

Empathy in Design

Interaction design shapes our everyday life through digital artefacts (Smith 2002) and aims to create experiences and interactions that enable humans to achieve their objective(s) in the most pleasant way possible. Thus, understanding human needs, behaviours, and values become the core abilities of an interaction designer (Teo 2020). As my job as a lecturer of interaction design, I have the entitlement to introduce the young soon to be interaction designers to what interaction design is. And being an empathic person is an attribute that we cherish seeing in interaction design students. However, when I read more into it, I became alarmed to the way empathy is discussed in the design world.

The concept of empathy-building is commonly employed as a user research tool in the field of design, through for example observation (Leonard and Rayport 1997), experience prototyping (Buchenau and Suri 2000), role-playing (Kaario et al 2009), and empathy maps (Gray 2017). Accordingly, over the past years, terms like user-centred design (Norman 1986) or human-centred design (Cooley 1989) became popular in the field. Hence, Empathize is the name commonly given to the initial stage of the human-centred design process in which designers seek to understand their intended users to inform technology development (Bennett and Rosner 2019). The following stages can be summarized as Define, Ideate, Prototype, and Test. In the Empathize stage, the goal is to gain an empathic understanding of the people you’re designing for. This process involves observing, engaging, and empathizing with the people you are designing for to understand their experiences and motivations (Dam and Teo 2019).

I stand that the realization of such empathic design methods may bring with it many uncertainties and unintended consequences that could cause potential harm in how designers perceive empathy and in an oversimplification of the term. Because, when empathy is understood as the experience of co-feeling, it risks obscuring the complicity in the wider relations of power in which marginalization, oppression, and suffering occur (Pedwell 2014). Therefore, Empathy is not and cannot be something to be done, built, or modelled within the design (Bennett and Rosner 2019).

In the light of my first-person, situated, embodied experience and my phenomenological experience of the research topic itself, I became sensitive to the way empathy is discussed, perceived and defined not only in my personal life but also in my professional life. With reference to my experience as a digital native (Prensky 2001) and Turkle’s study (2015), I aim to investigate new ways to design for digital natives who are in need to have remote conversations and experiencing the feeling of being distanced by the screen. The goal is not to come up with palpable solutions, rather create opportunity areas through design artefacts.

Different views on Empathy

Empathy is materialised in different ways in different spheres, through different media, and through different types of encounters (Pedwell 2014). Contrary to popular belief, the concept of empathy is not only being praised but also being criticised in academic venues. A range of recent scholarship has argued how empathy, and design thinking packages more generally, work as a means of assuring designers that they have superior training and ethical tools to quickly assess and innovate on problems in domains they are unfamiliar with, a phenomenon Lilly Irani aptly characterizes as the “design savior” complex (Bennett and Rosner 2019). Other scholars have expressed their criticism for empathy and its limits not in the context of design but in humanities, such as Paul Bloom (2017), Carolyn Pedwell (2014), Namwali Serpells (2019), and Candace Vogler (2018). 

Bloom (2017) argues that in order to criticise empathy, an understanding of its different facades is necessary: cognitive empathy and emotional empathy. Cognitive empathy is about understanding others’ emotions, whereas emotional empathy is about feeling and experiencing others’ emotions. Bloom illustrates an analogy to explain why empathy, of the emotional type, leads to unfair, biased, and tragic results. “Empathy is like a spotlight that has a narrow focus, one that shines most brightly on those we love and gets dim for those who are strange or different or frightening”.

Pedwell (2014) points out that “the possibilities of entering another’s subjective and psychic world accurately are both tenuous and ethically fraught.” Similar to Pedwell (2014), Serpells (2019) also claims that feeling what others are feeling and can empathize with their experience is not possible in real life, nor is it ethical.

Vogler (2018) discusses the barriers of empathy by claiming that humans are strongly inclined to use themselves as guides to comprehend how things are for others. She argues that this may cause a failed attempt for empathy as it is harder to try to get a sense for someone else than it is to stick with our own perspective.

In the light of different views on empathy, I inquired about a definition that illustrates my vision of the concept. When talking about how to strengthen the connection with the loved ones through the screen, empathy is perceived differently than the empathy that designers need to acquire in the interaction designer role while working with presumably strangers. With reference to Pedwell (2014), empathy is materialised in different ways in different spheres, through different media, and through different types of encounters. In my research, I would like to avoid materialising empathy and better understand and position myself in the landscape of empathy clearly as a designer, a digital native, and a scholar.  

I reject empathy as an accomplishment, as a means to an end, and accept its barriers and handle them as opportunities. Empathy is an analytical activity that compounds the sense-making of the reasons behind one’s actions and emotions, it can be nurtured and extended through imagination. On balance, empathy is not a technique for understanding what others feel and trying to feel the same, but for seeking to imagine how they may be feeling and why. Even though “over-empathy can be impairing. A doctor who feels his or her patients’ pain will not be able to function to the best of his/her ability. The same applies to a therapist whose emotional empathy may be detrimental to developing a productive relationship with a depressed client, for example.” (Bloom 2017), we can focus on understanding and not trying to feel what others are feeling. Accordingly, humans could make room for an effective cognitive partnership that may help us destabilize and reimagine imposed boundaries of “me” and “you”. 

OUTLINE OF THE STUDY

It may appear that I developed a particular interest in the feeling of being distanced by the screen just recently, however, I have been exploring this theme for a few years. In 2019, I worked on a similar topic for my Masters’ Degree final project (Akbulut 2019) and developed a design concept for connecting digital natives through haptic technology. The project was my attempt to capture attention to humanizing communication in a digital era. It was meant as a tool to imagine new ways and behaviours to empower the connection between two people and to close the long-distance gap through physical touch. I envisioned a new personal device, a piece of jewellery that is enhanced with haptic technology, to allow human beings to have a different experience of each other’s text-based messages, through the study of the long-distance relationship scenario. In this scenario, design, and especially interaction design had an inherent power to bring attention to how the way we interact with technology could affect our existence. In pursuance of understanding in which ways designing with, for and through the sensorial richness of interaction design might contribute to how empathy is produced, expressed and perceived in computer-mediated communications, my Masters’ Degree final project (Akbulut 2019) serves as a starting point.

Sensorial Richness of Interaction Design

Our perception of reality is determined by our senses. In other words, our experience of reality is a combination of sensory information and sense-making mechanisms related to that information in our brain (Narumi 2016). Even from our childhood, we learn how to use our senses to make sense of the world around us. Our human senses also play an essential role in our emotional processing, learning, and interpretation, our emotions can be influenced by sensory information (Thomson et al 2010). 

It is a common misconception that interaction design is seen as designing digital interfaces, however, interaction design shapes our everyday life through digital artefacts (Smith 2002) and does not necessarily dwell in digital interface creation. Interaction design is the approach of designing with a focus on the impacts and interactions between humans and technology. Even though the senses we call upon when interacting with technology are restricted (Obrist et. al 2016), it is crucial to determine the meaningful design space that unlocks the potential of the sensorial richness of interaction design.

Our interpersonal encounters with others are characterised by sensory exchanges, and this relation has and have long since been part of sociological analysis (Vannini et al 2012) with a focus in human-social interaction and also represented by the interest of designers. For instance, “Hug Shirt” pioneered sensorial digital communication with the world’s first haptic telecommunication wearable invented by CuteCircuit in 2002 (CuteCircuit 2020). “Hugvie” (Kuwamura et al 2013) was designed as a physical medium to build conversations with others through hugs. Although senses are being articulated in interaction design venues in the interest of communications, the potential of the experiences the sensorial interactions can deliver is still largely unexplored especially in the context of rethinking empathy. My goal is to understand in which ways to create richer experiences for human-technology interactions to tackle the feeling of being distanced by the screen along with rethinking empathy in a digital world. To meet this challenge, I need to determine which tactile, gustatory, and olfactory experiences we can design with, for and through and how to meaningfully stimulate them in CMC interactions. 

Research through Design

To explore in which ways designing with, for and through the sensorial richness of interaction design might help us contribute to how empathy is produced and expressed in CMC, I am conducting research through design (RtD) (Frayling 1993) which is a continuous and iterative process of designing and reflecting, both feeding into each other. The approach of RtD compounds a conscious investigation of the creative production of artefacts wherein the knowledge is generated by understanding the current state (Olson and Kellogg 2014). This approach guides the study to articulate the created knowledge from different angles with multiple tools, particularly, participatory methods, as well as other forms of engagement such as interviews and reading are conducted. To enrich the data collection, experimental ethnography and sensory ethnography guides study to research everyday life of digital natives with digital technologies. Accordingly, I have completed my cultural probe pilot study with 5 digital natives (international masters students born in 1990-1995 living in Tallinn) and will conduct a further probing activity developed in the light of the results gathered with the pilot study.

Sensory ethnography is not a technique for understanding what people do and say, but for seeking to imagine how they may feel (Pink et al 2019). Pink et al (2019) discuss how video and our engagements with material and technological artefacts of everyday life can play a key role in generating collaborative encounters between researcher and participant, through which normally unspoken and invisible, sensory and emotional ways of knowing are revealed. Sometimes these encounters enable understandings of sensory or unspoken ways of knowing or doing everyday tasks that surprise participants and researchers. They also enable new ways of sharing sensory and unspoken knowing with viewers, who likewise become co-implicated in encounters through video (Pink et al 2019). 

For the sensory ethnography activities, I am designing research experiments to bring forth new social, technological, fashionable, embodied (etc.) imaginaries, and assisting with the journey from hypothesis to real-world contribution. Research experiments are expected to dialect to theory extensions, which formulate a new experiment. As the research and design evolve, the generated knowledge is expected to evolve into a public showcase that is carefully designed to display visual and written information on the research question studied over the year. The visual information consists of design artefacts and the written information dwells in reflections. These artefacts can be understood by tangible objects not simply as a static material, physical and visual environment, but as a feeling, inviting sensory experiences.

References:

Candace Vogler (2018). Barriers to Empathy. Retrieved 02.01.2021, from https://thevirtueblog.com/2018/06/07/barriers-to-empathy/

Carolyn Pedwell (2014). Affective Relations: The transnational politics of empathy

Cristopher Frayling (1993). Research in Art and Design

Cynthia L. Bennett and Daniela K. Rosner (2019). The Promise of Empathy: Design, Disability, and Knowing the “Other”

CuteCircuit (2002). Hug Shirt. Retrieved 20.04. 2020, from https://cutecircuit.com/the-hug-shirt

David Gray (2017). Updated Empathy Map Canvas. Retrieved 03.06.2020, from https://medium.com/the-xplane-collection/updated-empathy-map-canvas-46df22df3c8a

David M. Thomson, Christopher Crocker, and Christopher G. Marketo (2010). Linking sensory characteristics to emotions: An example using dark chocolate

Don Norman (1986). User-Centered System Design: New Perspectives on Human-Computer Interaction

Eun-Ju Lee, Soo Youn Oh (2015). Computer-Mediated Communications. Retrieved 02.01.2021, from https://www.oxfordbibliographies.com/view/document/obo-9780199756841/obo-9780199756841-0160.xml

Dorothy A. Leonard and Jeffrey R. Rayport (1997). Spark Innovation Through Empathic Design

Gillian Crampton Smith (2020). The foreword of Designing Interactions

Judith S. Olson and Wendy A. Kellogg. 2014. Ways of Knowing in HCI

Kaiko Kuwamura, Kurima Sakai, Takashi Minato, Shuichi Nishio, and Hiroshi Ishiguro (2013). Hugvie: A Medium that Fosters Love

Kirsikka Vaajakallio, Vilma Lehtinen, Petri Kaario, Tuuli Mattelmäki, Kai Kuikkaniemi, and Vesa Kantola (2010). Someone Else’s Shoes — Using Role-Playing Games for Empathy and Collaboration in Service Design

Marc Prensky (2001). Digital Natives, Digital Immigrants

Marianna Obrist, Carlos Velasco, Chi Vi, Nimesha Ranasinghe, Ali Israr, Adrian Cheok, Charles Spence, and Ponnampalam Gopalakrishnakone (2016). Sensing the Future of HCI: Touch, Taste, and Smell User Interfaces

Marion Buchenau and Jane Fulton Suri (2000). Experience prototyping

Mike Cooley (1989). Designing Human-centred Technology: A Cross-disciplinary Project in Computer-aided Manufacturing

Namwali Serpells (2019). The Banality of Empathy Retrieved 02.01.2021, from https://www.nybooks.com/daily/2019/03/02/the-banality-of-empathy/

Nesli Hazal Akbulut (2019). Personal Touch: A Design Concept to Empower Connections

Paul Bloom (2016). Against Empathy: The Case for Rational Compassion

Phillip Vannini, Dennis Waskul, and Simon Gottschalk (2012). The Senses in Self, Society, and Culture: A Sociology of the Senses

Rebecca Rago (2014). Emotion and our Senses. Retrieved 08.06.2020, from https://sites.tufts.edu/emotiononthebrain/2014/10/09/emotion-and-our-senses/

Rikke Friis Dam and Teo Yu Siang (2019). Design Thinking: Getting Started with Empathy. Retrieved 07.06.2020, from https://www.interaction-design.org/literature/article/design-thinking-getting-started-with-empathy

Sarah Pink, Jolynna Sinanan, Heather Horst, and Larissa Hjorth (2019). Sensory Encounters and Mobile Technologies: Mundane Intimacies as a Site for Knowing

Sherry Turkle (2015). Reclaiming Conversation: The Power Of Talk In A Digital Age

Takuji Narumi (2016). Multi-sensorial Virtual Reality and Augmented Human Food Interaction

See my CV on the Estonian Research Information System: https://www.etis.ee/CV/Nesli_Hazal_Akbulut/

View All