(10,385 words)
Introduction
Until recently, if someone had asked me for advice on how to get into the field of UX design quickly, I had my usual answer ready. Besides the obligatory bibles of interaction design by Alan Cooper and Don Norman, the theory of realistic design by Jan Michl, a Czech academic working in Norway, and a study of the cognitive aspects of human-machine interaction, I also stressed the need to stop reading most online blogs and to be sceptical of websites such as Stack Exchange. Many times I have myself made use of this online “collective intelligence” when I was not sure whether the OK and CANCEL buttons in a modal window should be on the right or on the left. Having a living online archive of best practices is useful. After all, another widely cited book for UX designers, A pattern language by the architect and theorist Christopher Alexander, aimed at the same thing: to democratise the discipline of architecture so that anyone could use proven practices for their own projects, which was, according to Alexander, the way to improve architecture and with it people’s lives. To that extent there is no need to criticise cookbooks of quick recipes such as Stack Exchange as a source of UX education.
But a problem can arise when the practice of designers is reduced to applying proven methods. Design then loses the main thing for which designers do their work: the desire to find new opportunities for improving human lives through the creative and thoughtful use of technology. Moreover, the field of design as such necessarily suffers too. The more often designers resort to what is known and proven, without a deeper understanding of why the given solutions work, the more the field will lose the innovativeness and critical thinking that are needed to regenerate old theories and dogmas.
The argument of the market economy and the free market of ideas would most likely claim that although the commercial sector is not the friendliest towards purely theoretical research and inquiry, the market is always about profit, and if designers get to improve the quality and profit of their products through theoretical reading, they will surely do so. Undoubtedly this may be the case to a certain extent. But it is not possible to think that in the course of their commercial work designers will at the same time hold or replace the positions of academic researchers. These roles can overlap, and in a discipline oriented both towards theory and practice, such as design, this is a more than fruitful combination, but it would be naive, and above all historically incorrect, to think that it is predominantly the market that comes up with innovations. New theories need, parallel to the market, a non-commercial universe in the form of academic research, whose main task is intellectual production that provides both new theories and a critical analysis of what is happening in practice. As we know, today academic research can also be carried out outside universities, under the baton of digital giants such as Google, Microsoft, Apple, Amazon and others. But even these giants collaborate with individual academics or directly with universities. Many fields can benefit from this symbiosis of market rules and academic research. How does UX design fare?
An intellectual foundation for UX design?
What is specific to UX design is that it arose directly in a commercial environment, and only ex post did practising UX designers begin to look around for some intellectual foundation. Precisely because of its uncertain beginnings, UX design is still a term without a settled definition, which may also stem from the fact that it is a field that responds fluidly to new technologies and market demands. In the academic sphere, however, there are already several publications that try to capture how UX design differs from the related fields of interaction design and HCI. The literature agrees that UX design can be understood as a field that:
- goes beyond the pragmatic and utilitarian requirements of traditional HCI usability, which tried to grasp the user’s interaction with technology scientifically, as a kind of cognitive performance that can be measured and quantified. UX design emphasises the hedonic, emotional, experiential and aesthetic aspects and the fulfilment of user goals [HASSENZAHL and TRACTINSKY, VÄÄNÄNEN-VAINIO-MATTILA, LAW, ROTO, HASSENZAHL et al.]
- is anchored in the philosophy of User-centered design (UCD), which places users and their context of use, knowledge and culture at the centre of attention throughout the whole design cycle [SWARD and MACARTHUR], and in this it differs from traditional HCI by taking a human rather than a technological perspective [HASSENZAHL and TRACTINSKY]
- is subjective and holistic [BATTARBEE and KOSKINEN, HASSENZAHL and TRACTINSKY]
- requires empathetic understanding and “putting oneself” in the role of the users [BATTARBEE and KOSKINEN]
From the very beginning, however, UX design contained important theoretical questions, which are forgotten under the pressure of deadlines and of delivering good-looking prototypes in the right software. They were questions that anticipated a far more intimate relationship between people and technology, and at the same time posed, without any hyperbole, a philosophical problem: if designers are in some way to have a positive influence on user experiences and on how technology is experienced, what do we actually know about human experience? Or in other words, what must designers know about people and technology in order to design experience? And is it even possible to design something like that?
To trace the origin of these questions, it is worth recalling when people began to talk about UX design. The first mention can be traced to 1986, when the now classic book User Centered System Design: New Perspectives on Human-computer Interaction, edited by Don Norman with Stephen Draper, included the text Interface as Mimesis by Brenda Laurel. If, however, we wanted to find the modern meaning of the term, we arrive once again at Don Norman, who writes in an email exchange that he invented the term UX design in 1995, when he was working as an HCI researcher at Apple:
“I invented this term because I thought HCI and usability were too limiting: I wanted to cover all aspects of the human experience with the system, including industrial design, graphics, the interface, the physical interaction and the manual.” [MERHOLZ, Whither “User Experience”?, my emphasis]
Let us notice several things in the quotation. In 1995 Norman recognises that the academic field of HCI had long been under the influence of a dominant focus on usability, and that it does not provide the intellectual frameworks and methodologies needed to grasp the human experience of technology. And that includes experience which concerns not only the use of the main product — for example an (information) system — but also seemingly unrelated parts and features, which according to Norman UX design should take into account. Norman lists several other “touchpoints” that users encounter while using a system, and which can be called the wider and indirect context of the system. His list is not exhaustive. To complete it, we immediately think, for example, of how the architecture and interior of shops or the look of a television advertisement also influence the way we experience and think about technologies. The whole idea of marketing campaigns is precisely to create the right image of products by presenting selected facts and attractive visuals, where the function of the products recedes into the background, since it is often the same as the competition’s, and what decides is the narrative presented of how the person-user-customer can become part of some community that will give their life meaning.
Advertising works, among other reasons, because human experience rarely has to correspond to the physical shape of the world: we see things that do not exist, we dream of what we do not have, and so on. Our life consists not only of sets of scientific facts and logical inference but also of a rich (phenomenological) inner life, which, to put it in Heidegger’s terms, is marked from the start by the fact that it takes place in the world. A person cannot stand still in the world; they must act, and there is rarely time for logical deduction in order to act; they must find their way in the world on the basis of acquired experience and intuition, and rather than analysing every situation and all things, they turn to the meaning that the things and technologies surrounding them have for them, or that they assign to them.
Martin Heidegger as a philosopher of design
As is well known, Martin Heidegger, the central figure of the philosophical movement of phenomenology, also argued in his magnum opus Being and Time that things appear to us on the basis of meaning, specifically as tools, or more precisely as equipment “for something” (in-order-to), rather than as a set of properties of an object that simply lies before us without affecting us in any way. Specifically, Heidegger named two ways in which things appear to us: present-at-hand (Vorhandenheit) and ready-to-hand (Zuhandenheit).
The first, less common mode, present-at-hand, resembles the way a scientist looks at things. The scientist examines a thing objectively, in isolation from its natural environment, analyses its properties in detail, and all that such a scientist could say about the thing would be a very long list of properties. It is a view that represents a disinterested examination of the thing, as if we had removed ourselves from the world and were looking at it through a scientific lens.
Heidegger’s basic idea, however, is that we are incapable of precisely this removal from the world. We are always beings who exist in the world, and with us the things around us. That is why the second way in which things appear to us, and according to Heidegger the more common one, is the ready-to-hand mode. In that case we approach things as equipment that offers us some function we can perform with it. The ready-to-hand mode is part of the ordinary everydayness of our lives, in which we do not theorise but have goals that we want to fulfil by means of equipment. Our action is directed at goals (goal-oriented).
When someone needs to use a hammer during the day, they do not think about its physical properties; often they do not even really notice what colour it is. What matters to them is that they know what the hammer as a tool is for, and that it will serve them to drive in a nail. Heidegger writes that when equipment, for example a favourite hammer, becomes authentically ready-to-hand (zuhanden) [HEIDEGGER, 2006, §15 69/99], it is characteristic of it “that in its readiness-to-hand it somehow holds itself back, and precisely in order to be ready-to-hand in the proper sense” [HEIDEGGER, 1996, §15 69/99]. A further important insight is that however long we look at the hammer, we will never know exactly its function, its readiness-to-hand. Only when we use it for the first time will we have access to what it is like to use a hammer. [HEIDEGGER, 2006, §15 69/98].
When our work with the hammer goes without any problems, we do not consciously think about the hammer or the nail. The hammer becomes transparent for us. The more we are immersed in our work, the more we also lose awareness of ourselves. In the state that current psychological research would call flow, as described by the psychologist Mihaly Csikszentmihalyi, the clear separation between us, the hammer, the nail and the work ceases to hold. Phenomenologically speaking, the split between subject and object is suspended; the only thing that remains is the lived experience (experience) of the given act. [ZALTA, 2017].
In many respects it is precisely the state of flow — that is, total immersion in the act, without our perceiving the tools or the interface we are using — that we want to achieve through UX design. On the other hand, there are moments when the designer should, on the contrary, consider that there are suitable occasions for consciously interrupting the continuous flow of experience in order to draw the user’s attention to something. It need not be a warning “alert” in a browser window; in the same way, a traffic sign or a speed bump on the road can consciously alert a driver.
In both cases the designer must bear in mind that new, creative products can be so radical that users cannot imagine how to handle them until they try them out themselves (see ready-to-hand in Heidegger above). Even a radically new product should therefore make use of what users have already learned and, through its shape, contain a hint of how the product works, how it is operated, and thus also what its meaning and function may be in the given context. Both academic and popular design literature has a term for this visual hint: affordance.
Affordance
The term affordance expresses both the existing and the perceived properties of objects that the user recognises and that enable them to perform some activity. Norman introduced affordances to the HCI and design community when he took over the concept from the eminent psychologist James J. Gibson, who created the then non-existent noun “affordance” from the verb “to afford” (to provide, to make possible) in order to express a new idea: affordances are part of both the environment and organisms, and they must be analysed relative to the organism [GIBSON, 2015, p. 120], although many affordances may be the same for organisms if they live in the same ecological niche. An affordance can thus be understood not as a fixed, static property but rather as a relationship between the properties of the environment and the properties of the organism [NORMAN, 2013, p. 11], in the context of design, for example, the abilities of a person interacting with an object of industrial design or with a user interface. Affordances are both physical and psychological, and in this they go beyond the Cartesian dualism of body and mind and the subject-object dichotomy. [GIBSON, 2015, p. 121]
The prototypical example of an affordance given in the literature is a door handle. Its appearance suggests how it can be handled and also what function it performs. Norman deals with the design of doors, and in general of objects with which we have unnecessarily big problems every day, in his book The design of everyday things (formerly also published as The psychology of everyday things). Norman became so famous in the design community that badly or senselessly designed doors earned the nickname “Norman doors” [NORMAN, 2013, p. 1].
The conceptualisation of affordance nevertheless differs between Norman and Gibson. As is well known, Gibson claimed that we perceive affordances directly, without the need for mediation through symbolic representation; that is, we perceive the affordances of objects in the environment without needing to classify those objects into abstract, linguistic categories. It is precisely affordances that define meaning in our environment. For Gibson, an affordance is meaning, and if we take literally his thesis that we perceive affordances directly, without some interpretive intermediate process, it means that meanings and values lie outside the organism that perceives the affordances; as Gibson writes, they are external with respect to the observer. [GIBSON, 2015, p. 119]
But is Gibson consistent? Or rather, does this mean that meanings are independent of the observer? That they simply “lie” in our environment, waiting for someone to “pick them up” (Gibson uses the English verb “to pick up”)? Since Gibson rejects the subject-object dichotomy, meaning must be shared between the observer and the environment. After all, Gibson himself claims that affordances are always relative to the observer, so meaning must be relative to the observer too.
Norman responds to Gibson’s definition of affordances by dividing affordances into two types: those that exist without our perceiving them, and perceived affordances. In this way Norman formulates more explicitly than Gibson the thesis that there exist in the environment a priori affordances, whose ontological status is given not by an existing subject-object relation but by its mere potentiality. In other words, affordances are now no longer only the direct perception of affordances, as Gibson vehemently maintained, but also all the affordances that an organism can potentially perceive in contact with the environment.
Secondly, Norman departs radically from the original meaning of affordances when he disagrees with Gibson’s main thesis that we perceive affordances directly. Instead, Norman claims that affordances need to be interpreted [NORMAN, 2013, p. 12]. In my opinion, it is precisely this step aside from necessity that allows Norman to divide affordances into perceived and invisible ones, because it removes that dependence on direct perception. But whereas the ontological status of affordances is trivial in Gibson – they exist at the moment the organism perceives them in the environment –, in Norman the existence of affordances becomes complicated, especially since already in Gibson their existence is relative to the organism. Norman relativises their existence too, by arguing that the organism must first interpret affordances in order to perceive them. A logical question thus arises: if affordances are always relative to the organism, how can affordances exist even though the organism does not perceive them?
In a short article on his blog [NORMAN, Affordances and Design], Norman remarks that although the introduction of affordances quickly caught on in the design community, it was often misinterpreted. Norman admits the difficulties with understanding his notion of affordances and adds that he should rather have written about “perceived affordances”, which are much more interesting for designers, since it is precisely perceived affordances that designers build into their creations. As far as I know, however, Norman does not discuss in his work the ontological status of affordances that are invisible.
From my point of view, what is important about the theory of affordances in both Gibson and Norman is that affordances are closely connected with the meaning of things. At the same time, affordances, and thus meaning, are relative to the organism, the person, the user, on the basis of their experience with the given “ecological niche” in which they live. In other words, affordances and meaning depend on the context, society and culture of the person. They are thus to a certain extent socially constructed.
Experiences are socially constructed
Like affordances, the meaning of things mentioned above is not something we find hidden only in the bowels of programming code or in the material of the latest smartphone. Meaning is not a material but a metaphysical quantity. It is created only in reading, interpreting or interacting with a text, objects or an interface. It is a combination of material properties, social customs, and people’s life experiences and skills. Even the time and place in which we read a book or use a smartphone can significantly influence what things mean to us. A poem we had kept putting aside may for the first time evoke strong experiences in us, because we read it after the loss of a loved one. Similarly, a smartphone suddenly becomes an important companion in our life when we get lost in the mountains and, with a very poor signal, manage to bring up mobile maps or call for help.
What connects the questions of meaning, affordance and thus also of user experience is the fact that, according to the theories of symbolic interactionism, actor-network theory, but also, say, research on mental models and schemas in cognitive psychology, they are inherently socially constructed. This was also the claim of the sociologist Herbert Blumer, a student of G. H. Mead and the founder of the sociological theory of symbolic interactionism. Blumer and his followers hold that “people act towards things on the basis of the meanings that things have for them, and [t]he meaning of these things is derived from the interaction that takes place between the individual and society.” [Herbert Blumer, 2001].
This means that however hard the designer tries to instil in their work unambiguous properties which in their view should contribute to, or even determine, positive user experiences, they will never be able to capture the life histories of individual users, nor the social complexity in which their work will exist once released into the real world, and which will inevitably shape how users approach it and think about it. But they need not throw up their hands in despair. Realising that the designer does not wield divine powers can, on the contrary, help them focus on what the designer is able to influence.
Even if the designer cannot design user experiences, they design products and interactions between people and technologies that, in their judgement and experience, will form the best possible context for the desired user experiences to arise. For that, however, one needs to know what parameters a suitable context should have, and also why the user wants to use the given product at all.
The city and the Anthropocene: the world as a medium and a social system
The question why? is in general one of the most fruitful tools a designer can include in their repertoire, because it introduces them in a gentle way to the complex and at the same time fascinating problem of thinking about design, technology and society. In short, it goes roughly as follows: how is the relationship of people to their technological creations changing under the onslaught of smart, ambient devices, autonomous agents and deep-learning algorithms? And, inevitably, how is people’s relationship to their surroundings, to nature and to the world in general changing?
For the boundaries between the technical, organic, social and natural layers are blurring. Following the official declaration of the Stockholm Memorandum, the interconnectivity between these layers is so strong that human intervention now affects “all aspects of planetary ecology, whereby natural and ecological systems become socially influenced”. Our planet has thus, retrospectively from the beginning of the 19th century, entered the new era of the Anthropocene [JONES, 2014]. An era defined by the fact that human beings, and above all their technologies, influence and shape the world around them.
In new media theory and urban sociology we further learn that a smaller unit on the planetary scale — the city — , is also a unit that not only urban planners and architects but also designers must pay attention to. Not least because by 2030, according to estimates, more than 60 per cent of the world’s population will live in cities. Within the planet-wide interconnected network, the city will become an important node that will form a “digital urban panorama” [DEUZE, 2015, p. 20], against whose background the quality and quantity of interactions between people, technologies and nature will be defined. And these interactions do not take place only in the physical space of the city, because the city, much like the human being, consists of a physical and a digital (virtual) layer, the latter formed by mobile applications that can reduce the vastness and complexity of the city to its most interesting points, which they bring to life, as it were, in the application, since those points suddenly become interactive. Through an application, places offer us old information in a new way as well as entirely new experiences, which the city could never offer without the supporting digital layer in the form of the application.
The city is a hybrid space where “the perception of buildings, neighbourhoods and the bustle of street life cannot be seen separately from the direct involvement of media” [DEUZE, 2015, p. 20]. Theorists speak of the city as a media infrastructure [DE JONG and SCHUILENBURG, 2006, p. 15], a “machine” based on information processing, or an unfinished and constantly reshaped space which, after the model of modern web applications, is in a permanent state of beta [DEUZE, 2015, p. 27].
Despite its apparent rigidity, the city as a hybrid physical-digital whole is a malleable node of the complex planet-wide network. Its digital layer in particular is constantly open to innovations and design interventions that can solve technical as well as social problems, as we could see in the Czech context in the example of the mobile application Záchranka, which helps the emergency medical service locate a person and makes it possible to report quickly on the medical condition of the person affected.
Digital interventions can also have their dark side. Applications such as Uber or Airbnb are frequently cited examples. Although at the start they meet an unsatisfied demand “only” with good intentions, in a later phase of their operation, owing to their spread, they may turn out to be forces that negatively affect the functioning of the city. This may be a disruptive effect on taxi services in the case of Uber, or a noticeable increase in the price of long-term rents and a deepening of the effects of gentrification, as we observe in the discussion about the Airbnb service.
If the effects mentioned really are unplanned consequences of the massive use of a popular digital platform, could the negative impact on the city and its inhabitants have been prevented? Are the main culprits the technical, political or social aspects of the service?
Since we live at a time when the interconnection and growing together of people, technology and the world is not just a theoretical question but also a practical problem, it is also up to design, as the field that brings new products and services into the world, to deal with these questions. It used to be a matter of privilege, of a large budget and of design-minded managers, that the impact of new products and services on users was considered during their development. Now any new service, application or website that intervenes non-trivially in the social life of people and of the city becomes a new node of the complex, planet-wide social network of technologies, people and the world. That is why critical reflection on what corporations, but also the State, bring into the world through designers cannot be a privilege but must be a necessary precondition. The (UX) designer must design for the age of the Anthropocene, for a time in which cities themselves are media. But how should technology be created so that it does not cause catastrophic consequences in a complex, planet-wide social system? And what methods and theories do designers of complex systems have at their disposal?
HCI as the intellectual foundation of UX design
Answers to these questions should be provided, among others, by the field of Human-computer interaction. The history of HCI goes back to the early 1980s, when in the relatively young areas of personal computers and interactive computer graphics a demand arose to simplify the operation of these new technologies so that even the new target group of home users would be able to use personal computers.
The 1980s brought new techniques to the field of interaction with the computer: the direct manipulation of objects on the screen, the desktop graphical user interface, the WYSIWYG editor and interaction with the mouse. These were developed at research centres at Stanford University and at Xerox PARC, led by the pioneer Douglas Engelbart. But it was only in 1984 that Apple, with its Macintosh computer, managed to present these innovations to the wider public for the first time. Other systems from IBM and Microsoft followed.
In 1982 the ACM Special Interest Group on Computer-Human Interaction (SIGCHI) was founded, connected with both academia and industry. The group emphasised user-centred research. In 1983 the first conference on HCI was attended by more than 1,000 people. In the following years key publications appeared, such as The psychology of Human-Computer Interaction (S. K. Card, A. Newell and T. P. Moran, 1983), Understanding computers and cognition: a new foundation for design (T. Winograd and F. Flores, 1986) and the previously mentioned book by Norman and Draper, User centered system design: new perspectives on Human-Computer Interaction. [ROUSSEL, 2014]
From its beginnings the field of HCI was multidisciplinary and influenced by the methods and theories of related fields, above all computer science and cognitive science. And it was cognitive science, itself drawing on psychology, linguistics, philosophy of mind and anthropology, that for the years to come set HCI on a pronounced course focused on usability engineering and on the cognitive aspects of using information systems and computers.
Later, it is true, HCI began to draw also on alternative theories and fields such as activity theory, actor-network theory or phenomenological analyses of the situated and embodied aspects of interaction, which today’s theorists refer to collectively as post-cognitivist [KAPTELININ, 2003]. Before that happened, HCI was in the grip of the paradigm of the cognitive sciences. To understand the rise of post-cognitive approaches, it is worth looking at the shortcomings of the cognitive science approach to which post-cognitive theories respond, the so-called cognitivism.
Shortcomings of the first wave of HCI and of cognitivism
The more technology became part of people’s everyday lives, the more the limits of the first, so-called cognitive wave of HCI gradually began to show. It turned out that motivations, goals, user experiences and the complex interactions of people and inanimate objects cannot be studied only by analysing the cognitive aspects of human thinking in the artificial environment of laboratories.
HCI researchers and designers began to take an interest in where and when the person-user comes into contact with technology, what their motivations and goals are, and, last but not least, in examining what today’s world is like, the world in which users, machines, interactions and interfaces exist. The methods and theories of the first, cognitive wave of HCI are insufficient for such an inquiry. That is why theorists of HCI and design began to look elsewhere. The logical step was to start taking seriously the arguments of alternative theories within cognitive science itself. HCI further began to draw on the social sciences and the humanities, above all on sociology, anthropological methods, the theory of design for complex systems and philosophy. It was philosophy that dealt cognitivism in cognitive science, psychology, artificial intelligence research and HCI probably its heaviest blow.
By cognitivism we mean the theories and assumptions stemming from the so-called classical or cognitive research programme in cognitive science, artificial intelligence and philosophy of mind, which conceptualise the human mind by means of the metaphor of the digital computer. The paradigm of cognitivism understands thinking as follows:
- as the ability to solve problems on the basis of input and output “data”
- it presupposes the existence of symbolic processing, of encoding into some form of representation, and of computation with these symbols and representations
- it holds that cognition can be understood primarily by focusing on the organism’s internal cognitive processes involved in computation and representation [COWART]
In short, cognitivism assumes that mental states and intelligence can be understood as the manipulation of abstract symbols and the following of fixed formal rules. [BREY, c2001, p. 41].
Leading figures of the cognitivist approach to modelling artificial intelligence included Herbert Simon, Allen Newell and Marvin Minsky. Their work differed in how far they trusted that their research really modelled human intelligence (Minsky took AI research “only” as a theoretical contribution to the study of human cognition), but they agreed on the main point: for the classical paradigm of cognitive science, also referred to as “symbolic AI”, all intelligent processes, including perception, thinking, calculation and the use of language, are a kind of information processing, that is, obtaining information from the environment, processing or manipulating it, and providing an appropriate response [BREY, c2001, p. 39]. For information to be processed, it must be in some way represented: by an image, a sound or precisely by the symbols of some language, ideally one that is formal and consists of a finite set of independent elements [DREYFUS, 1967, p. 14]. With the arrival of digital computation, this set became de facto exclusively the language of binary numbers.
The requirement that the language be formal brings with it one important property: symbolic AI research sees information processing purely on the basis of the form of symbols, not their meaning. When a computer processes the symbol “dog”, further converted into the binary system, it knows what to do with the symbol not on the basis of its meaning but solely on the basis of its form [BREY, c2001, p. 40]. The assumption that all knowledge can be converted into formal shape, without the need to take meaning, or rather context, into account, is the philosophical a priori core of symbolic AI research. A core that the American philosopher and expert on Heidegger’s phenomenology Hubert Dreyfus began to criticise sharply at the start of the 1970s.
Hubert Dreyfus and the critique of cognitivism
The most important criticism Dreyfus directs at symbolic AI is the fact that intelligent behaviour involves not only knowledge of certain facts but also knowledge of how to apply these facts in a relevant situation in thinking, communication or behaviour. According to Dreyfus, knowing that fire is hot thus also includes the ability to handle fire appropriately. [BREY, c2001, p. 43] But if symbolic AI is to be truly formal, independent of context, then all relevant situations must be programmed into the system in advance, or else the system must have fixed rules capable of responding appropriately to all the given situations. It seems unlikely, however, that all the relevant situations to which the system is to respond could be listed in advance. Leaving aside a few exceptions that represent an idealised “world”, which symbolic AI can handle well, such as mathematical and formal logical problems, appropriate interpretation and response are always tied to contextual factors.
Brey [BREY, c2001, p. 33] illustrates the problem of context for symbolic AI with the example of interpreting a sentence of natural language. We are to imagine an English sentence in which the word “hot” occurs. In English, the word “hot” can mean “of high temperature”, “spicy” and, in slang, also “physically attractive”. To be able to understand the sentence correctly, we must look at the context in which the word is used. Sometimes it is enough to look at the whole sentence. In other cases larger units such as paragraphs must be analysed. Similarly, the interpretation of the meaning of a bunch of flowers lying on a table next to bottles of alcohol depends on the occasion on which, and the person to whom, the flowers were given. At one moment they may be wedding flowers, funeral flowers or just flowers found by chance in a park. Both examples point to the fact that an analysis of the symbols themselves —
the flowers — without the surrounding context is not enough to establish their meanings.
The problem of context can further be shown on a situation generally familiar to designers. It is the moment when a designer first embarks on the creative part of a project. They have before them all the possible facts gained through ethnographic research, SWOT analysis and other methods belonging to the analytical phase of the design process. With these facts, and with their creativity, they are to create something new that will be a suitable solution to the brief. The difficulty is that design problems are defined, if at all, very vaguely, and in many respects it is up to the designer to decide what the real problem of the given project is. For the fact that a client claims to need a new visual identity or a better website may not in reality be the ideal solution to the client’s product communicating its strengths poorly, or it may not be the website’s fault that the client has lately seen a drop in orders. The culprit may be poor communication on social networks, which repels customers with excessive vulgarity or, on the contrary, formality. A responsible designer understands that the design problem must be placed in the wider context of all the customer’s services and touchpoints with which customers interact.
Intuition and the “problem” of design problems
As was said above, the nature of design problems is that they are hard to define and therefore very hard to formalise. Problems of this type have earned the name wicked problems in the specialist literature. They are wicked, because, unlike problems that symbolic AI can solve very comfortably, wicked problems have no clearly defined brief and therefore no proper definition of a solution either. The design theorist Horst Rittel, in his article on social policy, which he defined as a whole as a wicked problem, lists the common properties of all wicked problems [RITTEL, 1973]:
- There is no definitive formulation of them
- It is not clear when we reach the end of their solution (unlike mathematical or chess problems, where we know unambiguously whether we have reached the required solution of the problem)
- Unlike well-structured problems, there is neither a fixed list of potential solutions nor a defined set of permissible operations that lead to the solution of the problem
- Solutions to wicked problems are not a right/wrong dichotomy but appropriate/inappropriate
- Every wicked problem is unique, which for Rittel means that there are no clearly given classes of problems
- Solving one part of the problem often leads to new problems
To put it pithily, with wicked problems, and creative problems in general, we find that “finding the problem is the same as finding the solution; a wicked problem cannot be defined until we find the solution” [FERENC, 2015, p. 54].
Dreyfus does not deal with wicked problems directly, but his term ill-defined problems does by its nature remind me of what design literature knows precisely under the name wicked problems. What specific difficulties does this type of problem present for cognitivism and symbolic AI?
Dreyfus writes that “since there is no limit to the amount of data that may be relevant to solving a badly defined problem [ill-defined], we cannot even in principle try all the permutations of possible data in order to find a solution. For in such a case we must decide not only which operations will lead to the desired transformations, but also which facts from the whole context are relevant”. [DREYFUS, 1967, p. 23]
How do people, creatives, designers solve wicked problems? In his work The Reasoning of Designers, Rittel proposes a conception of design thinking as argumentation. In Rittel’s account, the designer works on a wicked problem by “debating” with themselves. They look for the advantages and disadvantages of each of their steps. They choose their next step not on the basis of a logical truth value but by taking into account its appropriateness [FERENC, 2015, p. 54].
Practical intelligence
It is precisely the ability to find an appropriate, relevant, not necessarily logically correct solution for the given context that Dreyfus mentions as one of the important strategies people use when solving wicked problems. According to Dreyfus, solving problems of this type requires plain horse sense (common sense) and intuition. The need for practical, common-sense intelligence is the central argument of what has entered the literature of artificial intelligence research as the so-called “frame problem”.
Dreyfus recalls how Marvin Minsky and other experts were convinced that if artificial intelligence had millions of pieces of information and facts about the individual objects of its environment in its memory, it would be able to decide which objects are relevant. For example, in a situation where a computer has stored in memory a representation of some 3D scene in which something changes. For the computer to be able to decide what it should pay attention to or what needs to be recalculated, it must have some criterion of relevance.
Such a criterion, however, is closely tied to the environment and context in which the computer happens to be [LEIDLMAIR, 2009, p. 41]. Minsky proposed to solve this problem by creating for artificial intelligence hundreds or thousands of prototypical cases of contexts, which would have certain general properties as well as prescriptions for detecting relevant facts and for how the artificial intelligence system should behave in the given case. Minsky called this prescription a frame. Dreyfus notes that Minsky was most likely inspired by Husserl, who also called a similar structure of prescription a “frame” [LEIDLMAIR, 2009, p. 41].
AI researchers did not, however, get rid of the frame problem. The very choice of a “frame” presupposes that there will again be some criterion available that selects a suitable frame for the given context. The frame problem has not disappeared; it has only moved one level up. Instead of finding the relevant frame, one would now need to find a meta-frame that would define the suitable frame. We can thus say that the problem gets lost in a closed loop of infinite regress. [SHANAHAN, 2016]
The Heideggerian idea of a holistic world, in which meanings (and affordances) “shine” on us from all sides, moreover implies that even if something appears irrelevant at a given moment, and artificial intelligence could with great probability choose a suitable frame for that moment, we are never a priori able to say that what we explicitly ignored will not later in time become relevant to our decision after all. [SHANAHAN, 2016]
The world has no limits determined in advance. It is inherently open-ended. These are conditions that human intelligence can cope with by practical common sense. It is precisely openness and an a priori infinite number of relevant, context-dependent and interconnected parts of a system that are the properties which can also describe the super-complex wicked problems mentioned above, for whose solution experienced designers are well equipped.
The design theorist Nigel Cross, in his research into the work of professional designers, notes that experienced designers differ markedly from amateurs in being able to move fluidly from formulating a design problem to solving it, and in quickly finding, amid the flood of possible ways to approach the problem, strategies that lead to a suitable goal. The academic and, again, design theorist Bryan Lawson calls these strategies “gambits”: certain proven patterns that have worked in the past on the basis of experience and that are part of the designer’s standing repertoire. Lawson borrows the term “gambit” from the terminology of chess, where it is a move that is suitable in the given situation. [FERENC, 2015, p. 58]
The human ability to choose the suitable ones from a potentially unlimited amount of information, facts and available affordances, and in the case of experienced designers to choose even those that lead to new, creative activity, is something in which human cognition differs fundamentally from the practices of symbolic AI, and thus from the paradigm of the human mind as an information unit that processes and represents symbols.
Dreyfus’s critique, based on his reading of Heidegger, the later Wittgenstein and also Merleau-Ponty, consists in the claim that if AI researchers want to solve the “frame problem” at least at a theoretical level, they must begin to implement a vision in which beings exhibiting human-level intelligence form their goals, their intentionality and thus also the relevance of actions on the basis of a close interconnection between the environment of the world and the corporeality of those beings. That is why it was necessary to look more closely at how the body influences our cognition and the formation of conceptual schemas, and also at how corporeality determines our perception of the world. These questions began to be addressed by a new research programme collectively called embodied cognition, situated cognition, distributed or extended cognition.
Situated, embodied and distributed cognition versus Descartes
Research on embodied cognition and on the closely related programmes of situated cognition and distributed or extended cognition responds to the shortcomings of cognitivism by taking into account that thinking is separate neither from the body nor from the environment in which the bodily being finds itself. In general, research on embodied cognition argues that cognitive processes develop when goal-directed interactions between an organism and its environment give rise to a tightly interconnected system of mind, body and environment. [COWART]
The phenomenological critique of cognitivism and concepts such as the situatedness and embodiment of human cognition depart radically from the building blocks of Western philosophical thought, of rationalism, which we inherited from the 17th-century French philosopher René Descartes. In his Meditations on First Philosophy, Descartes analyses what in this world he can declare to be truly clear and real. With his method of extreme scepticism he rejects everything that comes into his consciousness through the senses; he rejects body, shape, extension and place as chimeras [DESCARTES, 2003, p. 27]. In doing so he also casts doubt on whether the external world could be a suitable source of true information. As Descartes continues with his method, he comes to the conclusion that the only thing he can be truly certain of in his doubting is that he himself, who does the doubting, exists. Here the well-known aphorism I think, therefore I am is born.
For Descartes, the mind does not need the body in order to function. The mind (res cogitans, thinking thing) and the body (res extensa, extended thing) are two distinct substances. Although we see a distinct conception of body and mind (soul) already in ancient philosophy, in Plato and his realm of ideal Forms, which we must recollect, or in the Christian metaphysical doctrine of the immortal soul, it is precisely Descartes’ division that established the dualism of body and mind in philosophy.
Contemporary science, including cognitive science, rejects Cartesian dualism. Most scientists today understand the mind as a process which is complicated but which we can ultimately explain by means of physical mechanisms, without appealing to esoteric theories. Science thus holds the philosophical position of materialism. On the other hand, because the materialist conception of cognitive science, following in Descartes’ footsteps, still refuses to grant the body a greater role in the constitution of thinking, it resembles Cartesian dualism more than it would be willing to admit. Symbolic AI resembles rather a certain synthesis of Cartesianism and materialism. [DAWSON, 2013, p. 59]
Situated and embodied cognition are at odds with the Cartesian separation of body and mind. “Embodied” cognitive science, which draws on these concepts, explicitly abandons the disembodied mind. It sees the goal of human cognition and intelligence not as the creation of a correct representation of the world but as a sequence of actions that respond to practical problems in the world around us. Embodied cognitive science argues that theory must include in the analysis of cognition not only the human body but also its environment and the experiences of this environment. Experiences depend on situatedness in the given environment, on how the environment is perceived by the senses.
Situatedness in embodied cognitive science means the ability of a person, or of an agent in general, to perceive the world through their senses. At the same time, however, it depends critically on the physical form of the agent, including the structure of its physical body. [DAWSON, 2013, p. 216] Among the key themes of embodied cognitive science, Lawrence Shapiro identifies so-called conceptualisation, a theme through which Shapiro explains that a person acquires concepts for interacting with the environment on the basis of the form of the agent’s body. If agents have different bodies, their understanding of the same environment will differ. [SHAPIRO, 2011 cited in DAWSON, 2013, p. 209].
The embodiment and situatedness of cognition of course refer back to the original ideas of Heidegger and his analysis of being-in-the-world, thanks to which we can regard Heidegger as the first theorist of embodied interaction and thus also, in my opinion, as a theorist of HCI/interaction/UX design. Let us recall that Heidegger argued that in everyday life we do not encounter the world as a world of abstract properties, devoid of meaning for us, which we must constantly re-constitute on the basis of an endless stream of sense data. Instead, the world is for us already structured in advance by familiar meanings. [CHOKR, 2014, p. 178]
Theorists of embodied cognitive science would, however, remark that even though the world is structured for us in advance, our mind still, in real time and in interaction with the world, actively selects what is directly relevant, depending precisely on the goals set, the acts, the body, but also on the current context, the environment in which we find ourselves. [COWART]
Research in embodied cognitive science directly invites us to consider how its findings may influence the theory and practice of design, and specifically of design focused on experiences in complex, heterogeneous systems made up of many physical and digital components.
So far I have dealt with the fact that it is the body and the environment that shape our cognition, our goals and the acts that seem relevant to us in a given situation. In other words, in the environment we find affordances that help us find problems to solve, but also to solve them. Embodiment is already a significant departure from the cognitivist understanding of the mind, yet many other theorists now go even further and believe that the environment influences human cognitive processing so fundamentally that they speak of the human mind “leaking into the world” [DAWSON, 2013, p. 208]. This is the position adopted by the theories of the distributed, or extended, mind introduced by Andy Clark and the well-known philosopher David Chalmers. In such a conception the human mind is not located only within the skin or the skull; rather, “the mind is a leaky organ, forever escaping its ‘natural’ confines and mingling shamelessly with body and with world” [CLARK, 1998, p. 53].
In their important article The extended mind, Andy Clark and David Chalmers argue that we should take seriously the idea that the surrounding environment matters for our cognition so much that without tools and technologies we would not be capable of certain cognitive acts. Clark and Chalmers claim that some cognitive states, such as memory or beliefs, can be realised outside our body by means of suitable external elements. If these cognitive states depend on the availability of these external elements, Clark and Chalmers argue, the organism and these elements are causally necessary for the cognitive states in question, and therefore form an inseparable unity, a kind of cognitive ecosystem.
Clark and Chalmers give the case of a person who has problems with memory. They can think only about what they have written down in a notebook. If their notebook says that Vinohradská Street is in the centre of Prague, and solely on the basis of their notebook they believe that this is so, we can say that the medium of their cognitive processes is precisely the notebook, and that together they form a cognitive ecosystem. This does not mean that the notebook itself acquires mental states; the notebook has no consciousness, but it functions as an externalisation of the memory in the brain, which today we uncontroversially assume to be necessary for our mental states.
To use further examples from the present, besides Google and mobile phones a useful example is the increasingly commercially available technology for virtual and augmented reality, which can be linked to the example of the game Tetris that Clark and Chalmers mention in the original article Extended mind. VR or AR technologies allow people to manipulate 3D objects in space at will, an activity very useful for architects, designers or engineers, who in their work often use computer software to visualise extremely complex 3D objects. Seeing an unfinished design in real space, as augmented reality technology allows, greatly reduces the demands on the imagination and helps to place test prototypes of projects in their real future context. Such visualisations are hard to imagine for an ordinary person. Technology, however, makes them accessible to the mass population. Since the ability to mentally rotate 3D objects is one of the two main components of the IQ test, which is still regarded as the most widely used approximation of the so-called g factor of human intelligence, it is possible to say that an increase in this ability, through the mediation of new media and technologies, may have an impact not only on professionals who use this technology for their work, where it will allow them to speed up existing processes, but at the same time changes the human being as such.
Just as, for example, the invention of writing and later of printing undoubtedly influenced the structure and evolution of human thought, culture and society, so too new technologies extending our cognitive abilities have and will have an impact on society and on what we as human beings are capable of.
Cognitivism holds that the body, social context and human technological tools and artefacts do not play a great role in human thinking. A brief summary of the critique of cognitivism from the side of the new approaches in cognitive science and philosophy shows, however, that this is not the case. This naturally calls for reflection in the fields of HCI, interaction design and UX design as well, which today take part in shaping the technological artefacts of our society. Technological artefacts shape societies not only on a superficial level but also on a much more intimate one. As the theorists of the approaches to extended cognition just discussed maintain, HCI academics and UX designers must necessarily include in their repertoire of thinking, methods and theories the fact that the body, social context and technological tools take a significant part in what relationship we have to the world. People, technology and the world do not form separate spheres but are interconnected parts of a holistic whole.
Technology as a medium: the consequences of rejecting cognitivism for HCI and design?
The role of technology in a world conceived so holistically can be neither neutral nor passive. Technology plays an active part in shaping our relationship to the world. Technology is a medium. And a medium is not neutral. A medium shapes our thinking, and that is why Medium is the message.
That is the dictum pronounced by the media theorist Marshall McLuhan, who thereby drew attention to the fact that although in the analysis of media we had so far attended mainly to content, the time had come to turn our attention also towards that part of the technical apparatus which conveys – mediates – the content. According to McLuhan, “[t]he message of any medium or technology […] is the change of scale, pace or pattern that it introduces into human affairs” [MCLUHAN, 1991, p. 20]. Every technology brings into the human world a certain logic of its own, according to which, if we yield to it, our society is structured. One of the reasons why this happens is the fact that technology offers us these possibilities – it provides affordances and we seize them. Technology seduces us. It has power over us. If anyone claims that technology rules the world, then it is the designers who pull the strings from behind the scenes. To a certain extent, because designers in turn respond to the demands of society and to already existing cultural and social customs, metaphors and needs. The relationship between technology and society could be described as a relationship of feedback. Each influences the other.
This is in direct contradiction to the thesis of technological determinism, which claims that technology develops independently of the culture and social context in which it arises. After the phenomenological analysis and the critique of cognitivism, this thesis must be rejected. The world, the body and the mind form a unity in which human experiences and meanings are not abstract, independently existing entities that people find independently of the world, or that a designer can design without taking the social and cultural context into account. The human being is thrown into a world in which they act, and meanings, and subsequently experiences, are disclosed to us in the world depending on our goals and on the actions that we perform and that the world offers us (“affords”). Distributed cognition, moreover, argues that technological artefacts play an active role in establishing some important cognitive processes.
Technology extends human abilities and brings new affordances into the world. It generates new meanings and retroactively redefines old ones. The history of interaction with technology is a gradual expansion of the range of human skills and abilities [DOURISH, 2014, p. 17]. Because technology is created by people, we can, indeed we must, ask what affordances and meanings we want in our lives. That is no longer a technical question, nor a question of the surface of technical artefacts, the way some still perceive design today, as the decorator of almost finished objects. It is a philosophical question, which also lies at the core of UX design: experiences derive from a certain shape of the world. What that world should be like is a question that goes beyond, yet includes, design. To deal with this question, it is not enough to read the UX StackExchange online forum and follow established practices. The answer to how to create technology so that it provides affordances that will have a positive impact on societies will be a new answer, constantly and fluidly changing, and normative, and therefore it will also fall within ethics. Technology as a medium that changes people and society is a philosophical problem. HCI academics and UX designers must draw on theories that overcome the shortcomings of cognitivism, illuminate the influence of context and include the active role of technology in our lives. Among such candidate theories I count those we find under the collective label of post-cognitivist theories of HCI. Relevant analyses, above all of the mediating role of technological artefacts, can be found in current research in the postphenomenological branch of the philosophy of technology. The last part of this text will be devoted precisely to a brief presentation of their most essential ideas.
The second wave of HCI: the rise of post-cognitivism and the philosophy of technology
The main intellectual foundation of the second wave of HCI is formed by the so-called post-cognitivist theories. They earn this label by clearly defining themselves against the shortcomings of cognitivist theories. Besides their critique, however, they bring many innovative approaches to how cognitivism can actually be overcome productively. The aim of this last section is to show that the premises of post-cognitivist theories offer fertile ground on the basis of which the fields concerned with creating and theoretically examining human interaction with computer systems can be “re-designed”.
Post-cognitivist theories include activity theory, built on the unfinished work of the Soviet psychologist Lev Vygotsky, actor-network theory, developed by the French sociologist Bruno Latour, phenomenology, and the already mentioned theory of the distributed, embodied and extended mind, which, above all thanks to the contribution of the academic Paul Dourish, can be read in the context of HCI as embodied interaction.
I will draw mainly on the book Acting with technology: Activity theory and Interaction design, which offers a low-threshold gateway to the study of post-cognitivist theories of HCI and interaction design. The authors of this book provide a summary and comparison of the individual theories. It is precisely in the comparison that it becomes apparent that although post-cognitivist theories share much in common, they do contain important differences that need to be mentioned. Among the most fundamental differences is a different view of the role of the subject and its agency, where activity theory and phenomenology attribute certain privileged properties to the human subject, whereas distributed cognition and actor-network theory place the human subject in an extensive interconnected network of living and non-living entities, or rather “nodes”, which they regard as symmetrically equal.
In this text I will focus primarily on presenting activity theory and current postphenomenological research in the philosophy of technology. For the other theories I will mention their most important characteristics.
The individual theories
Despite their differences, all the theories presented agree that technology influences a significant part of our lives. They further reject the dualist conception of body and mind, claiming that the individual is not defined by the boundaries of their body. In various ways they try to explicate the notion of a fundamental unity of mind and world [KAPTELININ and NARDI, 2006, p. 197]. No less important is that their opposition to cognitivism also lies in the fact that cognitivism very rarely understood technology as something worthy of detailed analysis in relation to human cognition. Post-cognitivist theories and postphenomenology analyse technology not only as an extension of the human senses after McLuhan’s example, but also as the medium of goal- and object-oriented activity, through which technology mediates interaction with the world in a non-trivial way. As was said earlier, all the theories agree on the thesis that technology actively participates in constituting our relationship to the world.
In this way technology and its use have a say in how we come to know the world, what things exist for us and what constitutes the good life we ought to lead, that is, in the traditional epistemological, ontological and ethical questions of philosophy.
Activity theory
Activity theory derives from the work of the brilliant Soviet psychologist Lev Vygotsky, who in his short but productive career brought about revolutionary ideas in educational psychology. Although Vygotsky published from the 1930s, it was only in 1960 that one of his books was first translated into English, namely Thought and Language. Among the reasons why Vygotsky’s work did not reach beyond the borders of the Soviet Union earlier were the sporadic publication of his work and also the fact that after his death Vygotsky’s theories were banned until 1956. [YAROSHEVSKY, 1989]
Activity theory is a socio-cultural theory of the development of the human mind. The thesis of this theory is that the human mind is constructed through interaction with the world, which means that rather than an attribute of the person, the mind can be understood as an attribute of the relationship between object and subject. In activity theory, consciousness is seen as the result of practical activity within the dynamic relationship between internal cognition and the external world of artefacts and people. [KAPTELININ and NARDI, 2006, p. 197]. That is why its key principle is so-called tool mediation, in which it is not only technological tools that help the socially contextualised subject with development and education. It should be recalled that Vygotsky was primarily interested in the psychological development of people in education.
In educational psychology he postulated two significant laws. The first of them, The Zone of Proximal development, defines the potential of what a person is able to learn if they are given help by a teacher-mentor. The second, universal law of psychological development says that new psychological functions are first distributed among people or tools, and with increased competence and experience the external help is replaced by internalisation, after which the subject manages in the given activities without outside help [KAPTELININ and NARDI, 2006, p. 204]. Activity theory, like distributed cognition, thus contains the concept of tool mediation, thanks to which human subjects are able to extend their cognitive skills and competences.
Tool mediation is a key element for activity theory. In relation to HCI, interaction design and UX, activity theory proposes that the design of technology be based on an analysis of the role that technology has in the activities we perform with it. The design of technologies should therefore concern itself with the moral and ethical problems, cultural diversity, social implications, emotions, feelings or spirituality that technology mediates [KAPTELININ and NARDI, 2006, p. 27].
Compared with the first, cognitivist wave of HCI, activity theory is interested in the social context in which activities take place and in the meaning of these activities. Activity theory thus explicitly denies that human-machine interactions can be studied in laboratory conditions, completely stripped of the real context of the activities.
The first to point out the use of activity theory in HCI was the Danish computer science academic Susanne Bødker, who was a key figure in the participatory design community. In 1982–1983, during her academic stay at Xerox PARC, and also on the basis of her experience with participatory design, she realised the limits of the then dominant information-cognitive approach to HCI. Seven years later she submitted her doctoral dissertation, in which she argued that activity theory could be a suitable foundation for HCI research and design, since it provides methods for analysing the human use of technology in the wider context of interaction with the world: interaction mediated by technology. [KAPTELININ and NARDI, 2006, p. 73].
Phenomenology and embodied interaction
In his book Where the Action Is Paul Dourish connects a deeply intellectual analysis of the philosophical currents of phenomenology with theoretical approaches in the field of HCI. Dourish focuses on two new currents in interaction design: tangible computing and social computing. Tangible computing (graspable, touchable computing), or the tangible user interface, is an approach to user interface design that emphasises the possibility of physical and spatial interaction with interactive objects, so that the user can interact with and manipulate the digital and data layer of the objects. Social computing is then an approach that emphasises that interaction with technologies is bound to the context in which this interaction takes place. That is why, as Dourish points out, it is advisable to draw on traditional sociological, anthropological and ethnographic methods to obtain information about how users really use technology. Dourish emphasises that social action is not an abstraction but is socially embedded [DOURISH, 2004, p. 96].
On the basis of his analysis of these two approaches to computing, Dourish continues with an analysis of phenomenological ideas, by means of which he points to what Heidegger had already argued, namely that a person is situated in a physical, pre-organised world of various demands and possibilities, where the human mind is embodied in a physical body, just as the body is embedded in the physical world [PEPPERELL, 2013]. Dourish’s own term “embodied interaction” builds on the phenomenological analysis, and he defines it as “the creation, manipulation and sharing of meaning through […] interaction with artefacts” [DOURISH, 2004, p. 126]. Dourish reflects the analysis I carried out earlier of the shortcomings of the first wave of HCI and of cognitivism, including the situatedness and embodiment of cognition, when he writes that “[o]ur actions cannot be separated from the meanings that we and others ascribe to them.” [DOURISH, 2004, p. 189]. Dourish is thus interested in a problem already mentioned several times, for HCI and for design in general: how to examine actions, activities and practical work in their social context so that designers are able to use this knowledge to design technologies that are functional, friendlier, aesthetically more interesting and that also provide meaningful experiences. His discussion of sociological and ethnographic methods for use in HCI and UX design is thus a concrete example of how to examine the wider socio-cultural context in which people use technology.
Actor-network theory and distributed cognition in HCI
Actor-network theory and distributed cognition, like the other post-cognitivist approaches, reject the subject-object dichotomy. ANT resolves it by proposing to place the human subject in heterogeneous networks in which there further appear artefacts, organisations, but also immaterial objects. All the components of such networks are called actors (agents, actants and the like), which interact with one another. ANT is by its nature a relational and post-structuralist theory, since it attributes fundamental weight to the relationships between the individual nodes of the networks. Agency is ascribed to people and to inanimate objects alike on the basis of the relationships they have with other nodes of the network.
Distributed cognition is a well-known approach in the field of HCI which, like activity theory, emphasises the importance of tools and technological artefacts for human cognition [KAPTELININ and NARDI, 2006, p. 195]. For the comparison with the other post-cognitivist approaches, it is important that according to distributed cognition people and tools are defined within the cognitive system as equivalent media. [KAPTELININ and NARDI, 2006, p. 195]
Whereas activity theory, phenomenology and postphenomenology attribute agency and intentionality only to human beings, ANT and distributed cognition understand the human being as an equivalent “node” in a network of living and non-living, material and immaterial actors, thereby establishing the relationships between the nodes of the network as symmetrical. Verbeek comments that with its radical symmetry of actors, ANT loses the ability to explain the experience of human actors, including how technological artefacts uniquely mediate the world for human actors. [VERBEEK, 2015]
Postphenomenology
In the philosophy of technology today we find active research in the so-called mediation theory of technology, pursued by Peter-Paul Verbeek. Mediation theory analyses and explains various types of relationships and interactions between people and technology. Verbeek’s research also deals with the ethical questions of technology, through which Verbeek wants to contribute to the discussion of how technology can improve people’s lives. This is a shift from the earlier focus of the philosophy of technology, which by and large saw technology as a negative element. Technology was analysed against the background of a dialectical view, in which technology is something people must come to terms with, must overcome, must defend themselves against, so as not to lose completely what remains of their humanity. Verbeek, however, indirectly builds on McLuhan’s “programme” of studying the mediating effects of technologies. At the same time his mediation theory goes further: not only does it study the mediating role of technologies from their technical point of view, it also examines how people appropriate these technological mediations, a topic that still remains largely unexplored [VERBEEK, 2015]. Verbeek’s work fits into the wider philosophical current within the philosophy of technology, postphenomenology. Other prominent authors of this current include the American philosopher Don Ihde, from whose contribution, a detailed analysis of the types of human-technology interaction, Verbeek starts.
First of all, postphenomenology too rejects the modernist split of the subject-object dichotomy, but unlike ANT and distributed cognition it does not agree that the human actor should be analysed on the same level as technological or immaterial objects.
Mediation theory, however, does not see people and technology as opposite and fixed poles between which interaction takes place, but as the result of this interaction. [VERBEEK, 2015a] Mediation theory redefines interaction design. Instead of “interaction” as the main unit to be designed, it proposes that interaction designers realise that they are not designing interactions with technology but the relationship of people to the world itself, which we access through technology. This is a familiar note that we have heard several times in this text. Technology as a medium, as a mediator. Verbeek adds:
“Interaction design […] implies not only the design of technological objects that enable specific interactions, but also the design of the human subjects who interact with these objects. Designing technology means designing human beings.” [VERBEEK, 2015a]
Postphenomenology and mediation theory are critical of seeing the human-technology relationship only from the point of view of functionality and usability, and in this they comment on the inadequacy of the first wave of HCI, which focused primarily on precisely these two aspects of interaction. Verbeek writes that in the age of smart mobile technologies, ambient technology and the various implants and technological prostheses used in healthcare, functionality and usability are limiting tools for analysing technology. If we have an implant in our brain, if we use a fully automated smart home, these are not, strictly speaking, used. On the contrary, they are designed so that a person comes into contact with them as little as possible. Postphenomenology thus claims that it is necessary to problematise in more detail how else the human-technology relationship can be seen. Verbeek lists three approaches: extension, dialectics and hybridity. [VERBEEK, 2015a]
In the case of the first approach, extension, we perceive technology primarily as tools, where we are interested above all in its instrumental value in fulfilling our goals and needs. Technology seems neutral in the sense that it does not actively take part in shaping our goals and needs; it only passively helps to fulfil them.
The dialectical approach emphasises technology as a force that acts on people as a negative element causing a sense of alienation. Examples are the Marxist critique of mechanisation [VERBEEK, 2015a] or the current debate about the influence of automation and artificial intelligence, which accelerate so-called technological unemployment, in which technologies increasingly replace low-skilled workers. Verbeek mentions the work of Ernst Kapp and Wilhelm Schmid, who understood the dialectical relationship to technology as a gradual liberation through the externalisation of certain human skills. Already in 1877, Kapp’s philosophical-anthropological approach to technology saw technology as a projection of human organs. [VERBEEK, 2015a]
Hybridity is an approach that draws attention to the dualistic subject-object tendencies of the previous approaches. Instead of the opposite poles of human/subject and technology/object, this approach proposes the concept of the hybrid, in which people and technology are mutually influencing forces that shape one another. At the same time, the hybrid approach explicitly declares that technology actively shapes our behaviour and interaction with the world, as a medium that connects us with the world.
Verbeek further cites the work of the American philosopher Don Ihde and his more detailed elaboration of the human-technology relationship. Ihde names the following relations: embodiment, hermeneutic, alterity and background relations. Verbeek supplements Ihde’s analysis with his own relations of the cyborg, immersion and augmentation. A brief description of the individual relations can be found in the article A short introduction for mediation theory, written by Verbeek himself.
Verbeek, however, is interested first and foremost in the ethical problems of technology. Like the post-cognitivist theories of HCI mentioned above, postphenomenological mediation theory too agrees that technology plays an active role in shaping our lives, and that designers therefore take on responsibility for the affordances they design into technologies. As I mentioned in another part of the text, however, affordances are a relative relationship between a person and their environment. Just like experiences, affordances are not fully under the designer’s control. That does not mean, though, that rigorous analysis and reflection on design work are unnecessary. On the contrary, Verbeek argues that the philosophy of technology is experiencing a renaissance right now, because the design of technologies needs a certain theoretical framework that would give designers firm ground from which they can push off in their creative work. Postphenomenological philosophy of technology is trying to do precisely this.
Conclusion
In this text I have tried to outline important aspects of current thinking about the design of technologies. First, I tried to problematise and show that the original premises of the first wave of HCI, based on a cognitivist and information-processing model of human cognition, ignored the socio-cultural context in which our cognition takes place. Further, through an analysis of the critique of cognitivism, I pointed to alternative, post-cognitivist theories that try to situate the person and their body in a wider context, where action, activity and meaning derive from practical, everyday activity and “interaction with the world”. Post-cognitivist theories include the phenomenology of Martin Heidegger and his successors, activity theory, actor-network theory and distributed cognition. Alongside these theories I presented the relevant current of postphenomenological research in the philosophy of technology.
The theories and approaches mentioned should provide the theoretical core of the so-called second wave of HCI, interaction and UX design. These fields must respond to the fact that the design of technologies is now becoming an activity that fundamentally affects human lives, for a reason on which all the post-cognitivist theories and postphenomenology agree: technology is a medium that actively shapes our relationship to the world. Designers hold power in their hands, but also an enormous potential to improve the present world. Without a theoretical foundation, however, this power may degenerate either into unused potential or, in the worse case, into a dystopian scenario in which technologies slip out of our hands, because for far too long we let designers “play” without their work being examined critically and theoretically-philosophically.
This text set itself, in all modesty, the goal of presenting topics that should not be missing from the theoretical and philosophical study of the influences of technology. A detailed analysis of post-cognitivist theories, above all of actor-network theory, was beyond the scope of this text. Partly for this reason, I intend to remedy these shortcomings in my master’s thesis.
References
BATTARBEE, KATJA and ILPO KOSKINEN. Co-experience: user experience as interaction. CoDesign[online]. 2005, 1(1): 5-18 [cited 2015-12-06]. DOI: 10.1080/15710880412331289917. ISSN 15710882. Available at: http://www.tandfonline.com/doi/abs/10.1080/15710880412331289917
BREY, Philip, c2001. Hubert Dreyfus — Human versus Machine. ACHTERHUIS, Hans. American philosophy of technology: the empirical turn. Bloomington: Indiana University Press, pp. 37–63. ISBN 0253214491.
CHOKR, Nader N., 2014. ‚Philosophy‘: After the End of Philosophy In a Globalizing and Glocalizing World. Cambridge Scholars Publishing. ISBN 9781443865388.
CLARK, A. and D. CHALMERS, 1998. The Extended Mind. Analysis [online]. 58(1), 7-19 [cited 2017-10-16]. DOI: 10.1093/analys/58.1.7. ISSN 0003-2638. Available at: https://academic.oup.com/analysis/article-lookup/doi/10.1093/analys/58.1.7
CLARK, Andy, 1998. Being there: putting brain, body, and world together again. Cambridge: The MIT Press. ISBN 0-262-53156-9.
COWART, Monica, Embodied Cognition. The Internet Encyclopedia of Philosophy [online]. [cited 2017-10-17]. Available at: http://www.iep.utm.edu/embodcog/
DAWSON, Michael R.W., 2013. Mind, body, world: foundations of cognitive science. [CEL version]. Edmonton: AU Press. ISBN 9781927356180.
DE JONG, Alex, Marc, SCHUILENBURG. 2006. Mediapolis: popular culture and the city. Rotterdam: 010 Publ. ISBN 9789064506284.
DESCARTES, René, 2003. Meditace o první filosofii: námitky a autorovy odpovědi. Translated by Tomáš MARVAN, translated by Petr GLOMBÍČEK, translated by Pavel ZAVADIL. Praha: OIKOYMENH. Knihovna novověké tradice a současnosti. ISBN 80-7298-084-x.
DOURISH, Paul, 2004. Where the action is: the foundations of embodied interaction. Cambridge, Mass. [u.a.]: MIT Press. ISBN 9780262541787.
DEUZE, Mark, 2015. Media life: Život v médiích. Translated by Petra IZDNÁ. Praha: Univerzita Karlova v Praze, nakladatelství Karolinum. Studia nových médií. ISBN 978–80–246–2815–8.
DREYFUS, Hubert L., 1967. Why Computers Must Have Bodies in Order to Be Intelligent. The Review of Metaphysics. Philosophy Education Society, 21(№1 (Sep.), 13–32.
ENGEL, Andreas K., K. J. FRISTON and Danica KRAGIC, 2015. The pragmatic turn: toward action-oriented views in cognitive science. Cambridge, Massachusetts: The MIT Press. ISBN 9780262034326.
FERENC, Jakub. Teorie současného web designu a její aplikace při návrhu responsivních www stránek. Praha, 2015. Bachelor’s thesis. Charles University in Prague, Faculty of Education, Department of Information Technology and Technical Education. Supervisor: PhDr. Josef Procházka, Ph.D.
GIBSON, James J., 2015. The Ecological Approach to Visual Perception. Classic Edition. 711 Third Avenue, New York, NY 10017: Psychology Press. ISBN 978-1-315-74021-8.
HASSENZAHL, Marc and Noam TRACTINSKY. User experience – a research agenda. In: Behaviour & Information Technology [online]. 2006,25(2): 91-97 [cited 2015-12-06]. DOI: 10.1080/01449290500330331. ISSN 0144-929x. Available at: http://www.tandfonline.com/doi/abs/10.1080/01449290500330331
HEIDEGGER, Martin, 1996. Bytí a čas. Praha: Oikoymenh. Oikúmené. ISBN 80–86–005–12–7.
HEIDEGGER, Martin, John MACQUARRIE and Edward ROBINSON, 2006. Being and time. Oxford: Blackwell. ISBN 0631197702.
Herbert Blumer, 2001. In: Wikipedia: the free encyclopedia [online]. San Francisco (CA): Wikimedia Foundation [cited 2017-10-21]. Available at: https://cs.wikipedia.org/wiki/Herbert_Blumer
JONES, Peter H., 2014. Systemic Design Principles for Complex Social Systems [online]. p. 91 [cited 2017–08–22]. DOI: 10.1007/978–4–431–54478–4_4. ISBN 978–4–431–54478–4. Available at: http://link.springer.com/10.1007/978-4-431-54478-4_4
KAPTELININ, Victor, Bonnie NARDI, Susanne BØDKER, John CARROLL, Jim HOLLAN, Edwin HUTCHINS and Terry WINOGRAD, 2003. Post-cognitivist HCI. In: CHI ’03 extended abstracts on Human factors in computing systems – CHI ’03 [online]. New York, New York, USA: ACM Press, p. 692- [cited 2017-10-15]. DOI: 10.1145/765891.765933. ISBN 1581136374. Available at: http://portal.acm.org/citation.cfm?doid=765891.765933
KAPTELININ, Victor. and Bonnie A. NARDI, 2006. Acting with technology: activity theory and interaction design. Cambridge, Mass.: MIT Press. Acting with technology. ISBN 9780262513319.
LEIDLMAIR, Karl, ed., 2009. After cognitivism a reassessment of cognitive science and philosophy. Dordrecht: Springer. ISBN 9781402099922.
MCLUHAN, Marshall, 1991. Jak rozumět médiím: extenze člověka. Praha: Odeon. Eseje (Odeon). ISBN 80-207-0296-2.
MERHOLZ, Peter. 1998. Whither “User Experience”? [online]. [cited 2015–05–11]. Available at: http://peterme.com/index112498.html
NORMAN, Donald A., 2013. The design of everyday things. Revised and expanded edition. New York, New York: Basic Books. ISBN 978-0-465-00394-5.
NORMAN, Don, Affordances and Design. Jnd.org [online]. [cited 2017-10-19]. Available at: http://www.jnd.org/dn.mss/affordances_and.html
PEPPERELL, Robert, 2003. Where The Action Is: The Foundations of Embodied Interaction by Paul Dourish. MIT Press, Cambridge, MA, U.S.A., 2001. 229 pp., illus. ISBN. Leonardo[online]. 36(5), 412-413 [cited 2017-10-21]. DOI: 10.1162/leon.2003.36.5.412. ISSN 0024-094x. Available at: http://www.mitpressjournals.org/doi/10.1162/leon.2003.36.5.412
RITTEL, Horst W. J. and Melvin M. WEBBER, 1973. Dilemmas in a general theory of planning. In: Policy Sciences. 4(2), pp. 155–169. DOI: 10.1007/BF01405730. ISSN 0032–2687. Also available at: http://link.springer.com/10.1007/BF01405730
RITTEL, Horst W. J. The Reasoning of Designers. In: Arbeitspapier zum International Congress on Planning and Design Theory in Boston, August 1987. Universitaet Stuttgart: Schriftenreihe des Instituts fuer Grundlagen der Planung, Universitaet Stuttgart, 1988. Available at: http://www.cc.gatech.edu/~ellendo/rittel/rittel-reasoning.pdf
ROUSSEL, Nicolas, 2014. Looking back: a very brief history of HCI. Inria Lille – Nord Europe. Also available at: http://direction.bordeaux.inria.fr/~roussel/publications/2014-looking-back.pdf
SHANAHAN, Murray, 2016. The Frame Problem. ZALTA, Edward N. The Stanford Encyclopedia of Philosophy (Spring 2016 Edition) [online]. Stanford University: Metaphysics Research Lab [cited 2017-10-17]. Available at: https://plato.stanford.edu/entries/frame-problem/
SHAPIRO, Lawrence, 2011. Embodied cognition. London: Routledge. ISBN 9780415773423.
VÄÄNÄNEN-VAINIO-MATTILA, Kaisa, Virpi ROTO and Marc HASSENZAHL. Now let’s do it in practice. In: Proceeding of the twenty-sixth annual CHI conference extended abstracts on
Human factors in computing systems – CHI ’08 [online]. New York, New York, USA: ACM Press, 2008, p. 3961- [cited 2015-12-06]. DOI: 10.1145/1358628.1358967. ISBN 978160558012x. Available at: http://portal.acm.org/citation.cfm?doid=1358628.1358967
VERBEEK, Peter-Paul, 2015. Toward a Theory of Technological Mediation: A Program for Postphenomenological Research. BERG, Jan Kyrre, O FRIIS and Robert P. CREASE. Technoscience and postphenomenology: the Manhattan papers. Lanham, MD: Lexington Books, pp. 189-204. ISBN 978-0-7391-8961-0.
VERBEEK, Peter-Paul, 2015a. Beyond interaction. Interactions [online]. 22(3), 26-31 [cited 2017-10-22]. DOI: 10.1145/2751314. ISSN 10725520. Available at: http://dl.acm.org/citation.cfm?doid=2767137.2751314
WINOGRAD, Terry and Fernando FLORES, 1987. Understanding computers and cognition: a new foundation for design. Reading, Mass.: Addison-Wesley Publishing Company. ISBN 0-201-11297-3.
YAROSHEVSKY, M, 1989. Lev Vygotsky. Moscow: Progress Publishers.
ZALTA, Edward N, 2017. Martin Heidegger. The Stanford Encyclopedia of Philosophy (Fall 2017 Edition) [online]. [cited 2017–10–09]. Available at: https://plato.stanford.edu/entries/heidegger/