1. Introduction
The main aim of this work is to propose a way to finally merge the digital world and real one in Augmented Reality by embedding it with elements coming from a phenomenological perspective.
It is well known Augmented Reality aims to merge the digital and real world [
1]. We will use the term “digital” as opposed to “real” in order to identify the elements generated through augmented reality. This fusion between two different worlds is not achieved by merely creating a link between a digital database and an object in the everyday world, but it is also made thanks to a visualisation of digital objects in the surroundings. The system provides perceptual digital objects as if it were part of the everyday world where subjects live [
2].
Since it is possible to develop different particular systems by focussing on the five senses, there are many ways to realise an augmented reality. The augmented reality can provide the perception of a digital object thanks to tactual, olfactory, acoustic, taste, and visual stimuli depending on the interest of the developers [
3,
4,
5]. Moreover, augmented reality can merely overlay the everyday world with a layer of information which refers to pre-existing objects, or it can produce new objects which are placed in the surroundings [
6]. However, despite the presence of these many ways of producing an augmented reality, we need to limit our analysis in order to be more precise and to provide examples which can be easily compared and understood. Since visual augmented realities are pervasively used, and phenomenology directly focusses mainly on the visual field [
7]. We can limit our analysis to the augmented reality producing new visual objects without taking into consideration other senses and other ways to merge the digital and real world. Even if we limit our analysis to visual objects, it is also possible to extend it to other sensible fields since our work tackles problems in augmented reality in general and not merely in visual applications.
There are many different examples of visual augmented reality generating digital objects which are visualised as part of the surroundings through head-mounted devices and mobile technologies. For example, one of the most famous applications for mobile devices is
Pokémon GO by
Niantic (
http://www.pokemongo.com/ accessed on 13 October 2017, [
8,
9,
10,
11]). This application generates Pokémon all around the subject to be visualised through the screen of a mobile device which merges the image of the surroundings captured by a camera and the digital image of a Pokémon. However, it seems that the people playing this game are not interacting with actual objects, but merely with fictions generated within the digital game-like world which is perceived only by the small community playing the game. Augmented reality seems to produce new fictions in our world more than producing new objects in the everyday world [
12]. Thus, augmented reality is facing an ontological problem, and it is not able to achieve its original goal to merge the digital and real world.
This paper will show some elements that augmented reality needs to have to finally achieve its goal. Moreover, we will provide an example of how to solve the problem by using specific devices.
The paper will be structured in two main sections:
The first section will focus on the introduction of digital objects in the everyday world. We will take into consideration postphenomenology and mediation theory in order to show how our world is embedded with digital objects. Moreover, we will show how augmented reality is willing to introduce other new digital objects in the everyday world, but, at the same time, it is facing ontological problems which preclude it from achieving its goal.
The second section will focus on the elements needed in order to provide an augmented reality which is finally capable of introducing digital objects as common objects in our everyday world. We will use the story The Emperor’s new clothes in order to highlight some elements of the digital objects needed in order to be part of the everyday world, and we will propose a possible solution in order to finally achieve augmented reality’s goal.
2. Digital Objects and Everyday World
2.1. Postphenomenology and Digital Objects
According to postphenomenology, our world is constituted through technologies. Technologies provide the perception of objects, and, through this perception, the objects become part of the world where we live from the moment they are visualised. Postphenomenology clearly points out the world and subjects are founded on technologies. The subject and the world are not two entities which are untouched and immutable through time. They are co-constituted through the use of technologies, and so they change according to the devices subjects use [
13].
Subjects are shaped in the perceptual capabilities since technologies allow them to modify their sensory organs. For example, optical glasses allow users to perceive an object with more details enriching their perceptual capability. Obviously, it is not just a mere enhancement since every technology also yields reductions. For example, in the case of glasses, the subject does see just more details thanks to the technology, but glasses also produces chromatic aberration and lens distortions [
14]. However, we will focus just on the technological mediation in general without getting into details on the enhancement produced.
At the same time, the world becomes inhabited by new objects made perceptual thanks to the use of technologies. They allow the users to perceive something which is hidden or unreachable, and, by doing so, they produce new perceptual objects. For example, a famous example related to optics is the telescope [
15,
16,
17]. The telescope does not only produce a magnification, but it allows the user to see unseen aspects of the universe we live in. New planets become visible thanks to the use this technology.
Digital technologies just move in the same direction. They allow subject to perceive the world around them in a different way, and so they shape what the subjects are and what “dwells” in the world. Digital technologies allow users to extend their visual capabilities by turning some part of the invisible spectrum of light into visible such as the infra-red or ultraviolet radiations. Moreover, they generate new objects in our world by making visible something which was hidden and unreachable like in the case of the brain activities with fMRI [
18,
19], unborn children with Ultrasound technologies [
20], and distant galaxies with radiotelescopes [
21,
22]. These entities are the production of the use of certain kind of digital technologies which produce visual images of objects which were invisible to the naked eye.
As mediation theory suggests, the introduction of new entities and the shaping activities of the devices directly moulds our everyday choices and our way of making ethical decisions [
23]. Since it is possible to have an image of unborn children which are visualised with the dimension of an actual child on a monitor, the parents are shaped into new subjects who have to make an ethical choice on the unborn child in case of malformations or any other problems emerging from the possibility of that image [
24]. Technologies, and so digital technologies too, are not neutral. They shape the way subjects perceive the world and the way they are called for actions and moral decisions because of what the devices do. Therefore, the impact of technologies is not merely seen in terms of what the world and the subjects are, but also on the way subjects act and structure their lives.
Augmented reality seems to introduce digital objects in the same way by allowing users to perceive new objects as part of the surroundings. These objects are generated by a digital system and made perceptual through the use of a device. For example, in the case of Pokémon GO, Pokémon are visualised as placed in the surroundings through the use of the screen of a smartphone. Therefore, they can aim to shape our world in the same way as other technologies like ultrasound technologies by introducing new objects. However, even if the way unborn children and Pokémon are generated is the same, the result achieved by the systems is completely different. Unborn children visualised as digital objects are different from the digital objects generated through augmented reality. The former are real entities, while the latter are not. The difference cannot be related to the fact that digital objects in augmented reality are generated through a digital system since even the other objects are generated through the use of digital technologies. Once objects generated through digital technologies are accepted as constituting part of the world as suggested by postphenomenology, we cannot found the difference between these two entities within their digital origins. Even if we try to anchor our distinction on real elements present in the objects, we cannot draw a clear line between them. The unborn children visualised on the screen have real elements since they are spatially located within the womb of their mothers. However, even the digital objects in augmented reality have real elements since they are spatially located thanks to GPS coordinates. Therefore, the difference cannot be related to their digital origins nor to their relation to the real world.
Understanding how these digital entities are different seems to be important in order to figure out if it is possible to live in an augmented world where digital objects produced with augmented reality are perceived as common objects around the users. More specifically, it is important to understand where this difference lays in order to know if we can design augmented objects which can become part of our everyday world as unborn children do.
2.2. Fantasies
Digital objects produced in augmented reality like
Pokémon GO are visualised as part of the surroundings, but they do not seem to be part of the everyday world where everybody lives. They are different from the unborn children or distant galaxies even if they are generated through a digital technology in the way subjects perceive and relate to them. We will follow a phenomenological perspective even if there are other ways to tackle the topic (on user acceptance of AR, see [
25,
26,
27,
28,
29], on fashion related to new digital technology, see [
30]).
Digital objects produced in augmented realities are merely valid within the small community that uses the right devices and the right applications. Therefore, they are perceived like mere fictional entities just like if they were fantasy objects generated within the game world of a child. AR users act like the children who play against monsters generated by their imagination. While playing they interact with these objects, but other people around simply do not fall in that illusionary game.
When children run in a park playing with their fictional objects, these fictions are quite effective within their game-like world, but they are not valid in the everyday world where everybody else lives [
31,
32]. In the case of AR users, we have a similar situation [
12,
33]. People who do not use Augmented Reality look at the AR users as mere players lost in their game-like world, even if this game takes place in their surroundings and it allows them to perceive their augmented objects instead of merely be the product of the gamers’ fantasy.
According to phenomenology, common objects are not only perceived, but they are experienced as things the subjects have to live with. As Alfred Schütz clearly pointed out, the world, and therefore the objects in it, resists to the subjects because with it they face something which is out of their power [
34,
35]. Thus even if subjects can work on it, and they can shape it through their work, they cannot simply ignore it [
36,
37].
The objects are part of the world in the sense they are not so dependant on what the subjects want. Maybe subjects would like to have billions of Euro in their Bank account, but this mere thought is not enough to materialise that amount of money in their account, and the subjects have to live with it. Common objects are made of something which is visible to everybody without the use of any particular device. It is not merely their perceptual presence which is important, but it is the way they impose themselves through their perceptual presence to the others. They impose themselves, and they are resistant in the sense they force their existence to people. The objects are given to the other subjects who cannot ignore them, but with which they have to co-live.
In the case of augmented reality, digital objects are visualised in the surroundings, but they are present just for the users. People around the users do not perceive any digital objects in their surroundings. They perceive just other people tapping on their smartphones and swaying their devices in the air without any apparent reason. Even in the case the people around the users know the compulsive tapping is part of an AR games, they do not have any access to that reality if not through an AR device which opens that digital world to them.
Real objects impose themselves to the people around, and so they force people to co-live with them. Digital objects like Pokémon do not impose themselves, and so they do not force this co-living. The difference between digital and real objects is not related to the fact one is perceptual, and the other is not, but it is founded on the co-living implied by the real objects.
The merge between real and digital world is achieved because these digital objects are visualised as part of the surroundings, but this mere visualisation is not enough to produce a perfect intertwinement between the two kinds of objects since the digital ones look like fantasies attached to the real world. Digital objects in augmented reality use the real world as mere background instead of becoming part of it.
3. Emperor Clothes and AR Clothes
The story
The Emperor’s new clothes by Hans Christen Andersen [
38] clearly highlights the limits and the problems of such kind of objects produced with these augmented realities. Thus, by drawing a parallel with the story, we will clarify what the elements which need to be changed in order to have an augmented reality which is part of the everyday world are.
In this story, a vain Emperor who only cares about expensive and fancy clothes hires two weavers who promise him extraordinary clothes which are invisible to whom is unfit for his position. When the Emperor wore these invisible clothes and marched down the streets, a child started to scream what everybody was seeing but was afraid to say. The child pointed out the Emperor was marching naked because he was not able to see any clothes. Because of this “obvious” statement, everybody around the Emperor started to laugh, and they started mocking the Emperor for its nakedness. There are different ends to the story. In one the Emperor simply ignored the comments and continued his walk as if nothing happened. In another one, he realised that he was hustled, and so he covered himself while running away from the crowd.
This story can be read in many different ways in order to highlight different elements. For example, a possible reading highlights the need for a moral imperative to say what a person thinks without caring too much of other social boundaries which limit their expression [
39]. However, we will focus our attention on the perceptions of these clothes by taking into account merely the fact the Emperor is wearing objects which are invisible to everybody around him because this element is more than enough to draw the parallel we need. In the story, not even the Emperor was able to see the clothes because these clothes were non-existent. However, we can think of the story focussing on the effect produced on the people around the Emperor. In this way, we will be able to analyse the existence of the clothes from their perspective.
The Emperor is wearing clothes which are not visible and valid outside a small community. Only a small community can perceive the clothes, but everyone else around cannot see them, and they can point out the Emperor is naked. Moreover, it is not merely the possibility of other people to perceive the naked Emperor, but their comment makes the “illusion” of the presence of the clothes end for everybody around the Emperor who was trying just to believe in the presence of these invisible objects. A simple statement made by a child is enough to crack the dream the people around have. Thus, it highlights how these illusionary clothes simply vanish the very moment people decide they are non-existent.
The problem the Emperor has with his clothes is not so much that he was hustled, but the absence of two main characteristics in those clothes:
In order to see under the common clothes of someone, subjects need to physically remove them, and they cannot simply vanish. The common clothes impose themselves on the surrounding subjects.
In the case of the Emperor’s clothes, they are not visible to everybody since the child is able to say that the king is walking naked. They do not impose themselves on the others. It is important to highlight it is not the perceptual aspect of the clothes which produces the imposition of their existence. Even invisible objects can impose themselves. However, common clothes impose themselves on who is around because of their visual presence. Thus, the perceptual presence is not necessary, but it can provide the ground to produce this imposition to the others. Moreover, they are not resistant because the moment the child says that the Emperor is naked, everybody starts to see him as naked, and they start to laugh at him. For these two reasons, the clothes cannot be considered part of the world since they are not perceivable by everybody and they do not oppose any resistance.
With augmented reality, we have the same kind of problem. We can just re-imagine the story by using augmented reality in it in order to see how the augmented objects are perceived by people around the wearer.
A person buys digital clothes on an application from a digital waiver who provides the 3D model of clothes which can be visualised as placed on the user’s body through augmented reality glasses or mobile devices. The person “wears” these clothes made of digital tissue and visualised through the devices, and the wearer decides to go out in the city showing the new clothes to other people.
In our story, these digital clothes are visible and valid for a small community using the right applications. However, other people do not see anything but the users’ naked body just like the child in the Emperor’s case who sees the naked body of the Emperor.
The digital clothes are not perceivable by everybody because only the ones using a particular digital device are able to perceive them. At the same time, the clothes are not even resistant. The moment the users decide to see the wearer naked, they can simply turn the device off, and the digital clothes cease to exist for them, allowing their gaze to reach the nudity the clothes were supposed to cover.
We have:
These clothes are not visible to everybody, but merely to the people having and using specific devices.
These clothes can be ignored quite easily by just turning the devices off.
Can we say augmented reality fulfils its goals in merging real and digital world even if it provides objects which look like the nonexistent clothes of the Emperor?
Obviously not, and the reason is related to how these augmented objects are produced by the devices.
3.1. Problems and Devices Dependencies
These digital objects are directly related to the existence of the technology which makes them visible: the AR devices like glasses and mobile devices producing the visualisation. The digital clothes are perceivable just because there is a device which makes them perceptual for the users.
This dependency relating the digital objects and the devices like glasses and mobile devices which produce the visualisation has two consequences.
For these reasons, these digital clothes are not part of the everyday world, but they are merely fantasies which are present only for who decide to live in that illusionary reality. At the same time, also the other augmented realities which produce the same kind of digital objects which do not impose themselves are fantasies too. However, these limits are not imposed by Augmented Reality by itself, but on the specific ways these digital objects are made visual by some technologies like glasses and mobile devices which generate digital objects with these two elements. There are other possible ways of producing augmented realities thanks to different technologies. Therefore, it is possible to elude these elements in order to provide digital objects which are not resistant and perceived by everybody by using different devices.
3.2. Possible Alternatives
A possible solution to surpass these limitations is the use of projectors instead of using glasses and mobile devices. Digital images are still digital objects, and so they can be treated as Augmented Reality. Projectors have been pervasively used in augmented reality in the past [
40,
41,
42,
43,
44].
Projectors visualise an image. Thus, as in the previous case, the projection is dependent on the device since it is visible only when the projector is active and it works correctly. However, it is not necessary to look through devices in order to perceive the image. This simple difference allows the objects to be perceived in a completely different way. The moment the projectors cannot be manipulated by the subjects, the images are imposed on them. Instead of perceiving the augmented objects because they decided to turn the mobile device on or to wear AR glasses, with projectors they are forced to perceive the digital objects. The perception of these objects are not limited to a small community using the AR devices, but they are visible to everybody around them.
In the visualisation of these images, there is a sort of imposition or violence involved towards the people around since they cannot erase the objects, but they have to cope with them. The example of the common clothes shows this “resistance” since people cannot take them off so easily, but there are other examples which make the resistance even more vivid. For example, the images produced through projectors can have political effects, and they impose themselves on the viewers. On his
Projection on South Africa House in Trafalgar Square (London) in 1985 [
45], Krzysztof Wodiczko projected the image of a swastika on the South Africa Embassy. He showed how images impose their resistance on others since only police were able to turn the projectors off. That image highlights the “violence” involved in the use of projectors since he forced the Embassy to turn their “face” into something they do not like before police stopped him.
We can re-think of our example with digital clothes by using projectors instead of glasses or mobile devices. Projectors can provide images which work as a layer on usual clothes of the wearer in order to change their colours, or they can project directly on the naked skin of the “wearer” hiding the nudity.
Even if these digital clothes are still dependant on the presence of the projectors, they are projected on the skin of the wearer, and they are visible to everybody. Moreover, these digital clothes are resistant to the subjects since they are not able to turn the projectors off. In the previous case, the devices producing the digital clothes were the personal device of the onlookers, and so people were able to de-activate them. In the case of projectors, the projectors are not in control of the onlookers, and so they have no power over them. The only person who can decide where and what to project is the person controlling the projectors. Not even the wearer has the power to decide what to wear, just like in the case of the South African Embassy, the South African Ambassador was not able to decide to immediately erase the swastika projected on his building, but he had to pass through the police in order to take it away.
The access to the projectors plays an important role because it determines the level of resistance the digital objects have. The moment the projectors are out of the control of the wearer, the digital clothes become resistant to the wearer too because the user is not able to choose what to wear. The digital objects are imposed on everybody. At the same time, if the projectors are under the control of everybody, the digital objects are not resistant since everybody is able to easily deactivate them. In order to provide digital clothes which mimicing the presence of real clothes made of real tissue, the access to the projectors should be open to the wearing subject only.
Just like in the case of the Emperor who chooses to wear the invisible clothes, the projectors creating the digital objects should be under the control of the wearer. The projectors should be personal in order to allow the user, and only the user, to decide what to wear. The wearer does not have to live with imposed digital clothes on their body, but they are the ones who decide to have them by using the projectors just like in the case of real clothes where the wearer decides what to wear. Obviously, the clothes are just a “layer of light” without any tactile presence, but they are enough to cover the body from another person’s gaze (
Figure 1).
Moreover, in order to improve this effect, the projectors should not be only personal but mounted on the wearer. Even if the projector is personal, people could interfere with the projection of the images such as by interposing an object between the projectors and the wearer of the digital clothes. To limit the access to the projectors to people who are not the wearer, the projectors should be mounted as close as possible to the body of the user, and, ideally, they should be worn by the wearer.
Digital clothes, if they are produced through personal projectors are visible to everybody. Moreover, the digital objects produced by the personal projector mounted on the user are still directly related to the existence of the digital technology which generates them. However, since it is possible to have a technology controlled directly by the wearer only and mounted on the body of the user, it is possible to provide digital objects which are resistant to the people around the wearer and which are under the control of the user only. They impose their existence on everybody but the user.
Thanks to these devices, the difference between digital objects provided by augmented reality and other digital objects like unborn children and galaxies gets thinner. The forced presence of these digital clothes makes them resistant for the onlookers, and so they impose themselves as objects they have to live with. Obviously, they are not like unborn children, but, at the same time, they are something more than mere fantasies because they are part of the everyday world and elements which people have to co-live with.
Augmented reality in itself does not produce mere fantasies. However, the use and the choice of devices can change the way subjects perceive the objects and the way they relate to them. Therefore, in augmented reality, it is possible to produce fantasy digital objects and resistant ones according to the technology we use. In order to produce something which is able to merge the digital and real world, we need to think of how the devices make the digital objects perceivable to the people. We need to choose the ones which are able to provide resistant objects visible by everybody. Personal projectors mounted on the user can provide such experience, and so they can finally generate digital objects which are able to merge the digital and real world.
4. Discussion
There are already examples which move in this direction by providing digital clothes worn by the users thanks to the use of projectors.
We could use the example of
Magic Leap which aims to generate augmented reality without the use of head-mounted devices (
https://www.magicleap.com/, accessed on 22 October 2017). However, we will not refer to it for two main reasons. Firstly, the company still have to produce a working prototype, and so it does not provide us with an actual technology we can use as an example. Secondly, it does not refer directly to clothes.
The performance by the Japanese pop group
Perfume [パフューム] (
http://www.perfume-web.jp/, accessed on 22 October 2017) is a famous example of how projections can become part of the clothes of the performers. Three singers dance while images are projected on their body by projectors mounted on the stage. Images are imposed and resistant to everybody at the concert because they cannot turn the projectors off. However, the projectors are not personal because they are not controlled by the three singers directly, but they are controlled by the staff. Therefore, the digital clothes projected are resistant to the singers too. Moreover, these projectors are not mounted on the singers, and so they allow possible interference with the projected images since it is possible, for example, to block the projection by interposing objects. We can find the same kind of problems in the case of other digital clothes produced by projectors. For example, in the performance
What is real? by the fashion designer Ece Özalp (
https://vimeo.com/143157544, accessed on 22 October 2017), the white clothes of a model are used as a screen for projections. The clothes of the model is a combination of white tissue and the digital image projected on it. Even in this case, the wearer does not control the projections, and the projectors are located far away from the user. Another example which shows even better the problem of devices not controlled by the wearer is the
Holographic Catwalk by Pinar & Viola (
http://pinar-viola.com/commissionsarchive/holographic-catwalk/, accessed on 22 October 2017). A model wears clothes produced through a hologram. In this case, the model has to follow the hologram because the images are pre-recorded. Thus the clothes cannot follow the movement of the wearer since their movement are already predetermined, but it is the model who has to move according to the movements of the clothes.
However, there are other examples which do not involve such restrictions because they use personal projectors mounted on the subject. For example, skin buttons can be projected on the wrist by using projectors placed within an armlet [
46]. Even if they are not clothes, but mere interactive buttons, it is clear how the problems we faced are solved by this design. The buttons are projected on the skin of the wearer. They are visible to everybody, and the wearer has complete control on them. They are resistant to the people around since they cannot interfere or erase them because only the wearer has the power to do it. The augmented reality jewellery produced by
Neclumi is another example of how digital objects become part of the everyday world thanks to projectors. A necklace is projected on the neck of the wearer thanks to a projector mounted on the user (
https://vimeo.com/110207736, accessed on 22 October 2017). These objects have limits depending on the light in the environment. Since the projectors are small, their power is limited, and so they cannot provide a good result in a bright environment. For example, the augmented jewellery cannot be used during daytime in outdoor spaces because the direct sun light makes the digital object invisible.
Personal projectors mounted on the wearer can solve the problems we had with the resistance opposed by the digital objects, but it is not enough to produce something which is like other usual objects. Part of the problem is related to the performance of the projectors. Digital objects like Neclumi are visible only in dark places because the projectors used are not strong enough to produce visible objects under the direct sunlight.
The ideal digital clothes should be something close to body painting which covers the body of the user without making other people notice the nudity of the person in any circumstance. A good example is the “experiment” made by body painting artists in New York in 2014. A model walked around the city without wearing any clothes but having her body completely covered by painted clothes (
http://www.nydailynews.com/news/national/model-strolls-manhattan-wearing-paint-pants-article-1.2011042, accessed on 12 October 2017). Even if the clothes used by the model were not “real” clothes, but painted ones, they were able to merge completely with the everyday world, and the model was able to walk around the street without being “naked” like the Emperor. This imposition and forced co-living between painted objects and common ones during the entire day is the effect to achieve in augmented reality.
At the beginning of this work, we did not know what these digital objects were. We showed how they are merely fantasies for the users and not part of the real world as other common objects. Moreover, we showed they are perceived as fantasies because of the device used. They do not provide the digital objects with the elements required to be perceived differently. Thus, it is possible to improve them by choosing different devices which force subjects to co-live with them.
Even if the digital objects we showed have some limits, they are examples of digital objects generated with personal projectors mounted on the users. They impose themselves on people around them, and they are controlled by the wearer. These forced presences impose themselves on the users to co-live with them, and so they are something more than mere fantasies even if maybe they are still far away from being objects like other “common” objects such as unborn children or galaxies.
5. Conclusions
This work aimed to solve the problem augmented reality is facing related to the reality of the objects produced. We divided the text into three main sections.
In the first section, we showed that postphenomenology and mediation theory suggest our world is deeply related to the technologies used. Digital technologies are already shaping what is around us, and augmented reality could play an important role in producing digital objects which are part of our everyday world. However, we also showed that augmented reality risks producing fictional digital objects instead of objects which are parts of our world.
In the second section, we analysed the story the Emperor’s new clothes, and we introduced augmented reality in it in order to highlight the limits of this technology which provides merely fantasies. We showed how the use of personal projectors mounted on the wearer helps in solving the problem. Since the images are imposed on the people around the wearer, they force the people to co-live with them.
As we showed, the perceptual presence of these objects can initiate the co-living which is necessary for turning the digital objects into part of our everyday world. Obviously, the perception of these objects is not the only way to achieve these results, but it is one way to realise it.
These digital clothes are real, not like unborn children. However, they are not fantasies either since they force themselves on the people around them just like everyday objects do.
Augmented reality objects are real enough to be worn.