Next Article in Journal
On a First Evaluation of ROMOT—A RObotic 3D MOvie Theatre—For Driving Safety Awareness
Next Article in Special Issue
On the Use of ROMOT—A RObotized 3D-MOvie Theatre—To Enhance Romantic Movie Scenes
Previous Article in Journal / Special Issue
The Cyborg Mermaid (or: How Technè Can Help the Misfits Fit In)
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Robotic Arts: Current Practices, Potentials, and Implications

Mind Music Machine Lab, Michigan Technological University, Houghton, MI 49931, USA
Multimodal Technol. Interact. 2017, 1(2), 5; https://doi.org/10.3390/mti1020005
Submission received: 20 January 2017 / Revised: 4 March 2017 / Accepted: 22 March 2017 / Published: 24 March 2017
(This article belongs to the Special Issue Love and Sex with Robots)

Abstract

:
Given that the origin of the “robot” comes from efforts to create a worker to help people, there has been relatively little research on making a robot for non-work purposes. However, some researchers have explored robotic arts since Leonardo da Vinci. Many questions can be posed regarding the potentials of robotic arts: (1) Is there anything we can call machine-creativity? (2) Can robots improvise artworks on the fly? and (3) Can art robots pass the Turing test? To ponder these questions and see the current status quo of robotic arts, the present paper surveys the contributions of robotics in diverse forms of arts, including drawing, theater, music, and dance. The present paper describes selective projects in each genre, core procedure, possibilities and limitations within the aesthetic computing framework. Then, the paper discusses implications of these robotic arts in terms of both robot research and art research, followed by conclusions including answers to the questions posed at the outset.

1. Introduction

The first digital, programmable robot, “Unimate,” was invented in 1954 and sold to General Motors in 1961. From the start, robots (which came from “robota” in Czech, meaning “work”) have been designed for work in manufacturing industry. Robot research has traditionally focused on navigation (locomotion) and manipulation, and further extended to include sensing and executing for performance [1]. It is not surprising that robot research has not focused on making a robot for non-work purposes. Therefore, the arts may not have been treated seriously in robotics and artificial intelligence (AI) because those disciplines have mainly focused on a problem solving paradigm. In problem solving, there may be an optimal solution. However, it might not be appropriate to mention optimal drawing or optimal composing in arts [2]. Creativity in arts may be a search for goals, rather than attaining a fixed goal [3]. Nevertheless, some researchers and artists have envisioned the integration of robots and arts. It can even be traced back to Leonardo da Vinci’s “Automaton” in the 16th Century.

1.1. Motivation and Scope

In robotics and AI research, there have been consistent attempts to create human-like robots, but those efforts have been limited to mainly mimicking human cognition and physicality. With affective computing and social robots becoming pervasive, research has been changing to including human emotions in robots, and thus, robotic arts are now more widespread than ever before. This trend is reflected in recently published books (e.g., Musical Robots and Interactive Multimodal Systems [4]), conference proceedings (e.g., International Conferences on Computational Creativity (http://computationalcreativity.net) and International Congress on Love and Sex with Robots (http://loveandsexwithrobots.org/)), workshops (e.g., Humanoid Robots and Creativity (http://cogsci.eecs.qmul.ac.uk/humanoids/) and Cognitive Robotics and Computational Creativity (http://icrc.asia.edu.tw/workshop-crcc2017/)), and journals (e.g., special issue on Robotics and Creativity of Robotics and Autonomous Systems e.g., [5]). Not only is interest increasing in academia, but also is it gaining attention in popular culture. For example, the IBM Watson drew an abstract piece with six artists; in Ars Electronica 2016, an artist suspended from a robot arm drew his work by moving around a large canvas on the wall; and in Rio 2016 Paralympics, a dancer performed with a robot partner.
The present paper traces some of the efforts to integrate robots and arts, connects it with the aesthetic computing framework, and identifies implications of robotic arts from both robotic and artistic research perspectives. There are uncountable art genres and practices, and new names of art forms are appearing even now. Therefore, reviewing all existing robotic arts is beyond the scope of the present paper. Even though there is high potential of robots or creative systems to create totally new, unforeseen forms of arts [2], the present paper will neither try to discover a new genre nor try to label it. It is beyond the scope of the paper. Before we get there, it would better if we recapitulate the current state first. Thus, this paper attempts to outline the landscape of robotic arts in existing genre of arts by exploring selective works to learn how human-robot interactions are heading to new potentials and implications for both sides. Furthermore, the focus will be only on the parts of these pieces, which are mostly relevant to the discussion of the present paper: machine-creativity, autonomy (improvisation), and intentionality (novel concept making) as defined below.

1.2. Working Definitions and Questions

Terms related to creativity or computational creativity are defined in various ways in different domains and these concepts are also not clearly distinguished from one another. However, for a clear presentation of ideas, here are specific operational definitions of each term in the context of the current paper.
In AI, Boden early attempted to make a framework on creative behaviors [6]. She suggested hierarchical concepts of creativity (e.g., historical and psychological; explorative and transformational creativity). However, it is not optimized for formalization and there has been some argument about her conceptualization (e.g., her transformational creativity may be exploratory creativity at the metal-level [7]). More simply, psychologists define creativity as the ability to produce outcome that is new and appropriate [8]. Thus, machine (or computational) creativity can also be defined and evaluated by its product, if it enables us to understand human creativity and to produce programs for creative people to use. Specifically, it also needs to serve as a creative collaborator rather than a mere tool [9]. Colton and Wiggins proposed a similar working definition in their later work [2]: “the philosophy, science and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviors that unbiased observers would deem to be creative.” When they emphasize responsibilities, it resonates with a “collaborator”, not merely a tool and thus, when unbiased people assess the outcome, they can attribute the source of originality to machines.
Autonomy is the characteristics of an agent that can experience self-direction and personal endorsement in the initiation and regulation of one’s behavior [10]. Autonomy of robotic arts can refer to creating art products with self-direction and initiation beyond the human input or predefined/predicted outcomes. For example, improvisation will represent a manifestation of autonomy of art robots.
Intentionality is one of the most important concepts in cognitive science and AI [11,12]. It often refers to meaning or consciousness [13]. In the current paper, it is related to novel concept/meaning making of the artwork. There is an argument that there are two aspects of creativity: syntactic and semantic creativity [14]. Even though some strong AI scholars believe that semantics can emerge from syntax, in the context of this paper semantics is used closer to intentionality.
Many questions can be posed regarding the status of robotic arts: (1) Is there anything we can call machine-creativity?; (2) Can robots improvise artworks going beyond drawing observed objects or playing predetermined orders? (autonomy); and (3) Can art robots pass the Turing test? (intentionality) Note that, here, the Turing test does not refer to merely asking people whether the outcome seems to be created by a human artist or robot artist. Audience or an evaluator needs to have a conversation with a partner (either human or robot) about the work produced. The dialogue can include a title of the work, motivation behind, and the process of the work [2,15]. All these discussions have long tradition in cognitive science and AI history. The present paper contributes to revisiting these constructs using examples of robotic arts. At the same time, this investigation can shed light on the development of expressivity of robotic arts. This multidisciplinary approach will not only guide robotic artists, but also inspire other roboticists and artists who seek a breakthrough of their research and practice. To see the current landscape of robotic arts and answer these questions, the present paper surveys the robotic artworks in diverse forms of arts, including robot drawing, theater playing, music playing, and dancing.

1.3. Framework: Aesthetic Computing

Integration between art and technology is not a new idea. There have been such attempts in art history (e.g., Futurism or Fluxus movement—both are avant-garde art movements of the 20th century, either celebrating advanced technology and demonstrating the beauty of the machine (Futurism) or combining and fusing all media and all artistic disciplines (Fluxus)). Human-Computer Interaction (HCI) researchers have also envisioned artistic aspects will be the core components of HCI e.g., [16]. However, a systematic approach to this topic has relatively recently started in the name of Aesthetic Computing, which is defined as “the application of art theory and practice to computing” [17]. Given that robotics inherently has a “computing” component in it, this notion can also be applied to robotics, as “aesthetic robotics”. The original aesthetic computing highlighted more the effects of art and aesthetics on computing, but the discussion has been expanded to view its reciprocal interactions between art and technology [8]. According to that research, computing and technology influence art and aesthetics by expanding our perceptual dimensions, whereas art and aesthetics also influence computing and technology by expanding representations of computing process and structure. To outline potentials and implications of these mutual effects in robotic arts, aesthetic computing is expected to serve as a good framework for the present paper.

2. Drawing

Given that when people hear about art, many of them think of “fine art” first, it would be natural to begin our discussion with drawings. Children are likely to draw similar pieces across cultures and nations. Perhaps, it is one of the most basic communication tools for people even prior to obtaining formal language. Artists and researchers have made drawing robots, e.g., [18,19,20] or used robots for drawing e.g., [21]. For drawing or painting, at least, two physical components are involved: observation (sensing in robotics) and drawing/painting (manipulation in robotics). These are iterative processes and so, artists look at the object and draw some part of it and repeat the cycle over again. Since research on computer vision and robot arms have been intense, these two areas have been advanced enough to mimic human capabilities. For example, a famous drawing robot, AARON’s visual representation system has evolved, including 2D, 2½D, and 3D knowledge base [19] just as in the human vision system modeling.
The definition of fine art includes not only aesthetic components but also something that expresses important ideas or feelings with the intellectual purposes (Merriam Webster dictionary). Here, aesthetics can be a “style” and the intellectual part can be “meaning”. In Colton’s [15] terms, these could correspond each to “imagination (originality)” and “appreciation (value)”. Of course, the two elements are often intermingled with each other and hard to separate one another. Aesthetics can be judged for its beauty and the intellectual part can be judged for its meaningfulness. Researchers have pondered how their robots can have their own style in drawings. To illustrate, artists have trained their robots just as they do human apprentices. To make creative artworks, students or apprentices need deliberate practice and systematic training with good teachers [22]. To make this happen in robotic arts, researchers and artists have made a feedback system to make the process a closed loop ([5,18,19,20], e.g., using a neural network based learning model). As a result, the drawings that robots have produced seem to provide comparable emotional, aesthetic, and artistic effects on the observer.
Sometimes, an observation stage can be omitted if the artwork only includes abstraction. Can robots make Abstract Expressionism or Informalism (pictorial movements that includes all the abstract and gestural tendencies in America (Abstract Expressionism) or Europe (Informalism))? For example, ArtSBot swarm [21], composed of small turtle type robots which have two pens and a pair of RGB sensors, was designed to make “action painting” just as Jackson Pollock did. In the Robot Action Painter (RAP) project, there are two modes to achieve unique machine creativity. First, RAP makes the initial random color spots, which serve as a seed for the second, reactive mode where the robot traces the presence, shape and intensity of color and plans the emergent composition. The results of ArtSBot look pretty much like Pollock’s artworks. One interesting, relevant note is that researchers and artists acknowledge that randomness (or even mistakes) provides an opportunity of “unique experience” or even aura of robotic arts as much as in human artworks [18]. Thus, an interesting intellectual experiment could be conducted to test whether the same robot is able to create the same masterpiece if the object (or the topic) is the same under the same environmental factors.
In contemporary art since Duchamp’s “Fountain”, a concept has become certainly much more important than traditional beauty in art appreciation. It is even said, artists can be judged by questions, such as “how much did they question the nature of art?” or “what did they add to the conception of art?” (Kosuth, 1969 cited in [23]). It is questionable if these drawing robots are capable of concept (i.e., meaning) making in this sense given that current robots do not have high level domain knowledge of the structures of the objects or the capability of learning expertise based on experience as people would from examples above [18,19,21]. For example, a drawing robot, Paul can detect/follow human face and draw it (syntax), but it does not know/process the meaning of the human face (semantics). In the Painting Fool case [15], however, researchers have tried to add semantic part. In their project, they incorporated about fifty artistic styles to its knowledge database and tagged each with an emotion keyword. Thus, the Painting Fool could draw some images with a particular emotion. However, the Painting Fool requires a trigger for a specific emotion expressed in the image to select the same emotion from its knowledge base. To this end, they combined a computer vision system to detect the emotion of the person in their video clip pools.
Therefore, here comes a question on the Turing test: Can the current art robot pass the Turing test? If the robot produces an artwork as comparable as a human artist’s work, can we call the robot an independent artist? More importantly, can the art robot make a conversation about the meaning of its work?

3. Theater

Robot theater is another active research platform e.g., [24,25,26]. These projects aimed to facilitate artistic performance and used this performance as a useful stepping stone for robotics research and further, as a platform for social futurology research. For example, Ishiguro’s Robot Actors Project tried to naturally depict in-house family situations where people ponder socio-cultural problems in the near future, by playing theater with robots [24]. Just as in other robotic art projects, they had an artist (drama writer) in their team. This type of experiment can vary, by having robots play different roles, such as a “robot”, “human”, or “animal”. Based on various roles robots play and the storylines, theater production can also serve as a parable about human existence. In this type of project, robots are able to dynamically interact with their environments and other people (e.g., shaking hands, playing soccer) and thus, people can have an impression that robots might have some free will and autonomy. In fact, Ogawa, Taura, and Ishiguro [26] showed that people were impressed much more by android (Geminoid) performance than by human performance. Their audience preferred the android over the human as a poetry-reciting agent in the collaborative theatrical project.
The level of theater plays could be divided into two: (1) having robots perform a prescripted, complex play and (2) having robots improvise performance based on the structure of scenes and characters. Just as in drawing, humans can instruct robots how to behave. Taking the first approach, Ishiguro’s group tried to improve robots’ movements and behaviors. In the theater play with a robot (robovie, a humanoid robot), Hirata, a professional stage director, provided precise instructions to the robovie so that its movements and gestures became more human-like and natural [27]. In the second approach, researchers at Carnegie Mellon University have investigated autonomy further and used comedy improvisation to create believable agents [25]. They tried to make robots that could improvise scripts on the fly instead of using a predetermined scenario. The robot can be designed to have different personality and accordingly, each character can have different types of behavior patterns. However, researchers thought that dramas are not driven by characters with indefinable emotions, but rather driven by the character’s objectives, goals, actions, and tactics. Therefore, they created scenarios which can be easily formalized (e.g., there are a hero, goal states, obstacles, and given circumstances, etc.). To implement this scenario by designing turn taking and conversation of the robots, timing and rhythm are important considerations just as in music playing and dancing. Even though their attempt was in a nascent stage, it provided a high potential for improvisation of the robot theater. Note that in their iterative robot theater productions, slight differences in the physical state resulted in significantly different performances, even when it was run with the same characters. Complicated chains ultimately made distinctive artistic outcomes, which can answer one of the questions at the outset. In other words, robots acted in different ways to achieve the same goal (overcoming obstacles and getting out the door) and so, it looks like the robot in that project has basic level of autonomy. However, the robot still acted to obtain the predefined goal. Therefore, it is questionable if we can say that the robot has its own intentionality.
Another approach to the robot theater play is to use it for STEAM (Science, Technology, Engineering, Arts and Design, and Mathematics) education (e.g., http://stemtosteam.org). Researchers used multiple robots with various types and different capabilities in the afterschool program of an elementary school (Figure 1) [28]. In their robot theater afterschool project, researchers tried to provide “computational thinking” experience to young students. In addition to traditional storyboarding procedures used both in script development and interface design, they devised tangible elements for students to iteratively refine their storytelling ideas. For example, students created their own robot models using sculpting material. They also attempted to have students create and implement their ideas by an event-based approach (e.g., touching the robot’s back can generate the next action) rather than the paper and pencil-based text diagram. While the actual coding was performed separately by experienced programmers (graduate and undergraduate students), rehearsals with both the programmers and students allowed for interactive debugging of readily observable problems. This embodiment of computing process allowed 5–8 year old children to experience the concept of computing and create their own unique story.
Depending on the context of the theater play or drama (e.g., the setting of the story can be the prehistoric with a dinosaur robot, the present with an animal-like robot, or the future with an android) and the role (human, animal, or robot) of the robots, audience’s expectancy would differ and the required implementation level (or naturalness) might be differentiated. For whatever situations, however, robots should be able to behave in ways that make sense within a narrative. Rather than merely behaving as they are supposed to, robots’ face, motion, timing, and intonation of the voice should express appropriate emotional aspects. In this line, Knight [29] has identified critical components to better design social robots from robot theater: relatable gestures, affect expressions, movement metaphors, outward emotion communication, social intelligence, multi-agent interactions, audience feedback, and machine humor. Just as drawing is not merely copying the object or taking a photo, robot acting should possess its own nuance and interpretation. All of these are the core subjects of social robotics research.

4. Music

Just as in other types of arts, sensing and manipulating processes are major functions in robotic musicianship. Based on these activities, robots analyze and generate music. While fine art is mainly using the visual modality in sensing, musical robots are also using the auditory modality, called machine listening [30]. Robots can easily process analog auditory signals. In addition, symbolic protocols can be used (e.g., MIDI: Musical Instrument Digital Interface or OSC: Open Sound Control) for efficient signal processing, computation, and communication. Musical robots can also use machine vision. They can detect human musicians’ presence, gesture, or position of the instrument and accordingly notice and respond to the musical context.
Robotic musicianship consists of two primary research domains, musical mechatronics and machine musicianship. Musical mechatronics is the study and construction of physical systems that generate sound through a mechanical means [31]. Machine musicianship focuses on developing algorithms and cognitive models about music perception, composition, performance, and theory [30]. Just as in robot theater productions, the robot should be able to play expressive music for itself and interact with human musicians.
Some robot musicians play traditional instruments (e.g., Cog [32] or Shimon [33]), whereas other robots play modified instruments or novel experimental instruments (Brainstem Cello or ActionclarInet) (http://zownts.com). Even in the case where they play traditional instruments, the playing methods for different instruments (e.g., percussion, wind, string, etc.) vary. For example, for percussion, the coordination among the wrist, elbow, and fingers for different types of strokes is important. For wind instruments producing the energy fluctuations to make the instrument resonate at desired frequencies and volumes is critical [34]. In any case, many musical robots try to mimic human instrumentalists. For example, the Waseda Flutist Robot No. 4 Refined IV consists of a mouth mechanism (lips), air supply mechanism (lung), and fingers [35]. The Robotic Drumming Prosthesis also has a generalized physical model for drum stroke generation to achieve human-like musical expressivity [36].
In addition to playing a predefined musical score, improvisation can also be performed by the current robot musicians. It would be interesting to compare improvisation in theater productions and improvisation in music playing. Just as robots memorize their action and speech, monitor others’ playing, and then take turns in theater play, musical robots analyze and interpret others’ playing and then, improvise their music in real-time. This improvisation requires all components of linguistics: syntax, semantics, and pragmatics. Spoken language should follow rules of grammar (syntax), deliver the precise meaning in a given turn (semantics), and go well with a given context (pragmatics). This linguistic rules are exactly applied to musical playing. Regardless of whether improvisation is based on artistic inspiration or complex algorithms, musical improvisation should follow the musical grammar and deliver the intended musical meaning with a context-appropriate progress. For example, in jazz music there are two ways to improvise. On one hand, musicians can have a preset chord progression. Then, within the chord scales musicians can vary their playing. On the other hand, they can improvise their music by modifying the main melody in various ways. The melody contour can be expanded, cut, shrunken, merged, etc. In robotic musicianship, generative algorithms have opened a new possibility of this type of improvisation and accompaniment. For example, Hidden Markov Models are often used for chord progression in Western tonal music [37]. Robot improvisation can also be based on the real-time data. Zhang et al. [38] designed and developed a robotic sonification platform (Figure 2). They used a server-client architecture using multiple robots and computer vision technology as clients. Depending on the interaction data between children and robots, the music server can generate/improvise sounds and music.

5. Dance

Dance performance has to properly represent key poses with appropriate timings. It is similar to music in that it generates sounds with appropriate timings, but in dance, gesture itself should express meanings and feelings. More importantly, dance includes locomotion, which is usually not included in music performance. Dancing robots can begin with mimicking human movements and translating those into robotic movements. However, it is fairly hard to directly translate human movements into robot movements because robots’ weight, balance, and the structure of arms and legs are different from humans’. To solve this issue, robot dance researchers have developed various strategies. For instance, Ikeuchi et al. [39] used two steps for their humanoid robot dancer. First, they made a predesigned task model to imitate (recognize and parse) human actions. Then, they adapt these actions to the robot’s morphology and dynamics.
Dance basically maps sounds and emotions to movements. In this process, the most critical part is intermodality mapping. This process can be divided into two phases. First, the robot observes predefined events together with associated sounds in the learning phase. The robot memorizes these sounds along with the motions of the sound source. Then, when the robot detects the same sounds, it associates this with learned dance movement and expresses it. In this process, machine learning techniques can be applied (e.g., the recurrent-neural-network model [39]). The key to success would be “expression” again. To this end, studying and implementing a specific dance genre (e.g., rhumba, waltz, or tango) may be an appropriate approach. Dance does not require verbal communication. It features regular, structured bodily rhythms externally reinforced by sound. Therefore, meaning in dance might be unclear, just as in music. However, dancers have their intention and allocate meanings to their gestures and the entire work. Just as in drawing, researchers developed a robot system that can learn dancing styles from human expert dancers. By employing an interactive version of Simple Genetic Algorithm (SGA) [40], Augello et al. [41] developed a learning and evolving dancing robot. One of the interesting parts of their robot dancer is that it creates choreographies according to its artistic motivation, influenced by its urges. Their system focused on cognitive urges, including competence and certainty, while more research is required to include other types of motivation, such as physiological and social urges. Nevertheless, this dancing robot is capable to learn from human dancers and create natural and original choreography, which ensures autonomy.
Dance involves visual, auditory, and tactile modalities and it is familiar and fun to most people. Because of these characteristics, researchers envisioned that dance can be used for pedagogical and therapeutic purposes, with a range of disorders from autism spectrum disorders (ASD) to schizophrenia. Indeed, the recent research attempts to use dancing robots for children with ASD. An ongoing research study (Figure 3) attempts to use a “dance freeze” game to facilitate social interactions of autistic children [42].

6. Discussion

To glean some hint as to the future direction of robotic arts, it is first necessary to look back on what has been done. To this end, various forms of robotic arts were surveyed for their approaches, methods, issues, and practices. Current art robots seem to have the components of creativity. Audiences of Paul (drawing robot) or Shimon (music playing robot) evaluate their works as equivalent to human artists’ works. Given that evaluation of creativity is conducted by observing how creative products are [8], we can admit machine (robot) creativity here. ArtSBot, other theater play robots, or musical robots even produce novel artworks, which go beyond the predetermined information by human artists or engineers. Thus, we can also see a certain type of autonomy. The remaining question is whether a robot can have “intentionality” (or consciousness) for making a new concept behind the artwork per se because it is the most critical part of contemporary arts. For instance, Pollock fulfills this intention (e.g., questioning the nature of art or breaking social/traditional art norms) by splashing ink onto the canvas with the purpose of representing nothing but the action itself. Even though ArtSBot can create a similar abstract work to Pollock’s, it is questionable if it can make an original “artist’s statement” about its own artwork. Art robots can obtain a “predetermined” goal (or goal state, e.g., drawing, coloring, music playing, etc.), but can they have clear motivation derived from personal characteristics or socio-cultural background?
This question is deeply related to growing research on machine consciousness [43,44]. Chella and Manzotti identified four critical features for machine consciousness, which can also be applied to robotic arts (e.g., jazz improvisation robot) [14]: embodiment, situatedness, emotions, and experience. Embodiment is a salient characteristic of robots and so, it partially fulfills situatedness [3]. However, given that both constructs are not only having body and location, but including cognitive functions (e.g., action-perception loop) and interactions carried out by physicality and being located, many issues remain unanswered. Thanks to rapid growth in the affective computing area, machines are getting more and more emotionally intelligent and capable (see recent handbooks [45,46]). Finally, given the speed of advances of machine learning techniques (and specifically recent application of deep learning, e.g., [47]), musical robots might not only learn “what is played” (classical symbolic approach) but also experience “how is played” (expressive performance elements). They could also directly or indirectly store life experiences (or even multiple human lives) and the zeitgeist (the spirit of the times) at some point.
So far, many art robots seem to have teachers (human artists) and appropriate feedback to develop their own style and aesthetics. In other words, they have preliminary learning capabilities from human artists. Therefore, we can pose that the source of creativity lies in between human-robot interaction; not just humans (programmers or artists) or not solely robots. This is good because this means that art robots do not replace human artists and this does not mean the end of art, but means the birth of a new collaborative artwork.

Implications for Robotics Research, Art Research, and Beyond

This attempt to combine robotics and arts provides new implications for both robotics research and art research. Developing robots going beyond making utilitarian slave robots can also open the new perspective to traditional robotics research. Topics critical to artistic functions, such as timing, anticipation, expression, mechanical dexterity, and social interaction, are also pivotal to robotics research. Robot researchers have begun to implement these elements more systematically, while they try to understand human drawing, painting, and playing, and model those. Therefore, through practices of this robotic arts, the robotics discipline will be able to leap one step further. Going back to notions of aesthetic computing, this phenomenon certainly expands representations and computing structure in robotics research.
The ultimate goal of robotic arts is not replacing human artists or human creativity. Researchers agree that introducing robotic arts can promote human creativity and enrich human artistic culture [18,21,34]. TV has affected a radio culture, but never replaced it. A different medium can bring new potentials and opportunities rather than hindering them. Enhanced precision, unlimited physical freedom and cognitive capabilities, which human performers cannot process in a timely manner due to innate limitations, will certainly extend our perceptual dimension and open new potential. The presence of robots in art performance also facilitates the shift of art paradigm from representative arts into performative arts [48]. In addition to the robots’ superior real-time capability, active robotic arts provide certainly a novel scene that people are likely to watch. Therefore, robots can even accelerate this change. Moreover, the audiences not only observe the robots’ performance, but they can also attend the creative process. Since the 1960s, integrating the audience as a key part of the artwork has been a critical milestone [49]. It is well witnessed in cases where artists have collaborated with other technologists or technologies [50]. Robotic arts researchers have also tried to involve audience in their works. For example, the Senster (drawing robot) [23] utilized the audiences’ response data to process its expressions in space. Breazeal et al. [51] also demonstrated interactive theater. In their work, audiences could engage in a loosely constrained storyline through sight, sound, scent, and touch.
Overall, robotic arts require collaborative efforts of a number of research communities, such as engineering, machine learning, computation, each genre of arts, sociology, psychology, etc. Therefore, robotic arts promote and foster transdisciplinary research. Interaction with robots is fun and engaging. Research studies have shown that children (and even adults) tend to pay more attention to robots than other products or humans, regardless of whether it is because of novelty, embodiment, or any other factors. As seen in robot theater and dancing projects, robotic arts can be used in STEM or STEAM education. Moreover, robotic music/sonification as well as robot theater and dancing can facilitate mental health, clinical sessions, or developmental disorders [39,42].

7. Conclusions

We now have a basis on which to answer the questions presented at the outset. (1) Is there anything we can call machine-creativity? (creativity) Yes. People appreciate the artworks created by robots as equivalent to the artworks by human artists. However, it would be more accurate to attribute the source of creativity to the interaction between humans and robots, rather than putting it solely on the robot side given that humans’ input plays a big role in both style and meaning making. (2) Can robots improvise artworks going beyond drawing observed objects or playing predetermined orders? (autonomy) Yes. Robots in various arts can improvise their artworks in real-time (e.g., abstractionist drawing, jazz playing, or even unscripted theater playing). (3) Can art robots pass the Turing test? (intentionality) The current state of robotic arts might not. Style or expression has been a necessary component in every genre of arts, but the contemporary arts focus more on “concept” rather than implementation. The robot arm can draw the pipe, but it would be hard for the same robot to annotate “this is not a pipe” under the pipe. (“The Treachery of Images” is a painting by the Belgian surrealist painter René Magritte. The picture shows a pipe. Below it, Magritte painted, “Ceci n’est pas une pipe”, French for “This is not a pipe.”) The robot arm can physically move a porcelain urinal to the art gallery (as in Duchamp’s “Fountain”), but it would be hard for the robot to come up with the idea to do so in the right timing. If any art robot attempts to do the exactly same performance now, it would not be appreciated as a radical masterpiece any more. This naming/labeling/concept making process may stem from an artist’s personal history, education, characteristics, personality, school of art (or community), interpretation on zeitgeist, etc. Given the rapid advances of artificial intelligence, robots may get to this stage in the near future. However, even in that era, the collaboration between a human artist and an art robot would be better than just an art robot alone.

Acknowledgments

The author would like to thank his graduate students, Jaclyn Barnes, Jason Sterkenburg, and Steven Landry for their proofreading and useful comments which have helped strengthen the article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Parsons, H.M.; Kearsley, G.P. Robotics and human factors: Current status and future prospects. Hum. Factors J. Hum. Factors Ergon. Soc. 1982, 24, 535–552. [Google Scholar]
  2. Colton, S.; Wiggins, G.A. Computational creativity: The final frontier? In Proceedings of the 20th European Conference on Artificial Intelligence, Montpellier, France, 27–31 August 2012; pp. 21–26. [Google Scholar]
  3. McGregor, S.; McGinity, M.M.; Griffiths, S. How many robots does it take? Creativity, robots and multi-agent systems. In Proceedings of the AISB15’s 2nd International Symposium on Computational Creativity (CC2015), Park City, UT, USA, 29 June–2 July 2015. [Google Scholar]
  4. Solis, J.; Ng, K. Musical Robots and Interactive Multimodal Systems; Springer: London, UK, 2011. [Google Scholar]
  5. Sasaki, K.; Noda, K.; Ogata, T. Visual motor integration of robot’s drawing behavior using recurrent neural network. Robot. Auton. Syst. 2016, 86, 184–195. [Google Scholar] [CrossRef]
  6. Boden, M. The Creative Mind; Weidenfeld and Nicolson: London, UK, 1990. [Google Scholar]
  7. Wiggins, G.A. A preliminary framework for description, analysis and comparison of creative systems. Knowl. Based Syst. 2006, 19, 449–458. [Google Scholar] [CrossRef]
  8. Sternberg, R.J. The Nature of Creativity: Contemporary Psychological Perspectives; Cambridge University Press: New York, NY, USA, 1988. [Google Scholar]
  9. Colton, S.; de Mántaras, R.L.; Stock, O. Computational creativity: Coming of age. AI Mag. 2009, 30, 11–14. [Google Scholar]
  10. Reeve, J. Understanding Motivation and Emotion; Wiley: Hoboken, NJ, USA, 2014. [Google Scholar]
  11. Posner, M.I. Foundations of Cognitive Science; MIT Press: Cambridge, MA, USA, 1989. [Google Scholar]
  12. Thagard, P. Mind: Introduction to Cognitive Science; MIT Press: Cambridge, MA, USA, 1996. [Google Scholar]
  13. Boden, M.A. Odd man out: Rely to reviewers. Artif. Intell. 2008, 172, 1944–1964. [Google Scholar] [CrossRef]
  14. Chella, A.; Manzotti, R. Jazz and machine consciousness: Towards a new Turing test. In Proceedings of the Revisiting Turing and His Test: Comprehensiveness, Qualia, and the Real World (AISB/IACAP) Symposium, Birmingham, UK, 2–6 July 2012; pp. 49–53. [Google Scholar]
  15. Colton, S. Creativity versus the perception of creativity in computational systems. In Proceedings of the AAAI Spring Symposium: Creative Intelligent Systems, Stanford, CA, USA, 26–28 March 2008. [Google Scholar]
  16. Packer, R.; Jordan, K. Multimedia: From Wagner to Virtual Reality; WW Norton & Company: New York, NY, USA, 2002. [Google Scholar]
  17. Shem-Shaul, N.B.; Bertelsen, O.W.; Bolter, J.; Bruns, W.; Bureaud, A.; Diehl, S.; Dombois, F.; Ebert, A.; Edmonds, E.; Entacher, K. Aesthetic computing manifesto. In Proceedings of the Dagstuhl Workshop on Aesthetic Computing, Dagstuhl, Germany, 5–19 July 2003. [Google Scholar]
  18. Tresset, P.; Leymarie, F.F. Portrait drawing by Paul the robot. Comput. Graph. 2013, 37, 348–363. [Google Scholar] [CrossRef]
  19. Cohen, H. The further exploits of AARON, painter. Stanf. Hum. Rev. 1995, 4, 141–158. [Google Scholar]
  20. Deussen, O.; Lindemeier, T.; Pirk, S.; Tautzenberger, M. Feedback-guided stroke placement for a painting machine. In Proceedings of the Eighth Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging, Annecy, France, 4–6 June 2012; pp. 25–33. [Google Scholar]
  21. Moura, L. A new kind of art: The robotic action painter. In Proceedings of the X Generative Art Conference, Politecnico di Milano University, Milano, Italy, 12–14 December 2007. [Google Scholar]
  22. Howe, M.J.; Davidson, J.W.; Sloboda, J.A. Innate talents: Reality or myth? Behav. Brain Sci. 1998, 21, 399–407. [Google Scholar] [CrossRef] [PubMed]
  23. Kac, E. Telepresence Art. Available online: http://www.ekac.org/telepresence.art_94.html (accessed on 20 January 2017).
  24. Paré, Z. Robot actors: Theatre for robot engineering. In Theatres du Posthumain; Ehwa Institute for Humanities Science & LABEX Arts-H2H Laboratory, Ed.; Arcarnet: Seoul, Korea, 2015; pp. 143–162. [Google Scholar]
  25. Bruce, A.; Knight, J.; Listopad, S.; Magerko, B.; Nourbakhsh, I.R. Robot improv: Using drama to create believable agents. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’00), San Francisco, CA, USA, 24–28 April 2000; pp. 4002–4008. [Google Scholar]
  26. Ogawa, K.; Taura, K.; Ishiguro, H. Possibilities of androids as poetry-reciting agent. In Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication, Paris, France, 9–13 September 2012; pp. 565–570. [Google Scholar]
  27. Nishiguchi, S.; Ogawa, K.; Yoshikawa, Y.; Chikaraishi, T.; Hirata, O.; Ishiguro, H. Theatrical approach: Designing human-like behaviour in humanoid robots. Robot. Auton. Syst. 2017, 89, 158–166. [Google Scholar] [CrossRef]
  28. Jeon, M.; FakhrHosseini, M.; Barnes, J.; Duford, Z.; Zhang, R.; Ryan, J.; Vasey, E. Making live theatre with multiple robots as actors bringing robots to rural schools to promote STEAM education for underserved students. In Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI’16), Christchurch, New Zealand, 8–10 March 2016; pp. 445–446. [Google Scholar]
  29. Knight, H. Eight lessons learned about non-verbal interactions through robot theater. In Proceedings of the International Conference on Social Robotics, Amsterdam, The Netherlands, 24–25 November 2011; pp. 42–51. [Google Scholar]
  30. Rowe, R. Machine Musicianship; The MIT Press: Cambridge, MA, USA, 2004. [Google Scholar]
  31. Kapur, A. A history of robotic musical instruments. In Proceedings of the International Computer Music Conference, Barcelona, Spain, 5–9 September 2005; pp. 21–28. [Google Scholar]
  32. Williamson, M.M. Robot Arm Control Exploiting Natural Dynamics. Doctoral Dissertation, Massachusetts Institute of Technology, Cambdrige, MA, USA, 1999. [Google Scholar]
  33. Hoffman, G.; Weinberg, G. Shimon: An interactive improvisational robotic marimba player. In Proceedings of the CHI’10 Extended Abstracts on Human Factors in Computing Systems, Atlanta, GA, USA, 10–15 April 2010; pp. 3097–3102. [Google Scholar]
  34. Bretan, M.; Weinberg, G. A survey of robotic musicianship. Commun. ACM 2016, 59, 100–109. [Google Scholar] [CrossRef]
  35. Solis, J.; Sugita, Y.; Petersen, K.; Takanishi, A. Development of an anthropomorphic musical performance robot capable of playing the flute and saxophone: Embedding pressure sensors into the artificial lips as well as the re-designing of the artificial lips and lung mechanisms. Robot. Auton. Syst. 2016, 86, 174–183. [Google Scholar] [CrossRef]
  36. Gopinath, D.; Weinberg, G. A generative physical model approach for enhancing the stroke palette for robotic drummers. Robot. Auton. Syst. 2016, 86, 207–215. [Google Scholar] [CrossRef]
  37. Chen, R.; Shen, W.; Srinivasamurthy, A.; Chordia, P. Chord recognition using duration-explicit Hidden Markov Models. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR2012), Porto, Portugal, 8–12 October 2012; pp. 445–450. [Google Scholar]
  38. Zhang, R.; Barnes, J.; Ryan, J.; Jeon, M.; Park, C.H.; Howard, A. Musical robots for children with ASD using a client-server architecture. In Proceedings of the 22nd International Conference on Auditory Display (ICAD2016), Canberra, Australia, 2–8 July 2016. [Google Scholar]
  39. Aucouturier, J.J.; Ikeuchi, K.; Hirukawa, H.; Nakaoka, S.I.; Shiratori, T.; Kudoh, S.; Kanehiro, F.; Ogata, T.; Kozima, H.; Okuno, H.G. Cheek to chip: Dancing robots and AI’s future. IEEE Intell. Syst. 2008, 23, 74–84. [Google Scholar] [CrossRef]
  40. Vose, M.D. The Simple Genetic Algorithm: Foundations and Theory; MIT Press: Cambridge, MA, USA, 1999. [Google Scholar]
  41. Augello, A.; Infantino, I.; Manfrè, A.; Pilato, G.; Vella, F.; Chella, A. Creation and cognition for humanoid live dancing. Robot. Auton. Syst. 2016, 86, 128–137. [Google Scholar] [CrossRef]
  42. Jeon, M. Aesthetic computing for representation of the computing process and expansion of perceptual dimensions: Cases for art, education, and interfaces. In Proceedings of the 5th International Conference on Arts and Technology (ArtsIT2016), Esbjerg, Denmark, 2–3 May 2016. [Google Scholar]
  43. Chella, A.; Manzotti, R. Artificial Consciousness; Imprint Academic: Exeter, UK, 2007. [Google Scholar]
  44. Chella, A.; Manzotti, R. Machine consciousness: A manifesto for robotics. Int. J. Mach. Conscious. 2009, 1, 33–51. [Google Scholar] [CrossRef]
  45. Calvo, R.A.; D’Mello, S.; Gratch, J.; Kappas, A. The Oxford Handbook of Affective Computing; Oxford University Press: New York, NY, USA, 2014. [Google Scholar]
  46. Jeon, M. Emotions and Affect in Human Factors and Human-Computer Interaction; Academic Press: San Diego, CA, USA, 2017. [Google Scholar]
  47. Zhou, X.; Lerch, A. Chord detection using deep learning. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Malaga, Spain, 26–30 October 2015. [Google Scholar]
  48. Penny, S.G. Someday We Will Have Shakespeare of Computer Games; Humanist: Seoul, Korea, 2009. [Google Scholar]
  49. Carvalho, R. The magical features of immersive audiovisual environments. Interaction 2013, 20, 32–37. [Google Scholar] [CrossRef]
  50. Jeon, M.; Landry, S.; Ryan, J.D.; Walker, J.W. Technologies expand aesthetic dimensions: Visualization and sonification of embodied penwald drawings. In Arts and Technology; Brooks, A.L., Ayiter, E., Yazicigil, O., Eds.; Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Springer International Publishing: Berlin, Germany, 2014; Volume 145, pp. 69–76. [Google Scholar]
  51. Breazeal, C.; Brooks, A.; Gray, J.; Hancher, M.; McBean, J.; Stiehl, D.; Strickon, J. Interactive robot theatre. Commun. ACM 2003, 46, 76–85. [Google Scholar] [CrossRef]
Figure 1. Robot theater program as part of an afterschool program at local elementary school. Children wrote the scenario, made props, and played live theater with robots.
Figure 1. Robot theater program as part of an afterschool program at local elementary school. Children wrote the scenario, made props, and played live theater with robots.
Mti 01 00005 g001
Figure 2. Robotic sonification system architecture. Each robot serves as a client. The interaction data between each robot and child are integrated in the sonification server.
Figure 2. Robotic sonification system architecture. Each robot serves as a client. The interaction data between each robot and child are integrated in the sonification server.
Mti 01 00005 g002
Figure 3. Dance freeze game for kids with autism spectrum disorders. Participants experienced human partner and robot partner for the game.
Figure 3. Dance freeze game for kids with autism spectrum disorders. Participants experienced human partner and robot partner for the game.
Mti 01 00005 g003

Share and Cite

MDPI and ACS Style

Jeon, M. Robotic Arts: Current Practices, Potentials, and Implications. Multimodal Technol. Interact. 2017, 1, 5. https://doi.org/10.3390/mti1020005

AMA Style

Jeon M. Robotic Arts: Current Practices, Potentials, and Implications. Multimodal Technologies and Interaction. 2017; 1(2):5. https://doi.org/10.3390/mti1020005

Chicago/Turabian Style

Jeon, Myounghoon. 2017. "Robotic Arts: Current Practices, Potentials, and Implications" Multimodal Technologies and Interaction 1, no. 2: 5. https://doi.org/10.3390/mti1020005

Article Metrics

Back to TopTop