Single-Sheet Separation from Paper Stack Based on Friction Uncertainty Using High-Speed Robot Hand

Abstract

The successful separation of a single sheet from a stack of paper is considered a paper-handling goal when using a robot hand. Under the condition of uncertain friction coefficients, a stochastic algorithm introducing randomness is formulated, which converges a paper stack to a state of single-sheet separation through the repetition of simple robot operations. This formulation is based on the proposed motion strategy for a robotic hand, which introduces a state of a partially separated paper bundle to temporarily allow the simultaneous separation of multiple sheets and a return operation to return the paper to the original paper bundle. The experimental results indicate that a single sheet can be completely separated from a vertically standing stack of business-card-sized papers by shifting the paper in a high-speed translational movement using two fingers of the robot hand that grasp the paper from both sides.

1. Introduction

The basic functions of paper are writing, wrapping, and wiping. Paper products such as memo paper, envelopes, and tissues are frequently used in our daily lives. Although electronic devices are being widely used in place of notepads in education and business, research has indicated that writing on paper is more effective for memory retention and memory reproduction than taking notes on a smartphone or other electronic devices, and the importance of paper has been confirmed from the perspective of brain science [1]. Paper has the characteristic of being so thin and deformable that it cannot hold its own shape under gravity. This characteristic varies in paper bundles made of the same type of paper, such as books, newspapers, and bank books. Because of the flexibility characteristics of paper, it is generally difficult to predict its deformation through modeling, recognize its state of deformation using sensors, or manipulate it using machines. Therefore, it is challenging to realize real-time automated paper handling at high speeds.

The geometric and mechanical behaviors of paper have been analyzed mathematically. Based on its geometric characteristics, paper is classified as a developable surface, which is a type of ruled surface that has the property of zero Gaussian curvature at all points [2]. Intuitively, a developable surface is a curved surface created by bending, cutting, rounding, and connecting planar surfaces without stretching or shrinking. Therefore, from an engineering viewpoint, paper exhibits strong deformation resistance to tension. During compressive deformation, paper buckles easily when compressed in the longitudinal direction and bounces in the direction perpendicular to the bending direction when bent in the direction of the paper surface [3]. This indicates that paper resists being bent in the backward direction.

These characteristics have led to the development of specialized handling devices such as printers, copiers, automatic teller machines, and page-turning systems. There are two main types of handling equipment: air and friction equipment. A noncontact mechanism using an air blast [4] and a mechanism using negative pressure [5] have been developed. The most common friction method involves pressing a roller against the surface of paper and drawing it out via rotation [6,7,8]. Other examples include a paper-moving system using electrostatic induction [9]; a page-turning system using a polydimethylsiloxane pad [10]; and a document-sorting system using an electroadhesive pad [11]. Although these devices excel at handling paper, they are generally problematic with regard to size and maintenance, and their functionality is limited.

To realize more general-purpose mechanisms for handling paper, multi-fingered robotic hands have been considered. In particular, origami, a complex paper-handling task, has been extensively studied to understand paper manipulation [12,13]. For example, in a study on folding operations, a four-fingered robotic hand is used, in which two fingers with claws are assigned for grasping the paper, and the other fingers are assigned for manipulating it [14]. Additionally, a valley folding method using a motion strategy that emphasizes the accuracy of the crease line has been developed [15]. In addition to origami, the task of picking up a sheet placed on a flat surface has been achieved mainly by holding the sheet down with one finger while rubbing the surface with the other finger to curve the sheet, and then slipping the finger into the gap created and lifting it up. As a research example, this kind of task has been realized by using two robot hands, even in the case of occlusion, owing to the real-time visual recognition of sheets with specific patterns printed on them and the robust computation of the paper shape using a three-dimensional physical model [16]. To realize sheet manipulation, researchers have evaluated the stability of bent sheet grasping based on resistance to large external forces [17]. Another study has demonstrated the motion of bending and turning over a sheet placed on a metal plate using three fingers with hard nails and soft skin [18]. As mentioned previously, most paper handling is executed via paper bending; however, there are cases in which folding or bending the paper is undesirable, such as in the case of business cards, which are somewhat rigid. In addition, few studies on robotic hands have focused on the manipulation of paper bundles, and robotic skills to manipulate paper bundles have not been established.

Therefore, the objective of this study is to pull out sheets of paper one by one from a multi-sheet bundle without bending the paper during handling using a multifingered robot hand, as shown in Figure 1. We propose a method for reliably separating a sheet of paper from a stack of paper by repeating a simple translational movement of a finger parallel to the surface of the paper in the forward and reverse directions. By formulating paper-to-paper friction as a random variable, this method provides a framework for robustly performing real-world paper manipulations involving uncertainty using a robot. In experiments, a high-speed robot hand and high-speed vision are used to achieve the operation of continuously pulling out the uppermost sheet by quickly executing finger movements and detecting paper overlap.

2. Modeling of Paper Bundle

2.1. Problem Establishment

Assume that a paper bundle ${\mathcal{P}}^0=\{{\mathrm{A}}_1,{\mathrm{A}}_2,\cdots ,{\mathrm{A}}_n\}$ consists of n sheets of the same type. Here, ${\mathrm{A}}_i$ denotes the ith sheet of paper. The friction coefficient between sheets ${\mathrm{A}}_i$ and ${\mathrm{A}}_{i+1}$ is defined as ${\mu }_i$, where ${\mu }_i$ can be either the static friction coefficient ${\mu }_i^{\prime }$ or the dynamic friction coefficient ${\mu }_i^{″}$. Because the papers are of the same type, the friction coefficient between them is ideally considered to be the same regardless of i. However, for actual papers, the friction coefficient may have a non-uniform distribution, even within a sheet of paper, and the friction coefficient may change depending on the humidity and temperature. Let the variation that occurs in each paper with respect to the nominal value of the friction coefficient ${\mu }_0^{\prime }$ be denoted as ${\delta }_i^{\prime }$. In this study, ${\delta }_i^{\prime }$ is modeled as a stochastic variable.

$$\begin{array}{c}\hfill {\mu }_i^{\prime }={\mu }_0^{\prime }+{\delta }_i^{\prime }\end{array}$$

(1)

The friction coefficient between the fingers of the robot hand and the paper is defined as ${\mu }_{\mathrm{H}}^{\prime }$, and its value is set to be larger than any static friction coefficient between the sheets of paper.

$$\begin{array}{c}\hfill {\mu }_{\mathrm{H}}^{\prime }>{\mu }_i^{\prime }>{\mu }_i^{″}\end{array}$$

(2)

Because the static friction always exceeds the dynamic friction regardless of the variation ${\delta }_i^{\prime }$, this can be achieved by selecting the material of the fingertips. In other words, the variation ${\delta }_i^{\prime }$ within the range where ${\mu }_{\mathrm{H}}^{\prime }$ > ${\mu }_i^{\prime }$ holds is set to $\left|{\delta }_i^{\prime }\right|\ll 1$ and $\left|{\delta }_i^{\prime }\right|\ll 1$ is assumed in this paper. Note that this is true even when ${\delta }_i^{\prime }$ is time-varying.

We assume that the robot finger feeds the paper without slipping in the $+x$ direction, which is defined as the direction parallel to the surface of the paper, to separate the paper while making contact with it with a normal force F. In addition, the deformation of the paper is not considered.

2.2. Number of Sheets Removed by Paper Feed Operation

Consider the number of sheets removed from a stack of paper. When the finger moves in the $+x$ direction while in contact with the paper, the frictional force ${\mu }_{\mathrm{H}}^{\prime }F$ acts on the finger in the $-x$ direction.

The surface of sheet ${\mathrm{A}}_1$ is subjected to a reaction force with respect to the frictional force ${\mu }_{\mathrm{H}}^{\prime }F$ in the $+x$ direction, as shown in Figure 2. This force exceeds the frictional force ${\mu }_1^{\prime }F$ between sheets ${\mathrm{A}}_1$ and ${\mathrm{A}}_2$ (see Equation (2)). Sheet ${\mathrm{A}}_1$ always moves in conjunction with the finger.

The surface of sheet ${\mathrm{A}}_2$ is subjected to a reaction force of ${\mu }_1^{\prime }F$ from sheet ${\mathrm{A}}_1$ in the $+x$ direction and a frictional force of ${\mu }_2^{\prime }F$ from sheet ${\mathrm{A}}_3$ in the $-x$ direction. If ${\mu }_1^{\prime }>{\mu }_2^{\prime }$, sheet ${\mathrm{A}}_2$ overcomes the frictional force and moves with sheet ${\mathrm{A}}_1$. In other words, sheets ${\mathrm{A}}_1$ and ${\mathrm{A}}_2$ move in overlapping states. In this case, ${\mu }_1$ becomes ${\mu }_1^{\prime }$ because it is relatively fixed. On the other hand, if ${\mu }_1^{\prime }\le {\mu }_2^{\prime }$, sheet ${\mathrm{A}}_2$ remains in place, because of static friction. In other words, only sheet ${\mathrm{A}}_1$ can be separated and removed. In this case, ${\mu }_1$ becomes ${\mu }_1^{″}$ because the sheets are slipping relative to each other.

$$\begin{array}{c}{\mu }_1=\left\{\begin{array}{cc}{\mu }_1^{\prime }& ({\mu }_1^{\prime }>{\mu }_2^{\prime })\\ {\mu }_1^{″}& ({\mu }_1^{\prime }\le {\mu }_2^{\prime })\end{array}\right.\end{array}$$

(3)

Similarly, for sheet ${\mathrm{A}}_3$, if ${\mu }_1^{\prime }>{\mu }_2^{\prime }$, the same argument holds if we replace the set of parameters $({\mathrm{A}}_1$, ${\mathrm{A}}_2$, ${\mathrm{A}}_3$, ${\mu }_1^{\prime }$, ${\mu }_2^{\prime })$ in the above discussion for sheet ${\mathrm{A}}_2$ with the set of parameters $({\mathrm{A}}_2$, ${\mathrm{A}}_3$, ${\mathrm{A}}_4$, ${\mu }_2^{\prime }$, ${\mu }_3^{\prime })$. In contrast, if ${\mu }_1^{\prime }\le {\mu }_2^{\prime }$, sheet ${\mathrm{A}}_2$ is stationary, whereas the reaction force ${\mu }_1^{″}F$ from sheet ${\mathrm{A}}_2$ in the $+x$ direction is always smaller than the maximum static friction force ${\mu }_3^{\prime }F$ in the $-x$ direction generated between sheets ${\mathrm{A}}_4$ and ${\mathrm{A}}_3$ because ${\mu }_1={\mu }_1^{″}$ holds. Therefore, the actual frictional force generated is ${\mu }_1^{″}F$, and ${\mathrm{A}}_3$ is also stationary.

$$\begin{array}{c}{\mu }_2=\left\{\begin{array}{cc}{\mu }_2^{\prime }& ({\mu }_1^{\prime }>{\mu }_2^{\prime }>{\mu }_3^{\prime })\hfill \\ {\mu }_2^{″}& ({\mu }_1^{\prime }>{\mu }_2^{\prime },{\mu }_2^{\prime }\le {\mu }_3^{\prime })\hfill \\ {\mu }_1^{″}& ({\mu }_1^{\prime }\le {\mu }_2^{\prime })\hfill \end{array}\right.\end{array}$$

(4)

As discussed above, the state of a sheet ${\mathrm{A}}_i$ is sequentially determined from that of the previous sheet ${\mathrm{A}}_{i-1}$. The friction coefficient ${\mu }_i$ is expressed as follows:

$$\begin{array}{c}{\mu }_i=\left\{\begin{array}{cc}{\mu }_i^{\prime }& ({\mu }_1^{\prime }>{\mu }_2^{\prime }>\cdots >{\mu }_{i+1}^{\prime })\hfill \\ {\mu }_i^{″}& ({\mu }_1^{\prime }>{\mu }_2^{\prime }>\cdots >{\mu }_i^{\prime },{\mu }_i^{\prime }\le {\mu }_{i+1}^{\prime })\hfill \\ {\mu }_{i-1}^{″}& ({\mu }_1^{\prime }>{\mu }_2^{\prime }>\cdots >{\mu }_{i-1}^{\prime },{\mu }_{i-1}^{\prime }\le {\mu }_i^{\prime })\hfill \\ ⋮\\ {\mu }_1^{″}& ({\mu }_1^{\prime }\le {\mu }_2^{\prime })\hfill \end{array}.\right.\end{array}$$

(5)

When k sheets of paper are removed from overlapping layers, that is, when sheet ${\mathrm{A}}_{k+1}$ is stationary, the lower sheet ${\mathrm{A}}_i(i>k+1)$ is also stationary, because the dynamic friction forces ${\mu }_k^{″}F$ act on the top and bottom surfaces of each sheet in an opposite manner.

When the bottom surface of the paper bundle is supported by the floor, the floor exerts a normal force F on the bottom sheet ${\mathrm{A}}_n$. Therefore, if the static friction coefficient ${\mu }_{\mathrm{G}}^{\prime }$ between the floor and sheet ${\mathrm{A}}_n$ is larger than that between the sheets of paper, sheet ${\mathrm{A}}_n$ is subjected to a frictional force ${\mu }_{n-1}^{\prime }F$ in the $+x$ direction from sheet ${\mathrm{A}}_{n-1}$, which is always smaller than the maximum static friction force ${\mu }_{\mathrm{G}}^{\prime }F$ generated in the $-x$ direction by the floor. From Equation (5), because ${\mu }_{n-1}^{\prime }F$ is at most ${\mu }_{n-1}^{\prime }F$, the frictional force from the floor exceeds ${\mu }_{n-1}^{\prime }F$ because ${\mu }_{n-1}^{\prime }<{\mu }_{\mathrm{G}}^{\prime }$, and sheet ${\mathrm{A}}_n$ remains fixed regardless of whether sheet ${\mathrm{A}}_{n-1}$ is sliding or fixed. The number of sheets removed k is always less than n because $1\le k\le n-1$ is satisfied.

Therefore, if feed operations on a paper bundle, separated from the previous feed operation, are repeatedly performed in sequence, it is possible to separate and extract only the topmost sheet ${\mathrm{A}}_1$ a finite number of times.

A summary is presented below:

• With a single feed operation, paper bundle $\mathcal{P}$ is divided into two sets: the separated paper bundle ${\mathcal{P}}_{\mathrm{move}}\equiv \{{\mathrm{A}}_1,{\mathrm{A}}_2,\cdots ,{\mathrm{A}}_k\}$ and the remaining paper bundle ${\mathcal{P}}_{\mathrm{fix}}\equiv \{{\mathrm{A}}_{k+1},{\mathrm{A}}_{k+2},\cdots ,{\mathrm{A}}_n\}$.
• The number of sheets removed k is determined stochastically and is not known in advance.

3. Trajectory Generation of Robot Hand for Paper Separation

3.1. Concept

In this study, paper handling is achieved using a one-handed robot. As the number of sheets removed is stochastic in a single feed operation, we consider two states: partial separation and complete separation, as shown in Figure 3, in the process of the continuous removal of a single sheet at a time. Partial separation is a state in which paper bundles partially overlap with each other, and it can be achieved by setting a small stroke for the paper feed operation. In contrast, complete separation is a state in which the paper bundles are not in contact with each other. By considering partial separation, the paper can be moved in either the forward or reverse direction.

The reverse movement of the paper is achieved by placing the finger of the robot on the back of the partially separated subset of the paper bundle and feeding the paper back to the remaining paper bundle ${\mathcal{P}}_{\mathrm{fix}}$. This motion is defined as the return operation, as shown in Figure 4. By repeatedly performing the return operation on the reverse side of the partial paper bundle ${\mathcal{P}}_{\mathrm{move}}$, $\{{\mathrm{A}}_2,{\mathrm{A}}_3,\cdots ,{\mathrm{A}}_k\}$ can be returned to the paper bundle ${\mathcal{P}}_{\mathrm{fix}}$ without returning the topmost sheet ${\mathrm{A}}_1$. When the return operation is performed, the finger used for the feed operation stops moving to obtain a reaction force similar to that of the floor. Therefore, by using two fingers to pinch the paper bundle from both sides and combining the forward and reverse return motions, a single, uppermost sheet of paper can be completely separated from the paper bundle.

Although multiple operations are needed to completely separate a sheet of paper, each operation consists of simple motions, such as moving the fingers in one direction while the fingers are in contact with the paper. Therefore, it is easy to accelerate the finger movement, which leads to a reduction in the paper handling time, which is usually difficult to control.

3.2. Motion Strategy

The finger for the feed operation on the front side and the finger for the return motion on the back side are hereafter called the “feed finger” and the “return finger”, respectively. Assuming that the paper bundle is sufficiently thin, the origin is set at the center of the bundle, with the x-axis in the direction of the paper surface and the y-axis in the direction perpendicular to the paper surface. The feed finger in the $+y$ region moves the paper in the $+x$ direction, and the return finger in the $-y$ region moves the paper in the $-x$ direction. To transition to a state of partial separation without completely separating multiple paper bundles with a single stroke, we set the stroke of the feed operation to be smaller than the paper width $x_{\mathrm{width}}$. We assume that the paper bundles ${\mathcal{P}}_{\mathrm{move}}^i$ and ${\mathcal{P}}_{\mathrm{fix}}^i$ are not displaced during one stroke of the feed and return operations.

For the $k_1$ sheets ${\mathcal{P}}^2\equiv {\mathcal{P}}_{\mathrm{move}}^1=\{{\mathrm{A}}_1,{\mathrm{A}}_2,\cdots ,{\mathrm{A}}_{k_1}\}$ removed in the 1-set feed operation, and return operations are performed from the $-y$ direction to the direction of the original paper bundle using the return finger. Because the number of papers to be returned is stochastic, $k_2$ sheets ${\mathcal{P}}_{\mathrm{move}}^2\equiv \{{\mathrm{A}}_{k_1-k_2+1},{\mathrm{A}}_{k_1-k_2+2},\cdots ,{\mathrm{A}}_{k_1}\}$ can be returned, and the paper bundle ${\mathcal{P}}_{\mathrm{fix}}^2\equiv \{{\mathrm{A}}_1,{\mathrm{A}}_2,\cdots ,{\mathrm{A}}_{k_1-k_2}\}$ remains at the current position when the feed finger is set to stop while holding sheet ${\mathrm{A}}_1$, as shown in Figure 5a. The normal force acts on the paper bundle ${\mathcal{P}}^2$ but not on the remaining paper bundle ${\mathcal{P}}_{\mathrm{fix}}^1={\mathcal{P}}^1-{\mathcal{P}}^2=\{{\mathrm{A}}_{k_1+1},{\mathrm{A}}_{k_1+2},\cdots ,{\mathrm{A}}_n\}$, and ${\mathcal{P}}_{\mathrm{fix}}^1$ remains stationary because there is no frictional force between ${\mathcal{P}}_{\mathrm{move}}^2$ and ${\mathcal{P}}_{\mathrm{fix}}^1$. If more than two sheets remain in bundle ${\mathcal{P}}_{\mathrm{fix}}^j$ for $j\ge 2$, the jth 1-set return operation is executed in the same manner, and finally, only the topmost sheet ${\mathcal{P}}_{\mathrm{fix}}^{j+1}\equiv \left\{{\mathrm{A}}_1\right\}$ can be partially separated. Leaving a single sheet can be realized with fewer than $(k_1-1)$ set return operations; that is, $j<k_1-1$ is true. When the number of sheets is reduced to one, the feed operation can be restarted and a single sheet can be removed, which can be completely separated into a single sheet of ${\mathrm{A}}_1$.

The center position $x_i$ with respect to sheet ${\mathrm{A}}_i$ satisfies the following relationship, even if multiple feed and return operations are performed:

$$\begin{array}{c}\hfill x_1\ge x_2\ge \cdots \ge x_n.\end{array}$$

(6)

The stroke length does not have to be the same for all the feed operations; it can be set to any value as long as partial separation can occur. The same condition holds even if the return finger contacts bundle ${\mathcal{P}}_{\mathrm{move}}^{i-1}$, which had returned 1-set earlier while returning bundle ${\mathcal{P}}_{\mathrm{move}}^i$ for $i\ge 3$. This is because the moment the contact point $x_{c_i}$ between the return finger and the paper bundle ${\mathcal{P}}_{\mathrm{move}}^i$ coincides with the edge position of the bundle ${\mathcal{P}}_{\mathrm{move}}^{i-1}$, and the contact point of the return finger is shifted onto the paper bundle ${\mathcal{P}}_{\mathrm{move}}^{i-1}$ under the assumption that the paper is sufficiently thin, as shown in Figure 5b. This is true regardless of the contact point $x_{c_i}$.

Based on the above discussion, paper separation is characterized by the fact that paper can be separated without strictly controlling finger movement. Therefore, in this study, the stroke length of a single feed-and-return operation is not explicitly controlled, and the strategy is to achieve paper separation by repeating simple motions of the fingertips. This allows a robotic hand with a narrow range of motion and short stroke-length of fingers to perform paper handling operations.

3.3. Trajectory Generation of Fingers

The fingertip behaviors of the feeding and returning fingers are generated using periodic trajectories. The trajectory is designed using a simple sinusoidal function for the joint trajectory, and the parameters are determined such that the fingertip performs a pseudo-elliptical motion. This allows the finger to move backward at the start of the stroke, enabling the paper to move repeatedly in the same direction. To prioritize high-speed movement, an elliptical trajectory is adopted rather than a linear trajectory parallel to the paper surface. For the two fingers of the robotic hand, let ${\theta }_i$ be the proximal and distal joint angles of the feed and return fingers, respectively. For amplitude ${\alpha }_i$, offset ${\beta }_i$, angular velocity ${\omega }_i$, and phase ${\varphi }_i$, the joint motion of ${\theta }_i$ can be expressed as follows for $i=1,\cdots ,4$:

$${\theta }_i={\alpha }_{i}sin({\omega }_{i}t+{\varphi }_i)+{\beta }_i.$$

(7)

The angular velocities of the finger joints are set to the same value, and the phases of the joint angles on the proximal and distal parts are shifted by ${\displaystyle \frac{\pi }{2}}$.

The camera detects the position of the partially separated paper bundle ${\mathcal{P}}_{\mathrm{move}}^i$ and switches from a feed operation to a return operation when two or more sheets of paper pass through a certain threshold position $x_{\mathrm{thd}}$ set in the x direction. The robot repeats the return operations until only ${\mathrm{A}}_1$ remains among the papers that have passed the position $x_{\mathrm{thd}}$ and then switches to the feed operation. By continuously performing these operations, it is possible to remove the topmost sheets of paper sequentially.

3.4. State Detection of Paper Bundle Using Camera

A high-speed camera is used to detect the paper overlaps. To facilitate the determination of the number of sheets of paper separated, all papers are marked in black on one edge, as shown in Figure 6. If two or more sheets of paper completely overlap without the slightest deviation, they are undetectable. Therefore, slightly shifting the initial position of each sheet of paper prevents the marker from being hidden by other sheets of paper, even when multiple sheets are removed, and the number of sheets of paper can be determined according to the area of the marker.

$$\begin{array}{c}\hfill x_1>x_2>\cdots >x_n\end{array}$$

(8)

In other words, for overlap detection, the experiments in this study are performed under the above conditions instead of using Equation (6).

4. System Configuration [19]

An outline of the system is shown in Figure 7.

4.1. High-Speed Multifingered Hand

This robotic hand is being developed with the assumption that it can perform high-speed dynamic movements compared to conventional hands, as shown in Figure 8. Since this hand is designed to be lightweight so that it can be mounted on an arm, the number of fingers is three, which is the minimum number required for stable grasping in the face of disturbance. The degrees of freedom (DOFs) are 2 for the middle finger, 3 for the right and left fingers, and 2 for the wrist, including flexion and rotation. In total, the hand has 10 DOFs. A compact actuator capable of instantaneous high output has been developed based on a backlash-free harmonic drive gear ^®. By increasing the winding density by more than 1.5 times compared to conventional commercial products, the instantaneous torque/weight ratio is more than 3.5 times higher than that of conventional commercial products. The actuators are used in an interphalangeal (IP) joint and a metacarpophalangeal (MP) joint of each finger, resulting in a strong grasping force.

4.2. High-Speed Vision

A Basler ace acA800-510uc sourced by Basler Japan KK, Tokyo, Japan with an 800 × 600 resolution and a maximum frame rate of 511 fps is used as a high-speed vision system. The images acquired by the high-speed camera are processed by a personal computer (PC) with an Intel(R) Core(TM) i7-9750H CPU @ 2.60 GHz processor with 16 GB of memory. The total area of black markers on white paper is calculated by image processing to recognize paper overlaps. Since this computation can be done at the maximum frame rate using only the CPU if the image processing is implemented well, the vision system in this paper is configured without using GPUs. Another monochrome high-speed camera (Phantom Miro C110 sourced by Nobby Tech. Ltd., Tokyo, Japan) with 8 GB memory is also used for debugging purposes for offline visual verification. The frame rate is 500 fps, and the resolution is 768 × 768.

4.3. Real-Time Controller

An Expansion Box PX10 sourced by dSPACE Japan Corporation, Tokyo, Japan consisting of modular hardware is used as the real-time controller. A dual-core QorIQ P5020 PowerPC processor (DS1007) sourced by dSPACE Japan Corporation, Tokyo, Japan running at 2 GHz is connected to 32ch 16bit A/D inputs (DS2003), 32ch 14bit D/A inputs (DS2103), and three 6ch 32bit encoder counters (DS3002). The processor is equipped with a 32 KB L1 data cache, a 512 KB L2 cache, and a 2 MB L3 cache per core, and its board has 128 MB of flash memory and 1 GB of DRAM global memory. The processor and a number of I/O boards are connected by a dedicated 32-bit bus. The system is suitable for machine control that requires high sampling rates and large-scale input/output processing capacity. The real-time controller communicates with the image processing PC via Ethernet and receives image features.

5. Experiments

Two of the three fingers on each side were used for paper separation. The left and right fingers were assigned as the feed and return fingers, respectively. To increase the friction coefficient between the finger and the paper ${\mu }_{\mathrm{H}}^{\prime }$, the fingertips were covered with rubber finger sacks. PD control according to Equation (7) was executed for the IP and MP joints of both fingers, and the other joints were fixed in a posture such that the motion of both fingers moved in the horizontal plane. The lengths of the fingertip links is 5.5 cm, the length of the root link is 6.5 cm, and the distance between the bases of both fingers is 12 cm. Torque commands to the robot hand were updated at 1 kHz. The parameters in Equation (7) were set as shown in

Table 1. Hand motion parameters.

i	${\mathit{\beta }}_{\mathit{i}}$ (rad)	${\mathit{\alpha }}_{\mathit{i}}$ (rad)	${\mathit{\omega }}_{\mathit{i}}$ (rad/s)	${\mathit{\varphi }}_{\mathit{i}}$ (rad)
1	0.20	0.10	8$\pi$	$\pi$
2	1.20	0.30	8$\pi$	$-{\displaystyle \frac{\pi }{2}}$
3	$-0.30$	0.12	8$\pi$	$\pi$
4	1.30	0.20	8$\pi$	${\displaystyle \frac{\pi }{2}}$

. A bundle of business-card-sized papers was used, and its width was approximately equal to the finger length of the robot hand. To ignore the effect of gravity and avoid interference between the floor and return finger, the paper bundle was placed in a cardcase and stood vertically, as shown in Figure 9, and the robot hand was positioned such that it could pinch the front and back of the paper bundle with two fingers in the horizontal plane. In addition, the paper bundle was placed at an angle to the extension direction of both fingers, as shown in Figure 9. The reason is that the stroke range of the return finger is shifted to the right relative to that of the feed finger so that the return finger does not contact the remaining paper bundle during the return operation. The high-speed vision system for hand control was positioned on the right side of the robot hand and measured the number of papers that passed the threshold position $x_{\mathrm{thd}}$. The frame rate was set to 200 fps, and paper overlap recognition was also performed at 200 fps. In general, increasing the frame rate tends to shorten the exposure time, darken the image, and reduce the accuracy of image processing, so it is not desirable to increase the frame rate unnecessarily. Therefore, 200 fps was selected for this experiment as a sufficient value corresponding to the speed of the sheet movement controlled by the robot hand.

6. Results

6.1. Experimental Results

A video of the experimental results was recorded [20]. Figure 10 shows the results of recognizing paper overlaps using high-speed vision. The vertical axis of the graph shows 1 when two or more sheets of paper are detected and 0 when there is only one sheet. Figure 11 shows the y-coordinates of the fingertip position of a robotic hand. It can be confirmed that the feed fingers operate at 10.5 and 11 s to feed the paper. Then, they stop when two or more sheets are detected, and the return fingers operate to return the paper to the paper stack. Then, the return finger stops because the number of sheets of paper removed has reached one, and the feed finger resumes the paper-feeding operation. As a result, 35 feed and return operations were performed during a 10 s operating period, and seven sheets of paper were completely separated. As described above, the paper is returned when overlapping papers are detected, and the paper is successfully separated individually by exploiting frictional uncertainty. Figure 12 shows the fingertip path of the feed finger in the x-y plane. It can be confirmed that the fingertip path is periodic in the x-y plane, although the feed finger repeats the stop-and-go motion due to the presence of the return motion by the return finger, as shown in Figure 11. Its shape forms a smooth path according to Equation (7), indicating that the motion follows the surface direction of the diagonally placed paper bundle. The two lines extending to the lower right represent the movement to and from the initial state with straight fingers at the beginning and end of the feed operation.

A sequence of pictures taken every 0.1 s from 11.3 to 11.6 s is shown in Figure 13. It can be seen that multiple sheets of paper are fed out when the paper feed operation is performed and that the paper is returned to the paper bundle by the return operation. In this experiment, the robotic hand fed approximately one-third of the paper during a single feed operation. Therefore, to completely separate a sheet of paper, three feed operations were required in addition to return operations. The maximum number of sheets fed out during a single operation was two.

6.2. Discussion

The proposed method took an average of 1.43 s to remove one sheet, and the speed could be improved considering the specifications of the robot hand used in the experiment. Continuing to feed operation with overlapping sheets requires a corresponding return stroke, resulting in a longer total operation time. Therefore, one solution to achieve higher speed is to detect overlapping paper at an early stage in the feed operation and continue a long stroke in the case of single sheet removal while switching to a trajectory that returns to the initial position of the stroke earlier in case of overlapping. In other words, by setting the overlap judgment position $x_{\mathrm{thd}}$ closer to the remaining paper bundle and switching the parameters in Table 1 based on whether or not there is paper overlap, the time-efficient trajectory of the robot hand can be generated. Moreover, by detecting overlaps continuously instead of at a single location $x_{\mathrm{thd}}$, it is possible to respond in real-time to changes in the number of sheets during a stroke.

In this experiment, visual markers were used to recognize paper overlaps, but this is a limited method because the markers must be attached to the paper in advance, and the initial position of the paper bundle must be slightly displaced. Reflective laser sensors are a possible alternative to vision sensors for detecting paper overlaps. One unit can detect the presence or absence of paper overlaps by step detection, and by using two units from both sides of a paper bundle, the thickness of the removed sheets can be detected, and consequently, the number of overlapping sheets can be calculated directly. While most laser sensors have an accuracy in the order of tens of micrometers, the thickness of ordinary paper is about 100 µm, and that of slightly thicker paper, such as a business card, is about 200 µm, making measurement possible. This can be a realistic method because it can robustly measure the distance to an object without being affected by the material or color of the object.

The single-sheet separation method formulated in this study can be considered the equivalent of the Las Vegas algorithm in computational sciences, which combines stochastic events and randomness. The Las Vegas algorithm is a random-choice algorithm that solves mathematical problems numerically using a computer to sample random variables. Our approach is a movement strategy that considers friction between papers as a stochastic random element whose value is sampled by the interaction with the robot. By modeling the uncertainty of real-world events as random elements and formulating a robot’s behavior as a random-choice algorithm, this method actively utilizes uncertainty, which cannot be eliminated in reality. Robot manipulation involving physical contact with an object is difficult to model rigorously, and even using sim2real reinforcement learning does not work well in the current situation. In contrast, in the proposed method, the robot does not execute actions sampled by the computer; rather, the results of the robot’s actions, which reflect the uncertainty of the real world, are taken as samples and formulated as a random-choice algorithm in which the real-world task converges to the desired action through repeated trials.

In general, stochastic algorithms are characterized by a simpler form of refinement and lower computational costs than deterministic algorithms for the same problem. In this study, the simplicity of the operation was demonstrated by the fact that paper separation could be achieved with only two types of operations—feed and return—without explicitly adjusting the contact force, the contact point, or the stroke length. Although this requires repetitive processing, the simplicity of each operation directly contributes to its ease of execution with a real-world robot and the ease of accelerating the operation. Randomness was assigned as the uncertainty of paper-to-paper friction in this study. However, there are other uncertainties, such as slippage, deformation at contact points between the robot and the object, and rattling and loosening of the robot’s moving parts. Accordingly, the proposed method can be developed as a framework for robust robot manipulation under uncertain conditions.

7. Conclusions

We propose a method for separating a sheet from a paper bundle using a robotic hand via a probabilistic approach based on the uncertainty of friction. The key point is that it introduces a state of partial separation that allows the paper to be returned even when multiple sheets are separated temporarily. A single sheet can be separated by repeating a simple operation that involves a combination of forward and reverse return movements. A high-speed camera was used to detect the presence or absence of paper overlap and to determine the switching timing between the feed and return operations. Paper handling was demonstrated experimentally. A sheet from a paper stack was successfully separated using two fingers of the robotic hand.

Further research is needed to improve the detection method for the number of completely overlapping papers, as the paper overlap was detected by shifting the initial paper position slightly for visual recognition. As described in Section 6.2, the use of reflective laser sensors instead of a camera is also a possible candidate. In the future, the validity of this single-sheet separation method based on the uncertainty should be demonstrated using various paper types and sizes to enhance the method’s robustness. Moreover, this method can be extended to remove sheets at any position from the paper bundle, not just the uppermost sheet.