FLIA: Architecture of Collaborated Mobile GPU and FPGA Heterogeneous Computing

Abstract

Accelerators, such as GPUs (Graphics Processing Unit) that is suitable for handling highly parallel data, and FPGA (Field Programmable Gate Array) with algorithms customized architectures, are widely adopted. The motivation is that algorithms with various parallel characteristics can efficiently map to the heterogeneous computing architecture by collaborated GPU and FPGA. However, current applications always utilize only one type of accelerator because the traditional development approaches need more support for heterogeneous processor collaboration. Therefore, a comprehensible architecture facilitates developers to employ heterogeneous computing applications. This paper proposes FLIA (Flow-Lead-In Architecture) for abstracting heterogeneous computing. FLIA implementation based on OpenCL extension supports task partition, communication, and synchronization. An embedded system of a three-dimensional waveform oscilloscope is selected as a case study. The experimental results show that the embedded heterogeneous computing achieves 21× speedup than the OpenCV baseline. Heterogeneous computing also consumes fewer FPGA resources than the pure FPGA accelerator, but their performance and energy consumption are approximate.

1. Introduction

In many applications nowadays, embedded SoCs (System-on-a-Chip) integrate mobile GPU (Graphics Processing Unit) that optimizes image rendering. As an accelerator of the host CPU (Central Processing Unit), FPGA (Field Programmable Gate Array) accelerates the algorithm and reduces energy consumption at the hardware level.

The motivation is that mobile GPU supports accelerating general algorithms by OpenCL development [1]. Therefore, the algorithms with diverse parallel characteristics can pertinently map to the heterogeneous GPU-FPGA accelerators. The heterogeneous computing platform successfully combines flexibility and efficiency [2]. Meanwhile, image processing tasks offloaded from the FPGA into GPU will reduce the usage of FPGA resources.

However, in the common scenario, the applications always utilize only one type of accelerator. The traditional development toolchain lacks portability and collaboration of heterogeneous accelerators. The collaborative GPU-FPGA design faces a series of challenges: (1) The development environment and the programming toolchain of these accelerators are reasonably different. A unified and comprehensive development model will facilitate the application of heterogeneous computing. (2) The programming languages should be portable to various accelerators while ensuring individual performance. Specific architectural optimizations are necessary to explore the performance potential of the accelerator. (3) Heterogeneous computing involves tasks partitioning, mapping, and other jobs, that lead to vast combinations of design space.

To solve the above problems, this paper proposes FLIA (Flow-Lead-In Architecture) to abstract heterogeneous computing that supports task partitioning, communication, and synchronization. This paper also extends OpenCL and presents two communication mechanisms.

This paper contributes in the following aspects:

• The comprehensible FLIA and its implementation facilitate developers to employ heterogeneous computing applications.
• A case study developed on the collaborated mobile GPU and FPGA platform proves the effectiveness of heterogeneous computing.

The paper is organized as follows: Section 2 overviews the background and related work. Section 3 presents the Flow-Lead-In architecture, including the model description and model analysis, then Section 4 describes the implementation of the FLIA architecture. Section 5 proposes the case study design, and Section 6 shows the evaluation results. Finally, Section 7 concludes the paper.

2. Background and Related Work

Many applications benefit from heterogeneous computing, e.g., the virtual reality field uses the global illumination algorithm on mobile SoC with CPU and GPU [3], whose approach increases CPU utilization from 13% to 80%. In the collaborated CPU and mobile GPU computing, [4] reduces the data initialization latency caused by the shared memory of heterogeneous processors. [5] profits from CPU-FPGA collaboration in the cloud-accelerating environment.

2.1. Heterogeneous Computing Workload Partioning

The current mainstream idea of heterogeneous computing is to partition the overall computing task into small workloads. [6] classifies the partitioning methods into two categories, task partitioning and data partitioning.

CED (Canny Edge Detection) is an edge detection algorithm that consists of four stages: (1) a Gaussian filter, (2) a Sobel filter, (3) non-maximum suppression, and (4) hysteresis. As shown in Figure 1, lateral data partitioning means the heterogeneous accelerators perform the identical task with different subsets of the data. Vertical task partitioning means each heterogeneous accelerator performs various sub-tasks and communicates between devices. Taking CED processing a stream of video frames as an example, in the case of data partitioning, each heterogeneous accelerator handles a mutually disjoint set of frames. As task partitioning, all frames are serially processed by the first two stages on device1, then by the last two stages on device2.

Each partitioning strategy entails its tradeoffs. The main challenge with data partitioning is determining the distribution of data workloads across devices that results in the workload balance. MultiFastFit [7] divides consecutive iterations into chunks, whose sizes are modeled to ensure performance and resource utilization. Similarly, HGuided [8] dynamically calculates partitioned data packets size with considering the power of various accelerators. In order to reduce the idle time of each accelerator, the data partitioning method tends to divide data workloads into smaller sizes, which increases the cost of sub-task synchronization.

On the other side, task partitioning does not need to deploy FPGA resources to all sub-tasks, because part of sub-tasks entirely runs on the GPU or CPU. Refs. [9,10] propose task-level schedulers to accelerate the modules running in parallel on FPGA. HRES is a task partitioning method based on DPR (dynamic partial reconfiguration) [11]. HRES implements an identical task with different scaling factors to facilitate the scheduling algorithm fills up the resource slots on FPGA [12].

As a result, task partitioning generally enables more task duplication on the FPGA than data partitioning, and gains better overall performance in most application scenarios [13]. FLIA adopts a task partitioning strategy that is suitable for the application scenario of the data processing pipeline. CAL provides a data flow programming language whose primary concern is converting the model to hardware description language [14]. CDFG adopts directed acyclic graph as an input format to high-level synthesis tools, which is always used to study the CPU and FPGA tasks partition [15], and positively inspires FLIA. However, FLIA focus on the heterogeneous accelerators collaboration and the communication mechanism between tasks.

2.2. OpenCL Runtime

OpenCL is a platform-independent open standard that supports compiling CPUs, GPUs, and FPGAs from various accelerator vendors [16]. Base64 encoding [17], Face recognition [18], object removal [19], and SIFT (scale-invariant feature transform) feature detector [20] are developed using OpenCL on mobile GPUs. K-Nearest neighbor and bitonic sorting algorithms have been modeled in OpenCL for both GPU and FPGA [21]. However, OpenCL does not focus on the hardware and software tasks partitioning, nor the communication between modules on heterogeneous accelerators [22].

Previous [7,12,13] perform data partitioning studies based on OpenCL. These experiments only accelerate one algorithm at a time, because OpenCL cannot natively support task-level scheduling in heterogeneous computing.

Although OpenCL is portable at the language level, the toolchains provided by accelerator vendors are hard to work together, especially for collaborated heterogeneous accelerators. Ref. [23] extracts kernel-level task parallelism from in-order queues by extending an open-source implementation of OpenCL. In addition, OpenCL natively lacks several critical mechanisms of heterogeneous computing, such as communication and synchronization across accelerators, which are manually implemented by many studies. Ref. [7] provides HBB library to fill the gap of OpenCL for heterogeneous computing. Therefore, this paper proposes FLIA that extends the OpenCL to simplify of heterogeneous computing applications.

3. Flow-Lead-In Architecture for Heterogeneous Computing

Heterogeneous computing faces problems with development complexity and high barriers to entry. FLIA abstracts an application as a data-flow-driven processing pipeline consisting of servants and execution-flows. Quoting the concepts from OS area of SEFM (Servant & Execution-Flow Model) [24,25], the servant stands for the functional module that accomplishes specific computational task in the system. The execution-flow represents the data-flow between the servants, and is responsible for communication and synchronization. The FLIA owns four characteristics: portability, encapsulation, parallelism, and reusability:

• Portability: As an abstraction of the computational unit, the servant does not dedicate to the specific accelerator.
• Encapsulation: The servant is a self-fulfilled module. The developers only focus on the algorithm of the servant, knowing neither the details of FLIA runtime nor the internal implementation of other servants.
• Parallelism: Multiple servants can run on heterogeneous accelerators in parallel to improve system performance.
• Reusability: The design of the servant can be reused on various accelerators.

3.1. Model Definition

The application is modeled as a sextuple:

$$M\left(\mathrm{S},\mathrm{Q},\mathrm{C},\delta ,Q_0,\mathrm{F}\right)$$

S is the servant, Q is the execution-flow, C is the computational resource, $\delta$ is the state transition function, $Q_0$ and F are the initial and terminated states. The definitions in more detail are as follows:

Definition: Servant S: An operational component that can perform a particular computing task.

The system has a set of N servants represented as:

$$\mathrm{S}=\left\{{\mathrm{S}}_{\mathrm{i}}|i\in N\right\}$$

This model specifies that a servant is a minimum unit that can be loaded and unloaded while the system runs. Thus, the servant is the fundamental unit of scheduling in heterogeneous computing.

Definition: Execution-flow Q: Message (data structure) transferred between servants. Execution-flow is represented as symbol ×, and indicated by an arrow in diagrams. As shown in Figure 2, the computing result of the servant S₁ transfers to S₂ for the subsequent processing. This execution-flow is represented as:

$${\mathrm{S}}_1\times {\mathrm{S}}_2$$

The set of execution-flows in the system is:

$$\mathrm{Q}=\left\{Q_i^j|i,j\in N \wedge Q_i^j\in \mathbb{N} \wedge Q_i^j\le M_i^j\right\}$$

$Q_i^j$ stands for the execution-flow ${\mathrm{S}}_{\mathrm{i}}\times {\mathrm{S}}_{\mathrm{j}}.$ Both i and j range from 1 to N (N is the total number of servants). $Q_i^j$ indicates the number of messages on the execution-flow that ${\mathrm{S}}_{\mathrm{i}}$ has produced but ${\mathrm{S}}_{\mathrm{j}}$ has not consumed. When the leading servant ${\mathrm{S}}_{\mathrm{i}}$ completes and generates its output, an additional message exists on the execution-flow, i.e., $Q_i^j=1$. After the subsequent servant ${\mathrm{S}}_{\mathrm{j}}$ consumes this message, $Q_i^j=0$.

An execution-flow has dual roles: (1) Data communication. The execution-flow describes the communication relationship between the servants. (2) Drive the successive servants to start execution. After a servant executes completed, the generated execution-flow will issue the successor servant to push the whole computing task. Therefore, the FLIA is driven by data flow, which is different from the Von Neumann architecture of instruction-driven.

Definition: FIFO-execution-flow: The execution-flow contains FIFO (First-In-First-Out) queue. The FIFO can buffer one or more messages. $M_i^j$ is defined as the maximum legal capacity of FIFO $Q_i^j$. Therefore, $Q_i^j=M_i^j$ means buffers of FIFO are full.

Definition: Resource C: The set of processing units is:

$${\mathrm{C}}_{set}=\left\{GPU,FPGA,CPU\dots \right\}$$

States of the resources can be:

$${\mathrm{C}}_{state}=\left\{BUSY,IDLE\right\}$$

Then, the set of resources is:

$$\mathrm{C}=\left\{{\mathrm{C}}_{\mathrm{i}}\langle R_i\rangle |i\in N\wedge {\mathrm{C}}_{\mathrm{i}}\in {\mathrm{C}}_{state}\wedge R_i\in {\mathrm{C}}_{set}\right\}$$

Definition: State transition function $\delta$:

$$\begin{array}{c}\delta \left({\mathrm{C}}_{\mathrm{i}}=IDLE,{\bigwedge }_j\left(Q_i^j<M_i^j\right)\wedge {{\displaystyle \prod }}_k{Q}_k^i\ne 0\right)\stackrel{\Delta }{\to }{\mathrm{C}}_{\mathrm{i}}=BUSY\\ \delta \left({\mathrm{C}}_{\mathrm{i}}=BUSY,{\mathrm{C}}_{\mathrm{i}}=IDLE\right)\stackrel{\Delta }{\to }\left(\forall Q_k^i\in Q,Q_k^i--\right)\left(\forall Q_i^j\in Q,Q_i^j++\right)\end{array}$$

As shown in Figure 3, the first state transition function means the servant ${\mathrm{S}}_{\mathrm{i}}$ will occupy resource and issue execution while ${\mathrm{S}}_{\mathrm{i}}$ meets the following three conditions simultaneously: (1) ${\mathrm{S}}_{\mathrm{i}}$ is under resource idle state; (2) ${\wedge }_j\left(Q_i^j<M_i^j\right)$, i.e., any output port has free space; (3) ${{\displaystyle \prod }}_k{Q}_k^i\ne 0$, i.e., all input ports are not empty.

The second state transition function means that when the resource’s state of servant ${\mathrm{S}}_{\mathrm{i}}$ changes from BUSY to IDLE, the space count of each input buffer decreases and the space count of output buffer increases.

3.2. Model Analysis

FLIA is analyzed using the directed graph of graph theory, which defines:

• Vertex: The servant is regarded as the vertex of the graph.
• Edge: The execution-flow is regarded as the edge of the graph.
• Weight of edge: The sum of a servant’s execution duration and its initiated execution-flow’s communication duration.
• Path: A path between servants ${\mathrm{S}}_{\mathrm{A}}$ and ${\mathrm{S}}_{\mathrm{B}}$ means that the execution result of ${\mathrm{S}}_{\mathrm{A}}$ is processed by one or many servants finally sent to ${\mathrm{S}}_{\mathrm{B}}$. Such as Figure 4, there are two paths between the servants ${\mathrm{S}}_{\mathrm{A}}$ and ${\mathrm{S}}_{\mathrm{B}}$.

To simplify following analysis, the weights of all edges are assumed to be 1, i.e., the execution duration of all servants is identical. For a graph with multiple paths, like Figure 4, the result from starting point transfers through all paths and eventually arrive at ${\mathrm{S}}_{\mathrm{B}}$. Considering the spanning tree with ${\mathrm{S}}_{\mathrm{B}}$ as the root vertex, weighted depths of ${\mathrm{S}}_{\mathrm{A}}$ are 2 and 3, respectively. Different depths mean that the operands issued by ${\mathrm{S}}_{\mathrm{A}}$ along two paths do not arrive at ${\mathrm{S}}_{\mathrm{B}}$ simultaneously.

Analysis 1 Synchronous flow controlling: If there are multiple paths between head-end and tail-end servants, even if each path has different execution duration, the tail-end servant’s inputs always consume the same operand from the head-end servant.

Taking Figure 4 as an example, operands arrived at ${\mathrm{S}}_{\mathrm{A}}$ are marked by α, β, γ… in chronological order.

Table 1. Space-time diagram.

SA	α	β		γ	δ		ε
SX		α		β	γ		δ
SY		α	β		γ	δ
SZ			α	β		γ	δ
SB				α	β		γ
	t0	t1	t2	t3	t4	t5	t6

shows the execution space-time diagram of Figure 4.

At the moment t1, operand α is transferred from ${\mathrm{S}}_{\mathrm{A}}$ to ${\mathrm{S}}_{\mathrm{X}}$ and ${\mathrm{S}}_{\mathrm{Y}}$ simultaneously while a new operand β arrives at ${\mathrm{S}}_{\mathrm{A}}$.

At the moment t2, α from ${\mathrm{S}}_{\mathrm{X}}$ arrives ${\mathrm{S}}_{\mathrm{B}}$, while α from ${\mathrm{S}}_{\mathrm{Y}}$ arrives ${\mathrm{S}}_Z$ without reaching ${\mathrm{S}}_{\mathrm{B}}$. According to the state transition conditions, ${\mathrm{S}}_{\mathrm{B}}$ does not meet the third condition (execution-flows of all input ports are ready). Thus, ${\mathrm{S}}_{\mathrm{B}}$ does not run, causing that ${\mathrm{S}}_{\mathrm{X}}$ not to meet the second condition (execution-flows of all output ports transfer to the successive servant). Consequently, ${\mathrm{S}}_{\mathrm{A}}$ does not run.

At the moment t3, α goes through both paths and arrives at ${\mathrm{S}}_{\mathrm{B}}$. ${\mathrm{S}}_{\mathrm{X}}$ processes β then ${\mathrm{S}}_{\mathrm{A}}$ processes $\mathsf{\gamma }$.

The following process is similar to those mentioned above.

According to the initial analysis of Table 1, the servant ${\mathrm{S}}_{\mathrm{B}}$ in the tail never fetches different operands from two paths simultaneously. Therefore, it can be conjectured that if there are multiple paths between two servants, the operands of the tail-end servant are always from the same operand transferred by the head-end servant. This phenomenon is called synchronous flow controlling.

The inductive proof: The pipeline is empty at the start-up. Then the first operand α issues and arrives ${\mathrm{S}}_{\mathrm{B}}$ along the less weighted path in advance of the path with heavy weight. However, ${\mathrm{S}}_{\mathrm{B}}$ does not meet the third state transition condition that ‘all inputs are ready’. Therefore, ${\mathrm{S}}_{\mathrm{B}}$ will not run until the operands from all children vertices arrive. (1) For the first operand, the above conjecture is true. (2) Assuming the conjecture is also true for the operand n. Then the computing resource of ${\mathrm{S}}_{\mathrm{B}}$ is occupied by operand n. Two servants before ${\mathrm{S}}_{\mathrm{B}}$ can execute the operand (n + 1), according to the second state transition condition. (3) ${\mathrm{S}}_{\mathrm{B}}$ runs when two operands (n + 1) both arrive from the previous servants, according to the third state transition condition. So, the conjecture for the two paths is true.

If there is only one path between two servants, i.e., a simple serial flow without any branch, such as Figure 2, the above conjecture is true. Using the inductive method again, it is easily obtained that the conjecture remains true when there are multiple paths between two servants.

Due to the intrinsic feature of synchronous flow controlling, there is no write-conflict problem caused by the multiple-issuing of the Von Neumann processor. Thus, the FLIA is beneficial to developing a parallel computing system.

Analysis 2 The elimination of different execution duration of paths: The execution duration analysis of the intersected paths benefits from a spanning tree, which is derived from the intersected vertex as the root. The weighted depth of the spanning tree’s leaf vertex equals its path’s total running duration. Therefore, the total running duration of the whole spanning tree depends on the path with the deepest leaf. According to Amdahl’s law, optimization efforts aim at the path with the largest weight.

Analysis 3 Cyclic execution-flows and deadlock: Figure 5 is an example of execution-flows with a loop, where ${\mathrm{S}}_{\mathrm{B}}$ and ${\mathrm{S}}_{\mathrm{D}}$ are two vertices on the loop. In the initialization, since ${\mathrm{S}}_{\mathrm{B}}$ is the successor of ${\mathrm{S}}_{\mathrm{D}}$, ${\mathrm{S}}_{\mathrm{B}}$ must wait for the output of ${\mathrm{S}}_{\mathrm{D}}$ to start. At the same time, ${\mathrm{S}}_{\mathrm{D}}$ must wait for ${\mathrm{S}}_{\mathrm{B}}$. Both ${\mathrm{S}}_{\mathrm{B}}$ and ${\mathrm{S}}_{\mathrm{D}}$ are waiting for the resource held by each other, which originates a classic deadlock problem.

Being similar with the process of solving the deadlock in the operating system field, one of the cyclic servants should first acquire all necessary resources. For instance, an initial execution-flow should be output by ${\mathrm{S}}_{\mathrm{D}}$ when the system boots. By this time, ${\mathrm{S}}_{\mathrm{B}}$ starts after the execution-flow arrival from ${\mathrm{S}}_{\mathrm{A}}$. Then the ${\mathrm{S}}_{\mathrm{B}}$ output execution-flow starts ${\mathrm{S}}_{\mathrm{D}}$. This loop will continue till the entire operation completed.

4. Implementation of FLIA

FLIA is suitable for the data processing pipeline application that always contains multiple stages, and each stage can run on an accelerator. Thus, numerous stages can run in parallel on heterogeneous accelerators. FLIA implements this feature based on an extension of OpenCL, whose architecture is shown in Figure 6. FLIA involves variety of heterogeneous accelerators, each with a vendor-dedicated OpenCL runtime. The scheduler of FLIA integrates these runtimes for collaborative work.

4.1. Servant Implementation

A servant consists of two critical elements: kernel function and parametric-instantiation. In practice, the kernel function is usually the innermost iteration of large nested loops [23]. Vendor-specific OpenCL toolchain compiles the source code of the kernel then the kernel maps to various accelerators. The scheduler inserts the kernel into the commands queue of the corresponding accelerator runtime and issues the kernel execution. The optimization of the generated logic circuit and executable code inner a servant is not done by FLIA, but a task of the OpenCL compiler.

4.2. Execution-Flow Implementation

Typically, there are communication and coordination overheads involved in heterogeneous computing. The execution-flow is responsible for this critical work. The scheduler of FLIA listens to the events from all accelerator vendor-specific runtimes. When the vendor-specific OpenCL runtime calls clFinish() hook function to indicate the completed signal of a servant, the scheduler performs the state transition functions, including transactions as buffer copy, multiple servants execution synchronizations, new servant issuing, etc.

A hardware implementation of execution-flow, referring to the bus-style communication architecture of OneSys [26], is presented in Figure 7. All FPGA-based servants and a memory controller with MMU (Memory Management Unit) are attached to the bus.

FLIA maps the data inner an execution-flow to an area of memory. ${\mathrm{S}}_{\mathrm{A}}\times {\mathrm{S}}_{\mathrm{B}}$ is taken as an example for illustration: The servant ${\mathrm{S}}_{\mathrm{A}}$ writes computing results to the memory as an output execution-flow. After that, the FLIA scheduler switches this memory area to the address space of the subsequent servant ${\mathrm{S}}_{\mathrm{B}}$ using MMU. After that, the data packet is transferred from ${\mathrm{S}}_{\mathrm{A}}$ to ${\mathrm{S}}_{\mathrm{B}}$. This design reduces memory duplication between FPGA servants.

In the case of bus-style communication, the execution-flow is sent to the subsequent servant ${\mathrm{S}}_{\mathrm{B}}$, then ${\mathrm{S}}_{\mathrm{B}}$ starts executing. At this time, if ${\mathrm{S}}_{\mathrm{A}}$ produces a new execution-flow that requires a memory writing operation, the bus will conflict with ${\mathrm{S}}_{\mathrm{B}}$ reading operation. It is necessary to extend the bus-style execution-flow.

The architecture of the post-style execution-flow is shown in Figure 8. It contains multiple buses, and all servants share each bus. When the writing operation of ${\mathrm{S}}_{\mathrm{A}}$ conflicts with the reading operation of ${\mathrm{S}}_{\mathrm{B}}$, the FLIA scheduler will choose a new bus for ${\mathrm{S}}_{\mathrm{A}}$ to reduce the latency of bus waiting.

5. Case Studies

The 3D waveform oscilloscope displays the time-amplitude information of the signal. Furthermore, it can use the pseudo-color and luminance to present the waveform’s statistical information. Through this method, the 3D waveform takes advantage of capturing incidental events such as burrs [27]. The application’s block diagram is shown in Figure 9.

3D waveforms oscilloscope is built by four commonly used algorithms, string matching, matrix addition, histogram, and contrast stretching. Among these algorithms, string matching and matrix addition belong to the field of digital processing, and histogram and contrast stretching belong to the field of image processing.

Servant1 String Matching: This stage servant matches the trigger pattern in the input sample stream and aligns each waveform segment onto that trigger point to stabilize the afterglow display. The sample stream is treated as a string, which is processed by the Brute Force string matching algorithm. The matched waveform segment is a two-dimensional plot with time as the horizontal axis.

Servant2 Matrix Accumulator: This stage servant stacks a series of 2D waveform segment plots from the previous stage onto a 3D surface map by the matrix addition algorithm.

Servant3 Histogram: This stage servant converts the 3D surface map into a pseudo-color image through the histogram algorithm.

Servant4 Contrast stretching: This stage servant transforms the pseudo-color image into a thermal map using the contrast stretching algorithm, which enhances the dark brightness of low-probability events.

6. Results

6.1. Experimental Setup

The experiments are conducted on an embedded system with FPGA and mobile SoC collaboration. The SoC is Freescale i.MX6Q includes quad-core ARM Cortex-A9 CPU and Vivante GC2000 GPU, with 2-channels 32-bits memory bandwidth. This ARM processor supports the NEON vector instruction set, essentially an implementation of SIMD (Single Instruction/Multiple Data), which supports four 32-bit integers, four single-precision floating-point, and eight 8-bit integers operations within a single instruction [28]. The Xilinx Kintex-7 series FPGA contains four independent 16-bit memory banks. The FPGA board connects to the SoC via a 1-lane PCI-e 2.0 interface.

Four commonly used algorithms, S₁ string matching, S₂ matrix addition, S₃ histogram, and S₄ contrast stretching, are selected as benchmarks. OpenCV library with OpenMP support [29] is the experimental baseline.

6.2. Servants Acceleration

The servants are developed on GPU and FPGA using OpenCL language [30]. FPGA adopts the declarative optimization methods provided by OpenCL. These optimization methods include: UL (Loop Unrolling) and SIMD. UL method unrolls inner loop to increase concurrent execution paths and reduce iterations. ULw represents w loops are transformed into parallel computational units, i.e., w units run on FPGA simultaneously. OpenCL also supports combining v scalar arithmetic operations into a vector operation to be SIMDv.

The measurement results of the throughput of each servant are shown in Figure 10. The mobile GPU achieves 1.9–3.8× speedup over the baseline (traditional CPU). FPGA can change the hardware architecture to meet the characteristics of the algorithms, which is different from GPU. The pipeline in FPGA owns extremely high parallelism of the arithmetic units. For example, the suffix of SIMD in Figure 10, ‘8’ represents that the computing unit on the FPGA processes 8 data simultaneously. By duplicating more computing units, FPGA obtains greater acceleration, while more resources are costed as an expense. Minimum parallelism optimized FPGA achieves 2.7–10.5× speedup over the baseline.

6.3. Heterogeneous Computing Evaluation

FLIA is suitable for the application scenario of the data processing pipeline. According to Amdahl’s Law: The overall performance of a data processing pipeline is determined by the bottleneck stage with the lowest throughput. Because FLIA improves the entire performance through the pipelined parallel, the execution duration of each stage should be as balanced as possible [31]. The throughput advantage of just one stage is not confident of enhancing overall performance.

In this case study 3D waveform oscilloscope, the ADC can sample up to 1G points in a second, and the image processing servants should handle 60 frames per second. Therefore, considering the coordination of modules in this application, the execution frequency of each servant is different. To accomplish these tasks, the execution duration required for each servant is normalized, as Figure 11 shows. For image processing servants (histogram and grayscale stretching), their performance on GPU approximates that on FPGA with SIMD8 optimization. This experiment uses a fixed-point calculation to utilize the DSP arithmetic unit in FPGA [32].

The following experiment is that all servants use only one type of processing resource, CPU, GPU, or FPGA. As Figure 12, FPGA can achieve 22× speedup, but the higher speedup is at the cost of higher parallelism and more resource usage. Then, as a comparison, FPGA and GPU collaborate to implement the same application. There is always an ample design exploration space for the FPGA and GPU collaborating acceleration. For example, these four servants are mapped to FPGA and GPU with combinations of ${\mathrm{C}}_4^1+{\mathrm{C}}_4^2+{\mathrm{C}}_4^3=14$. Automatic scheduling based on the throughput of servants should be future work. To keep the pipeline stages balanced, according to the single servant required execution duration in Figure 11, the data processing algorithms (S₁ string matching, S₂ matrix addition) run on FPGA, and the image processing algorithms (S₃ histogram, S₄ grayscale stretching) run on GPU. It is also reasonable to place adjacent servants in the same accelerator, which can reduce memory duplication. The overhead of copying memory data between heterogeneous accelerators is still significant. An efficient communication mechanism should be future work.

In Figure 12, comparing execution-flow communication implementation, the post-style achieves an average 1.3× speedup over the bus-style. Taking the 16 computing units parallelism on FPGA as an example, collaborated GPU-FPGA achieves 21× and 10.5× performance than the CPU baseline and mobile GPU. The heterogeneous computing approaches the performance of the pure FPGA accelerator, while it only consumes about 15% DSP resource of the pure FPGA accelerator.

6.4. Power Evaluation

In Figure 13, the power increases in the order of CPU, GPU, then FPGA, and the energy consumption is in the reverse turn. The energy consumption of the CPU is about 5 times that of FPGA. The energy efficiency of collaborated GPU-FPGA is slightly lower (<10%) than the pure FPGA.

7. Conclusions

This paper proposes FLIA for the collaboration of mobile GPU and FPGA to accelerate the data processing pipeline. Evaluations are conducted on the case study embedded system, 3D waveform oscilloscope. From the experimental results, the benchmark algorithms are accelerated by 1.9–3.8× and 2.7–10.5× on GPU and FPGA respectively, over the OpenCV baseline. The post-style execution-flow achieves an average 1.3× better performance than bus-style. The embedded heterogeneous computing achieves 21× speedup than the OpenCV baseline. Moreover, the collaborate GPU-FPGA consumes fewer FPGA resources than pure FPGA accelerators, but their performance and energy consumption are approximate.

In the future, the work will focus on the following aspects: (1) Iteratively improve FLIA by researching more practical applications. (2) FLIA should support more accelerators, such as multi-core CPUs. (3) Automatic scheduling. The heterogeneous mapping schemes of servants influence the performance enormously. Exploring huge design spaces ought to be more efficient.