NanoBoot: A Field-Programmable Gate Array/System-on-Chip Hardware Boot Loader for IoT Devices

Abstract

This paper presents a new boot loader scheme for embedded devices with file system support built as a hardware module. The work focuses on improving the boot loader hardware and the possibility of carrying out a full boot-up process from the dedicated on-chip hardware, using a light file system to store an operating system kernel. To do so, the new full-hardware boot loader is integrated into two Field-Programmable Gate Array (FPGA) System-on-Chip (SoC), capable of launching a Linux kernel from a formatted removable media.

1. Introduction

Today, some embedded systems are commonly used as Internet of Things (IoT) devices. These devices usually have limited hardware and software resources. Therefore, it is often necessary to create custom designs that optimize performance and/or area.

A significant part of these devices are based on System-on-Chip designs, which are complete computer systems capable of performing complex tasks. Each SoC includes a power-on reset device, which ensures that the core and peripherals reach a known state before starting the pre-programmed firmware.

According to [1], then firmware can be defined as the first code executed in an embedded system, which is stored in a Non-Volatile Memory (NVM) such as Flash or Read-Only Memory (ROM). Among the objectives of the firmware may be to provide the final standalone application software of the system (like in simple Micro-Controller Units (MCUs)), but in most cases, one of the main tasks of the firmware is to retrieve the application code from an external peripheral.

In embedded systems that run a full operating system, the firmware typically works as a boot loader or fetches the boot loader from some pre-defined peripheral device. According to [1], the boot loader is an application to fetch the OS or a standalone application from an external device. Boot loaders are also used in simple embedded systems that do not run an OS. For instance, MCUs include a boot section in their non-volatile Flash memory, whose main purpose is to support firmware updates.

Boot loader technology is constantly evolving to improve features such as performance [2,3], reliability [4], development effort [5] and portability [6]. In general, the scheme of the boot process in complex SoCs is similar to general-purpose computer systems. Initially, designs lacked the ability to update firmware, as it was stored in ROM. In such systems, only the secondary software loaded by the firmware could be customized.

In this situation, the boot loader code is divided into stages, where the role of each stage is the initialization of some peripherals that are necessary to obtain the next stage from another peripheral. The firmware only implements the first stage of the boot-up process, which is commonly referred to as the preloader. A similar procedure is used in general-purpose computer systems, but currently the ROM-BIOS is upgradeable. The main differences in the boot-up process for MCUs, SoCs, and general-purpose computer systems are summarized in Figure 1.

The majority of embedded systems based on System-on-Chip use Linux as their operating system due to its flexibility and support for a wide range of architectures. The Linux system exposes a set of well-defined system-callable subroutines to manage peripherals. These subroutines are implemented in the OS as specific device drivers for each peripheral. Over these drivers and on peripherals that can work as storage elements, file systems are built. File systems are a fundamental part of the OS and an almost mandatory requirement to run applications in most SoC solutions. The file systems store both the application programs and also the OS components in the form of executable files, libraries, configuration files, etc. Storing these resources in a file system greatly simplifies application and OS updates including most of the boot loader stages.

In order for the boot loader to be able to fetch the kernel, it needs to include support to read the file system type used in the underlying device. Due to the limited resources available to the first stage of the boot loader, only simple and lightweight file systems may be read from this stage. For this reason, the old and limited File Allocation Table (FAT) file system is widely used in embedded computing systems. But FAT has important deficiencies, the most important being as follows: (1) when used on Flash memory devices, the Flash is quickly degraded, since the file metadata table is frequently updated and the same Flash pages are recurrently written; (2) FAT consistency is compromised for unfinished or failed write operations. To overcome the limitations of FAT more capable later stages of the boot loader have to be added that support more robust and complex file systems.

The current performance of Flash memory has driven SoC designs to adopt it as the main mass-storage technology. Traditional limitations of Flash memory have been overcome by the addition of the so-called Flash Translation Layer (FLT). FTL is a hardware/software driver that makes Flash memory linear, appearing to the system like a disk drive that emulates a block device [7]. FTLs are present in most commercial Flash mass-storage devices [8].

This paper presents NanoBoot, a hardware boot loader which is capable of booting an OS (Linux) or a standalone application from Flash removable media (SD card) using a lightweight file system for resource-restricted IoT devices, which are modeled in this paper as SoCs implemented in Field-Programmable Gate Array chips (FPGA/SOCs). The main goal to achieve with NanoBoot is to develop a fast, full hardware, generic and open source boot loader with a small footprint capable of booting an SO or standalone application. Linux was chosen because it has become the most used OS in SoC systems, and SD cards are an adequate solution to hold the system software in an SoC because of its low cost, high availability, and easiness to provide updates and fixes along the lifetime of the product. Nevertheless, many ideas and contributions in this paper may be extended to other operating systems or storage devices. The core of NanoBoot is the NanoFS file system introduced in [9] and now extended in this work. NanoFS is a lightweight file system that is suitable for implementation in hardware due to its small footprint.

The paper is organized as follows: Section 2 includes some background about booting Linux on SoCs and the role of file systems in the boot-up process; Section 3 describes the design and internal layout of the NanoFS file system in detail; the design of NanoBoot is developed in Section 4; Section 5 details the main results of the proposed approach and compares it to the standard solutions. Conclusions are summarized in Section 6.

2. Background

This paper focuses on booting a Linux system from removable Flash media. The following subsections outline some background about the different technologies involved, such as Flash media storage, FPGA SoCs, and the role of file systems in the Linux boot process.

2.1. SoC Booting from Flash Media

In modern SoCs, most firmware preloaders include support for a removable storage setup mode that permits obtaining the first boot loader stage from sources such as NAND Flash, SD/TF Card, eMMC, NOR Flash, USB, etc. The most common type of removable Flash storage is NAND-based, which has certain limitations. These include large page data, asymmetric read/write times, and low write-cycle endurance. Despite the limitations, NAND Flash mass storage reduces the cost of the embedded system. Currently, most NAND Flash is manufactured with an FTL that assumes the use of a file system. Therefore, most devices include FTL features such as garbage collection and a fine-grained address translation layer to efficiently manage the Flash memory [8,10,11]. Thus, today, standard consumer-grade Flash devices are ready to drive file systems with acceptable endurance and have adequate performance to boot up an OS and act as the main storage for the OS at run-time. These devices are optimized to be used with existing file systems (e.g., FAT, ext4, NTFS, exFAT, etc.) without any modifications. In this way, the Secure Digital (SD) standard [12] of the SD Association is one of the most widely used storage devices in general consumer electronics. SD cards are supported by many SoC boot loaders and have great compatibility with desktop computers, allowing the user to update or modify the system’s software and/or data in a friendly way.

For testing purposes, FPGA devices are a good choice because they are flexible and allow custom hardware and software development. Most FPGA vendors include ARM cores in their high-end products, offering a powerful and very flexible SoC platform for the development of embedded systems. There is also a range of processors that can be used on FPGA chips as soft cores: RISC-V [13], Nios [14], Microblaze [15], OpenRisc [16], Leon, OpenSparc, LowRisc, etc. Some of these processors are distributed under an open hardware license and can be synthesized on FPGA chips from different manufacturers. Since custom hardware can be included in FPGA/SOCs, these platforms offer new options for boot loader design and optimization, especially when the SoC is based on a soft-core processor. There exists the possibility to include boot-loading-specific hardware or to develop a full-hardware boot-loading solution, which is proposed later in this paper.

For this work, two soft-core processors have been selected to exemplify the design and implementation of a full-hardware boot-loading solution, OpenRisc and Microblaze, since both are 32-bit microprocessors capable of running Linux. OpenRISC is an open source microprocessor developed by the OpenCores community [17], implemented using a GCC-based toolchain and written in Verilog HDL synthesizable code. Its main characteristics are as follows: 32/64-bit load/store Reduced Instruction Set Computer (RISC) architecture with 32-bit uniform-length instruction format; virtual memory support through a Memory Management Unit (MMU); internal Wishbone B3 bus [18], which is an open specification. Instead, MicroBlaze is a soft-core processor designed by AMD Inc. and distributed as Intellectual Property Core (IP Core) under a proprietary license. It is a 32-bit RISC Harvard architecture processor, and the main difference compared to OpenRISC is the internal Advanced eXtensible Interface 4 (AXI4) [19] bus used. Both microprocessors can be synthesized on mid-range AMD devices, leaving plenty of spare FPGA resources to add extra internal peripherals needed to support the Linux boot-up process from removable media. FPGA/SOC booting will be discussed in detail in Section 4.

2.2. Embedded Linux

Despite the available alternatives, Linux is the preferred OS for SoC development. Although Linux was originally created to run on general-purpose computers, it is extremely flexible and configurable and supports a very wide range of computer architectures [20], making it ideal for custom embedded computing.

Linux for embedded systems has some specific features compared to Linux for general-purpose computers, especially related to the boot-up process. Embedded Linux covers a wide range of devices starting at small embedded systems with about 4 MB of ROM and 8MB of RAM [20] and a variety of custom boot loaders are available for these kinds of device.

The Linux boot-up process for both general-purpose computers and embedded systems is divided into three general stages [21]: boot loader, kernel startup, user space initialization. The boot loader stage is fully dependent on the platform, and although a custom boot loader is needed for each type of hardware, once the process reaches the kernel startup stage, the boot-up process is similar on all platforms. The Linux boot loader stage for current SoCs is thoroughly described in [22] and, despite some differences, its final target always consists of launching an executable Linux kernel image that was previously fetched from an external peripheral and copied to RAM. The differences among different SoC boot loaders are in firmware capabilities and limitations. For instance, the Linux kernel image is compressed in some configurations to reduce the fetch time, and other configurations add an extra RAM disk image that is also loaded before the kernel is launched. Both decompression and RAM disk loading are extra tasks carried out by the firmware. Moreover, in a removable media setup mode, the boot loader must also include the driver to access the external massive storage, increasing the firmware footprint.

Linux boot performance is highly platform-dependent, but the Linux boot process is flexible and can be optimized with multiple techniques, as introduced in [23,24,25]. In these works, the boot time is reduced by testing multiple platforms and scenarios with different configurations, but the optimization heavily depends on the scheme used, with or without RAM disk. For both boot-up schemes, the boot loader stage is divided into multiple sub-stages heavily dependent on the hardware platform, but the Linux community tries to reuse boot loader code pieces while making boot loaders compatible with multiple SoCs. Despite the variety of SoCs and custom boot loaders, Linux has achieved compatibility with a variety of platforms, continuously decreasing the number of custom boot loaders and taking preference for generic and open solutions such as U-Boot and Redboot [20], which are the preferred solutions for embedded systems because they minimize hardware dependency by reusing a generic first stage where it is only necessary to adapt a small piece of code for each platform.

Currently, Das U-Boot is the most widely used solution, since, like Linux, it is open software supported by a community and is compatible with a variety of ASIC-based SoCs, Single-Board Computers (SBCs) and FPGA/SOCs. Also, it supports development boards with microprocessors like ARM, AVR32, Blackfin, x86, Motorola 68K, AMD Microblaze, MIPS, Altera Nios, Nios2, PowerPC, Super-H, etc., and it is a good starting point for new boot loader developments [26].

All Linux boot loaders support file systems in some sub-stage of the process, since the Linux kernel image is size-variable and is frequently updated along the device’s lifetime. In addition, the availability of the file system improves the update capabilities of the system. Das U-Boot follows a standard Linux boot-up process that expects a kernel stored as a regular file in a file system. Thus, Das U-boot supports all modern file systems included in current Linux kernels.

2.3. File Systems in Embedded Linux

The Linux user space initialization stage is started when a large file system with the user data and applications is mounted, though sometimes it is more complex since multiple file systems may be mounted across the file system hierarchy. In Linux distributions, the kernel image and the RAM disk, if present, are stored in a file system. In this way, the kernel can be replaced or updated by writing or overwriting these files during the Linux run-time. Moreover, in removable setup mode, the kernel can be easily changed from an external desktop computer, but the file system makes the boot loader code complex since it must search for and read the files from a file system at boot time as has already been mentioned. Thus, the performance of the boot loader is heavily dependent on the ability to manage the file system efficiently, and for this reason, the file system selected for the boot process is of the utmost importance to improve the performance of the Linux boot process [24]. However, it is not always possible to change the initial file system used by the boot loader. Some SoC manufacturers integrate file system support into the on-chip firmware. This increases the firmware size since two levels of software drivers must be included: storage medium drivers (SD card, eMMC, etc.) and file system drivers. The size limitations of the firmware make it not possible to support complex file systems like ext4, btrfs and others, or use advanced features like journaling, snapshots, compression, etc. Other SoCs, choose to include only the medium access drivers, accessing the storage medium in raw mode, and leaving file system support for a secondary customizable stage.

In boot loaders, file systems designed without complex data structures are the best option. Therefore, there are some file systems with a simple layout whose implementation has a low software footprint aimed to be used by Linux in the boot-up process, like Read-Only File System (ROMFS) or Compressed ROM File System (CRAMFS). They are valid for embedded systems, but they have some limitations, such as the size of the file system and the size of individual files. In fact, ROMFS is very limited and can only be used for very small kernels and small initial RAM disks. Unlike, CRAMFS (developed by Linus Torvald) is more advanced, including features like EXIP for executable files, while keeping it simple. However, these types of file systems are not applicable to all kinds of embedded systems because any change in the contents, such as a driver or the kernel, requires a full file system image regeneration. With these types of file systems, an upgrade to the kernel or some boot loader stage requires two complex steps: (1) a full file system image regeneration and (2) a replacement of the old image safely. The second step is critical since any failure makes the system unbootable. Thus, it is very common to find firmware with FAT16/FAT32 support mainly for two reasons: FAT has a simple internal layout despite it being an old design and, due to its ubiquity in most computer systems, facilitates the sharing of data across their devices.

As already explained, the removable setup mode is the preferred configuration for general-purpose SBCs and for SoC-based designs in general, with SD cards being the current preferred medium. Most SBC Linux distributions store the kernel in a file system that can be accessed in read/write mode by Linux itself, mounted in the /boot directory. The main reason for this configuration is to make it easy to change or upgrade the OS using a desktop computer, even auto-update it from the running OS. Custom designs such as those implemented in FPGA/SOCs can be fully customized to support different booting strategies, like booting Linux from SD cards in a single boot stage.

In any scenario, the boot process will benefit from having the kernel and a possible RAM disk image in a file system that is easy to handle from a resource-limited firmware code, for this task, the NanoFS has been chosen due to being specifically designed for that.

3. Nano File System

NanoFS is a light file system aimed at being used with reduced software or directly embedded in hardware due to its small footprint. It is designed with simple internal data structures to be handled with low-memory resources and has an optimized internal layout to reduce file lookup and file read algorithms. NanoFS is a good choice for the first boot loader stage, even for the kernel fetch stage, since only read operations are used. The internal structure of this file system facilitates the construction of a low-footprint hardware module that can replace the internal firmware for reading files from a NanoFS. Hence, it is a suitable option for use in IoT devices where minimizing area and resources to implement a hardware boot loader is crucial.

At the user level, it is similar to other file systems where the files are organized in a hierarchical structure building a directory tree. Also, at the physical level, it is quite similar to other file systems that use the logical block addressing scheme, which consists of dividing the available space into fixed-size blocks called logical block (l-block), but in the internal data structures, NanoFS is quite different from other file systems like FAT or Extended series. The main design differences are summarized as follows:

• Small l-block size: 1 byte and 512 bytes.
• Reduced types of l-blocks named: dir node, data node and metadata node.
• Decentralized file system data structure to minimize implementation resources.
• Internal layout structured around forward lists to simplify file system algorithms implementations.

The following subsections detail the internal layout of NanoFS that is necessary to understand the simplified algorithms for NanoFS operations, such as file lookup and file reading.

3.1. NanoFS Layout

Like other file systems, NanoFS divides the storage device into an array of l-blocks. All l-blocks are referenced by a 32-bit unsigned integer number; thus, the file system is limited to 4G l-blocks. The size of the l-block is set when the file system is created, taking the values shown in

Table 1. NanoFS valid block sizes.

s_Blocksize	Block Size	File System Size Limit
0	1 byte	4 G bytes
1	512 bytes	2 T bytes
2	1024 bytes	4 T bytes
3	4096 bytes	16 T bytes

. Valid block size values were chosen to achieve the best performance according to the storage technology used. The relationship between physical block and logical block size directly impacts file system performance; thus, a primary rule for most file systems is to match their sizes or at least to use a multiple number.

The size of the l-block also impacts the complexity of the software required to manage the file system. In this sense, NanoFS allows small l-block sizes like 1 byte or 512 bytes, designed mainly to be used in devices with limited resources. The 1-byte size is proposed to be used for on-board ROM memories due to minimize the wasted space by the file system’s internal structures. The second fits with the SD card protocol block size and is intended to optimize the temporal data used in the file system driver for this storage media.

To reduce the resources to manage the file system, the NanoFS specification includes a small amount of data structures stored in Little-Endian format: superblock, dir node, data node and metadata node. In block 0, the super-block record is located that contains key information to manage the file system. The later l-blocks contain data structures called node with the following characteristics:

• Each node is referenced by a unique 32-bit unsigned integer called node number. It corresponds to the l-block where the node starts.
• All l-blocks are part of some node. This is required to preserve the file system integrity.
• The file system nodes can be of three types: dir node, data node and metadata node.
• Dir nodes are intended to compose the file system hierarchy.
• Data nodes stores the user data of a file and a header with file system internal data. In this way, the data node header is used to navigate to the next allocated data node for the current file.
• The data node is size-variable and consists of single or multiple consecutive l-blocks. It can be extended to consecutive l-blocks to improve the sequential file read/write performance and to reduce the file system fragmentation.
• Metadata nodes are optional, reducing the file system weight if they are not included.

A NanoFS formatted device is built over two lists of nodes starting at the superblock. Both are forward lists referenced by the node numbers called the data allocated nodes and free nodes. The whole file system is a forward list that can be navigated keeping only a single file system handle, the current node number. Each node always contains a record with a node number referencing the next node in the file system structure. In addition, the dir nodes and data nodes structures contain the records detailed in

Table 2. Records in the dir node data structure.

Offset	Size	Record	Description
(Bytes)	(Bytes)
0	1	d_flags	Dir node flags
1	4	d_next_ptr	Node of next dir node
5	4	d_data_ptr	Node of first child node
9	4	d_meta_ptr	Node of first metadata node
13	1	d_fname_len	Length in bytes of filename
14	256	f_name	Name of file
270	34	f_meta	Standard metadata

and

Table 3. Records in the data node data structure.

Offset	Size	Record	Description
(Bytes)	(Bytes)
0	4	d_next_ptr	Node of next data node
4	4	d_len	Data length
8	d_len	d_data	Data

, respectively.

Figure 2 contains an example of a NanoFS formatted device using a 512-byte l-block size. The example has a directory structure that starts in the root directory with one subdirectory and four files (Figure 2a). Additionally, it enumerates the sequence of l-blocks along the device highlighting the dir nodes. Starting at superblock (l-block 0), the file system has the two lists referenced by its node numbers: s_alloc_ptr references the list of data allocated nodes (l-block 1) and s_free_ptr references the free nodes list (l-block 13). The file system hierarchy begins at the root dir node (l-block 1) and is a list of dir nodes linked by two records: d_next_ptr and d_data_ptr. The d_next_ptr record always addresses the next dir node in the current directory until the last dir node is reached; in the last case, the value is 0 (end mark).

The d_data_ptr record can point to another dir node or to a data node. To distinguish it, each dir node is flagged in the d_flags record as a file or directory. When a dir node is a file, d_data_ptr addresses the first data node of the file. Instead, whether it is flagged as a directory, it represents a subdirectory and d_data_ptr points to the dir node where the subdirectory starts. The files are stored in data nodes that contains a small header followed by the file data, as is detailed in Table 3. A file consists of a single or multiple data nodes organized in a forward list and a data node can be extended to multiple consecutive l-blocks.

Figure 2a has an example of two files with allocated data: f1 and f2. The d_data_ptr records point to node 9 (l-block 9) and node 5 (l-block 5), respectively. Figure 2b details the nodes for both files, where f2 is a fragmented file in two nodes (5 and 7) and each node fits with a single l-block. Instead, f1 is an unfragmented file with a single node (9) that is expanded through four consecutive l-blocks (from 9 to 12). For large files, read/write operations can be sped up because data are allocated in consecutive physical blocks, especially for storage technologies with accelerated sequential access modes support, such as CMD8 protocol command on SD cards.

NanoFS manages the free nodes in another forward list of data nodes referenced from the superblock. In Figure 2, the free nodes start at node 13 (l-block 13), referenced by the s_free_ptr record of the superblock, and it has only one expanded data node (l-blocks 13 to 16). Finally, the intended purpose of the metadata nodes list is to extend the file system with custom features if a specific application requires them. Consequently, they are not mandatory and are not covered in this work.

3.2. NanoFS Implementations

The complexity and resources required for the implementation of file system operations depend on the internal layout, internal data structures, and the storage media used. The relationship between the file system l-block size not only affects the performance, but there is also a direct relationship between the resources required for the read/write algorithms. There are two recommended rules that are generally accepted to achieve optimal behavior: (1) set the l-block size to a multiple of the physical block size, (2) set the start offset of the file system to be aligned with a physical block, especially for devices with a large physical block size, such as Flash memory.

Referring to SD cards, they work with a serial bit stream protocol in 1-bit or 4-bit transference mode. The SD protocol also includes a 1-bit SPI mode, which is a reduced subset of the protocol aimed at supporting limited host devices. In both modes, the protocol assumes a 512-byte block size for data transference; therefore, the SD optimal l-block size for a partition formatted with NanoFS is this size. When matching the l-block size, the procedures to manage NanoFS are optimal, with the result of a simplified algorithm that is able to run with very-low-memory resources.

In this way, Algorithms 1 and 2 depict a possible implementation for file lookup and file read procedures. The file lookup procedure can be implemented using a single handle that is the current node number (a 32-bit unsigned integer). It can be seen in Algorithm 1, which describes a non-recursive file lookup starting at the root directory. The file lookup ends once the file is located; after that, the d_data_ptr record has the dir node number of the file, which will be used later as a handle to read the file data in the read procedure.

Algorithm 1 Lookup file procedure

Algorithm 2 Read file procedure

Hence, the Algorithm 2 depicts the read procedure taking the previous handle (d_data_ptr) and needs two additional 32-bit numbers to read the file content. Both procedures call the get_bytes function taking as the second argument N, returning only N bytes from the storage media. This function is optimized for the SD protocol since the read operation captures the data from the input stream, and a memory buffer of 512 bytes is not necessary.

Though write support is not required for boot loaders, it is used to prepare NanoFS partitions on external storage media. For Linux, the available NanoFS version is implemented using the Filesystem in Userspace (FUSE) [27] interface and supports read/write operations. The NanoFS fuse driver is intended to configure the SD cards used in the boot loaders tested in the following sections.

4. NanoBoot Implementation

This section describes the implementation of NanoBoot and analyzes how the use of NanoFS simplifies the complexity of the boot loader when applied to booting FPGA/SOCs. The objective is to minimize the number of stages and the complexity of booting a Linux kernel from a NanoFS partition. Since the hardware in FPGA/SOCs can be customized, it can be used to implement the boot loader in hardware. It will boot a Linux kernel in a single stage that will run on a soft-core processor. The SD card removable media is used as a convenient storage device as explained in previous sections.

The selected testing environments are two full FPGA/SOCs: OpenRISC 1200 and Microblaze 9.6. Both are intended to be implemented into AMD FPGA chips and deployed on prototype boards with at least 128MB of DDR2 off-chip memory, which is enough to run an OS such as Linux. The implementation results are presented later in Section 5.1, while the NanoBoot design is fully detailed in this section. The hardware is developed in SystemVerilog Hardware Description Language (HDL), and the source code is available on the authors’ Github [28].

The OpenRISC SoC in Figure 3 shows a block diagram with the complete proposed SoC to test the new boot loader procedure. It includes the processor and some controllers for off-chip peripherals that connect to a Wishbone bus. Wishbone defines a master-slave interface for connected devices. In this configuration, the OpenRISC processor and the Boot Loader Unit work as masters and the DDR controller has a slave role. When the boot process is triggered, the Boot Loader Unit halts the microprocessor (asserting its reset signal) and becomes the master of the bus in order to use the DDR controller in exclusive mode. Then, the boot loader reads the Linux kernel from the SD card and copies it to a specific address in the DDR memory. After that, the Boot loader Unit de-asserts the reset signal of the microprocessor and goes into an idle state while the processor starts executing the OS kernel.

The boot loader process depends on the hardware architecture, the included peripherals, and the behavior of the processor after a reset. Despite this, the proposed SoC with Microblaze is very similar to the one presented above, and a minimal set of changes is needed to support the new processors. Although both processors have the same peripherals, some changes are necessary: the internal bus used with Microblaze is AXI4, and both microprocessors must be reconfigured to set the program counter register (PC) to point to DDR memory after reset. Concerning the DDR controller, it was generated by the Memory Interface Generator (MIG) tool included in the Vivado 2019.1 suite and is compatible with the AXI4 bus protocol used in the MicroBlaze architecture. The same DDR controller is used in OpenRISC with a minor change that consists of adding a bridge with some glue logic.

Another peripheral added to the SoCs is an SD controller (SDHC-SPI Unit) [29] which is used by the Boot Loader Unit to boot up from a removable media. This unit is a light SD card host controller that implements part of the SD-SPI protocol specified in [12] since the boot loader only performs read operations. The Boot Loader Unit assumes that the SD card is prepared using a classic Master Boot Record (MBR) with the first partition formatted with NanoFS.

Also, for debugging purposes, some LEDs on the development board were connected to the boot loader status register and an RS232 controller was included to obtain a Linux terminal and check whether the boot process was successful.

Figure 4 shows the block diagram of the Boot Loader Unit. Internally, it is designed using multiple Finite State Machine (FSM) described using the Verilog HDL. The Boot Loader Unit divides the process into four steps:

• Check the MBR and obtain the address of the first partition.
• Check the first partition for a valid NanoFS file system.
• Look up the file vmlinuz.
• Copy vmlinuz to DDR memory.

In this way, the design is split into the four FSMs depicted in the figure: MBR Lookup Unit, NanoFS Init Unit, NanoFS Lookup Unit and Kernel Fetch Unit. The four FSMs are governed by the Main FSM which starts each one sequentially and covers the full boot loader process. FSMs are controlled by a handshake protocol with three signals connected to the Main FSM unit. Each step is triggered when its signal start_* is asserted and ends when its output signals are asserted: the finish_* signal on success; or the error_* signal when error. The error_* signals are captured in a status register to check for possible errors during the process. Internally, the FSMs only need a reduced set of resources: a few 32-bit registers for file system handlers and a small additional ROM to store the file name looked up (vmlinuz).

Figure 5 presents a flow diagram of the Main FSM including the main signals that govern the other four FSMs. Additionally, a flow diagrams for the MBR Lookup Unit, NanoFS Init Unit, NanoFS Lookup Unit and Kernel Fetch Unit are depicted in Figure 6 and Figure 7. As well, in the next paragraphs, the design and internal operation of the Boot Loader Unit for OpenRISC are described in detail, followed by the necessary modifications to use it with Microblaze.

The OpenRISC boot process is detailed as follows:

• As depicted in Figure 5, the boot process starts after the external signal start is asserted. Then the Boot Loader Unit asserts the boot_rst and start_mbr signals. The first signal stops the microprocessor which remains halted until the last step; hence, the Main FSM handles the boot process. The second starts the FSM MBR Lookup Unit.
• The MBR Lookup Unit performs two tasks: it initializes the SD card controller and reads block 0 of the SD card to analyze the MBR. The MBR is valid when it has the right 16-bit signature value at address 1FEh. In that case, the first partition offset is the 16-bit value located at offset 1C6h. This last value is saved in a register to be used in the next stage to access a NanoFS-formatted partition. Figure 6a shows the FSM functional flow diagram for this unit which saves the first partition offset in a register called partition.
• The next step is implemented in the NanoFS Init Unit and described in Figure 6b. This FSM checks the first partition for a valid NanoFS file system. It reads the NanoFS superblock at a partition offset of 0 to compare the magic number (614Eh) and, when successful, it obtains the file system handler of the root directory. The file system handler is a 32-bit value stored at offset 04h, which will be used in the next stage.
• The third FSM is the NanoFS Lookup Unit and it internally implements the Algorithm 1 that looks up the file vmlinuz along the root directory of the file system. As can be seen in the flow diagram of Figure 7a, this unit starts at the root directory and navigates through the directory structure until the file is located. When this stage is successfully completed, the file system handler points to the start of the file and it will be used in the next step.
• The last step is the Kernel Fetch Unit FSM that implements the Algorithm 2 to transfer the vmlinuz file directly to DDR memory through the Memory Bus Adapter. This adapter acts as a Wishbone bus master device, taking control of the DDR controller until the kernel is fully transferred. The flow diagram of this unit is depicted in Figure 7b.
• Finally, once the vmlinuz file has been copied to DDR memory, the microprocessor’s reset signal is de-asserted and the execution starts at the reset interruption vector, launching the kernel.

The last FSM (Kernel Fetch Unit) takes control of the microprocessor bus to write to the DDR memory through the Bus Adapter component highlighted in Figure 4. In general, for each SoC, only the Bus Adapter module must be adapted because the differences between the architectures are mainly in the internal bus. For Microblaze, the Boot Loader Unit is reused with minimal changes, which consists mainly of defining the bus control signals and conversion of the byte order from Big-Endian to Little-Endian due to the AXI4/AXI4-Lite [19] bus used by Microblaze.

5. Results

This section includes the resource utilization for the OpenRISC and Microblaze FPGA/SOC implementations described in the previous section. It also presents a functional and performance test of the NanoBoot implemented in both FPGA/SOCs.

5.1. NanoBoot Resources Implementation Results

The implementation of the two FPGA/SOCs presented in Section 4 has been tested on the prototype boards shown in

Table 4. Prototype boards used to test PSoCs.

Name	FPGA Chip	RAM Type	RAM Size
Arty	Artix-7 (XC7A35T)	DDR3	256 MB
Nexys4 DDR	Artix-7 (XC7A100T)	DDR2	128 MB
Genesys 2	Kintex-7 (XC7K325T)	DDR3	1 GB

. All boards include an AMD FPGA chip, off-chip DDR memory, and several peripherals. Nexys4 DDR [30] and Arty [31] are boards from Digilent Inc. (Pullman, WA, USA) with a mid-grade FPGA chip: XC7A35T and XC7A100T [32], respectively. Both are intended to prove the feasibility of the proposed design in a scenario with limited resources. Instead, the Genesys2 prototype board [33], also from Digilent Inc., includes a high-grade AMD Kintex-7 (XC7K325T) [32] chip, intended to demonstrate the small footprint of the hardware boot loader when implemented in state-of-the-art devices.

Table 5. OpenRSIC 1200 SoC resource utilization by FPGA.

FPGA	Parts	Slices	Slices	LUTs	Slices
			Reg.		(%)
XC7A35T	Full SoC	3317	8200	10,682	40.70
	NanoBoot	330	837	649	4.05
	DDR Controller	1837	4996	5479	22.54
	OpenRISC	1177	2020	4135	14.44
XC7A100T	Full SoC	3593	7937	9508	22.67
	NanoBoot	315	855	593	1.99
	DDR Controller	1974	4767	5154	12.45
	OpenRISC	1186	1939	3387	7.48
XC7K325T	Full SoC	3871	8252	10,579	7.60
	NanoBoot	320	857	613	0.63
	DDR Controller	2320	5090	6165	4.55
	OpenRISC	1116	1950	3398	2.19

and

Table 6. MicroBlaze 9.6 SoC resource utilization by FPGA.

FPGA	Parts	Slices	Slices	LUTs	Slices
			Reg.		(%)
XC7A35T	Full SoC	3712	10,279	10,482	45.55
	NanoBoot	320	871	604	3.93
	DDR Controller	1555	4049	4711	19.08
	Microblaze	1098	2904	3099	13.47
XC7A100T	Full SoC	5406	14,691	14,587	34.11
	NanoBoot	353	837	752	1.19
	DDR Controller	1346	3051	3859	5.18
	MicroBlaze	1264	3090	3245	4.67
XC7K325T	Full SoC	6084	15,952	16,645	11.94
	NanoBoot	367	858	771	0.72
	DDR Controller	2811	6525	8591	5.52
	Microblaze	1137	2829	3071	2.23

summarize the resource utilization of the entire PSoCs and its main components for the XC7A35T, XC7A100T, and XC7K325T chips. The peripherals and the glue logic at the top level are not included in the tables, hence the difference between the full SoC and the sum of the parts. Microblaze and OpenRISC SoCs have been synthesized for these chips using the AMD Vivado 2019.1 design suite with the following options: Synthesis strategy = Area Optimized-high and Implementation strategy = Flow RunPhysOpt.

It can be seen that the NanoBoot unit takes $4.05\%$ and $3.93\%$ of the FPGA resources in an OpenRISC SoC and a Microblaze SoC, respectively, on a mid-grade chip (XC7A35T), making it adequate for systems with limited hardware resources. On a high-grade chip (XC7K325T), resource utilization has a nearly negligible impact (≈0.7%).

It is interesting to note that the significant resource difference for the DDR controller in Table 5 and Table 6 is because Vivado uses different IP cores for each FPGA chip.

5.2. Boot Loader Verification on OpenRISC and Microblaze FPGA/SOCs

Functional verification consists of booting a Linux kernel in both FPGA/SOCs on the three prototype boards introduced previously in this section. For this purpose, a Linux kernel has been compiled and stored in an SD card partition formatted with NanoFS. The kernel used is Linux 4.4.0 released on 10 January 2016 for OpenRISC and Linux 4.19.0 released on 22 October 2018 for Microblaze. Both have been compiled with default options using a cross-compiler, obtaining a size of 6.2 MB for OpenRISC and 8.3 MB for Microblaze.

The compiled kernel has been copied to a file named vmlinuz in the root directory of a NanoFS partition, where the Boot Loader Unit expects to find it. After the boot process is finished, a Linux console is opened in the serial port included in both SoCs that serves to check the correct system function. The functional verification has been carried out on all chips listed in Table 4 with a wide variety of SD cards.

The time required by the Boot Loader Unit to load the kernel into DDR memory (fetch time) is shown in

Table 7. Linux kernel fetch time (initial configuration).

FPGA	Processor	System Clk (MHz)	SD Clk (MHz)	DDR Clk (MHz)	Fetch Time (s)
XC7A35T	OpenRisc	83	20.75	333.33	3.87
	MicroBlaze	83	20.75	333.33	5.01
XC7A100T	OpenRisc	83	20.75	333.33	3.88
	MicroBlaze	83	20.75	333.33	5.00
XC7K325T	OpenRisc	83	20.75	333.33	3.86
	Microblaze	83	20.75	333.33	4.96

. Time measurements were taken using a logic analyzer connected to two General Purpose InputOutput (GPIO) pins. The first GPIO pin is asserted after a hardware reset and the second when the Boot Loader Unit has copied the kernel to DDR memory. This fetch time adds to the platform’s configuration time, that is, the time elapsed to load the FPGA bitstream from ROM, but this time is approximately two orders of magnitude smaller than the fetch time (≈90 ms for Arty, ≈150 ms for Nexys4 and ≈200 ms for Genesys2) and has not been considered in Table 7. The fetch time has been measured with Class 10 SD cards, but the results shown in Table 7 are only for the Sandisk Ultra UHS-I Class 10 model. Table 7 contains the results with the clock frequencies set to the same value on all SoCs. It makes the fetch time to be similar for the three platforms, showing that it is not dependent on the platform. The fetch time difference between processors is mainly due to the different sizes of the kernel to be loaded.

The fetch time can be improved when the platforms are configured with their optimal clock frequencies. These results are summarized in

Table 8. Linux kernel fetch time (optimal configuration).

FPGA	Processor	System Clk (MHz)	SD Clk (MHz)	DDR Clk (MHz)	Fetch Time (s)
XC7A35T	OpenRisc	83	20.75	333.33	3.87
	MicroBlaze	83	20.75	333.33	5.01
XC7A100T	OpenRisc	100	25	200	3.22
	MicroBlaze	100	25	200	4.17
XC7K325T	OpenRISC	200	25	800	2.60
	Microblaze	200	25	800	3.70

. This configuration improves the performance of the boot loader on the XC7K325T chip: 34% for MicroBlaze and 48% for OpenRISC. Although the DDR speeds are different in different boards, the fetch time depends mainly on the SD card read speed, since the SD-SPI protocol is a serial protocol limited to 25 MHz.

The complete Boot time is the time elapsed until the Linux kernel starts the init process (process 0). It has been measured for the three platforms and the results are shown in

Table 9. Linux kernel boot time.

FPGA	Processor	System Clk (MHz)	Fetch Time (s)	Kernel Boot (s)	Boot Time (s)
XC7A35T	OpenRISC	83	3.9	0.06	3.96
	MicroBlaze	83	5.0	1.1	6.10
XC7A100T	OpenRISC	100	3.2	0.05	3.25
	MicroBlaze	100	4.2	1.0	5.20
XC7K325T	OpenRISC	200	2.6	0.04	2.64
	MicroBlaze	200	3.7	0.8	4.50

. This time is the sum of Fetch time and Kernel boot and is measured by using a logic analyzer that monitors the GPIO pin that is asserted when the init process starts.

Most factory solutions for FPGA/SOCs do not support booting from an SD card, since they are intended to boot from an internal Block RAM (BRAM) or using Joint Test Action Group (JTAG) to transfer the kernel. Thus, the closest to a standard solution is PetaLinux [34] from AMD, which is used as a reference to compare with NanoBoot. Petalinux has been designed to boot a Linux system on a Microblaze processor from an SPI Flash memory. Petalinux is a non-free system, but it uses a conventional boot loader based on U-Boot which is preprogrammed as internal firmware (FPGA BRAM) that acts as a preloader. The Petalinux boot process has multiple stages, starting at the BRAM preloader, which obtains the next stage from an off-chip Flash memory.

A Microblaze system booted using Petalinux with the kernel stored in the on-board SPI Flash memory has been deployed on the three prototype boards. The Petalinux preloader is not able to use any file system; therefore, the kernel is stored in raw mode in the off-chip SPI Flash. The Petalinux kernel used is a Linux 5.15.36 with 10.4 MB size, similar to the one used in the OpenRISC and Microblaze SoCs are used as examples in this paper.

Table 10. Power-on time.

SoC	Boot Time (s)
Arty—OpenRISC	4.0
Arty—Microblaze	6.1
Arty—Petalinux	42.6
Nexys4—OpenRISC	3.3
Nexys4—MicroBlaze	5.2
Nexys4—Petalinux	36.4
Genesys2—OpenRISC	2.6
Genesys2—Microblaze	4.5
Genesys2—Petalinux	51.2

compares boot time with the measures taken using the same GPIO-based method described above, consisting of asserting a GPIO when the kernel launches the init process. The OpenRISC and Microblaze rows correspond to results using the proposed hardware boot loader using NanoFS, while the Petalinux rows correspond to the factory solution using Petalinux.

The results obtained in Table 10 clearly show that the use of a full hardware boot loader with a file system does not have a negative impact on the OS boot time compared to the standard Petalinux AMD solution. Moreover, the boot times of the hardware solutions are much better with the added benefits of booting from a file system.

6. Conclusions

This paper introduces NanoBoot, a novel boot loader for embedded devices that uses the NanoFS light file system on removable SD card media. The simplicity of NanoFS makes it possible to boot a Linux system using a full-hardware boot loader in FPGA/SOCs and offers a convenient and easy procedure for system OS updates.

Full-hardware boot-loading capabilities are demonstrated with OpenRISC and Microblaze FPGA/SOCs. The hardware resources needed by the boot loader are low (about 7% of the whole system) and the boot time is greatly reduced compared to the standard solution using Petalinux on SPI Flash.