Advanced Embedded System Modeling and Simulation in an Open Source RISC-V Virtual Prototype^†

Round 1

Reviewer 1 Report

The paper presents a complete emulation platform based on simulating a complete system based around RISCV processor. The source code is open sourced and the system looks complete.

The paper is well written and clear to understand. There is no true "scientific" novelty even though the paper describes a significant engineering effort of practical interest.

I don't have much to comment on the paper. I would be interested the authors to include answers to the following questions.

- why Lua is selected relative to a python interface?

- Do you offer any processor customization features? i.e., to add custom instructions to the RISCV processor

- Is there any application of this material to relevant courses?

 

Author Response

Thank you for your review.

1. why Lua is selected relative to a python interface?

We added a paragraph to clarify that we mainly chose Lua because of the execution speed and comparatively lesser integration overhead:

--------------------------------- content.tex ---------------------------------
index efea6c1..7c6361c 100644
@@ -418,9 +418,16 @@ For rapid prototyping purposes we also implemented a Lua-scripting interface for
 \section{Rapid Prototyping using Lua Scripting}\label{sec:lua_interface}
 %
-As Lua is a dynamically typed language, the offered interfaces are activated in a \emph{"duck typing"} way.
-If a script is loaded, it is checked whether it implements certain functions that are expected by our framework.
-These can then be used by the configuration to enable/connect the following currently implemented functions:
+To increase the usability of our VP Environment Modeling tool, we added a device scripting engine.
+This allows developers to focus on the actual behavior of devices, without having to understand the whole system, to not have to re-build the framework for each change in a device, and to increase modularity for an easier community-driven library of devices.
+
+Such a scripting engine has to be fast, memory efficient, and easily learnable.
+Without a particular scientific relevance, we chose Lua as the driving scripting language, as it is widely used in games and other applications where execution speed and a low memory footprint is key.
+While the Python language was considered as is being used widely nowadays in more high level applications, its interpreters for C/C++ programs compare rather laborious and (slightly) slow.
+For the interface between Lua and C/C++, we kept the dynamically typed language style, and activate offered interfaces in a \emph{"duck typing"} way.
+This means, if a script is loaded, it is checked whether it implements certain functions that are expected by our framework.
+These can then be used by the configuration mechanism to enable/connect the following currently implemented functions:

2. Do you offer any processor customization features? i.e., to add custom instructions to the RISCV processor

Yes, we do. This was also done in the past, so we added two papers in references. I do not feel strongly about citing them, so if you dislike the change, we can undo this again.

--------------------------------- content.tex ---------------------------------
index 7c6361c..062d3d8 100644
@@ -45,11 +45,12 @@ A key property of VPs is their binary compatibility with the hardware platform,
 Beside a functional validation, VPs also enable design space exploration by evaluating different design decisions early in the design flow.
 
-It provides an extensive feature set, such as support for the 32 and 64 bit RISC-V ISA with all standard instruction set extensions, several operating systems (such as Zephyr and Linux), advanced debugging capabilities  and configurations to create different platforms such as the HiFive1 board from SiFive~\cite{sifive-hifive1-manual}.
+It provides an extensive feature set, such as support for the 32 and 64 bit RISC-V ISA with all standard instruction set extensions, several operating systems (such as Zephyr and Linux), advanced debugging capabilities and configurations to create different platforms such as the HiFive1 board from SiFive~\cite{sifive-hifive1-manual}.
+The main benefit of the RISC-V VP is, however, its ease of expandability: from custom RISC-V instructions with dynamic dataflow analysis extensions~\cite{PieperDAC20} to a symbolic execution engine~\cite{symex-vp}.

3. Is there any application of this material to relevant courses?

Yes, it was originally built for a lecture. As the other Reviewer commented on this topic as well, we added a subsection into the evaluation where we talk about our experiences in teaching with it. Also we made it more clear that other, unrelated, universities use this as well in their lectures.

-------------------------------- content.tex ---------------------------------
index f782a62..ed95780 100644

@@ -94,12 +95,13 @@ Beside the two virtual environments, we have also built the corresponding two ...
-%
-Besides our own positive experience in using our VP platform for teaching lectures on system-level design and virtual prototyping, we are also already aware of another academic group that has leveraged our RISC-V VP infrastructure for teaching an embedded systems lecture with laboratory sessions in the RISC-V context~\cite{vp-bb_being-used}.
+
+
+Besides our own positive experience in using our VP platform for teaching lectures on system-level design and virtual prototyping, we are also already aware of an other academic group that has leveraged our RISC-V VP infrastructure for teaching an embedded systems lecture with laboratory sessions in the RISC-V context~\cite{vp-bb_being-used}. @@ -826,7 +826,7 @@ end/*@\label{lst:oled+spi-end}@*/
 
+\section{Evaluation}
+In this section, we will show some use-cases for our VP Environment by modeling two example environments along with their interacting software (\refsec{case-studies}), give a performance evaluation of different modeling strategies (and comparing to the baseline RISC-V VP, \refsec{performance}), and lastly a short demonstration on how we used it in our own lectures (\refsec{teaching}).

@@ -976,6 +976,7 @@ The overall impact of our approach on execution speed can be observed against a...

+\subsection{Educational Tool for Teaching}
+\label{sec:teaching}
+
+Among others, we offer a system-level design lecture that also covers programming embedded systems.
+During the corona pandemic, there was no possibility for the students to interact with physical prototype boards like the Sifive Hifive1.
+As the students covered implementing their own small VPs, it was easy to show them the principles of the more complex RISC-V VP.
+The students then could use and program our digital version of the Hifive1 board to understand the basic concepts of interrupt handling and how embedded systems interact with their environment.
+As the RISC-V VP can be analyzed using normal software-based debuggers like GDB, we could show the detailed steps of different control flows during runtime, and how software and hardware modules interact between each others.
+The small exercises were laid out in incremental steps to program an interrupt-triggered blinking LED while reacting to button presses.
+One year, the final lectures could be held in person, where the students could test their own programs on real Hifive1 boards supplied by the university.
+
+Overall, we noted that the RISC-V VP with the Environment Model extension, while posing an initial learning curve, was very helpful during remote-teaching and still nice to have in in-person teaching as every student could test and build their programs at home without having to supply real hardware.
+We suppose that it will also be beneficial for more practical-focused embedded programming courses; especially when using hardware that is either too costly/complex to be supplied to every student or hardware that requires special programming devices.

We hope, that this addresses all of your comments. For a complete list of changes, see to the updated, diffed, manuscript.

Author Response File: Author Response.pdf

Reviewer 2 Report

This manuscript presents an extension to model external I/O modules for an open source RISC-V virtual prototype, along with its design and examples. The system has been well designed, and explained properly in the manuscript. Basically the manuscript should be accepted.

Let me point out room for improvement about the structure of the paper. The author refered to the potential of the proposed system in educational use in the introduction section, while discussion from an educational aspect is missing in the subsequent sections. (In fact, at first the reviewer misinterpreted "case-studies" as educational ones, rather than modeling ones.) If the authors are willing to make such a claim, it is desirable to introduce previous studies on computer systems education, compare with them, and describe the details of the authors' own experience. Or, if it simply means furure prospects, it is appropriate to mention it in the conclusion section.

Author Response

Thank you for your review.

Let me point out room for improvement about the structure of the paper. The author refered to the potential of the proposed system in educational use in the introduction section, while discussion from an educational aspect is missing in the subsequent sections. (In fact, at first the reviewer misinterpreted "case-studies" as educational ones, rather than modeling ones.) If the authors are willing to make such a claim, it is desirable to introduce previous studies on computer systems education, compare with them, and describe the details of the authors' own experience. Or, if it simply means furure prospects, it is appropriate to mention it in the conclusion section.

As a similar response came from Reviewer 1, we added a subsection to the evaluation where we talk about our experience with the RISC-V VP and its extension. This is, however, not a full study.
Also, we stated more prominently that an unrelated university uses our system for their lectures, where an actual study can be found.

The following changes were made to address your comment:

-------------------------------- content.tex ---------------------------------
index f782a62..ed95780 100644

@@ -94,12 +95,13 @@ Beside the two virtual environments, we have also built the corresponding two ...
-%
-Besides our own positive experience in using our VP platform for teaching lectures on system-level design and virtual prototyping, we are also already aware of another academic group that has leveraged our RISC-V VP infrastructure for teaching an embedded systems lecture with laboratory sessions in the RISC-V context~\cite{vp-bb_being-used}.
+
+
+Besides our own positive experience in using our VP platform for teaching lectures on system-level design and virtual prototyping, we are also already aware of an other academic group that has leveraged our RISC-V VP infrastructure for teaching an embedded systems lecture with laboratory sessions in the RISC-V context~\cite{vp-bb_being-used}. @@ -826,7 +826,7 @@ end/*@\label{lst:oled+spi-end}@*/
 
+\section{Evaluation}
+In this section, we will show some use-cases for our VP Environment by modeling two example environments along with their interacting software (\refsec{case-studies}), give a performance evaluation of different modeling strategies (and comparing to the baseline RISC-V VP, \refsec{performance}), and lastly a short demonstration on how we used it in our own lectures (\refsec{teaching}).

@@ -976,6 +976,7 @@ The overall impact of our approach on execution speed can be observed against a...

+\subsection{Educational Tool for Teaching}
+\label{sec:teaching}
+
+Among others, we offer a system-level design lecture that also covers programming embedded systems.
+During the corona pandemic, there was no possibility for the students to interact with physical prototype boards like the Sifive Hifive1.
+As the students covered implementing their own small VPs, it was easy to show them the principles of the more complex RISC-V VP.
+The students then could use and program our digital version of the Hifive1 board to understand the basic concepts of interrupt handling and how embedded systems interact with their environment.
+As the RISC-V VP can be analyzed using normal software-based debuggers like GDB, we could show the detailed steps of different control flows during runtime, and how software and hardware modules interact between each others.
+The small exercises were laid out in incremental steps to program an interrupt-triggered blinking LED while reacting to button presses.
+One year, the final lectures could be held in person, where the students could test their own programs on real Hifive1 boards supplied by the university.
+
+Overall, we noted that the RISC-V VP with the Environment Model extension, while posing an initial learning curve, was very helpful during remote-teaching and still nice to have in in-person teaching as every student could test and build their programs at home without having to supply real hardware.
+We suppose that it will also be beneficial for more practical-focused embedded programming courses; especially when using hardware that is either too costly/complex to be supplied to every student or hardware that requires special programming devices.

We hope, that this change is satisfactory. For a full list of changes, please refer to the diffed version of the updated manuscript that we uploaded.

Author Response File: Author Response.pdf