Execute and debug on hardware with FPGA-in-the-loop

4. Execute and debug on hardware with FPGA-in-the-loop#

4.1. Required files#

  • SpaceStudio Project

You will also need an FPGA development board — this tutorial uses a ZedBoard — connected to your computer with a JTAG programming cable and reachable over TCP/IP. No serial terminal emulator is needed: the application’s console is presented directly inside SpaceStudio.

4.2. Introduction#

This tutorial presents SpaceStudio’s FPGA-in-the-loop (FIL) workflow with a simple application. The application will be thoroughly explained in the next section.

FIL builds upon Architecture Implementation, the part of the SpaceStudio environment that implements or synthesizes your system for your preferred Computer Aided-Design (CAD) or Electronic Design Automation (EDA) implementation tool. Architecture Implementation generates the EDA project, runs high-level synthesis on the modules mapped to hardware, produces the bitstream and builds the boot image. FIL goes the extra mile: it also programs the board, boots it, deploys the application, runs it, and brings its console back into SpaceStudio. Building, programming and running therefore become a single launch — the same one you already use to simulate.

Because FIL is driven by a launch configuration, the very same configuration can be started in Run, Debug and Profile mode. In other words, you do not only execute on the board: you can also debug and profile the application while it runs on the board, with real hardware timing rather than a model of it.

This tutorial is divided into two sections. In the first section, the attendee will use the provided architecture and simulate it at the system architecture level. In the concluding section, the attendee will create a new system architecture where the computation modules are mapped to dedicated hardware co-processors, and will execute that architecture on a real FPGA board through the FPGA-in-the-loop workflow.

Important

The EDA and the HLS tools used by FPGA-in-the-loop need to be installed and, if applicable, a valid license for them is required. In this tutorial, the HLS tool used is Vitis HLS, and the EDA tool used is Vivado, both developed by Xilinx. These tools require a license.

4.3. Application#

The application is a 32-bit signed integer calculator. Figure 4.12 presents the application’s flow. In the figure, there are 7 modules, each with one input and one output. In general, the data/command comes from outside the system — over a serial link, for instance; in this tutorial it is produced by the command_generator testbench bundled with the input_reader, both in simulation and on the board. The data/command enters the application at the left of the figure. The input_reader module reads the data/command and forwards it to the controller. The controller is responsible for decoding each operand and operation. The controller enables the proper operation module by sending both operands and then waits for the result. The controller takes the result and sends it to the output_writer, and the result data exits the application flow at the right of the figure. In general, the result is read back on the host computer — with FPGA-in-the-loop, directly in the SpaceStudio console.

Calculator application data flow

Figure 4.12 Calculator Application data flow#

4.3.1. Modules#

4.3.1.1. Input Reader#

4.3.1.1.1. Description#

The input_reader module is responsible for abstracting the communication layer from the computation layer. This enables designers to change I/O peripherals without changing the application. In other words, for this case, it transforms the RS232 communication into a FIFO-based communication.

4.3.1.1.2. Behaviour#

This module polls the UART for data. When data becomes available, it reads the data from the UART and sends the data to the controller through a FIFO.

4.3.1.2. Output Writer#

4.3.1.2.1. Description#

The output_writer module is responsible for abstracting the communication layer from the computation layer. This enables designers to change I/O peripherals without changing the application. In other words, for this case, it transforms a FIFO-based communication into an RS232 communication.

4.3.1.2.2. Behaviour#

This module waits for data sent by the controller. When no data is present in the FIFO, this module goes into a dormant state until data becomes available in the FIFO. When data is inserted into the FIFO by the controller, this module wakes up from an interrupt launched by the FIFO. It then reads the data from the FIFO and sends it to the UART.

4.3.1.3. Controller#

4.3.1.3.1. Description#

The controller is responsible for decoding each operand and the operation from the string obtained by the input_reader. The controller will only process valid strings. It forwards the execution of the operation to the proper module and waits for the result. The result is sent to the output_writer.

4.3.1.3.2. Behaviour#

This module waits for data sent by the input_reader. When no data is present in the FIFO, this module goes into a dormant state until data becomes available in the FIFO. When data is inserted into the FIFO by the input_reader, this module wakes up from an interrupt launched by the FIFO. It then reads the data from the FIFO. The input data is stored into an array of fixed length. If the input data overflows the array, a message is thrown. The controller stops storing data into the array when a line feed or carriage return character is detected. This array is then evaluated for correctness. If the input string is invalid, a message is thrown. If the input string is valid, the controller decodes each operand and the operation. The controller sends both operands to the proper module (adder, subtractor, multiplier or divider) and waits for the result. When the result becomes available, the controller forwards the result to the output_writer.

4.3.1.4. Operation modules#

4.3.1.4.1. Description#

The four operation modules — adder, subtractor, multiplier and divider — each perform one arithmetic operation on two operands and return the result to the controller. They differ only in the operation they apply and in the operator characters that select them:

Table 4.22 Operation modules#

Module

Operation

Operator character(s)

adder

Addition

+

subtractor

Subtraction

-

multiplier

Multiplication

*, x, X

divider

Division

/

Because these four modules share one simple, self-contained and stateless behaviour, they are natural candidates for hardware mapping — which is exactly what Part 2 of this tutorial does with adder and divider.

4.3.1.4.2. Behaviour#

All four modules share the same behaviour. The module waits for data sent by the controller. When no data is present in the FIFO, the module goes into a dormant state until data becomes available in the FIFO. When data is inserted into the FIFO by the controller, the module reads the data from the FIFO. The data read from the FIFO are the operands: for each operation, there are two operands in the FIFO. When both operands are read, the module performs its operation and returns the result to the controller via another FIFO.

4.3.2. Input string#

The input string is the character string fed to the system by the input_reader. That string designates the command to be executed. A command is formed by two operands and one operation.

4.3.2.1. Supported commands#

The fully supported commands have the following structure :

<command> ::= <number><operator><number><new-line>
<operator> ::= "+" | "-" | "*" | "x" | "X" | "/"
<new-line> ::= CR | LF

Where <number> represents a 32-bit signed integer (e.g. -42). The tokens may also be separated by space characters.

4.3.2.2. Simplified supported command#

Even though the command supports spaces, that does not mean it has to contain spaces. Furthermore, since the generated commands already end with a new line character, the command can be simplified. Such simplification gives the following structure:

<command> ::= <number><operator><number>

Valid examples :

  • 4x2

  • -5+3

  • -6--4

Invalid examples (yielding an "Invalid input string" message) :

  • 4x--2

  • -5+3a

  • hello+world

4.3.2.3. Error messages#

When a command cannot be executed, the controller discards the accumulated input and reports one of the following messages:

Message

Cause

Invalid input string

A complete line was received, but it does not contain a valid operator and two valid operands.

Division by zero detected

The command is well-formed, but it is a division whose second operand is 0.

String too long, try again

The input filled the fixed-length input buffer before any end-of-line character was received.

In every case the controller simply resets its input buffer and waits for a new command, so the application keeps running.

4.4. Manipulation - Part 1#

4.4.1. Running the simulation#

  1. Open the application by double clicking the Calculator.spacestudio file.

  2. Open the microblaze_all_sw architecture by double clicking the microblaze_all_sw.diagram file from the Project Explorer.

  3. Configure the memory for the application’s requirements:

    1. From the diagram, select the microblaze_soc0 instance by clicking on it.

    2. Open the Properties tab and reveal the Parameter category.

    3. Make sure that the property Memory size is set to 64K.

    Important

    Correctly configure the BRAM size because the Architecture Implementation will use this setting. If you set this value much too high, the mapping process may fail due to over-utilization of resources.

  4. Configure the task’s stack size:

    1. Open the implementation file (i.e., .cpp file) for each module.

    2. In the constructor of the class, make sure 1024 bytes is requested as stack in the request_stack() function call.

  5. Launch the simulation:

    1. From the tool menu, click on the Run icon_execute button.

    2. The following messages should appear:

      [sw] Software is initializing ...
[sw] Software initialization done !

***LOTS OF OPERATIONS***

Simulation has ended @0.116307 s
Simulation wall clock time: 8 seconds.

[Completed]

4.4.1.1. What just happened ?#

SpaceStudio has run the calculator application based on the microblaze_all_sw architecture. The user did not have to feed any input since the data (operands and operators) came from the command_generator.cpp (imports of input_reader module) that acts like a testbench for the application.

The same testbench drives the application when you validate the design on the FPGA board with FPGA-in-the-loop: the commands are generated on the target itself, and the results are printed in the SpaceStudio console instead of the simulation console — which makes the two runs directly comparable.

4.5. Manipulation - Part 2#

4.5.1. Create a new architecture#

  1. Click on Solution

  2. In the drop-down menu, click on New Architecture…

  3. Enter zynq_add_div for the name of this new architecture.

  4. Click on OK.

  5. The diagram zynq_add_div is opened.

  6. In the Palette panel on the right, click on zynq in the SoC category, then drag and drop it into the diagram.

  7. Drag and drop modules controller, input_reader, multiplier, output_writer and subtractor from the Palette to the zynq0 instance so they are mapped in software (they will stay green).

  8. Drag and drop modules adder and divider from the Palette to empty areas in the diagram so they are mapped in hardware (they will become red).

Note

adder and divider are mapped to hardware while the other modules stay in software. The input_reader, output_writer and controller modules are tied to the UART and to string parsing, which is control-oriented code best left on the processor. The operation modules, by contrast, are small and self-contained, so they are the ones worth accelerating. Note that you do not have to modify a single line of module code to change this mapping — SpaceStudio generates the required interfaces and drivers for you.

4.5.2. Running on hardware with FPGA-in-the-loop#

4.5.2.1. Configure the EDA and HLS tools#

Note

This configuration only needs to be done once per EDA/HLS toolchain after having installed SpaceStudio. This will enable the EDA and HLS tools and specify to SpaceStudio where they are installed.

FPGA-in-the-loop uses Electronic Design Automation (EDA) primitives so it is important that the target EDA tool must be configured accordingly. To do so, follow the steps below:

  1. Click on Tools from the menu item

  2. In the drop-down menu, click on Preferences…

  3. Expand the path SpaceStudio > EDA > Xilinx - Vivado 2025.2

  4. Make sure that the EDA is enabled checkbox (which enables the use of Vivado) and High-level synthesis is enabled checkbox (which enables the use of Vitis HLS) are checked, and that the Xilinx Vivado installation directory is correctly configured.

  5. Click Apply and Close

4.5.2.2. Prepare the board#

FIL drives the board over two channels: JTAG to program the FPGA, and TCP/IP over SSH to deploy the application, start it and stream its console back to SpaceStudio. Both must be in place before launching (refer to fig_zedboard_parts for the ZedBoard connectors):

  1. Connect the JTAG programming cable between the board and the host computer.

  2. Connect the board to the same IP network subnet as the host computer.

  3. Power on the board.

The embedded Linux image that SpaceStudio builds for the target must also carry your SSH public key and a network configuration, so that SpaceStudio can log into the board without a password:

  1. Open the zynq_add_div diagram and select the zynq0 instance.

  2. In the Properties view, open the Linux tab.

  3. Under the File System section, enable Enable SSH public key and enter the absolute path to your SSH public key file (the .pub half of your RSA key pair).

  4. Under the Network section, enter the static IP address, subnet mask and gateway matching your local network.

Note

If you do not have an RSA key pair yet, generate one with ssh-keygen -t rsa -b 4096. FIL supports RSA SSH-2 public keys (the ones starting with ssh-rsa), and 4096-bit keys are recommended.

Target provisioning is described in full in the FPGA-in-the-loop section of the user guide.

4.5.2.3. Create the FIL launch configuration#

The FPGA execution is described by a launch configuration, exactly like a simulation is:

  1. Right-click the zynq_add_div architecture in the Project Explorer.

  2. Select Run As > Run Configurations….

  3. Right-click Execute on FPGA and select New Configuration. A default name is proposed, which you may change at the top of the window.

  4. In the General section:

    1. Make sure Architecture is set to zynq_add_div.

    2. Select Xilinx - Vivado 2025.2 as Electronic Design Automation (EDA) tool.

    3. Select the ZedBoard Zynq Evaluation and Development Kit as Board.

    4. Optionally raise Number of threads to speed up synthesis and implementation on a multi-core machine.

  5. In the HLS Options section:

    1. Ensure that the selected High-level synthesis tool is Vitis HLS.

    2. Both module instances to synthesize, adder0 and divider0, should be checked.

  6. In the Setup section:

    1. Enter the Host IP: the address of your computer on the network interface that reaches the board.

    2. Enter the path to your SSH private key — the private half of the key pair whose public key you provisioned above.

    3. Click Test connection to validate both the JTAG link and the network path to the board before launching.

  7. Click Run.

Important

The generated EDA and HLS projects live under the project’s implementation directory. Keep your SpaceStudio project in a path that is not overly long (less than 100 characters should be fine), since operating systems have path length limitations and the EDA and HLS tools create deeply nested subdirectories. Overly long module names can cause the same failure.

4.5.2.4. Launch the application#

The launch runs the whole flow unattended. SpaceStudio exports the virtual platform to Vivado, runs high-level synthesis on adder0 and divider0, synthesizes and implements the platform, generates the bitstream, builds the FSBL (First Stage Boot Loader) and the embedded Linux boot image, programs the board through JTAG, boots it, copies the application over SSH and starts it.

Follow the progress in the Jobs view: the FPGA-in-the-loop job is the parent of every process it spawns, and selecting a process shows its own output in the Console view. The EDA invocations take a while on the first run, but much of the result is cached by SpaceStudio to accelerate future compatible launches.

Once the application is started on the board, its console is presented directly in SpaceStudio — there is no serial terminal to set up. The application is driven by the very same command_generator used in Part 1: it feeds the input_reader with a sweep of commands, and each result is printed in the console as the output_writer produces it. You therefore get the same stream of operations you saw in simulation, except that 1+5 and 1/5 are now computed by real hardware on the FPGA fabric instead of by software on the ARM processor. When the generator has exhausted its commands it stops the application, the FIL job completes and the board is released.

Note

The command generator drives the application in both flows, so no input is typed during the run. To exercise other commands — a division by zero such as 4/0, or a malformed command such as hello+world, which produce the error messages described earlier — edit command_generator.cpp (imports of the input_reader module) and launch again. Because the same generator feeds the simulation and the board, the two consoles can be compared line by line.

You can end a run early at any time with the terminate button of the Jobs view (or of the console), which stops the application and releases the board.

4.5.2.5. Debugging and profiling on the board#

The configuration you just created is not limited to running the application. The same FIL configuration can be started in the two other execution modes, from Debug As > Debug Configurations… and Profile As > Profile Configurations…:

  • Debug attaches the debugger to the software part of the application while it executes on the board. You can set breakpoints in the controller and step through the decoding of a command, with the adder and divider co-processors answering from the FPGA fabric as real hardware.

  • Profile collects the execution profile of the run on the board, so the measured time is the SoC’s own time rather than a model of it.

This is the main practical benefit of FPGA-in-the-loop over a pure implementation flow: the board becomes just another execution target of your project, and the debugging and profiling habits you use in simulation carry over to real hardware unchanged.

4.5.2.6. What just happened ?#

SpaceStudio has created a Vivado project and performed all the steps needed for the bitstream generation. SpaceStudio, through Architecture Implementation technology, has generated all the required files, glue logic, firmware, Linux Drivers and has correctly configured all the cores to work together. In particular, the process has transparently implemented the adder and divider modules as hardware modules.

Communication has been refined through hardware components (such as processor_fifo_adapter & processor_fifo). These hardware components connect to the interconnect. The software modules communicate with these hardware component using optimized drivers.

Then, going beyond Architecture Implementation, the FPGA-in-the-loop workflow programmed the FPGA over JTAG, booted the target, deployed and started the application over SSH, and connected its console to SpaceStudio — so that a single launch took the architecture from source code to a running, interactive application on real hardware.

4.6. Result files#

  • SpaceStudio Project