5. Scale a hardware kernel with the Architect Dashboard

5.1. Required files

SpaceStudio Project

5.2. Introduction

A single hardware co-processor accelerates one piece of work at a time. When the workload is embarrassingly parallel, the natural next step is to instantiate the co-processor many times and run those instances side by side. The question then becomes an architectural one: how many instances give the best return before the FPGA runs out of room? This is exactly the kind of trade-off the Architect Dashboard is built to explore.

This tutorial uses matrix multiplication as the workload. The output elements of a product are independent, so they can be computed in any order and on any number of engines. We will build a small dot-product co-processor, instantiate it as a group, have a software controller fan work out across the group, and then use the Architect Dashboard to sweep the number of instances (2, 4, 8, 16, 32) and compare the resulting run time and FPGA utilization at a glance.

Two mechanisms carry the tutorial:

• Module groups, introduced in Tutorial 3 — instantiating the same module several times and addressing the instances through the generated *_GROUP macros.

• The explicit DMA (conduit) form of StreamWrite / StreamRead, where the controller names a memory_region_t instead of a local buffer, and its non-blocking (asynchronous) transfer model, used here to keep every instance busy. Only the two calls appearing in the fan-out loop below are needed to follow this tutorial; the mechanism is covered in depth in Tutorial 6.

5.3. The matrix-multiplication co-processor

The product C = A × B is defined element-by-element as C[i][j] = sum_k A[i][k] * B[k][j]:

\[ \begin{align}\begin{aligned}\begin{split}A = \begin{bmatrix} a_{1,1} & \dots & a_{1,n} \\ \vdots & \ddots & \vdots \\ a_{m,1} & \dots & a_{m,n} \end{bmatrix} B = \begin{bmatrix} b_{1,1} & \dots & b_{1,p} \\ \vdots & \ddots & \vdots \\ b_{n,1} & \dots & b_{n,p} \end{bmatrix}\end{split}\\\begin{split}A \times B = \begin{bmatrix} a_{1,1} b_{1,1} + \dots + a_{1,n} b_{n,1} & \dots & a_{1,1} b_{1,p} + \dots + a_{1,n} b_{n,p} \\ \vdots & \ddots & \vdots \\ a_{m,1} b_{1,1} + \dots + a_{m,n} b_{n, 1} & \dots & a_{m,1} b_{1,p} + \dots + a_{m,n} b_{n,p} \end{bmatrix}\end{split}\end{aligned}\end{align} \]

To keep the co-processor small enough to synthesise many times over, matrix_mult does not hold whole matrices. It computes one output element per iteration: it receives one row of A immediately followed by one column of B — packed into a single block — and returns their dot product. Reading the whole block in one StreamRead makes the transfer size a single compile-time constant, so SpaceStudio can size the channel (and the explicit DMA) exactly.

Listing 5.1 matrix_mult co-processor

void matrix_mult::thread(SPACECOMP_THREAD_PARAMS(matrix_mult, INDEX, thread)) {
    spacecomp_thread_initialize();
    uint32_t operand[MATRIX_ROWS + MATRIX_COLUMNS];   // row | column
    while (1) {
        const bool initializing = spacecomp_thread_loop_start();
        if (initializing) {}
        StreamRead(CONTROLLER0_ID, SPACE_BLOCKING, operand,
            MATRIX_ROWS + MATRIX_COLUMNS);
        const uint32_t* row = operand;
        const uint32_t* col = operand + MATRIX_COLUMNS;
        StreamWrite(CONTROLLER0_ID, SPACE_BLOCKING, dot_product(row, col));
    }
}

uint32_t matrix_mult::dot_product(const uint32_t* row, const uint32_t* col) {
    uint32_t acc = 0;
    L1: for (int k = 0; k < MATRIX_COLUMNS; k++) {
        acc += row[k] * col[k];
    }
    return acc;
}

The tutorial uses a single 1000x1000 problem size (solution matrix_1000). The two operand matrices are 8 MB in total, too large to keep as literals, so they are generated deterministically at start-up (see matrix.c); because the generators are deterministic, the controller can recompute any element on the CPU to validate the hardware.

5.4. Fanning work across a group of instances

Instantiating matrix_mult several times creates a module group. As in Tutorial 3, SpaceStudio generates a MATRIX_MULT_GROUP array of instance ids and a compile-time MATRIX_MULT_GROUP_SIZE count, so the same controller code works for any instance count without edits — only the architecture changes.

The controller walks the output matrix in batches of MATRIX_MULT_GROUP_SIZE elements. For each batch it dispatches one element to every instance with a non-blocking explicit DMA, waits for the operand transfers, then collects the results. Each instance gets its own staging window in the PS DDR, so N transfers can be in flight at once and the N dot products overlap:

Listing 5.2 Async fan-out across the group (controller)

constexpr unsigned N = MATRIX_MULT_GROUP_SIZE;
for (unsigned base = 0; base < MATRIX_ELEMENTS; base += N) {
    const unsigned batch = min(N, MATRIX_ELEMENTS - base);

    // Dispatch one element to each instance.
    for (unsigned k = 0; k < N; ++k) {
        if (k >= batch) continue;
        gather_element(a, b, (base + k) / MATRIX_COLUMNS, (base + k) % MATRIX_COLUMNS);
        DeviceWrite(STAGE_MEM_ID, OP_OFFSET(k), m_operand, MATRIX_ROWS + MATRIX_COLUMNS);
        m_send[k] = StreamWrite(MATRIX_MULT_GROUP[k], SPACE_NON_BLOCKING,
            memory_region_t{STAGE_MEM_ID, OP_OFFSET(k),
                            (MATRIX_ROWS + MATRIX_COLUMNS) * 4});
    }
    for (unsigned k = 0; k < N; ++k) { if (k >= batch) continue; space::wait(m_send[k]); }

    // Collect one result from each instance.
    for (unsigned k = 0; k < N; ++k) {
        if (k >= batch) continue;
        m_recv[k] = StreamRead(MATRIX_MULT_GROUP[k], SPACE_NON_BLOCKING,
            memory_region_t{STAGE_MEM_ID, RES_OFFSET(k), 4});
    }
    for (unsigned k = 0; k < N; ++k) {
        if (k >= batch) continue;
        space::wait(m_recv[k]);
        uint32_t c = 0;
        DeviceRead(STAGE_MEM_ID, RES_OFFSET(k), &c, 1);
        /* validate sampled positions against expected_element(...) */
    }
}

The controller times the whole product with std::chrono::steady_clock and prints the wall-clock in the most readable unit, followed by [OK] / [FAIL] from the sampled validation and the instance count:

matrix 1000x1000, 16 instances: 52.60 s [OK]

Note

STAGE_MEM_ID is a controller module parameter, set by each architecture rather than hard-coded in the source: the Zynq-7000 architectures set it to ZYNQ_DDR0_ID and the Zynq UltraScale+ ones to ZYNQ_ULTRASCALE_DDR0_ID, since the two SoCs expose their PS DDR under different memory ids. That is why moving the design from Zynq-7000 to UltraScale+ leaves the application code untouched.

5.5. Sweeping the instance count with the Architect Dashboard

The completed project ships one solution, matrix_1000, with five architectures that differ only in how many matrix_mult instances they contain:

Table 5.2 Architectures in the matrix_1000 solution

Architecture	Target SoC	matrix_mult instances
2_instances	Zynq-7000 (Zedboard)	2
4_instances	Zynq-7000 (Zedboard)	4
8_instances	Zynq-7000 (Zedboard)	8
16_instances	Zynq UltraScale+	16
32_instances	Zynq UltraScale+	32

The Zedboard’s small Zynq-7000 (xc7z020) holds the 2-, 4- and 8-instance designs; eight instances is the most it can take, so from 16 instances on the design is moved to the larger Zynq UltraScale+. This move is a one-line change in the architecture (the component type and the software target), not a change to the application code.

Note

The ceiling is easy to see coming in the utilization reports: each matrix_mult instance costs roughly 3,100 LUTs, so 4 instances take about 28 % of the Zedboard’s 53,200 LUTs and 8 instances about 52 %. Doubling once more would need essentially the whole device — which is why 16 instances is the row that moves to the UltraScale+ rather than the one that fills the Zedboard up.

The Architect Dashboard executes several architectures on FPGA boards and compares their results side by side, so you do not have to build → program → run each one by hand. Open it, select the architectures to run, and launch them; each row fills in its Runtime and FPGA Utilization (max) once it completes.

Figure 5.5 The Architect Dashboard after the matrix_1000 sweep

Figure 5.5 shows the dashboard at the end of the sweep. One row per architecture, each bound to its own FIL launch configuration, with the toolbar acting on the selection: Run selected launches the checked rows, Clean selected discards their build outputs, and Refresh re-reads the results from disk. The per-row Action button starts a single architecture, and turns into a stop button while that row is running.

The FIL is board aware: several boards can stay attached to the host at the same time — here a Zedboard and a ZCU102 — and each launch configuration programs the board its architecture targets. The FIL identifies the board from the SoC of the architecture and drives only that board’s JTAG cable, so a Zynq-7000 run never touches the UltraScale+ board and vice versa. There is nothing to plug, unplug or select between runs.

For this tutorial, that is what makes the sweep a single operation: with both boards connected, select all five architectures in the Architect Dashboard and launch them in one batch, even though three of them run on the Zedboard and two on the ZCU102. Once the full system compilation of the batch is done, every row has been programmed onto its own board, executed, and measured — all the results are ready in the same table.

Note

Board awareness distinguishes the attached boards by SoC family, so it needs at most one connected board per family (one Zynq-7000 and one Zynq UltraScale+ here). Two boards of the same family connected at once cannot be told apart, and the launch stops with an error naming the ambiguous JTAG cables rather than programming the wrong one.

Note

The values below are the ones measured for Figure 5.5. Your own run will produce numbers of the same shape, but not identical ones: run time depends on the board and the serial link, and utilization on the Vivado version and the implementation strategy.

Table 5.3 Architect Dashboard — matrix_1000 instance sweep

Architecture	Status	Launch Configuration	Runtime	FPGA Utilization (max)
matrix_1000/2_instances	Completed	matrix_1000_2_instances_fil	1:59	11.41 %
matrix_1000/4_instances	Completed	matrix_1000_4_instances_fil	1:49	27.98 %
matrix_1000/8_instances	Completed	matrix_1000_8_instances_fil	1:20	51.61 %
matrix_1000/16_instances	Completed	matrix_1000_16_instances_fil	52.60 s	18.05 %
matrix_1000/32_instances	Completed	matrix_1000_32_instances_fil	55.42 s	35.60 %

Two columns tell the story. Runtime falls as the instance count rises, because more dot products run in parallel — but it falls by less and less: 2 → 4 instances buys 10 seconds, 4 → 8 another 29, and 8 → 16 (already on the bigger device) takes the run below a minute. At 32 instances the curve has flattened out completely: 55.42 s is no better than the 52.60 s of the 16-instance design, even though the fabric cost has doubled. That is the plateau, and it is the reason the sweep stops there — past this point the engines are no longer the bottleneck, the data path feeding them is, so adding more of them buys nothing.

FPGA Utilization tells the other half. It climbs with the instance count on a given SoC and drops back down at the row where the design moves to the bigger device: the Zedboard rows run 11.41 % → 27.98 % → 51.61 % and stop at 8 instances, then the UltraScale+ picks the sweep back up at 18.05 % for 16 instances and 35.60 % for 32.

Note

The dashboard reports the most utilized single resource, not an average — hover a cell to see which one it is. That is why the number can go down between two rows: the 8-instance Zedboard design is limited by one resource type, the 16-instance UltraScale+ design by another, on a much larger device.

Reading the two columns on the same row is how you pick the sweet spot. Here they agree on 16_instances: it is the fastest architecture of the sweep, and it reaches that run time using barely a fifth of the device — the 32-instance design spends twice the fabric to be marginally slower.

Tip

The Runtime column reports the same wall-clock the controller prints on the serial console, so the dashboard value and the console value should agree. The first run on a fresh board includes one-time set-up; re-run to read the steady-state number.

5.6. Conclusion

The attendee built a deliberately small dot-product co-processor, instantiated it as a group of 2 to 32 engines, fanned the output elements of a 1000x1000 product across the group with non-blocking explicit DMA, and let the Architect Dashboard build, run and compare the five architectures on the board.

Done by hand, that comparison is five build-program-run-record cycles, each one long enough to interrupt the train of thought — and two of them on a different board than the other three, so the sweep also means swapping hardware halfway through — with the results ending up in a notebook or a spreadsheet where nothing guarantees they were measured the same way. The Architect Dashboard turns it into a single selection and a single launch: because the FIL is board aware, both boards stay attached and each architecture is programmed onto the one it targets, so all five are built and executed unattended, and Runtime and FPGA Utilization land side by side in the same table, measured identically, ready to read as a curve rather than as five isolated numbers.

That is what makes the answer readable here. Neither column answers the question alone: runtime tells you how much faster this architecture is, utilization whether it still fits. Read on the same row, they locate the knee — the Zedboard runs out of fabric past 8 instances, the UltraScale+ takes the sweep to its fastest point at 16, and the 32-instance row buys nothing for twice the resources. And because every row was built and measured the same way, the rows can be compared directly. That is how “add more instances until it stops helping” becomes a specific device with a specific instance count.

The workload happened to be matrix multiplication, but the method does not depend on it. Define a set of architectures that differ in the one parameter you want to settle, let the Architect Dashboard build and measure them all on real hardware, and read the trade-off from the table. Any parameter you can express as an architecture can be settled this way, and each additional question you ask costs you one more row.

5.7. Result files

SpaceStudio Project