Reconfigurable Controller Module

CNN Co-processor for RISC-V Architecture

System: CNN Co-processor

Designer: Digital Design IC Team
Supervisor: Prof. Dr. Sherif Fathy
Project: Graduation Project - CNN Unit

Table of Contents

Pages 1-3: Fundamentals

 System Overview

 Module Purpose

 Interface Overview

 Basic Architecture

Pages 4-6: Core Implementation

 Finite State Machine Design

 Instruction Processing

 Resource Coordination

 Timing Control Systems

Pages 7-10: Analysis and Verification

 Performance Characteristics

 Resource Analysis

 System Integration

 Validation Results

Page 1: System Overview

CNN Coprocessor Context

The Reconfigurable Controller serves as the organizational core of the CNN

acceleration coprocessor designed for the RISC-V E203 processor,

1 | Page
orchestrating every internal operation within the accelerator system. This
module determines when each processing layer is activated, directs where
data are loaded and stored in accordance with input instructions, and
enforces precise temporal coordination of all data and control signals
throughout the system. The controller manages the entire computational
flow from data storage in caches to processing element activation, and
finally to result delivery back to the RISC-V processor.

The controller operates within a comprehensive CNN accelerator architecture

that achieves up to 6.27x performance improvement over software-based
CNN implementations through dedicated hardware processing elements and
optimized data flow management. The system includes four reconfigurable
processing elements that can be configured for convolution, pooling, ReLU,
and addition operations, all coordinated through the sophisticated control
mechanisms implemented in the Reconfigurable Controller.

System Integration Role

The Reconfigurable Controller integrates with all major system components

including the Decoder for instruction interpretation, Data Fetcher for memory
management, Data Selector for data routing, Crossbar for interconnection
control, and four processing elements for computational operations. This
central coordination role makes the controller essential for maintaining
efficient operation throughout complex CNN processing sequences while
ensuring proper timing and resource utilization.

The integration approach emphasizes comprehensive coordination where the

controller provides centralized management of distributed processing
resources. The module receives decoded instructions from the Decoder and
translates them into detailed control sequences that coordinate cache
operations, processing element activation, data routing configuration, and
result collection. This approach enables complex CNN algorithms to be
executed through simple instruction sequences while maintaining optimal
resource utilization and performance characteristics.

Architectural Significance

The controller implements a sophisticated finite state machine architecture

that manages five distinct operational states: IDLE for system readiness,
CONFIG for instruction processing and system configuration, EXECUTE for
active CNN layer computations, WAIT_COMPLETION for monitoring processing
element completion, and WRITE_BACK for result storage and system cleanup.

2 | Page
This state-based approach provides deterministic behavior essential for real-
time CNN inference applications.

The architectural design enables the system to execute sixteen different CNN
operation sequences ranging from simple single-layer operations to complex
four-layer pipelines including convolution, pooling, ReLU, and addition
combinations. This flexibility supports implementation of standard CNN
architectures including LeNet-5, AlexNet-style networks, and other common
neural network topologies while maintaining efficient resource utilization and
predictable performance characteristics.

Page 2: Module Purpose

Control Path Coordination

The primary purpose of the Reconfigurable Controller involves managing

sixteen different CNN operation sequences and layer combinations that
enable comprehensive neural network processing capabilities. The controller
coordinates complex data flow patterns including convolution-only
processing, pooling-only operations, ReLU-only activation, addition-only
operations, and various sequential combinations such as convolution
followed by ReLU, convolution followed by pooling, and complete four-layer
pipelines.

The coordination function extends beyond simple sequencing to include

intelligent resource allocation where processing elements are dynamically
assigned based on current operation requirements. The controller manages
crossbar configuration to enable appropriate data routing between
processing stages, coordinates cache operations to ensure data availability
when required, and manages completion monitoring to ensure proper
sequencing of multi-layer operations.

Layer coordination logic implements sophisticated decision-making

capabilities that determine optimal processing sequences based on
instruction types and system conditions. The controller automatically
configures filter sizes for convolution operations, sets pooling parameters for
pooling stages, and manages completion-based progression through CNN
layer sequences while ensuring intelligent routing of intermediate and final
results.

Data Path Management

3 | Page
The Reconfigurable Controller manages seven distinct data flow paths
throughout the CNN system, enabling comprehensive data movement
coordination that supports various CNN processing patterns. The data paths
include image cache storage for input data routing, kernel cache storage for
filter coefficient management, old data cache storage for intermediate result
handling, processing element data loading for computation coordination,
crossbar RAM storage for intermediate results, result cache storage for
output management, and RISC-V data transfer for final result delivery.

Data path management includes sophisticated addressing capabilities that

provide memory addressing for convolution, pooling, ReLU, and addition
operations through dedicated address generation logic. The controller
coordinates memory read and write operations to ensure data availability
when processing elements require input data and proper result storage when
computations complete.

The management strategy implements intelligent resource sharing where

memory bandwidth and processing element capabilities are optimized to
maximize overall system throughput. The controller coordinates
simultaneous operations including address block activation for concurrent
memory addressing, data fetcher control for parallel instruction dispatch,
cache enable sequencing for proper timing coordination, and crossbar RAM
addressing for synchronized memory operations.

Instruction Processing Coordination

The Reconfigurable Controller interprets and executes nine custom CNN

instructions that provide comprehensive control over CNN processing
operations. The instruction set includes [Link] for local reset control,
[Link] for filter storage, [Link] for image data storage,
[Link] for result loading, [Link] for result transfer, [Link]
for old data storage, [Link] for old data loading, [Link] for working
mode configuration, and [Link] for CNN operation initiation.

Instruction processing coordination includes parameter control for runtime

configuration of filters, pooling parameters, and addressing modes that
enable dynamic adaptation to different CNN algorithms and processing
requirements. The controller implements high-level operations where single
instructions can trigger complex CNN sequences, reducing software
overhead while maintaining precise control over processing operations.

State management provides comprehensive control of system initialization

and operation through sophisticated timing coordination that ensures proper

4 | Page
sequencing of instruction execution phases. The controller coordinates
instruction reception from the Decoder, parameter extraction and validation,
system configuration based on instruction requirements, processing element
activation and monitoring, and completion signaling for subsequent
instruction processing.

Page 3: Interface Overview

Input Interface Specifications

The Reconfigurable Controller receives inputs through several functional

groups that provide comprehensive control over CNN processing operations.
Clock and reset signals include the system clock for synchronous operation
and an active-low reset signal for system initialization. Instruction interface
signals include a 7-bit funct7 operation code that specifies the CNN
instruction type, 5-bit rs1, rs2, and rd register specifiers for source and
destination information, and cfg_valid and cfg_ready handshaking signals for
coordination with the Decoder.

Processing element completion interfaces include 4-bit finish signals for each
processing element type including finish_conv for convolution completion,
finish_pool for pooling completion, finish_relu for ReLU completion, and
finish_adder for addition completion. These completion signals enable the
controller to monitor processing progress and coordinate proper sequencing
of multi-stage operations.

Cache and memory interface inputs include various wait request signals from
different cache banks, readdata valid signals indicating data availability, and
hit detection signals from cache subsystems. Additional input signals include
index values for operation identification, indicator signals for data change
detection, and validation signals for error detection and system coordination.

Output Interface Specifications

The output interface provides comprehensive control signals that coordinate

all aspects of CNN accelerator operation. Processing element control outputs
include 5-bit crossbar selection signals, 5-bit processing element mode
signals, individual enable signals for convolution, addition, pooling, and ReLU
operations, plus parameter signals including filter dimensions, number of
parallel processing elements, and pooling window size parameters.

5 | Page
Memory interface control outputs include 10-bit address signals for
convolution, addition, ReLU, and pooling data memory, plus various read and
write enable signals for memory access coordination. Data routing control
outputs include processing element selection signals, source enable signals
for data path control, and bank selection signals for result storage
coordination.

Cache interface outputs include read and write control signals for multiple
cache banks, memory request signals for external memory access, and
address generation outputs for cache coordination. Local reset outputs
provide selective reset capabilities for different system components including
processing elements, data management modules, and memory subsystems.

Protocol Design and Communication

The Reconfigurable Controller implements sophisticated protocol design that

manages multiple concurrent communication relationships while maintaining
deterministic timing behavior throughout all control operations. Protocol
implementation includes proper handshaking with the Decoder for
instruction reception, coordination with processing elements for operation
control, and interface management with cache and memory subsystems for
data coordination.

Handshaking protocols ensure reliable coordination between the controller

and other system components through explicit ready and valid signal pairs
that confirm successful operation initiation and completion. These protocols
prevent timing violations and coordination errors while enabling efficient
operation under various system conditions including variable processing
element timing and memory access patterns.

The communication protocol design emphasizes reliability while maintaining

efficient operation that supports high-throughput CNN processing
applications. Reset protocols provide immediate initialization of all control
outputs and state machines, ensuring deterministic startup behavior and
supporting system recovery from error conditions while maintaining proper
coordination with system-level reset and initialization procedures.

Page 4: Finite State Machine Design

Five-State Architecture

6 | Page
The Reconfigurable Controller implements a comprehensive five-state finite
state machine that provides structured management of CNN processing
operations through clearly defined operational phases. The IDLE state serves
as the system ready condition where the controller awaits cfg_valid signals
from the Decoder indicating new instruction arrival while maintaining all
control outputs in appropriate states for system readiness.

The CONFIG state processes nine CNN instruction types and configures
system parameters based on instruction requirements, including processing
element mode configuration, crossbar routing setup, memory address
initialization, and parameter extraction for filter sizes, pooling dimensions,
and other operational settings. This state implements comprehensive
instruction decoding that translates high-level CNN instructions into detailed
control sequences.

The EXECUTE state manages active CNN layer computations and crossbar
routing by enabling appropriate processing elements, configuring data paths
through crossbar settings, initiating memory operations for data access, and
monitoring processing element status for completion detection. The
WAIT_COMPLETION state coordinates layer transitions by monitoring
processing element completion flags and ensuring all required operations
complete before proceeding to result storage.

The WRITE_BACK state handles result storage, cache operations, and system
cleanup by coordinating result collection from processing elements,
managing write-back operations to appropriate memory locations,
performing system cleanup operations, and preparing the system for
subsequent instruction processing. This five-state architecture provides
sufficient functionality for comprehensive CNN processing while maintaining
design clarity and verification efficiency.

7 | Page
State Transition Control

State transition logic implements carefully designed conditions that ensure

proper sequencing and reliable operation under all supported CNN
processing scenarios. The IDLE to CONFIG transition occurs when cfg_valid
signal indicates new instruction arrival from the Decoder, ensuring that
processing begins only when valid instructions are available and the system
is prepared for operation.

CONFIG to EXECUTE transition depends on instruction type identification,

specifically when funct7 equals 7'b1000000 indicating [Link] start
computation instruction, ensuring that execution phase begins only for
appropriate instruction types while other instructions complete their
configuration operations and return to IDLE state. EXECUTE to
WAIT_COMPLETION transition occurs when processing element operations
are initiated and monitoring phase begins.

WAIT_COMPLETION to WRITE_BACK transition requires completion signal

assertion from all active processing elements, ensuring that result collection
begins only when all required computations complete successfully.
WRITE_BACK to IDLE transition occurs when operation cleanup completes
and the system is ready for subsequent instruction processing.

8 | Page
The transition logic includes comprehensive validation that ensures state
changes occur only when appropriate system conditions are present,
preventing invalid transitions that could affect processing accuracy or
system reliability. Default state handling provides graceful recovery when
unexpected conditions occur, ensuring robust operation under various
operational scenarios.

Timing Coordination and Synchronization

The finite state machine operates synchronously with the system clock,
ensuring deterministic timing behavior that integrates properly with other
CNN accelerator components while maintaining predictable performance
characteristics essential for real-time processing applications. State
transitions occur on positive clock edges with appropriate timing margins for
reliable operation across process and temperature variations.

Timing coordination includes sophisticated counter management with

twenty-nine independent counters that provide dedicated timing control for
different operations. These counters include hold and update mechanisms for
sophisticated timing management, address read counters for specialized
memory read operations, and delay counters for precise timing coordination
between cache operations and processing element timing.

Synchronization mechanisms ensure proper coordination between state

machine timing and external component timing requirements including
cache access patterns, processing element computation cycles, and memory
interface timing. The timing design enables overlapped operation of multiple
CNN layers while maintaining proper data dependency management and
result integrity throughout complex processing sequences.

Reset synchronization provides immediate state machine initialization

regardless of current state, ensuring reliable system startup and error
recovery capabilities. Both global and local reset signals are supported,
enabling comprehensive system initialization as well as selective component
reset for debugging and recovery operations.

Page 5: Instruction Processing

CNN Instruction Set Implementation

The Reconfigurable Controller implements comprehensive processing for

nine custom CNN instructions that provide complete control over CNN

9 | Page
accelerator operations. The [Link] instruction (function code 1) controls
local resets throughout the system, enabling selective initialization of
different system components without affecting overall system operation. The
[Link] instruction (function code 2) manages filter storage in COE
cache, coordinating filter loading operations with cache timing and data
availability.

The [Link] instruction (function code 3) handles image data storage

in image cache, managing input data routing and cache allocation for
optimal processing efficiency. The [Link] instruction (function code 4)
loads results from crossbar to result cache, coordinating intermediate result
storage for multi-stage processing operations. The [Link] instruction
(function code 5) transfers results from cache to RISC-V processor, managing
final result delivery and interface coordination.

The [Link] instruction (function code 6) manages old data storage in

dedicated cache areas, supporting CNN algorithms that require previous
computation results for subsequent processing stages. The [Link]
instruction (function code 7) handles old data loading from cache to RISC-V
processor, enabling bidirectional data movement for complex CNN
processing patterns.

The [Link] instruction (function code 8) sets processing element working

mode and crossbar configuration, enabling dynamic reconfiguration of
processing elements for different CNN operations. The [Link] instruction
(function code 9) starts CNN operations with configured parameters,
initiating the actual computational processing after appropriate system
configuration.

Parameter Extraction and Configuration

Instruction processing includes sophisticated parameter extraction that

derives operational settings from instruction fields and system configuration
requirements. The controller extracts filter dimension parameters from
instruction fields to configure convolution operations appropriately, sets
pooling window sizes based on instruction specifications, and determines the
number of parallel processing elements to activate based on computational
requirements.

Configuration management includes crossbar routing setup where selector

signals are configured to enable appropriate data paths between processing
elements, memory addressing configuration where base addresses and
offset calculations are established for different memory regions, and

10 | P a g e
processing element mode configuration where individual processing
elements are set to perform specific operations.

The parameter extraction process includes validation mechanisms that

ensure extracted parameters fall within supported ranges and that
instruction combinations are logically consistent with system capabilities.
Error detection logic identifies invalid parameter combinations and provides
appropriate error responses while maintaining system stability and
preventing undefined behavior.

Configuration coordination ensures that all system components receive

appropriate configuration information in proper sequence, preventing timing
conflicts and ensuring that processing elements are properly configured
before computational operations begin. This coordination includes cache
preparation, crossbar setup, address generation initialization, and processing
element mode configuration.

Instruction Execution Coordination

Instruction execution coordination manages the complex sequencing

required to translate high-level CNN instructions into detailed control
sequences that coordinate multiple system components effectively. The
coordination process begins with instruction reception and validation,
proceeds through parameter extraction and system configuration, continues
with processing element activation and monitoring, and concludes with result
collection and storage.

Execution sequencing ensures that CNN operations proceed in appropriate

order with proper data dependencies maintained throughout multi-stage
processing operations. The controller coordinates simultaneous operations
where multiple processing elements operate on different data streams while
ensuring that data dependencies are respected and result integrity is
maintained.

Completion monitoring provides real-time tracking of processing element

status to ensure proper sequencing of complex CNN operations that involve
multiple processing stages. The controller monitors completion flags from all
active processing elements and coordinates layer transitions based on
completion status while managing timeout conditions and error detection.

Result coordination manages the collection and routing of computation

results to appropriate storage locations based on instruction requirements
and subsequent processing needs. This includes intermediate result storage

11 | P a g e
for multi-stage operations, final result preparation for delivery to RISC-V
processor, and cache management for efficient memory utilization.

Page 6: Resource Coordination and Timing Control Systems

Processing Element Coordination

The Reconfigurable Controller implements sophisticated processing element

coordination that manages four independent processing elements, each
capable of performing convolution, pooling, ReLU, and addition operations
based on configuration requirements. The coordination includes dynamic
allocation where processing elements are assigned to current operation
requirements, mode configuration where each processing element is set to
perform specific operations, and completion monitoring where processing
status is tracked for proper sequencing.

Processing element enable sequencing provides coordinated activation of

multiple processing elements based on instruction requirements and data
availability. The controller manages enable signal timing to ensure
processing elements begin operation when input data is available and
coordination requirements are met. Completion monitoring includes real-time
tracking of processing element operation status to enable proper
coordination of subsequent operations.

The coordination strategy supports both sequential processing where

operations proceed through processing elements in series and parallel
processing where multiple processing elements operate concurrently on
different data streams. This flexibility enables optimization for various CNN
algorithms while maintaining efficient resource utilization and predictable
timing characteristics.

Pipeline management enables overlapped operation of multiple CNN layers

where different processing stages operate simultaneously on different data
sets, maximizing processing throughput while maintaining proper data
dependency management and result integrity throughout complex
processing sequences.

Cache Interface Timing Coordination

Cache interface timing coordination manages sophisticated interaction with

four specialized cache banks plus additional memory systems that provide
data storage and retrieval capabilities for CNN processing operations. The

12 | P a g e
coordination includes hit detection windows with 3-cycle timing for cache hit
detection, enable and disable sequences for coordinated cache activation
and deactivation, multi-cache coordination for simultaneous management of
multiple cache banks, and flow control integration with cache timing
synchronized to processing element operations.

Memory addressing coordination provides synchronized address generation

for different memory regions including convolution data memory, addition
data memory, ReLU data memory, and pooling data memory. The controller
coordinates read and write operations to ensure data availability when
processing elements require input data and proper result storage when
computations complete.

Cache timing optimization includes proper coordination with cache hit

detection mechanisms, efficient handling of cache miss conditions, and
appropriate buffering for timing tolerance between different system
components. The coordination ensures that cache operations align properly
with processing element timing while maintaining efficient memory
bandwidth utilization.

Memory interface timing includes coordination with external memory access

patterns, proper handling of wait request conditions, and efficient
management of memory bandwidth resources. The controller implements
appropriate retry mechanisms for memory access conflicts while maintaining
overall system performance and reliability.

Multi-Counter Timing System

The Reconfigurable Controller implements a sophisticated timing system

based on twenty-nine independent counters that provide dedicated timing
control for different operational aspects of CNN processing. These counters
include specialized functions for hold and update mechanisms that provide
sophisticated counter management, address read counters for specialized
memory read operations, delay counters for precise timing coordination
between cache and processing element operations, and general-purpose
counters for various timing requirements.

Counter coordination ensures that timing relationships between different

system components are maintained appropriately while supporting various
operational patterns including sequential processing, parallel processing, and
complex multi-stage operations. The timing system enables precise
coordination of data flow between components while maintaining optimal
throughput and resource utilization.

13 | P a g e
Timing system management includes appropriate counter initialization for
different operational phases, proper counter sequencing for multi-cycle
operations, and timeout detection for error handling and system reliability.
The system supports both fixed timing sequences for predictable operations
and adaptive timing for variable processing requirements.

Performance timing optimization includes overlapping of configuration,

execution, and write-back phases across successive instructions when
system resources and processing element availability permit. This
optimization maximizes system throughput while maintaining proper
coordination and data integrity throughout complex CNN processing
sequences.

Page 7: Performance Characteristics

Instruction Processing Throughput

The Reconfigurable Controller achieves efficient instruction processing

throughput through sophisticated pipeline management that enables
overlapped execution of different processing phases. The five-state finite
state machine typically processes instructions in predictable timeframes that
support sustained CNN processing without creating bottlenecks in the overall
system pipeline. Instruction processing latency is optimized through efficient
state transitions and parallel resource management.

Throughput analysis demonstrates that the controller can coordinate multiple

concurrent operations including processing element management, cache
coordination, memory addressing, and data routing without introducing
significant overhead in the CNN processing pipeline. The multi-counter timing
system enables precise coordination of simultaneous operations while
maintaining optimal resource utilization across all system components.

The controller supports maximum throughput through intelligent resource

allocation where processing elements and memory resources are utilized
efficiently to minimize idle time and maximize computational efficiency.
Pipeline coordination enables continuous operation where configuration,
execution, and write-back phases overlap across successive instructions
when system conditions permit.

Performance optimization includes sophisticated timing management that

coordinates processing element completion with memory operations, cache
management, and result storage to maintain high processing element

14 | P a g e
utilization rates while ensuring data integrity and proper sequencing of CNN
operations.

Resource Management Efficiency

Resource management efficiency analysis indicates optimal coordination of

system resources including four processing elements, multiple cache banks,
memory interfaces, and crossbar networks. The controller implements
intelligent allocation strategies that assign resources dynamically based on
current processing requirements while maintaining efficient utilization across
varying workload conditions.

Memory bandwidth utilization is optimized through coordinated access

patterns that minimize conflicts while maximizing effective bandwidth
utilization across different memory systems. The controller coordinates read
and write operations to prevent bottlenecks while ensuring data availability
when processing elements require input data and proper result storage when
computations complete.

Processing element utilization is maximized through appropriate workload

distribution and timing coordination that ensures processing elements
operate at optimal efficiency. The controller manages processing element
allocation to minimize idle time while maintaining proper coordination with
data availability and result collection requirements.

Cache efficiency is optimized through intelligent cache management that

maximizes hit rates while minimizing access conflicts and timing violations.
The controller coordinates cache operations with processing element timing
to ensure optimal cache utilization while supporting various CNN processing
patterns and memory access requirements.

System Integration Performance

System integration performance demonstrates effective coordination

between the Reconfigurable Controller and all other CNN accelerator
components including optimal timing alignment that ensures efficient
operation without introducing bottlenecks or timing violations. Integration
efficiency includes proper handshaking protocols that minimize coordination
overhead while ensuring reliable operation under various system conditions.

Real-time performance characteristics include deterministic timing behavior

that enables accurate scheduling of CNN processing phases, predictable
response to instruction requests that supports efficient control system
operation, and reliable completion signaling that enables proper coordination

15 | P a g e
with subsequent processing operations. These characteristics support real-
time CNN applications where timing predictability is essential.

Power efficiency analysis indicates optimized operation through selective

activation of system components based on current processing requirements,
efficient state machine operation that reduces unnecessary switching
activity, and appropriate resource management that minimizes power
consumption during idle periods without affecting operational performance.

System scalability analysis demonstrates that the controller architecture can

be extended to support additional processing elements, memory banks, or
specialized processing units without fundamental architectural changes. The
modular control approach enables efficient scaling while maintaining optimal
performance characteristics and reliable coordination capabilities.

16 | P a g e