Associative memory is a cornerstone of cognitive intelligence within the human brain. The Bayesian confidence propagation neural network (BCPNN), a cortex-inspired model with high biological plausibility, has proven effective in emulating high-level cognitive functions like associative memory. However, the current approach using GPUs to simulate BCPNN-based associative memory tasks encounters challenges in latency and power efficiency as the model size scales. This work proposes a scalable multi-FPGA high performance computing (HPC) architecture designed for the associative memory system. The architecture integrates a set of hypercolumn unit (HCU) computing cores for intra-board online learning and inference, along with a spike-based synchronization scheme for inter-board communication among multiple FPGAs. Several design strategies, including population-based model mapping, packet-based spike synchronization, and cluster-based timing optimization, are presented to facilitate the multi-FPGA implementation. The architecture is implemented and validated on two Xilinx Alveo U50 FPGA cards, achieving a maximum model size of 200x10 and a peak working frequency of 220 MHz for the associative memory system. Both the memory-bounded spatial scalability and compute-bounded temporal scalability of the architecture are evaluated and optimized, achieving a maximum scale-latency ratio (SLR) of 268.82 for the two-FPGA implementation. Compared to a two-GPU counterpart, the two-FPGA approach demonstrates a maximum latency reduction of 51.72x and a power reduction exceeding 5.28x under the same network configuration. Compared with the state-of-the-art works, the two-FPGA implementation exhibits a high pattern storage capacity for the associative memory task.

Efficiently synthesizing an entire application that consists of multiple algorithms for hardware implementation is a very difficult and unsolved problem. One of the main challenges is the lack of good algorithmic libraries. A good algorithmic library should contain algorithmic implementations that can be physically composable, and their cost metrics can be accurately predictable. Physical composability and cost predictability can be achieved using a novel framework called SiLago. By physically abutting small hardware blocks together like Lego bricks, the SiLago framework can eliminate the time-consuming logic and physical synthesis and immediately give post-layout accurate cost estimation. In this paper, we build a library for matrix-matrix multiplication algorithm based on the SiLago framework as a case study because matrix-matrix multiplication is a fundamental operation in scientific computing that is frequently found in applications such as signal processing, image processing, pattern recognition, robotics, and so on. This paper demonstrates the methodology to construct such a library containing composable and predictable algorithms so that the application-level synthesis tools can utilize it to explore the design space for an entire application. Specifically, in this paper, we present an algorithm for matrix decomposition, several mapping strategies for selected kernel functions, an algorithm to construct the mapping of each matrix-matrix multiplication, and finally, the method to calculate the cost estimation of each solution.

Coarse-grain reconfigurable architectures, which provide high computing throughput, low cost, scalability, and energy efficiency, have grown in popularity in recent years. SiLago is a new VLSI design framework comprised of two coarse-grain reconfigurable fabrics: a dynamically reconfigurable resource array and a distributed memory architecture. It employs the Vesyla compiler to map streaming applications on these fabrics. Binding is a critical step in the high-level synthesis that maps operations and variables to functional units and storage elements in the design. It influences design performance metrics such as power, latency, area, etc. The current version of Vesyla does not support automatic binding, and it has to be specified manually through pragmas, which makes it less flexible. This paper proposes various approaches to automate the binding in Vesyla. We present a list scheduling-based approach to automate functional unit binding and an integer linear programming approach to automate register binding. Furthermore, we determine the binding of various basic linear algebraic subprogram and image processing tasks using the proposed approaches. Finally, a comparative analysis has been made between the automatic and manual binding concerning the power dissipation and latency for various benchmarks. The experimental results show that the proposed automatic binding consumes significantly less power for nearly the same latency as manual binding.

The remarkable advancements in AI algorithms over the past three decades have been paralleled by an exponential growth in their complexity, with parameter counts soaring from 60,000 in LeNet during the late 1980s to a staggering 175 billion in ChatGPT 3.0. To mitigate this surge in memory footprint, approximate computing has emerged as a promising strategy, focusing on deploying the minimal resolution necessary to maintain acceptable accuracy. Yet, current practices are hindered by two major challenges: a) the process of identifying the optimal resolution and representation format for each tensor remains a manual, ad hoc task, and b) the representation, typically in floating point (FP) format, is confined to standardized norms predominantly supported by commercial-off-the-shelf (COTS) products like GPUs. This paper tackles these issues by introducing a systematic approach to exploring the FP representation design space to find the ideal FP format for each tensor, thereby leveraging the full potential of FP quantization techniques. It is designed for custom hardware, enabling access to arbitrary FP formats, but also allows users to limit their exploration to standard FP formats, making it compatible with COTS. Additionally, the proposed method explores the Block Floating-Point (BFP) and automatically decides on the size of the blocks. A heuristic-based search method is proposed to handle the large design space. The proposed approach is general, and the heuristic is not biased towards any specific category of algorithms. We apply this method to a Self-Organizing Map (SOM) for bacterial genome identification and LeNet-5 neural network, demonstrating a significant reduction in memory footprint by around 94% and 96%, respectively, compared to the conventional 32-bit FP baseline.

Coarse-Grained Reconfigurable Array (CGRA) architectures are potential high-performance and power-efficient platforms. However, mapping applications efficiently on CGRA, which includes scheduling and binding operations on functional units and variables on registers, is a daunting problem. SiLago is a recently developed VLSI design framework comprising two large-scale reconfigurable fabrics: Dynamically Reconfigurable Resource Array (DRRA) and Distributed Memory Architecture (DiMArch). It uses the Vesyla compiler to map applications on these fabrics. The present version of Vesyla executes binding and scheduling sequentially, with binding first, followed by scheduling. In this paper, we proposed an Integer Linear Programming (ILP)-based exact method to solve scheduling and binding simultaneously that delivers better solutions while mapping applications on these fabrics. The proposed ILP combines two objective functions, one for scheduling and one for binding, and both of these objective functions are coupled with weightage factors $\alpha $ and $\beta $ so that the user can have the flexibility to prioritize either scheduling or binding or both based on the requirements. We determined the binding and execution time of image processing tasks and various routines of the Basic Linear Algebraic Subprogram (BLAS) using the proposed ILP for multiple combinations of weightage factors. Furthermore, a comparison analysis has been conducted to compare the latency and power dissipation of several benchmarks between the existing and proposed approaches. The experimental results demonstrate that the proposed method exhibits a substantial reduction in power consumption and latency compared to the existing method.

True Random Number Generators (TRNGs) and Physical Unclonable Functions (PUFs) are two important types of cryptographic primitives. TRNGs create a hardware-based, non-deterministic noise that is often used for generating keys, initialization vectors, and nonces for various applications that require cryptographic protection. PUFs have been proposed as a tamper-resistant alternative to the traditional secret key generation and challenge-response authentication methods. A compromised TRNG or PUF can lead to a system-wide loss of security.

The conventional TRNG or PUF designs are challenged by new attack vectors such as deep learning-based side-channel analysis. In this dissertation, we propose several new PUF and TRNG designs and evaluations of their performance and security.

The first PUF we introduce is called threshold PUF. We show that, in principle, any n-input threshold logic gate can be used as a base for building an n-input PUF. We implement and evaluate a threshold PUF based on recently proposed threshold logic flip-flops using SPICE simulation as a proof of concept. Threshold PUFs open up the possibility of using the rich body of knowledge on threshold logic implementations for designing PUFs. 

The second proposed design is a lightweight PUF construction called CRC-PUF, which focuses on protecting PUFs against machine learning-based modeling attacks. In CRC-PUF, input challenges are de-synchronized from output responses to make the PUF model difficult to learn. The input transformation which does the de-synchronization is based on a Cyclic Redundancy Check (CRC), thus the name CRC-PUF. By changing the CRC generator polynomial for each new response, we assure that recovering the transforming challenge has a success probability of at most 2^-86 for 128-bit challenge-response pairs.

The first TRNG design we introduce is based on a Non-Linear Feedback Ring Oscillator (NLFRO). The proposed NLFRO-TRNG structure harvests randomness from noise and unpredictable variations in delay cells and bi-stable elements, which is further amplified by the formation of non-linear feedback loops. The NLFRO outputs have chaotic behavior, allowing the construction of TRNGs with high entropy and speed. We implement three NLFRO-TRNGs on FPGA and evaluate the properties of the implementations with the NIST 800-90B entropy estimation and NIST 800-22 statistical test suits. 

The second proposed TRNG design is based on a strong PUF. The PUF based TRNG exploits the inherent determinism of PUF to enable in-field testing of the entropy sources by known answer tests. We present a prototype FPGA implementation of the proposed TRNG based on an arbiter PUF that passes all NIST 800-22 statistical tests and has the minimal entropy of 0.918 estimated according to NIST 800-90B recommendations.

Apart from TRNG and PUF designs, it is crucial to consider potential attack vectors that can be created leveraging recently emerged technologies. To that end, in the second part of this dissertation, we introduce a novel attack on FPGA-based PUF and TRNG implementations that combines bitstream modification along with deep learning-based side-channel analysis. We evaluate this new attack vector on the design of an arbiter PUF and a ring oscillator-based TRNG implemented on Xilinx Artix-7 28nm FPGAs. In both cases, we are able to achieve close to 100% classification accuracy to recover the output or response. In the case of the arbiter PUF, the attack can even overcome countermeasures that are based on encrypting the challenges or responses.

With such potent attack vectors readily available, the construction of strong countermeasures is necessary. Unfortunately, many of the state-of-the-art countermeasures are one-sided. In the final part of the dissertation, we use a countermeasure proposed for the protection of the Advanced Encryption Standard as an example. We conduct experiments and conclude that it can assist another type of side-channel attack that is not considered by the countermeasure.

True Random Number Generators (TRNGs) create a hardware-based, non-deterministic noise that is used for generating keys, initialization vectors, and nonces in a variety of applications requiring cryptographic protection. A compromised TRNG may lead to a system-wide loss of security. In this brief, we show that an attack combining power analysis with bitstream modification is capable of classifying the output bits of a TRNG implemented in FPGAs from a single power measurement. We demonstrate the attack on the example of an open source AIS-20/31 compliant ring oscillator-based TRNG implemented in Xilinx Artix-7 28nm FPGAs. The combined attack opens a new attack vector which makes possible what is not achievable with pure bitstream modification or side-channel analysis.

Bayesian Confidence Propagation Neural Network is a biologically plausible spiking model of the cortex. The human cortex is comprised of 100 billion neurons, a human-scale model of BCPNN in real-time requires 162 TFLOPS, 50 TB of synaptic weight storage to be accessed with a bandwidth of 200 TB. The spiking bandwidth is relatively modest at 200 GB/s. In this report, we present the initial results of an ASIC implementation of the BCPNN model. This work is in progress, and here we showcase how we can explore the BCPNN’s inherit data locality. The base-line implementation, called eBrainII, consumes 3 kW for real-time, human-scale BCPNN model. We improve in the base-line by adopting a lazy column update model that eliminates the expensive column access to DRAM and reduces the power consumption to 1.7 kW. We further exploit the significant temporal locality of input spikes to reduce the DRAM power to 17% and the total power consumption to 700 Watts. The implementation is highly regular and tiled, requiring modest engineering effort.

Adversarial machine learning is an emerging threat to security of Machine Learning (ML)-based systems. However, we can potentially use it as a weapon against ML-based attacks. In this paper, we focus on protecting Physical Unclonable Functions (PUFs) against ML-based modeling attacks. PUFs are an important cryptographic primitive for secret key generation and challenge-response authentication. However, none of the existing PUF constructions are both ML attack resistant and sufficiently lightweight to fit low-end embedded devices. We present a lightweight PUF construction, CRC-PUF, in which input challenges are de-synchronized from output responses to make a PUF model difficult to learn. The de-synchronization is done by an input transformation based on a Cyclic Redundancy Check (CRC). By changing the CRC generator polynomial for each new response, we assure that success probability of recovering the transformed

Physical Unclonable Functions (PUFs) have been proposed as a tamper-resistant alternative to the traditional methods for secret key generation and challenge-response authentication. Although many different types of PUFs have been presented, the search for more efficient, reliable and secure PUFdesigns continues. In this paper, we introduce a new class of PUFs, called threshold PUFs. We show that, in principle, any n-input threshold logic gate can be used as a base for building an n-input PUF. This opens up the possibility of using a rich body of knowledge on threshold logic implementations for designing PUFs. As a proof of concept, we implement and evaluate binary and ternary PUFs based on recently proposed threshold logic flip-flops.