In Computer Vision (CV), the deployment of Convolutional Neural Networks (CNNs) is often hindered by their substantial computational requirements and large labeled datasets. Self-supervised learning (SSL) serves as an effective approach to reducing the reliance on labeled data with the option of augmentation methods to infer and train CNNs. Excluding irrelevant features accelerates learning and improves optimization. We propose a Field-Programmable Gate Array (FPGA)-based hardware accelerator architecture tailored for SSL framework, leveraging its parallelism and reconfigurability to expedite block matching, optimize sparse convolutions, and manage data reuse, significantly improving resource and energy efficiency. The implementation and evaluation of our work on Xilinx ZCU102 FPGA working at 200 MHz confirm that the similarity finding part's FPGA accelerations with a low hardware overhead generates a latency of 0.0106 seconds, surpassing GPU and CPU, and in the sparse CNN's FPGA acceleration part, with the processing of VGG16 and ResNet50, compared with the related FPGA-based works, our design claims a maximum of 3.08× throughput improvement and 1.5× in energy efficiency.

Associative memory is a cornerstone of cognitive intelligence within the human brain. The Bayesian confidence propagation neural network (BCPNN), a cortex-inspired model with high biological plausibility, has proven effective in emulating high-level cognitive functions like associative memory. However, the current approach using GPUs to simulate BCPNN-based associative memory tasks encounters challenges in latency and power efficiency as the model size scales. This work proposes a scalable multi-FPGA high performance computing (HPC) architecture designed for the associative memory system. The architecture integrates a set of hypercolumn unit (HCU) computing cores for intra-board online learning and inference, along with a spike-based synchronization scheme for inter-board communication among multiple FPGAs. Several design strategies, including population-based model mapping, packet-based spike synchronization, and cluster-based timing optimization, are presented to facilitate the multi-FPGA implementation. The architecture is implemented and validated on two Xilinx Alveo U50 FPGA cards, achieving a maximum model size of 200x10 and a peak working frequency of 220 MHz for the associative memory system. Both the memory-bounded spatial scalability and compute-bounded temporal scalability of the architecture are evaluated and optimized, achieving a maximum scale-latency ratio (SLR) of 268.82 for the two-FPGA implementation. Compared to a two-GPU counterpart, the two-FPGA approach demonstrates a maximum latency reduction of 51.72x and a power reduction exceeding 5.28x under the same network configuration. Compared with the state-of-the-art works, the two-FPGA implementation exhibits a high pattern storage capacity for the associative memory task.

Evaluating the computational accuracy of Spiking Neural Network (SNN) implemented as in-situ learning on large-scale memristor crossbars remains a challenge due to the lack of a versatile model for the variations in non-ideal memristors. This brief proposes a novel behavioral variation model along with a four-stage pipeline for physical memristors. The proposed variation model combines both absolute and relative variations. Therefore, it can better characterize different memristor cycle-to-cycle (C2C) variations in practice. The proposed variation model has been used to simulate the behavior of two physical memristors. Adopting the non-ideal memristor model, the trace-based spiking-timing dependent plasticity (STDP) unsupervised in-memristor learning system is simulated. Although the synaptic-level weight simulation shows a performance degradation of 7.99% and 4.07% increase in the relative root mean square error (RRMSE), the network-level simulation results show no accuracy loss on the MNIST benchmark. Furthermore, the impacts of absolute and relative C2C variations on network performance are simulated and analyzed through two sets of univariate experiments.

The memristor has been extensively used to facilitate the synaptic online learning of brain-inspired spiking neural networks (SNNs). However, the current memristor-based work can not support the widely used yet sophisticated trace-based learning rules, including the trace-based Spike-Timing-Dependent Plasticity (STDP) and the Bayesian Confidence Propagation Neural Network (BCPNN) learning rules. This paper proposes a learning engine to implement trace-based online learning, consisting of memristor-based blocks and analog computing blocks. The memristor is used to mimic the synaptic trace dynamics by exploiting the nonlinear physical property of the device. The analog computing blocks are used for the addition, multiplication, logarithmic and integral operations. By organizing these building blocks, a reconfigurable learning engine is architected and realized to simulate the STDP and BCPNN online learning rules, using memristors and 180 nm analog CMOS technology. The results show that the proposed learning engine can achieve energy consumption of 10.61 pJ and 51.49 pJ per synaptic update for the STDP and BCPNN learning rules, respectively, with a 147.03× and 93.61× reduction compared to the 180 nm ASIC counterparts, and also a 9.39× and 5.63× reduction compared to the 40 nm ASIC counterparts. Compared with the state-of-the-art work of Loihi and eBrainII, the learning engine can reduce the energy per synaptic update by 11.31× and 13.13× for trace-based STDP and BCPNN learning rules, respectively.

Optoelectronic memristors are suitable candidates for hardware implementation of optical synapses in spiking neural networks (SNNs), thanks to their electrical and optical characteristics. To study the feasibility of memristor-based optical synapses in SNNs, a behavior model for optoelectronic memristors is proposed in this paper, including electrical programming modeling and photocurrent read modeling. Based on the model, the behavior of a molecular ferroelectric (MF)/semiconductor interfacial memristor is simulated. This paper also proposes an optical synaptic circuit for trace-based spike-timing-dependent plasticity (STDP) learning rule. The electrical characteristics of the memristor are explored and exploited to emulate the trace in the pairwise nearest-neighbor STDP, while the optical characteristics are utilized for non-destructive readout and weight calculation. Synaptic-level simulation results show a 99.96% correlation coefficient (CC) and a 1.91% relative root mean square error (RRMSE) in the weight approximate computation. Extending the simulation to the network level, the optoelectronic memristor-based unsupervised STDP learning system can achieve a 92.07± 0.64% accuracy on the MNIST benchmark.