Due to the complexity and diversity of deep convolutional neural networks (CNNs), Network-on-chip (NoC) based CNN accelerators have grown in popularity to improve inference efficiency and flexibility. Current optimization approaches focus on computational-heavy layers. Therefore, pooling layers are often ignored and processed individually using general processing units. In this work, we explore the acceleration of pooling layers by in-network processing. We propose a pooling on-the-go method to do the pooling operations while transmitting its prior layer outputs. Consequently, we combine the pooling layer with its prior convolution layer to remove unnecessary data movements. We demonstrate our method on a cycle-accurate NoC-CNN accelerator simulator on two CNN models, LeNet and VGG16. The results show that the processing time of individual pooling layers is almost eliminated by around 99%. Compared with the pooling standalone baseline, we can achieve 1.09x speedup in the full LeNet model, and up to 1.16x speedup in the combined layers that our approach applies.

Network-on-Chip (NoC) based architectures are recently proposed to accelerate deep neural networks in specialized hardware. Given that the hardware configuration is fixed post-manufacture, proper task mapping attracts researchers’ interest. We propose a travel time-based task mapping method that allocates uneven counts of tasks across different Processing Elements (PEs). This approach utilizes the travel time recorded in the sampling window and implicitly makes use of static NoC architecture information and dynamic NoC congestion status. Furthermore, we examine the effectiveness of our method under various configurations, including different mapping iterations, flit sizes, and NoC architectures. Our method achieves up to 12.1% improvement compared with even mapping and static distance mapping for one layer. For a complete NN example, our method achieves 10.37% and 13.75% overall improvements to row-major mapping and distance-based mapping, respectively. While ideal travel time-based mapping (post-run) achieves 10.37% overall improvements to row-major mapping, we adopt a sampling window to efficiently map tasks during the running, achieving 8.17% (sampling window 10) improvement.

Network-on-Chip (NoC) based Deep Neural Net-work (DNN) accelerators are widely adopted, but their performance is still not satisfactory as the network congestion may enlarge the inference latency. In this work, we leverage the idea of in-network processing and propose a computation-while-blocking method to conduct activation in network that improves inference latency for NoC-based DNN accelerators. Our approach offloads the non-linear activation from processing elements (PEs) to network routers. Based on a cycle-accurate NoC-DNN simulator, we experiment on a popular neural network model LeNet. The proposed approach can achieve up to 12% speedup in the first layer, and an overall around 6% decrease in total cycles compared to the baseline.

Gait analysis helps clinical assessment and achieves comfortable prosthetic designs for lower limb amputees, in which accurate gait phase recognition is a key component. However, gait phase detection remains a challenge due to the individual nature of prosthetic sockets and limbs. For the first time, we present a gait phase recognition approach for transfemoral amputees based on intra-socket pressure measurement. We proposed a multiple GMM-HMM (Hidden Markov Model with Gaussian Mixture Model emissions) method to label the gait events during walking. For each of the gait phases in the gait cycle, a separate GMM-HMM model is trained from the collected pressure data. We use gait phase recognition accuracy as a primary metric. The evaluation of six human subjects during walking shows a high accuracy of over 99% for single-subject, around 97.4% for multiple-subject, and up to 84.5% for unseen-subject scenarios. We compare our approach with the widely used CHMM (Continuous HMM) and LSTM (Long Short-term Memory) based methods, demonstrating better recognition accuracy performance across all scenarios.

Object detection is one of the most fundamental problems in computer vision, and image sensors are commonly used for this. In this paper, we present the impact of image sensor faults on pruned neural networks for object detection. We compare the error sensitivities of networks after network slimming, networks after magnitude-based pruning, and native compact models. We also explore different spatial fault types with three intensities. Furthermore, we have developed a temporal error model based on realistic aging image sensor faults. The results illuminate that the performance on clean images is important as the mean Average Precision (mAP) experiences a decrease with an increase in injected faults. Additionally, we demonstrate that the size of the model does not invariably yield a decisive impact on error tolerance when comparing small models such as pruned models and native compact models.

The use of artificial intelligence (AI) in sensor analytics is entering a new era based on the use of ubiquitous embedded connected devices. This transformation requires the adoption of design techniques that reconcile accurate results with sustainable system architectures. As such, improving the efficiency of AI hardware engines as well as backward compatibility must be considered. In this paper, we present the Hybrid-Float6 (HF6) quantization and its dedicated hardware design. We propose an optimized multiply-accumulate (MAC) hardware by reducing the mantissa multiplication to a multiplexor-adder operation. We exploit the intrinsic error tolerance of neural networks to further reduce the hardware design with approximation. To preserve model accuracy, we present a quantization-aware training (QAT) method, which in some cases improves accuracy. We demonstrate this concept in 2D convolution layers. We present a lightweight tensor processor (TP) implementing a pipelined vector dot-product. For compatibility and portability, the 6-bit floating-point (FP) is wrapped in the standard FP format, which is automatically extracted by the proposed hardware. The hardware/software architecture is compatible with TensorFlow (TF) Lite. We evaluate the applicability of our approach with a CNN-regression model for anomaly localization in a structural health monitoring (SHM) application based on acoustic emission (AE). The embedded hardware/software framework is demonstrated on XC7Z007S as the smallest Zynq-7000 SoC. The proposed implementation achieves a peak power efficiency and run-time acceleration of 5.7 GFLOPS/s/W and 48.3x, respectively.

Non-negative matrix factorization (NMF) is an ef-fective method for dimensionality reduction and sparse decom-position. This method has been of great interest to the scien-tific community in applications including signal processing, data mining, compression, and pattern recognition. However, NMF implies elevated computational costs in terms of performance and energy consumption, which is inadequate for embedded applications. To overcome this limitation, we implement the vector dot-product with hybrid logarithmic approximation as a hardware optimization approach. This technique accelerates floating-point computation, reduces energy consumption, and preserves accuracy. To demonstrate our approach, we employ a design exploration flow using high-level synthesis on an embedded FPGA. Compared with software solutions on ARM CPU, this hardware implementation accelerates the overall computation to decompose matrix by 5.597 × and reduces energy consumption by 69.323×. Log approximation NMF combined with KNN(k-nearest neighbors) has only 2.38% decreasing accuracy compared with the result of KNN processing the matrix after floating-point NMF on MNIST. Further on, compared with a dedicated floating-point accelerator, the logarithmic approximation approach achieves 3.718× acceleration and 8.345× energy reduction. Compared with the fixed-point approach, our approach has an accuracy degradation of 1.93% on MNIST and an accuracy amelioration of 28.2% on the FASHION MNIST data set without pre-knowledge of the data range. Thus, our approach has better compatibility with the input data range.

Automated driving vehicles have shown glorious potential in the near future market due to the high safety and convenience for drivers and passengers. Image sensors' reliability attract many researchers' interests as many image sensors are used in autonomous vehicles. We propose an online image sensor fault detection method based on comparing the historical variances of normal pixels and defective pixels to detect faults. For fault pixels without uncertainty, with a detecting window of more than 30 frames, we get 100% accuracy and 100% recall on realistic continuous traffic pictures from the KITTI data set. We also explore the influence of fault pixel values' uncertainty from 0% to 25% and study different fixed thresholds and a dynamic threshold for judgments. Strict threshold, which is 0.1, has a high accuracy (99.16%) but has a low recall (34.46%) for 15% uncertainty. Loose threshold, which is 0.3, has a relatively high recall (83.78%) but mistakes too many normal pixels with 18.17% accuracy for 15% uncertainty. Our dynamic threshold balances the accuracy and recall. It gets 100% accuracy and 58.69% recall for 5% uncertainty and 78.38% accuracy and 55.39% recall for 15% uncertainty. Based on the detected damage pixel rate, we develop a health score for evaluating the image sensor system intuitively. It can also be helpful for making decision about replacing cameras.

The irregular pressure exerted by a prosthetic socket over the residual limb is one of the major factors that cause the discomfort of amputees using artificial limbs. By deploying the wearable sensors inside the socket, the interfacial pressure distribution can be studied to find the active regions and rectify the socket design. In this case study, a clustering-based analysis method is presented to evaluate the density and layout of these sensors, which aims to reduce the local redundancy of the sensor deployment. In particular, a Self-Organizing Map (SOM) and K-means algorithm are employed to find the clustering results of the sensor data, taking the pressure measurement of a predefined sensor placement as the input. Then, one suitable clustering result is selected to detect the layout redundancy from the input area. After that, the Pearson correlation coefficient (PCC) is used as a similarity metric to guide the removal of redundant sensors and generate a new sparser layout. The Jenson-Shannon Divergence (JSD) and the mean pressure are applied as posterior validation metrics that compare the pressure features before and after sensor removal. A case study of a clinical trial with two sensor strips is used to prove the utility of the clustering-based analysis method. The sensors on the posterior and medial regions are suggested to be reduced, and the main pressure features are kept. The proposed method can help sensor designers optimize sensor configurations for intra-socket measurements and thus assist the prosthetists in improving the socket fitting.

Pressure sensing in prosthetic sockets is valuable as it provides quantified data to assist prosthetists in designing comfortable sockets for amputees. We present a wearable pressure sensing system for lower limb amputees. The full system consists of three essential elements from sensing scheme (wearable sensors, sensor calibration and deployment), electronic measurement system (embedded hardware and software), to time-series database and visualization. The full system has been successfully applied in clinical trials to effectively collect pressure data in real-time.