In order to improve the accuracy of road information change detection, a new road information change detection method based on fractional integral and spatial neighborhood fuzzy C-means (FCM) algorithm was presented. Firstly, a new difference image was generated by the gray difference calculation of the dual phase remote sensing images after registration and geometric correction. Then, a smaller fractional integral order was used to construct the denoising image mask with eight directions on the upper and lower, left and right, and four diagonals, and the fractional integral calculation were applied to the difference images, which improved the image signal-to-noise ratio (SNR) while preserving the edge and texture details of the image. Finally, the FCM clustering method combined with neighborhood spatial information was used to calculate the difference image after denoising. The highest and lowest points of the difference image gray values were selected as the center point of cluster initialization. The Euclidean Metric of the neighborhood were used to depict different weight values, so as to characterize the influence degree of domain pixels on central pixels and eliminate invalid isolated points. Detecting probability, false alarm rate and missed alarm rate of the algorithm were evaluated by the experiment. The results show that FCM road information change detection method based on fractional integral and neighborhood spatial information can effectively extract road change information. When the integral fractional order is 0.2, the FCM smoothing parameter is 2.5, the detection probability is higher than the comparison algorithm by 18% to 46%, the false alarm rate is lower than the comparison algorithm by 15% to 38%, and the missed alarm rate is lower than the comparison algorithm by 3% to 7%. The present algorithm can achieve better results in suppressing noise information and enhancing texture details. Especially, when the center pixel is noise, due to the introduction of neighborhood information, and it is affected by the neighborhood normal pixels. The proposed method could avoid misclassification by adjusting the membership automatically, it can effectively suppress the influence of neighborhood noise points on the normal pixel classification, and reduce the false alarm rate. 2 tabs, 4 figs, 28 refs. 

We describe an approach to recognizing location from mobile devices using image-based Web search. We demonstrate the usefulness of common image search metrics applied on images captured with a camera-equipped mobile device to find matching images on the World Wide Web or other general-purpose databases. Searching the entire Web can be computationally overwhelming, so we devise a hybrid image-and-keyword searching technique. First, image-search is performed over images and links to their source Web pages in a database that indexes only a small fraction of the Web. Then, relevant keywords on these Web pages are automatically identified and submitted to an existing text-based search engine (e.g. Google) that indexes a much larger portion of the Web. Finally, the resulting image set is filtered to retain images close to the original query. It is thus possible to efficiently search hundreds of millions of images that are not only textually related but also visually relevant. We demonstrate our approach on an application allowing users to browse Web pages matching the image of a nearby location.

Finding information based on an object’s visual appearance is useful when specific keywords for the object are not known. We have developed a mobile image-based search system that takes images of objects as queries and finds relevant web pages by matching them to similar images on the web. Image-based search works well when matching full scenes, such as images of buildings or landmarks, and for matching objects when the boundary of the object in the image is available. We demonstrate the effectiveness of a simple interactive paradigm for obtaining a segmented object boundary, and show how a shape-based image matching algorithm can use the object outline to find similar images on the web.

We demonstrate an image-based approach to specifying location and finding location-based information from camera-equipped mobile devices. We introduce a point-by-photograph paradigm, where users can specify a location simply by taking pictures. Our technique uses content-based image retrieval methods to search the web or other databases for matching images and their source pages to find relevant location-based information. In contrast to conventional approaches to location detection, our method can refer to distant locations and does not require any physical infrastructure beyond mobile internet service. We have developed a prototype on a camera phone and conducted user studies to demonstrate the efficacy of our approach compared to other alternatives.

Covering objects with masking tapes is a common process for surface protection in processes like spray painting, plasma spraying, shot peening, etc. Manual taping is tedious and takes a lot of effort of the workers. The taping process is a special process which requires correct surface covering strategy and proper attachment of the masking tape for an efficient surface protection. We have introduced an automatic robot taping system consisting of a robot manipulator, a rotating platform, a 3D scanner and specially designed taping end- effectors. This paper mainly talks about the surface covering strategies for different classes of geometries. The methods and corresponding taping tools are introduced for taping of following classes of surfaces: Cylindrical/ extended surfaces, freeform surfaces with no grooves, surfaces with grooves, and rotational symmetrical surfaces. A collision avoidance algorithm is introduced for the robot taping manipulation. With further improvements on segmenting surfaces of taping parts and tape cutting mechanisms, such taping solution with the taping tool and the taping methodology can be combined as a very useful and practical taping package to assist humans in this tedious and time costly work.

Nonprehensile rearrangement is the problem of controlling a robot to interact with objects through pushing actions in order to reconfigure the objects into a predefined goal pose. In this work, we rearrange one object at a time in an environment with obstacles using an end-to-end policy that maps raw pixels as visual input to control actions without any form of engineered feature extraction. To reduce the amount of training data that needs to be collected using a real robot, we propose a simulation-to-reality transfer approach. In the first step, we model the nonprehensile rearrangement task in simulation and use deep reinforcement learning to learn a suitable rearrangement policy, which requires in the order of hundreds of thousands of example actions for training. Thereafter, we collect a small dataset of only 70 episodes of real-world actions as supervised examples for adapting the learned rearrangement policy to real-world input data. In this process, we make use of newly proposed strategies for improving the reinforcement learning process, such as heuristic exploration and the curation of a balanced set of experiences. We evaluate our method in both simulation and real setting using a Baxter robot to show that the proposed approach can effectively improve the training process in simulation, as well as efficiently adapt the learned policy to the real world application, even when the camera pose is different from simulation. Additionally, we show that the learned system not only can provide adaptive behavior to handle unforeseen events during executions, such as distraction objects, sudden changes in positions of the objects, and obstacles, but also can deal with obstacle shapes that were not present in the training process.

A dominant approach to brain mapping is to define functional regions in the brain by analyzing brain activation images obtained by PET or fMRI. In [1], it has been shown that the scale-space primal sketch provides a useful tool for such analysis. Some attractive properties of this method are that it only makes few assumptions about the data and the process for extracting activations is fully automatic.

In the present version of the scale-space primal sketch, however, there is no method for determining p-values. The purpose here is to present a new methodology for addressing this question, by introducing a descriptor referred to as the -curve, which serves as a first step towards determining the probability of false positives, i.e. alpha.

In this paper we present a modification to a latent topic model, which makes themodel exploit supervision to produce a factorized representation of the observeddata. The structured parameterization separately encodes variance that is sharedbetween classes from variance that is private to each class by the introduction of anew prior over the topic space. The approach allows for a more efficient inferenceand provides an intuitive interpretation of the data in terms of an informative signaltogether with structured noise. The factorized representation is shown to enhanceinference performance for image, text, and video classification.

We present an extension to the Hierarchical Dirichlet Process (HDP), which allows for the inclusion of supervision. Our model marries the non-parametric benefits of HDP with those of Supervised Latent Dirichlet Allocation (SLDA) to enable learning the topic space directly from data while simultaneously including the labels within the model. The proposed model is learned using variational inference which allows for the efficient use of a large training dataset. We also present the online version of variational inference, which makes the method scalable to very large datasets. We show results comparing our model to a traditional supervised parametric topic model, SLDA, and show that it outperforms SLDA on a number of benchmark datasets.

Supervised topic models are important machine learning tools whichhave been widely used in computer vision as well as in other domains. However,there is a gap in the understanding of the supervision impact on the model. Inthis paper, we present a thorough analysis on the behaviour of supervised topicmodels using Supervised Latent Dirichlet Allocation (SLDA) and propose twofactorized supervised topic models, which factorize the topics into signal andnoise. Experimental results on both synthetic data and real-world data for computer vision tasks show that supervision need to be boosted to be effective andfactorized topic models are able to enhance the performance.

In this paper, we present the Inter-Battery Topic Model (IBTM). Our approach extends traditional topic models by learning a factorized latent variable representation. The structured representation leads to a model that marries benefits traditionally associated with a discriminative approach, such as feature selection, with those of a generative model, such as principled regularization and ability to handle missing data. The factorization is provided by representing data in terms of aligned pairs of observations as different views. This provides means for selecting a representation that separately models topics that exist in both views from the topics that are unique to a single view. This structured consolidation allows for efficient and robust inference and provides a compact and efficient representation. Learning is performed in a Bayesian fashion by maximizing a rigorous bound on the log-likelihood. Firstly, we illustrate the benefits of the model on a synthetic dataset. The model is then evaluated in both uni- and multi-modality settings on two different classification tasks with off-the-shelf convolutional neural network (CNN) features which generate state-of-the-art results with extremely compact representations.

Learning about activities and object affordances from human demonstration are important cognitive capabilities for robots functioning in human environments, for example, being able to classify objects and knowing how to grasp them for different tasks. To achieve such capabilities, we propose a Labeled Multi-modal Latent Dirichlet Allocation (LM-LDA), which is a generative classifier trained with two different data cues, for instance, one cue can be traditional visual observation and another cue can be contextual information. The novel aspects of the LM-LDA classifier, compared to other methods for encoding contextual information are that, I) even with only one of the cues present at execution time, the classification will be better than single cue classification since cue correlations are encoded in the model, II) one of the cues (e.g., common grasps for the observed object class) can be inferred from the other cue (e.g., the appearance of the observed object). This makes the method suitable for robot online and transfer learning; a capability highly desirable in cognitive robotic applications. Our experiments show a clear improvement for classification and a reasonable inference of the missing data.

In this work we introduce an approach for modeling and analyzing collective behavior of a group of agents using moments. We represent the occupation measure of the group of agents by their moments and show how the dynamics of the moments can be modeled. Then approximate trajectories of the moments can be computed and an inverse problem is solved to recover macro-scale properties of the group of agents. To illustrate the theory, a numerical example with interactions between the agents is given.

Constructing discriminative features is an essential issue in developing face recognition algorithms. There are two schools in how features are constructed: hand-crafted features and learned features from data. A clear trend in the face recognition community is to use learned features to replace hand-crafted ones for face recognition, due to the superb performance achieved by learned features through Deep Learning networks. Given the negative aspects of database-dependent solutions, we consider an alternative and demonstrate that, for good generalization performance, developing face recognition algorithms by using handcrafted features is surprisingly promising when the training dataset is small or medium sized. We show how to build such a face identifier with our Block Matching method which leverages the power of the Gabor phase in face images. Although no learning process is involved, empirical results show that the performance of this “designed” identifier is comparable (superior) to state-of-the-art identifiers and even close to Deep Learning approaches.

Constructing discriminative features is an essential issue in developing face recognition algorithms. There are two schools in how features are constructed: hand-crafted features and learned features from data. A clear trend in the face recognition community is to use learned features to replace hand-crafted ones for face recognition, due to the superb performance achieved by learned features through Deep Learning networks. Given the negative aspects of database-dependent solutions, we consider an alternative and demonstrate that, for good generalization performance, developing face recognition algorithms by using hand-crafted features is surprisingly promising when the training dataset is small or medium sized. We show how to build such a face identifier with our Block Matching method which leverages the power of the Gabor phase in face images. Although no learning process is involved, empirical results show that the performance of this “designed” identifier is comparable (superior) to state-of-the-art identifiers and even close to Deep Learning approaches.

In this paper, we introduce the Block Matching (BM) as an alternative patch-based local matching approach for solving the face recognition problem. The Block Matching enables an image patch of the probe face image to search for its best matching from displaced positions in the gallery face image. This matching strategy is very effective for handling spatial shift between two images and it is radically different from that of the widely used LBP type patch-based local matching approaches. Our evaluations on the FERET and CMU-PIE databases show that the performance of this simple method is well comparable (superior) to that of the popular LBP approach. We argue that the Block Matching could provide face recognition a new approach with more flexible algorithm architecture. One can expect that it could lead to much higher performance when combining with other feature extraction techniques, like Gabor wavelet and deep learning.

Gabor features have been widely employed in solving face recognition problems in controlled scenarios. To construct discriminative face features from the complex Gabor space, the amplitude information is commonly preferred, while the other one — the phase — is not well utilized due to its spatial shift sensitivity. In this paper, we address the problem of face recognition in controlled scenarios. Our focus is on the selection of a suitable signal representation and the development of a better strategy for face feature construction. We demonstrate that through our Block Matching scheme Gabor phase information is powerful enough to improve the performance of face identification. Compared to state of the art Gabor filtering based approaches, the proposed algorithm features much lower algorithmic complexity. This is mainly due to our Block Matching enables the employment of high definition Gabor phase. Thus, a single-scale Gabor frequency band is sufficient for discrimination. Furthermore, learning process is not involved in the facial feature construction, which avoids the risk of building a database-dependent algorithm. Benchmark evaluations show that the proposed learning-free algorith outperforms state-of-the-art Gabor approaches and is even comparable to Deep Learning solutions.

Predicting attributes from face images in the wild is a challenging computer vision problem. To automatically describe face attributes from face containing images, traditionally one needs to cascade three technical blocks — face localization, facial descriptor construction, and attribute classification — in a pipeline. As a typical classification problem, face attribute preiction has been addressed using deep learning. Current state-of-the-art performance was achieved by using two cascaded Convolutional Neural Networks (CNNs), which were specifically trained to learn face localization and attribute description. In this paper, we experiment with an alternative way of employing the power of deep representations from CNNs. Combining with conventional face localization techniques, we use off-the-shelf architectures trained for face recognition to build facial descriptors. Recognizing that the describable face attributes are diverse, our face descriptors are constructed from different levels of the CNNs for different attributes to best facilitate face attribute prediction. Experiments on two large datasets, LFWA and CelebA, show that our approach is entirely comparable to the state-of-the-art. Our findings not only demonstrate an efficient face attribute prediction approach, but also raise an important question: how to leverage the power of off-the-shelf CNN representations for novel tasks

Automatic tracking and video of surveillance on a farm could help to support farm management. In this project, an automated detection system is used to detect sows in surveillance videos. This system is based upon deep learning and computer vision methods. In order to minimize disk storage and to meet the network requirements necessary to achieve the real-performance, tracking in compressed video streams is essential.

The proposed system uses a Discriminative Correlation Filter (DCF) as a classifier to detect targets. The tracking model is updated by training the classifier with online learning methods. Compression technology encodes the video data, thus reducing both the bit rates at which video signals are transmitted and helping the video transmission better adapt to the limited network bandwidth. However, compression may reduce the image quality of the videos the precision of our tracking may decrease. Hence, we conducted a performance evaluation of existing visual tracking algorithms on video sequences with quality degradation due to various compression parameters (encoders, target bitrate, rate control model, and Group of Pictures (GOP) size). The ultimate goal of video compression is to realize a tracking system with equal performance, but requiring fewer network resources.

The proposed tracking algorithm successfully tracks each sow in consecutive frames in most cases. The performance of our tracker was benchmarked against two state-of-art tracking algorithms: Siamese Fully-Convolutional (FC) and Efficient Convolution Operators (ECO). The performance evaluation result shows our proposed tracker has similar performance to both Siamese FC and ECO.

In comparison with the original tracker, the proposed tracker achieved similar tracking performance, while requiring much less storage and generating a lower bitrate when the video was compressed with appropriate parameters. However, the system is far slower than needed for real-time tracking due to high computational complexity; therefore, more optimal methods to update the tracking model will be needed to achieve real-time tracking.

The dynamic window approach (DWA) is a well-known navigation scheme developed by Fox et al. and extended by Brock and Khatib. It is safe by construction, and has been shown to perform very efficiently in experimental setups. However, one can construct examples where the proposed scheme fails to attain the goal configuration. What has been lacking is a theoretical treatment of the algorithm's convergence properties. Here we present such a treatment by merging the ideas of the DWA with the convergent, but less performance-oriented, scheme suggested by Rimon and Koditschek. Viewing the DWA as a model predictive control (MPC) method and using the control Lyapunov function (CLF) framework of Rimon and Koditschek, we draw inspiration from an MPC/CLF framework put forth by Primbs to propose a version of the DWA that is tractable and convergent.

Two broad classes of robot controllers are the modular, and the model based approaches. The modular approaches include the Reactive or Behavior Based designs. They do not rely on mathematical system models, but are easy to design, modify and extend. In the model based approaches, a model is used to design a single controller with verifiable system properties. The resulting designs are however often hard to extend, without jeopardizing the previously proven properties. This paper describes an attempt to narrow the gap between the flexibility of the modular approaches, and the predictability of the model based approaches, by proposing a modular design that does the combination, or arbitration, of the different modules in a model based way. By taking the (model based) time derivatives of scalar, Lyapunov-like, objective functions into account, the arbitration module can keep track of the time evolution of the objectives. This enables it to handle objective tradeoffs in a predictable way by finding controls that preserve an important objective that is currently met, while striving to satisfy another, less important one that is not yet achieved. To illustrate the approach a UAV control problem from the literature is solved, resulting in comparable, or better, performance.

We introduce a reconstruction framework that can account for shape related prior information in imaging-related inverse problems. It is a variational scheme that uses a shape functional, whose definition is based on deformable template machinery from computational anatomy. We prove existence and, as a proof of concept, we apply the proposed shape-based reconstruction to 2D tomography with very sparse and/or highly noisy measurements.

This report provides a mathematically thorough review and investigation of Metric Multidimensional scaling (MDS) through the analysis of Euclidean distances in input and output spaces. By combining a geometric approach with modern linear algebra and multivariate analysis, Metric MDS is viewed as a Euclidean distance embedding transformation that converts between coordinate and coordinate-free representations of data. In this work we link Mercer kernel functions, data in infinite-dimensional Hilbert space and coordinate-free distance metrics to a finite-dimensional Euclidean representation. We further set a foundation for a principled treatment of non-linear extensions of MDS as optimization programs on kernel matrices and Euclidean distances.