Change search
Link to record
Permanent link

Direct link
BETA
Alternative names
Publications (10 of 70) Show all publications
Lv, Z., ur Réhman, S., Khan, M. S. & Li, H. (2020). An anaglyph 2D-3D stereoscopic video visualization approach. Multimedia tools and applications, 79(1-2), 825-838
Open this publication in new window or tab >>An anaglyph 2D-3D stereoscopic video visualization approach
2020 (English)In: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721, Vol. 79, no 1-2, p. 825-838Article in journal (Refereed) Published
Abstract [en]

In this paper, we propose a simple anaglyph 3D stereo generation algorithm from 2D video sequence with a monocular camera. In our novel approach, we employ camera pose estimation method to directly generate stereoscopic 3D from 2D video without building depth map explicitly. Our cost-effective method is suitable for arbitrary real-world video sequence and produces smooth results. We use image stitching based on plane correspondence using fundamental matrix. To this end, we also demonstrate that correspondence plane image stitching based on Homography matrix only cannot generate a better result. Furthermore, we utilize the structure-from-motion (with fundamental matrix) based reconstructed camera pose model to accomplish visual anaglyph 3D illusion. The anaglyph result is visualized by a contour based yellow-blue 3D color code. The proposed approach demonstrates a very good performance for most of the video sequences in the user study.

Place, publisher, year, edition, pages
Springer, 2020
Keywords
2D to 3D conversion, 3D video, Anaglyph, Structure from motion, Cameras, Cost effectiveness, Stereo image processing, Video recording, 3-D videos, Camera pose estimation, Cost-effective methods, Generation algorithm, Homography matrices, Three dimensional computer graphics
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-268547 (URN)10.1007/s11042-019-08172-1 (DOI)2-s2.0-85073832433 (Scopus ID)
Note

QC 20200323

Available from: 2020-03-23 Created: 2020-03-23 Last updated: 2020-03-23Bibliographically approved
Shao, W.-Z., Liu, Y.-Y., Ye, L.-Y., Wang, L.-Q., Ge, Q., Bao, B.-K. & Li, H. (2020). DeblurGAN plus: Revisiting blind motion deblurring using conditional adversarial networks. Signal Processing, 168, Article ID UNSP 107338.
Open this publication in new window or tab >>DeblurGAN plus: Revisiting blind motion deblurring using conditional adversarial networks
Show others...
2020 (English)In: Signal Processing, ISSN 0165-1684, E-ISSN 1872-7557, Vol. 168, article id UNSP 107338Article in journal (Refereed) Published
Abstract [en]

This work studies dynamic scene deblurring (DSD) of a single photograph, mainly motivated by the very recent DeblurGAN method. It is discovered that training the generator alone of DeblurGAN will result in both regular checkerboard effects and irregular block color excursions unexpectedly. In this paper, two aspects of endeavors are made for a more effective and robust adversarial learning approach to DSD. On the one hand, a kind of opposite-channel -based discriminative priors is developed, improving the deblurring performance of DeblurGAN without additional computational burden in the testing phase. On the other hand, a computationally more efficient while architecturally more robust auto -encoder is developed as a substitute of the original generator in DeblurGAN, promoting DeblurGAN to a new state-of-the-art method for DSD. For brevity, the proposed approach is dubbed as DeblurGAN+. Experimental results on the benchmark GoPro dataset validate that DeblurGAN+ achieves more than 1.5 dB improvement than DeblurGAN in terms of PSNR as trained utilizing the same amount of data. More importantly, the results on realistic non -uniform blurred images demonstrate that DeblurGAN+ is really more effective than DeblurGAN as well as most of variational model-based methods in terms of both blur removal and detail recovery.

Place, publisher, year, edition, pages
ELSEVIER, 2020
Keywords
Blind deconvolution, Dynamic scene deblurring, Discriminative priors, Adversarial learning, Encoder-decoder
National Category
Signal Processing
Identifiers
urn:nbn:se:kth:diva-266416 (URN)10.1016/j.sigpro.2019.107338 (DOI)000503095100005 ()2-s2.0-85073690656 (Scopus ID)
Note

QC 20200123

Available from: 2020-01-23 Created: 2020-01-23 Last updated: 2020-01-23Bibliographically approved
Qian, Y., Xie, S., Zhuang, W. & Li, H. (2020). Sparse-View CT Reconstruction Based on Improved Re-Sidual Network. In: Computational and Experimental Simulations in Engineering: Proceedings of ICCES2019 (pp. 1069-1080). Springer
Open this publication in new window or tab >>Sparse-View CT Reconstruction Based on Improved Re-Sidual Network
2020 (English)In: Computational and Experimental Simulations in Engineering: Proceedings of ICCES2019, Springer, 2020, p. 1069-1080Chapter in book (Refereed)
Abstract [en]

With the development of CT imaging, people have higher requirements for the quality of CT image reconstruction. It is desirable to use as low as reasonably achievable X-ray dose while meeting the quality of imaging requirements. Sparse-view reconstruction is a valid measure to resolve the radiation dose problem. Owing to the angular range of projection data does not satisfy the data completeness condition, sparse-view reconstruction has always been a conundrum in CT image reconstruction. In this paper, we introduces a new CT sparse-view reconstruction algorithm, which bases on the residual network. We optimize traditional residual models by improving the superfluous modules and reducing unnecessary calculations. Compared to several other classic methods, the experimental results with our network obtained better consequent, regarding artifact reduction, feature preservation, and computational speed.

Place, publisher, year, edition, pages
Springer, 2020
Series
Mechanisms and Machine Science, ISSN 2211-0984 ; 75
Keywords
CT image reconstruction, Residual network, Sparse-view
National Category
Mechanical Engineering
Identifiers
urn:nbn:se:kth:diva-267863 (URN)10.1007/978-3-030-27053-7_92 (DOI)2-s2.0-85075553053 (Scopus ID)978-3-030-27052-0 (ISBN)
Note

QC 20200227

Available from: 2020-02-27 Created: 2020-02-27 Last updated: 2020-02-27Bibliographically approved
Cheng, X., Yang, B., Tan, K., Isaksson, E., Li, L., Hedman, A., . . . Li, H. (2019). A Contactless Measuring Method of Skin Temperature based on the Skin Sensitivity Index and Deep Learning. Applied Sciences, 9(7), Article ID 1375.
Open this publication in new window or tab >>A Contactless Measuring Method of Skin Temperature based on the Skin Sensitivity Index and Deep Learning
Show others...
2019 (English)In: Applied Sciences, E-ISSN 2076-3417, Vol. 9, no 7, article id 1375Article in journal (Refereed) Published
Abstract [en]

Featured Application The NISDL method proposed in this paper can be used for real time contactless measuring of human skin temperature, which reflects human body thermal comfort status and can be used for control HVAC devices. Abstract In human-centered intelligent building, real-time measurements of human thermal comfort play critical roles and supply feedback control signals for building heating, ventilation, and air conditioning (HVAC) systems. Due to the challenges of intra- and inter-individual differences and skin subtleness variations, there has not been any satisfactory solution for thermal comfort measurements until now. In this paper, a contactless measuring method based on a skin sensitivity index and deep learning (NISDL) was proposed to measure real-time skin temperature. A new evaluating index, named the skin sensitivity index (SSI), was defined to overcome individual differences and skin subtleness variations. To illustrate the effectiveness of SSI proposed, a two multi-layers deep learning framework (NISDL method I and II) was designed and the DenseNet201 was used for extracting features from skin images. The partly personal saturation temperature (NIPST) algorithm was use for algorithm comparisons. Another deep learning algorithm without SSI (DL) was also generated for algorithm comparisons. Finally, a total of 1.44 million image data was used for algorithm validation. The results show that 55.62% and 52.25% error values (NISDL method I, II) are scattered at (0 degrees C, 0.25 degrees C), and the same error intervals distribution of NIPST is 35.39%.

Place, publisher, year, edition, pages
MDPI, 2019
Keywords
contactless measurements, skin sensitivity index, thermal comfort, subtleness magnification, deep learning, piecewise stationary time series
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-254118 (URN)10.3390/app9071375 (DOI)000466547500110 ()2-s2.0-85064083775 (Scopus ID)
Note

QC 20190624

Available from: 2019-06-24 Created: 2019-06-24 Last updated: 2019-06-24Bibliographically approved
Xie, S., Xu, H. & Li, H. (2019). Artifact Removal Using GAN Network for Limited-Angle CT Reconstruction. In: 2019 9th International Conference on Image Processing Theory, Tools and Applications, IPTA 2019: . Paper presented at Ninth International Conference on Image Processing Theory, Tools and Applications, IPTA 2019, Istanbul, Turkey, November 6-9, 2019.
Open this publication in new window or tab >>Artifact Removal Using GAN Network for Limited-Angle CT Reconstruction
2019 (English)In: 2019 9th International Conference on Image Processing Theory, Tools and Applications, IPTA 2019, 2019Conference paper, Published paper (Refereed)
Abstract [en]

Computed tomography (CT) plays an increasingly important role in clinical diagnosis. However, in practical applications of CT, physical limitations on acquisition lead to some blind regions where data cannot be sampled. CT image reconstruction from limited-angle would enable a rapid scanning with a reduced x-ray dose delivered to the patient. As it is known, Generative Adversarial Networks (GAN) can keep the original information and details of the sample very well. In this paper, we propose an end-to-end Generative Adversarial Networks model used for removing artifacts from limited-angle CT reconstruction images. The proposed GAN is based on the conditional GAN with the joint loss function, which .can remove the artifacts while retaining the complete details and sharp edges. The intensity of reconstruction using the proposed method is very close to the full-view projective reconstructed image. Compared to several other classic methods, our GAN model shows better consequent, in terms of artifact reduction, feature preservation, and computational efficiency for limited-angle CT reconstruction.

National Category
Medical Image Processing
Identifiers
urn:nbn:se:kth:diva-268219 (URN)10.1109/IPTA.2019.8936113 (DOI)2-s2.0-85077956188 (Scopus ID)
Conference
Ninth International Conference on Image Processing Theory, Tools and Applications, IPTA 2019, Istanbul, Turkey, November 6-9, 2019
Note

QC 20200406

Available from: 2020-04-06 Created: 2020-04-06 Last updated: 2020-04-06Bibliographically approved
Cheng, X., Yang, B., Hedman, A., Olofsson, T., Li, H. & Van Gool, L. (2019). NIDL: A pilot study of contactless measurement of skin temperature for intelligent building. Energy and Buildings, 198, 340-352
Open this publication in new window or tab >>NIDL: A pilot study of contactless measurement of skin temperature for intelligent building
Show others...
2019 (English)In: Energy and Buildings, ISSN 0378-7788, E-ISSN 1872-6178, Vol. 198, p. 340-352Article in journal (Refereed) Published
Abstract [en]

Human thermal comfort measurement plays a critical role in giving feedback signals for building energy efficiency. A contactless measuring method based on subtleness magnification and deep learning (NIDL) was designed to achieve a comfortable, energy efficient built environment. The method relies on skin feature data, e.g., subtle motion and texture variation, and a 315-layer deep neural network for constructing the relationship between skin features and skin temperature. A physiological experiment was conducted for collecting feature data (1.44 million) and algorithm validation. The contactless measurement algorithm based on a partly-personalized saturation temperature model (NIPST) was used for algorithm performance comparisons. The results show that the mean error and median error of the NIDL are 0.476 degrees C and 0.343 degrees C which is equivalent to accuracy improvements of 39.07% and 38.76%, respectively.

Place, publisher, year, edition, pages
Elsevier, 2019
Keywords
Contactless method, Thermal comfort measurement, Vision-based subtleness magnification, Deep learning, Intelligent building
National Category
Building Technologies
Identifiers
urn:nbn:se:kth:diva-255723 (URN)10.1016/j.enbuild.2019.06.007 (DOI)000477091800027 ()2-s2.0-85067305627 (Scopus ID)
Note

QC 20190814

Available from: 2019-08-14 Created: 2019-08-14 Last updated: 2020-03-09Bibliographically approved
Xie, S., Zheng, X., Shao, W.-Z., Zhang, Y.-D., Lv, T. & Li, H. (2019). Non-Blind Image Deblurring Method by the Total Variation Deep Network. IEEE Access, 7, 37536-37544
Open this publication in new window or tab >>Non-Blind Image Deblurring Method by the Total Variation Deep Network
Show others...
2019 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 7, p. 37536-37544Article in journal (Refereed) Published
Abstract [en]

There are a lot of non-blind image deblurring methods, especially with the total variation (TV) model-based method. However, how to choose the parameters adaptively for regularization is a major open problem. We proposed a very novel method that is based on the TV deep network to learn the best parameters adaptively for regularization. We used deep learning and prior knowledge to set up a TV-based deep network and calculate the parameters of regularization, such as biases and weights. Therefore, we used the idea of a deep network to update these parameters automatically to avoid sophisticated calculations. Our experimental results by our proposed network are significantly better than several other methods, in respect of detail retention and anti-noise performance. At the same time, we can achieve the same effect with a minimum number of training sets, thus speeding up the calculation.

Place, publisher, year, edition, pages
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2019
Keywords
Non-blind image deblurring, total variation model, deep learning
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:kth:diva-249827 (URN)10.1109/ACCESS.2019.2891626 (DOI)000463637800001 ()2-s2.0-85064741851 (Scopus ID)
Note

QC 20190423

Available from: 2019-04-23 Created: 2019-04-23 Last updated: 2020-03-09Bibliographically approved
Shao, W.-Z., Ge, Q., Wang, L.-Q., Lin, Y.-Z., Deng, H.-S. & Li, H. (2019). Nonparametric Blind Super-Resolution Using Adaptive Heavy-Tailed Priors. Journal of Mathematical Imaging and Vision, 61(6), 885-917
Open this publication in new window or tab >>Nonparametric Blind Super-Resolution Using Adaptive Heavy-Tailed Priors
Show others...
2019 (English)In: Journal of Mathematical Imaging and Vision, ISSN 0924-9907, E-ISSN 1573-7683, Vol. 61, no 6, p. 885-917Article in journal (Refereed) Published
Abstract [en]

Single-image nonparametric blind super-resolution is a fundamental image restoration problem yet largely ignored in the past decades among the computational photography and computer vision communities. An interesting phenomenon is observed that learning-based single-image super-resolution (SR) has been experiencing a rapid development since the boom of the sparse representation in 2005s and especially the representation learning in 2010s, wherein the high-res image is generally blurred by a supposed bicubic or Gaussian blur kernel. However, the parametric assumption on the form of blur kernels does not hold in most practical applications because in real low-res imaging a high-res image can undergo complex blur processes, e.g., Gaussian-shaped kernels of varying sizes, ellipse-shaped kernels of varying orientations, curvilinear kernels of varying trajectories. The paper is mainly motivated by one of our previous works: Shao and Elad (in: Zhang (ed) ICIG 2015, Part III, Lecture notes in computer science, Springer, Cham, 2015). Specifically, we take one step further in this paper and present a type of adaptive heavy-tailed image priors, which result in a new regularized formulation for nonparametric blind super-resolution. The new image priors can be expressed and understood as a generalized integration of the normalized sparsity measure and relative total variation. Although it seems that the proposed priors are simple, the core merit of the priors is their practical capability for the challenging task of nonparametric blur kernel estimation for both super-resolution and deblurring. Harnessing the priors, a higher-quality intermediate high-res image becomes possible and therefore more accurate blur kernel estimation can be accomplished. A great many experiments are performed on both synthetic and real-world blurred low-res images, demonstrating the comparative or even superior performance of the proposed algorithm convincingly. Meanwhile, the proposed priors are demonstrated quite applicable to blind image deblurring which is a degenerated problem of nonparametric blind SR.

Place, publisher, year, edition, pages
Springer, 2019
Keywords
Super-resolution, Blind deconvolution, Camera shake deblurring, Discriminative models, Convolutional neural networks, Normalized sparsity, Relative total variation
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-255571 (URN)10.1007/s10851-019-00876-1 (DOI)000475765100008 ()2-s2.0-85062626918 (Scopus ID)
Note

QC 20190802

Available from: 2019-08-02 Created: 2019-08-02 Last updated: 2019-08-02Bibliographically approved
Shao, W.-Z., Xu, J.-J., Chen, L., Ge, Q., Wang, L.-Q., Bao, B.-K. & Li, H. (2019). On potentials of regularized Wasserstein generative adversarial networks for realistic hallucination of tiny faces. Neurocomputing, 364, 1-15
Open this publication in new window or tab >>On potentials of regularized Wasserstein generative adversarial networks for realistic hallucination of tiny faces
Show others...
2019 (English)In: Neurocomputing, ISSN 0925-2312, E-ISSN 1872-8286, Vol. 364, p. 1-15Article in journal (Refereed) Published
Abstract [en]

Super-resolution of facial images, a.k.a. face hallucination, has been intensively studied in the past decades due to the increasingly emerging analysis demands in video surveillance, e.g., face detection, verification, identification. However, the actual performance of most previous hallucination approaches will drop dramatically when a very low-res tiny face is provided, due to the challenging multimodality of the problem as well as lack of an informative prior as a strong semantic guidance. Inspired by the latest progress in deep unsupervised learning, this paper focuses on tiny faces of size 16 x 16 pixels, hallucinating them to their 8 x upsampling versions by exploring the potentials of Wasserstein generative adversarial networks (WGAN). Besides a pixel-wise L2 regularization term imposed to the generative model, it is found that our advocated autoencoding generator with both residual and skip connections is a critical component for WGAN representing the facial contour and semantic content to a reasonable precision. With the additional Lipschitz penalty and architectural considerations for the critic in WGAN, the proposed approach finally achieves state-of-the-art hallucination performance in terms of both visual perception and objective assessment. The cropped CelebA face dataset is primarily used to aid the tuning and analysis of the new method, termed as tfh-WGAN. Experimental results demonstrate that the proposed approach not only achieves realistic hallucination of tiny faces, but also adapts to pose, expression, illuminance and occluded variations to a great degree.

Place, publisher, year, edition, pages
ELSEVIER, 2019
Keywords
Super-resolution, Face hallucination, Wasserstein GAN, Autoencoding, ResNet, Skip connections
National Category
Computer Systems
Identifiers
urn:nbn:se:kth:diva-260159 (URN)10.1016/j.neucom.2019.07.046 (DOI)000484070700001 ()2-s2.0-85071384887 (Scopus ID)
Note

QC 20191001

Available from: 2019-10-01 Created: 2019-10-01 Last updated: 2019-10-01Bibliographically approved
Yang, B., Cheng, X., Dai, D., Olofsson, T., Li, H. & Meier, A. (2019). Real-time and contactless measurements of thermal discomfort based on human poses for energy efficient control of buildings. Building and Environment, 162, Article ID UNSP 106284.
Open this publication in new window or tab >>Real-time and contactless measurements of thermal discomfort based on human poses for energy efficient control of buildings
Show others...
2019 (English)In: Building and Environment, ISSN 0360-1323, E-ISSN 1873-684X, Vol. 162, article id UNSP 106284Article in journal (Refereed) Published
Abstract [en]

Individual thermal discomfort perception gives important feedback signals for energy efficient control of building heating, ventilation and air conditioning systems. However, there is few effective method to measure thermal discomfort status of occupants in a real-time and contactless way. A novel method based on contactless measurements of human thermal discomfort status was presented. Images of occupant poses, which are related to thermoregulation mechanisms, were captured by a digital camera and the corresponding 2D coordinates were obtained. These poses were converted into skeletal configurations. An algorithm was developed to recognize different poses related to thermal discomfort, such as hugging oneself or wiping sweat off the brow. The algorithm could recognize twelve thermal discomfort related human poses. These poses were derived from a questionnaire survey of 369 human subjects. Some other human subjects participated in the validation experiments of the proposed method. All twelve thermal discomfort related poses can be recognized effectively.

Place, publisher, year, edition, pages
PERGAMON-ELSEVIER SCIENCE LTD, 2019
Keywords
Contactless measurement, Thermal discomfort, Human pose, Machine learning
National Category
Engineering and Technology
Identifiers
urn:nbn:se:kth:diva-261035 (URN)10.1016/j.buildenv.2019.106284 (DOI)000484514400018 ()2-s2.0-85070109030 (Scopus ID)
Note

QC 20191002

Available from: 2019-10-02 Created: 2019-10-02 Last updated: 2019-10-04Bibliographically approved
Projects
Green Video Sharing [2008-06212_VR]; Umeå UniversityGreen Video Sharing [2008-08035_VR]; Umeå UniversityIs Wyner-Ziv coding a core technique enabling next generation face recognition technology for large-scale face image retrieval? [2009-04489_VR]; Umeå University
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-3779-5647

Search in DiVA

Show all publications