Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud

Guo Han Lin; Chun Hsiang Chang; Ming Chun Chung; Yu Cheng Fan

doi:10.1109/ICCE-Taiwan49838.2020.9258010

Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud

Guo Han Lin
, Chun Hsiang Chang
, Ming Chun Chung
, Yu Cheng Fan

Department of Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

We present a 'Self-driving deep learning system based on depth image based rendering and LiDAR point clouds' in this paper. Using the depth image based rendering (DIBR) to obtain complete point clouds, combining with the color image modified by histogram equalization and optimal profile compression (HEOPC), and further using MobileNet-YOLO neural network to achieve accurate 3D environment detection and identification. Using the DIBR to generate parallax map information, and obtaining the depth image through the conversion formula. Then combining LiDAR point cloud and the depth image to repair the objects in the point cloud image. We combine HEOPC and the accuracy of deep learning to optimize the color image enhancement of the night environment. According to the restored point cloud information, we perform a cutting algorithm to divide the area of interest. Then through MobileNet-YOLO neural network training, to identify the area of interest, including people, cars and bus.

Original language	English
Title of host publication	2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728173993
DOIs	https://doi.org/10.1109/ICCE-Taiwan49838.2020.9258010
State	Published - 28 Sep 2020
Event	7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020 - Taoyuan, Taiwan Duration: 28 Sep 2020 → 30 Sep 2020

Publication series

Name	2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020

Conference

Conference	7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020
Country/Territory	Taiwan
City	Taoyuan
Period	28/09/20 → 30/09/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Keywords

3D Environment Detection
DIBR
LiDAR
Point Clouds
Self-Driving Deep Learning System

Access to Document

10.1109/ICCE-Taiwan49838.2020.9258010

Cite this

Lin, G. H., Chang, C. H., Chung, M. C., & Fan, Y. C. (2020). Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud. In 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020 Article 9258010 (2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCE-Taiwan49838.2020.9258010

Lin, Guo Han ; Chang, Chun Hsiang ; Chung, Ming Chun et al. / Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud. 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020. Institute of Electrical and Electronics Engineers Inc., 2020. (2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020).

@inproceedings{bc859abf819247998f1b322fea401c01,

title = "Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud",

abstract = "We present a 'Self-driving deep learning system based on depth image based rendering and LiDAR point clouds' in this paper. Using the depth image based rendering (DIBR) to obtain complete point clouds, combining with the color image modified by histogram equalization and optimal profile compression (HEOPC), and further using MobileNet-YOLO neural network to achieve accurate 3D environment detection and identification. Using the DIBR to generate parallax map information, and obtaining the depth image through the conversion formula. Then combining LiDAR point cloud and the depth image to repair the objects in the point cloud image. We combine HEOPC and the accuracy of deep learning to optimize the color image enhancement of the night environment. According to the restored point cloud information, we perform a cutting algorithm to divide the area of interest. Then through MobileNet-YOLO neural network training, to identify the area of interest, including people, cars and bus.",

keywords = "3D Environment Detection, DIBR, LiDAR, Point Clouds, Self-Driving Deep Learning System",

author = "Lin, \{Guo Han\} and Chang, \{Chun Hsiang\} and Chung, \{Ming Chun\} and Fan, \{Yu Cheng\}",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020 ; Conference date: 28-09-2020 Through 30-09-2020",

year = "2020",

month = sep,

day = "28",

doi = "10.1109/ICCE-Taiwan49838.2020.9258010",

language = "???core.languages.en\_GB???",

series = "2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020",

}

Lin, GH, Chang, CH, Chung, MC & Fan, YC 2020, Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud. in 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020., 9258010, 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020, Institute of Electrical and Electronics Engineers Inc., 7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020, Taoyuan, Taiwan, 28/09/20. https://doi.org/10.1109/ICCE-Taiwan49838.2020.9258010

Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud. / Lin, Guo Han; Chang, Chun Hsiang; Chung, Ming Chun et al.
2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020. Institute of Electrical and Electronics Engineers Inc., 2020. 9258010 (2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud

AU - Lin, Guo Han

AU - Chang, Chun Hsiang

AU - Chung, Ming Chun

AU - Fan, Yu Cheng

PY - 2020/9/28

Y1 - 2020/9/28

N2 - We present a 'Self-driving deep learning system based on depth image based rendering and LiDAR point clouds' in this paper. Using the depth image based rendering (DIBR) to obtain complete point clouds, combining with the color image modified by histogram equalization and optimal profile compression (HEOPC), and further using MobileNet-YOLO neural network to achieve accurate 3D environment detection and identification. Using the DIBR to generate parallax map information, and obtaining the depth image through the conversion formula. Then combining LiDAR point cloud and the depth image to repair the objects in the point cloud image. We combine HEOPC and the accuracy of deep learning to optimize the color image enhancement of the night environment. According to the restored point cloud information, we perform a cutting algorithm to divide the area of interest. Then through MobileNet-YOLO neural network training, to identify the area of interest, including people, cars and bus.

AB - We present a 'Self-driving deep learning system based on depth image based rendering and LiDAR point clouds' in this paper. Using the depth image based rendering (DIBR) to obtain complete point clouds, combining with the color image modified by histogram equalization and optimal profile compression (HEOPC), and further using MobileNet-YOLO neural network to achieve accurate 3D environment detection and identification. Using the DIBR to generate parallax map information, and obtaining the depth image through the conversion formula. Then combining LiDAR point cloud and the depth image to repair the objects in the point cloud image. We combine HEOPC and the accuracy of deep learning to optimize the color image enhancement of the night environment. According to the restored point cloud information, we perform a cutting algorithm to divide the area of interest. Then through MobileNet-YOLO neural network training, to identify the area of interest, including people, cars and bus.

KW - 3D Environment Detection

KW - DIBR

KW - LiDAR

KW - Point Clouds

KW - Self-Driving Deep Learning System

UR - https://www.scopus.com/pages/publications/85098451998

U2 - 10.1109/ICCE-Taiwan49838.2020.9258010

DO - 10.1109/ICCE-Taiwan49838.2020.9258010

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85098451998

T3 - 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020

BT - 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020

Y2 - 28 September 2020 through 30 September 2020

ER -

Lin GH, Chang CH, Chung MC, Fan YC. Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud. In 2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020. Institute of Electrical and Electronics Engineers Inc. 2020. 9258010. (2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020). doi: 10.1109/ICCE-Taiwan49838.2020.9258010

Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this