Active Sensor Planning for Multiview Vision Tasks

An active robot system can change its visual parameters in an intentional manner and perform its sensing actions purposefully. A general vision task thus can be performed in an efficient way by means of strategic control of the perception process. The controllable processes include 3D active sensing, sensor configuration and recalibration, automatic sensor placement, and 3D sensing. This book explores these important issues in studying for active visual perception.

Vision sensors have limited fields of views and can only "see" a portion of a scene from a single viewpoint. To make the entire object visible, the sensor has to be moved from one place to another around the object to observe all features of interest. The sensor planning presented in this book describes an effective strategy to generate a sequence of viewing poses and sensor settings for optimally completing a perception task. Several methods are proposed to solve the problems in both model-based and nonmodel-based vision tasks. For model-based applications, the method involves determination of the optimal sensor placements and a shortest path through these viewpoints for automatic generation of a perception plan. A topology of viewpoints is achieved by a genetic algorithm in which a min-max criterion is used for evaluation. A shortest path is also determined by graph algorithms. For nonmodel-based applications, the method involves determination of the best next view and sensor settings. The trend surface is proposed as the cue to predict the unknown portion of an object or environment.

The 11 chapters in Active Vision Planning draw on recent work in robot vision over ten years, particularly in the use of new concepts of active sensing, reconfiguration, recalibration, sensor model, sensing constraints, sensing evaluation, viewpoint decision, sensor placement graph, model based planning, path planning, planning for robot in unknown environment, dynamic 3D construction,surface prediction, etc. Implementation examples are also provided with theoretical methods for testing in a real robot system. With these optimal sensor planning strategies, this book will give the robot vision system the adaptability needed in many practical applications.

Xu Cheng (Senior Member, IEEE) received his Ph.D. degree in Engineering from the Department of Ocean Operations and Civil Engineering, Intelligent Systems Laboratory, Norwegian University of Science and Technology (NTNU), Ålesund, Norway, in June 2020. From June 2020 to March 2022, he worked as a postdoctoral fellow, and researcher at the Department of Manufacturing and Civil Engineering, Gjøvik, Norway. From April 2022, he worked at Smart Innovation Norway as a permanent researcher. He has applied for and coordinated more than 5 projects supported by the Norwegian Research Council (NFR), the EU, and industry. He has published more than 130 papers as first and co-author and 1 book in Springer as first author in his research interests, including data analysis and artificial intelligence in maritime operations, time series analysis, and predictive maintenance of wind turbines. Mengna Liu is a Ph.D. candidate at the School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China, in 2023. With four years of experience as an algorithm engineer, she has developed expertise in designing algorithms and optimizing data processes. Her research interests include time series modeling and data mining. Fan Shi (Member, IEEE) is a professor at the School of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China. Dr. Shi received his Ph.D. degree from Nankai University, Tianjin, China, in 2012. From June 2018 to August 2019, he was a research scholar in West Virginia University. His research interests include machine vision, pattern recognition and optics. Xiufeng Liu received the Ph.D. degree in computer science from Aalborg University, Denmark, in 2012. He was a postdoctoral researcher at the University of Waterloo and a research scientist at IBM, Canada, from 2013 to 2014. He is currently a senior researcher at the Department of Technology, Management and Economics at the Technical University of Denmark. His research interests include smart meter data analysis, data warehousing, energy informatics, and big data. Houxiang Zhang (Senior Member, IEEE) received the Ph.D. degree in mechanical and electronic engineering and the Habilitation degree in informatics from the University of Hamburg, Hamburg, Germany, in 2003 and February 2011, respectively. He is currently a full professor with the Department of Ocean Operations and Civil Engineering, Faculty of Engineering, Norwegian University of Science and Technology (NTNU), Trondheim, Norway. Since 2004, he has been a postdoctoral fellow and a Senior Researcher with the Department of Informatics, Faculty of Mathematics, Informatics and Natural Sciences, Institute of Technical Aspects of Multimodal Systems, University of Hamburg, Hamburg, Germany. He was with NTNU, where he is currently a professor of Mechatronics in April 2011. From 2011 to 2016, he also hold a Norwegian National GIFT Professorship on product and system design funded by the Norwegian Maritime Centre of Expertise. Shengyong Chen (Senior Member, IEEE) is a full professor at Tianjin University of Technology and the director of the Engineering Research Center of Learning-Based Intelligent System (Ministry of Education). He has been conducting research on vision sensors for robotics for more than 20 years. He obtained the Ph.D. degree in computer vision from City University of Hong Kong. From 2006 to 2007, he received a fellowship from the Alexander von Humboldt Foundation of Germany and worked at University of Hamburg, Germany. From 2008 to 2012, he worked as a visiting professor at Imperial College London and University of Cambridge, U.K. He has published over 300 scientific papers in international journals and conferences, including 80 papers in IEEE Transactions. He also published 10+ books in the past years and applied 100+ patents. He received the National Outstanding Youth Foundation Award of NSFC.