MATLAB Computer Vision Toolbox™ Users Guide (The MathWorks, Inc.) (z-library.sk, 1lib.sk, z-lib.sk)

Computer Vision Toolbox™ User's Guide R2020a

How to Contact MathWorks Latest news: www.mathworks.com Sales and services: www.mathworks.com/sales_and_services User community: www.mathworks.com/matlabcentral Technical support: www.mathworks.com/support/contact_us Phone: 508-647-7000 The MathWorks, Inc. 1 Apple Hill Drive Natick, MA 01760-2098 Computer Vision Toolbox™ User's Guide © COPYRIGHT 2004–2020 by The MathWorks, Inc. The software described in this document is furnished under a license agreement. The software may be used or copied only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written consent from The MathWorks, Inc. FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification, reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or other entity acquiring for or through the federal government) and shall supersede any conflicting contractual terms or conditions. If this License fails to meet the government's needs or is inconsistent in any respect with federal procurement law, the government agrees to return the Program and Documentation, unused, to The MathWorks, Inc. Trademarks MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders. Patents MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for more information.

Revision History July 2004 First printing New for Version 1.0 (Release 14) October 2004 Second printing Revised for Version 1.0.1 (Release 14SP1) March 2005 Online only Revised for Version 1.1 (Release 14SP2) September 2005 Online only Revised for Version 1.2 (Release 14SP3) November 2005 Online only Revised for Version 2.0 (Release 14SP3+) March 2006 Online only Revised for Version 2.1 (Release 2006a) September 2006 Online only Revised for Version 2.2 (Release 2006b) March 2007 Online only Revised for Version 2.3 (Release 2007a) September 2007 Online only Revised for Version 2.4 (Release 2007b) March 2008 Online only Revised for Version 2.5 (Release 2008a) October 2008 Online only Revised for Version 2.6 (Release 2008b) March 2009 Online only Revised for Version 2.7 (Release 2009a) September 2009 Online only Revised for Version 2.8 (Release 2009b) March 2010 Online only Revised for Version 3.0 (Release 2010a) September 2010 Online only Revised for Version 3.1 (Release 2010b) April 2011 Online only Revised for Version 4.0 (Release 2011a) September 2011 Online only Revised for Version 4.1 (Release 2011b) March 2012 Online only Revised for Version 5.0 (Release 2012a) September 2012 Online only Revised for Version 5.1 (Release R2012b) March 2013 Online only Revised for Version 5.2 (Release R2013a) September 2013 Online only Revised for Version 5.3 (Release R2013b) March 2014 Online only Revised for Version 6.0 (Release R2014a) October 2014 Online only Revised for Version 6.1 (Release R2014b) March 2015 Online only Revised for Version 6.2 (Release R2015a) September 2015 Online only Revised for Version 7.0 (Release R2015b) March 2016 Online only Revised for Version 7.1 (Release R2016a) September 2016 Online only Revised for Version 7.2 (Release R2016b) March 2017 Online only Revised for Version 7.3 (Release R2017a) September 2017 Online only Revised for Version 8.0 (Release R2017b) March 2018 Online only Revised for Version 8.1 (Release R2018a) September 2018 Online only Revised for Version 8.2 (Release R2018b) March 2019 Online only Revised for Version 9.0 (Release R2019a) September 2019 Online only Revised for Version 9.1 (Release R2019b) March 2020 Online only Revised for Version 9.2 (Release R2020a)

Featured Examples 1 Localize and Read Multiple Barcodes in Image . . . . . . . . . . . . . . . . . . . . . 1-2 Monocular Visual Odometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-22 Track Vehicles Using Lidar: From Point Cloud to Track List . . . . . . . . . . 1-35 Semantic Segmentation Using Dilated Convolutions . . . . . . . . . . . . . . . . 1-55 Define Custom Pixel Classification Layer with Tversky Loss . . . . . . . . . . 1-59 Track a Face in Scene . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-66 Create 3-D Stereo Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-71 Measure Distance from Stereo Camera to a Face . . . . . . . . . . . . . . . . . . . 1-72 Reconstruct 3-D Scene from Disparity Map . . . . . . . . . . . . . . . . . . . . . . . 1-74 Visualize Stereo Pair of Camera Extrinsic Parameters . . . . . . . . . . . . . . . 1-77 Remove Distortion from an Image Using the Camera Parameters Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-80 Point Cloud Processing 2 Getting Started with Point Clouds Using Deep Learning . . . . . . . . . . . . . . 2-2 Import Point Cloud Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 Augment Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 Encode Point Cloud Data to Image-like Format . . . . . . . . . . . . . . . . . . . . . 2-3 Train a Deep Learning Classification Network with Encoded Point Cloud Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3 Point Cloud Registration Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 Point Cloud Registration Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 Point Cloud Registration Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5 Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6 The PLY Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8 File Header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8 v Contents

Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9 Common Elements and Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10 Using the Installer for Computer Vision System Toolbox Product 3 Install Computer Vision Toolbox Add-on Support Files . . . . . . . . . . . . . . . 3-2 Install OCR Language Data Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 Pretrained Language Data and the ocr function . . . . . . . . . . . . . . . . . . . . 3-3 Install and Use Computer Vision Toolbox OpenCV Interface . . . . . . . . . . . 3-6 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6 Support Package Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6 Create MEX-File from OpenCV C++ file . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Use the OpenCV Interface C++ API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Create Your Own OpenCV MEX-files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8 Run OpenCV Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8 Install and Use Computer Vision Toolbox OpenCV Interface for Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10 Import OpenCV Code into Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 Smile Detection by Using OpenCV Code in Simulink . . . . . . . . . . . . . . . . 3-19 Required Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 Set Up Your C++ Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 Step 1: Import OpenCV Function to Create a Simulink Library . . . . . . . . 3-20 Step 2: Use Generated Subsystem in Simulink Model . . . . . . . . . . . . . . . 3-25 Step 3: Simulate the Smile Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-26 Step 4: Generate C++ Code from the Smile Detector Model . . . . . . . . . . 3-26 Deploy the Smile Detector on the Raspberry Pi Hardware . . . . . . . . . . . . 3-27 Convert RGB Image to Grayscale Image by Using OpenCV Importer . . . 3-29 Required Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29 Set Up Your C++ Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29 Step 1: Import OpenCV Function to Create a Simulink Library . . . . . . . . 3-30 Step 2: Use Generated Subsystem in Simulink Model . . . . . . . . . . . . . . . 3-33 Step 3: Simulate the RGB to Gray Convertor . . . . . . . . . . . . . . . . . . . . . . 3-34 Draw Different Shapes by Using OpenCV Code in Simulink . . . . . . . . . . 3-36 Required Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-36 Set Up Your C++ Compiler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-36 Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-36 Step 1: Import OpenCV Function to Create a Simulink Library . . . . . . . . 3-37 Step 2: Use Generated Subsystem in Simulink Model . . . . . . . . . . . . . . . 3-38 Draw Atom on Image by Using C Caller Block . . . . . . . . . . . . . . . . . . . . . 3-38 vi Contents

Input, Output, and Conversions 4 Export to Video Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2 Setting Block Parameters for this Example . . . . . . . . . . . . . . . . . . . . . . . . 4-2 Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3 Import from Video Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4 Setting Block Parameters for this Example . . . . . . . . . . . . . . . . . . . . . . . . 4-4 Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5 Batch Process Image Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6 Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6 Convert R'G'B' to Intensity Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7 Process Multidimensional Color Video Signals . . . . . . . . . . . . . . . . . . . . . 4-10 Video Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12 Defining Intensity and Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12 Video Data Stored in Column-Major Format . . . . . . . . . . . . . . . . . . . . . . 4-12 Image Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13 Binary Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13 Intensity Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13 RGB Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13 Display and Graphics 5 Display, Stream, and Preview Videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 View Streaming Video in MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 Preview Video in MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 View Video in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 Draw Shapes and Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4 Rectangle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4 Line and Polyline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4 Polygon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6 Circle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6 Registration and Stereo Vision 6 Fisheye Calibration Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2 Fisheye Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3 Fisheye Camera Calibration in MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4 vii

Single Camera Calibrator App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8 Camera Calibrator Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8 Single Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8 Open the Camera Calibrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-9 Prepare the Pattern, Camera, and Images . . . . . . . . . . . . . . . . . . . . . . . . . 6-9 Add Images and Select Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11 Calibrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-15 Evaluate Calibration Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17 Improve Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-20 Export Camera Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23 Stereo Camera Calibrator App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-25 Stereo Camera Calibrator Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-25 Stereo Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-25 Open the Stereo Camera Calibrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-26 Prepare Pattern, Camera, and Images . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-26 Add Image Pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-29 Calibrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-31 Evaluate Calibration Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-31 Improve Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-35 Export Camera Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-37 What Is Camera Calibration? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-39 Camera Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-39 Pinhole Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-40 Camera Calibration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-41 Distortion in Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-42 Structure from Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-45 Structure from Motion from Two Views . . . . . . . . . . . . . . . . . . . . . . . . . 6-45 Structure from Motion from Multiple Views . . . . . . . . . . . . . . . . . . . . . . 6-46 Object Detection 7 Getting Started with SSD Multibox Detection . . . . . . . . . . . . . . . . . . . . . . 7-2 Predict Objects in the Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2 Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3 Design an SSD Detection Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3 Train an Object Detector and Detect Objects with an SSD Model . . . . . . . 7-4 Code Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4 Label Training Data for Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4 Getting Started with Object Detection Using Deep Learning . . . . . . . . . . 7-6 Create Training Data for Object Detection . . . . . . . . . . . . . . . . . . . . . . . . 7-6 Create Object Detection Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7 Train Detector and Evaluate Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7 Detect Objects Using Deep Learning Detectors . . . . . . . . . . . . . . . . . . . . . 7-7 How Labeler Apps Store Exported Pixel Labels . . . . . . . . . . . . . . . . . . . . . 7-9 Location of Pixel Label Data Folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9 View Exported Pixel Label Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9 viii Contents

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10 Anchor Boxes for Object Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14 What Is an Anchor Box? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14 Advantage of Using Anchor Boxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14 How Do Anchor Boxes Work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-15 Anchor Box Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-18 Getting Started with YOLO v2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-19 Predicting Objects in the Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-19 Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20 Design a YOLO v2 Detection Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-20 Train an Object Detector and Detect Objects with a YOLO v2 Model . . . . 7-21 Code Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-21 Label Training Data for Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 7-21 Getting Started with R-CNN, Fast R-CNN, and Faster R-CNN . . . . . . . . . 7-23 Object Detection Using R-CNN Algorithms . . . . . . . . . . . . . . . . . . . . . . . 7-23 Comparison of R-CNN Object Detectors . . . . . . . . . . . . . . . . . . . . . . . . . 7-24 Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-25 Design an R-CNN, Fast R-CNN, and a Faster R-CNN Model . . . . . . . . . . . 7-25 Label Training Data for Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 7-27 Getting Started with Semantic Segmentation Using Deep Learning . . . 7-29 Train a Semantic Segmentation Network . . . . . . . . . . . . . . . . . . . . . . . . 7-29 Label Training Data for Semantic Segmentation . . . . . . . . . . . . . . . . . . . 7-29 Training Data for Object Detection and Semantic Segmentation . . . . . . 7-31 Create Automation Algorithm for Labeling . . . . . . . . . . . . . . . . . . . . . . . . 7-35 Create New Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-35 Import Existing Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-36 Custom Algorithm Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-36 Label Pixels for Semantic Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . 7-39 Start Pixel Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-39 Label Pixels Using Flood Fill Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-40 Label Pixels Using Smart Polygon Tool . . . . . . . . . . . . . . . . . . . . . . . . . . 7-41 Label Pixels Using Polygon Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-43 Label Pixels Using Assisted Freehand Tool . . . . . . . . . . . . . . . . . . . . . . . 7-44 Replace Pixel Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-45 Refine Labels Using Brush Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-45 Visualize Pixel Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-46 Tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-47 Get Started with the Image Labeler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-49 Load Unlabeled Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-49 Create Label Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-49 Label Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-56 Export Labeled Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-58 Save App Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-60 Choose an App to Label Ground Truth Data . . . . . . . . . . . . . . . . . . . . . . . 7-62 ix

Get Started with the Video Labeler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-64 Load Unlabeled Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-64 Set Time Interval to Label . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-64 Create Label Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-65 Label Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-72 Export Labeled Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-74 Label Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-76 Save App Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-77 Use Custom Image Source Reader for Labeling . . . . . . . . . . . . . . . . . . . . 7-79 Create Custom Reader Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-79 Import Data Source into Video Labeler App . . . . . . . . . . . . . . . . . . . . . . 7-79 Import Data Source into Ground Truth Labeler App . . . . . . . . . . . . . . . . 7-80 Use Sublabels and Attributes to Label Ground Truth Data . . . . . . . . . . . 7-81 When to Use Sublabels vs. Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-81 Draw Sublabels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-81 Copy and Paste Sublabels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-82 Delete Sublabels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-83 Sublabel Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-84 Temporal Automation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-85 Create Temporal Automation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 7-85 Run Temporal Automation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-85 View Summary of Ground Truth Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-87 View Label Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-87 Compare Selected Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-89 Share and Store Labeled Ground Truth Data . . . . . . . . . . . . . . . . . . . . . . 7-91 Share Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-91 Move Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-94 Store Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-95 Keyboard Shortcuts and Mouse Actions for Image Labeler . . . . . . . . . . . 7-97 Label Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-97 Image Browsing and Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-97 Labeling Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-97 Polyline Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-98 Polygon Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-98 Zooming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-99 App Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-99 Keyboard Shortcuts and Mouse Actions for Video Labeler . . . . . . . . . . 7-100 Label Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-100 Frame Navigation and Time Interval Settings . . . . . . . . . . . . . . . . . . . . 7-100 Labeling Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-100 Polyline Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-101 Polygon Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-101 Zooming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-102 App Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-102 Point Feature Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-103 Functions That Return Points Objects . . . . . . . . . . . . . . . . . . . . . . . . . . 7-103 Functions That Accept Points Objects . . . . . . . . . . . . . . . . . . . . . . . . . . 7-105 x Contents

Local Feature Detection and Extraction . . . . . . . . . . . . . . . . . . . . . . . . . 7-109 What Are Local Features? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-109 Benefits and Applications of Local Features . . . . . . . . . . . . . . . . . . . . . 7-109 What Makes a Good Local Feature? . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-110 Feature Detection and Feature Extraction . . . . . . . . . . . . . . . . . . . . . . 7-110 Choose a Feature Detector and Descriptor . . . . . . . . . . . . . . . . . . . . . . 7-111 Use Local Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-112 Image Registration Using Multiple Features . . . . . . . . . . . . . . . . . . . . . 7-114 Train a Cascade Object Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-122 Why Train a Detector? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-122 What Kinds of Objects Can You Detect? . . . . . . . . . . . . . . . . . . . . . . . . 7-122 How Does the Cascade Classifier Work? . . . . . . . . . . . . . . . . . . . . . . . . 7-122 Create a Cascade Classifier Using the trainCascadeObjectDetector . . . 7-123 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-126 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-128 Train Stop Sign Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-132 Train Optical Character Recognition for Custom Fonts . . . . . . . . . . . . . 7-135 Open the OCR Trainer App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-135 Train OCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-135 App Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-137 Troubleshoot ocr Function Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-139 Performance Options with the ocr Function . . . . . . . . . . . . . . . . . . . . . 7-139 Create a Custom Feature Extractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-140 Example of a Custom Feature Extractor . . . . . . . . . . . . . . . . . . . . . . . . 7-140 Image Retrieval with Bag of Visual Words . . . . . . . . . . . . . . . . . . . . . . . 7-143 Retrieval System Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-144 Evaluate Image Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-145 Image Classification with Bag of Visual Words . . . . . . . . . . . . . . . . . . . . 7-146 Step 1: Set Up Image Category Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-146 Step 2: Create Bag of Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-146 Step 3: Train an Image Classifier With Bag of Visual Words . . . . . . . . . 7-147 Step 4: Classify an Image or Image Set . . . . . . . . . . . . . . . . . . . . . . . . . 7-148 Motion Estimation and Tracking 8 Multiple Object Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2 Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2 Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2 Data Association . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3 Track Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4 Video Mosaicking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5 Pattern Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-10 xi

Pattern Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-15 Geometric Transformations 9 Nearest Neighbor, Bilinear, and Bicubic Interpolation Methods . . . . . . . 9-2 Nearest Neighbor Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-2 Bilinear Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3 Bicubic Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3 Filters, Transforms, and Enhancements 10 Adjust the Contrast of Intensity Images . . . . . . . . . . . . . . . . . . . . . . . . . . 10-2 Adjust the Contrast of Color Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-6 Remove Salt and Pepper Noise from Images . . . . . . . . . . . . . . . . . . . . . 10-10 Sharpen an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-14 Statistics and Morphological Operations 11 Correct Nonuniform Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-2 Count Objects in an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-8 Fixed-Point Design 12 Fixed-Point Signal Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-2 Fixed-Point Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-2 Benefits of Fixed-Point Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-2 Benefits of Fixed-Point Design with System Toolboxes Software . . . . . . . 12-2 Fixed-Point Concepts and Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-4 Fixed-Point Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-4 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-5 Precision and Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-6 xii Contents

Arithmetic Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-8 Modulo Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-8 Two's Complement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-8 Addition and Subtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-9 Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-10 Casts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-12 Fixed-Point Support for MATLAB System Objects . . . . . . . . . . . . . . . . . 12-15 Getting Information About Fixed-Point System Objects . . . . . . . . . . . . . 12-15 Setting System Object Fixed-Point Properties . . . . . . . . . . . . . . . . . . . . 12-15 Specify Fixed-Point Attributes for Blocks . . . . . . . . . . . . . . . . . . . . . . . . 12-17 Fixed-Point Block Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-17 Specify System-Level Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-19 Inherit via Internal Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-19 Specify Data Types for Fixed-Point Blocks . . . . . . . . . . . . . . . . . . . . . . . 12-26 Code Generation and Shared Library 13 Simulink Shared Library Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . 13-2 Accelerating Simulink Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-3 Portable C Code Generation for Functions That Use OpenCV Library . . 13-4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-4 xiii

Featured Examples • “Localize and Read Multiple Barcodes in Image” on page 1-2 • “Monocular Visual Odometry” on page 1-22 • “Track Vehicles Using Lidar: From Point Cloud to Track List” on page 1-35 • “Semantic Segmentation Using Dilated Convolutions” on page 1-55 • “Define Custom Pixel Classification Layer with Tversky Loss” on page 1-59 • “Track a Face in Scene” on page 1-66 • “Create 3-D Stereo Display” on page 1-71 • “Measure Distance from Stereo Camera to a Face” on page 1-72 • “Reconstruct 3-D Scene from Disparity Map” on page 1-74 • “Visualize Stereo Pair of Camera Extrinsic Parameters” on page 1-77 • “Remove Distortion from an Image Using the Camera Parameters Object” on page 1-80 1

Localize and Read Multiple Barcodes in Image This example shows how to use the readBarcode function from the Computer Vision Toolbox™ to detect and decode 1-D and 2-D barcodes in an image. Barcodes are widely used to encode data in a visual, machine-readable format. They are useful in many applications such as item identification, warehouse inventory tracking, and compliance tracking. For 1-D barcodes, the readBarcode function returns the location of the barcode endpoints. For 2-D barcodes, the function returns the locations of the finder patterns. This example uses two approaches for localizing multiple barcodes in an image. One approach is clustering-based, which is more robust to different imaging conditions and requires the Statistics and Machine Learning Toolbox™. The second approach uses a segmentation- based workflow and might require parameter tuning based on the imaging conditions. Barcode Detection using the readBarcode Function Read a QR code from an image. I = imread("barcodeQR.jpg"); % Search the image for a QR Code. [msg, ~, loc] = readBarcode(I); % Annotate the image with the decoded message. xyText = loc(2,:); Imsg = insertText(I, xyText, msg, "BoxOpacity", 1, "FontSize", 25); % Insert filled circles at the finder pattern locations. Imsg = insertShape(Imsg, "FilledCircle", [loc, ... repmat(10, length(loc), 1)], "Color", "red", "Opacity", 1); % Display image. imshow(Imsg) 1 Featured Examples 1-2

Read a 1-D barcode from an image. I = imread("barcode1D.jpg"); % Read the 1-D barcode and determine the format.. [msg, format, locs] = readBarcode(I); % Display the detected message and format. disp("Detected format and message: " + format + ", " + msg) Detected format and message: EAN-13, 1234567890128 % Insert a line to show the scan row of the barcode. xyBegin = locs(1,:); imSize = size(I); I = insertShape(I,"Line",[1 xyBegin(2) imSize(2) xyBegin(2)], ... "LineWidth", 7); % Insert markers at the end locations of the barcode. I = insertShape(I, "FilledCircle", [locs, ... repmat(10, length(locs), 1)], "Color", "red", "Opacity", 1); Localize and Read Multiple Barcodes in Image 1-3

% Display image. imshow(I) Improving Barcode Detection For a successful detection, the barcode must be clearly visible. The barcode must also be as closely aligned to a horizontal or vertical position as possible. The readBarcode function is inherently more robust to rotations for 2-D or matrix codes than it is to 1-D or linear barcodes. For example, the barcode cannot be detected in this image. I = imread("rotated1DBarcode.jpg"); % Display the image. imshow(I) 1 Featured Examples 1-4

% Pass the image to the readBarcode function. readBarcode(I) ans = Rotate the image using the imrotate so that the barcode is roughly horizontal. Use readBarcode on the rotated image. % Rotate the image by 30 degrees clockwise. Irot = imrotate(I, -30); % Display the rotated image. imshow(Irot) Localize and Read Multiple Barcodes in Image 1-5

% Pass the rotated image to the readBarcode function. readBarcode(Irot) ans = "012345678905" Detect Multiple Barcodes The readBarcode function detects only a single barcode in each image. In order to detect multiple barcodes, you must specify a region-of-interest (ROI). To specify an ROI, you can use the drawrectangle function to interactively determine the ROIs. You can also use image analysis techniques to detect the ROI of multiple barcodes in the image. 1 Featured Examples 1-6