Drone Capturing and UAV LIoN Recovery {#sec2.3} --------------------------------------- For this experiment, the UAV is flown around a large area using a large, geotagged point cloud. The goal is to create a 3D map using LiDAR data that can later be registered to the camera data with reasonable accuracy. The drone is equipped with a downward-facing camera and GPS for navigation. A DJI Matrice 600, which is made up of multiple drones, is used for this experiment. The DJI Matrice 600 is a small aircraft used to capture low altitude images with a downward facing camera. A DJI Matrice M210 is flown for this experiment. The M210 model has 6-axis stabilization, a downward-facing camera, and has a maximum speed of 7 m/s. It has an estimated battery duration of 20 min at full charge, and a transmission speed of 868 Mbps. The M210 is equipped with a Pix4D Pix4Dmapper that provides an accurate 3D point cloud at a frequency of up to 2 Hz. We use our method to obtain the 3D point cloud of an indoor environment. We follow the procedures described in the [Methods](#sec2){ref-type="other"} section. All of the components of the system are deployed together (e.g., the Matrice 600, Pix4D, and the computer with our Matlab simulation running) but no additional personnel are needed outside of the operator. [Figure 5](#fig5){ref-type="fig"} shows some examples of the map created from the 3D point cloud generated using the sensor. This shows that the data produced by the sensor can be used to create a reasonable map with respect to the camera data. It is also possible to calculate and create a relative map from the GPS location. However, a relative map does not provide any absolute position and orientation information. This provides the ability to use GPS to navigate in an unknown location. Note, however, that GPS based navigation assumes that the GPS satellite constellation is functioning properly. If this is not the case, then this navigation technique will fail. 3. Discussion {#sec3} ============= We have presented the hardware and software components of a novel and convenient system to create maps and georegistration solutions that are both accurate and easy to use. Our system utilizes a UAV for flying the cameras and capturing images as well as capturing the LiDAR data, and it uses an autonomous driving computer to process this data. We have described a robust workflow that employs a 3D point cloud mapping system to provide an accurate mapping result. We have also presented two ways of navigating the UAV while capturing images and point cloud data. We have demonstrated our approach by performing experiments, including data capture and processing in both urban and rural settings. The systems are robust and easy to use due to its relatively small size and lightweight components. This system provides many advantages over other methods of mapping and georegistration. First, because the LiDAR sensors operate in a point cloud format, it is possible to obtain a very detailed 3D map of an area that can be used for many different types of experiments. Second, the system can be easily deployed in any type of environment, including indoor environments. Third, unlike other systems, the proposed system can operate using very little in-field calibration, so there is no need to use large calibration objects to obtain high quality results. In fact, because the system uses the same sensor for both tasks (i.e., for navigation and capturing images), the system can be used without any calibration process to obtain high quality results. This eliminates a significant source of error in obtaining calibration information. Our system can also be used to provide absolute location and orientation information using the GPS sensor. If GPS navigation fails, then it is possible to use the inertial navigation system (INS) within the UAV to determine a position and orientation. The use of a GPS sensor and an INS together increases the accuracy of the UAV by increasing the number of sources of information to calculate the actual pose. It is also possible to use image processing methods to obtain the absolute pose of the UAV. Unfortunately, this approach requires an accurate calibration of the camera and is much more complex than our proposed system. Using the GPS sensor, an accurate location can be determined by using a global navigation satellite system (GNSS), whereas using the image-based approach requires a complicated process of solving the pose and obtaining a precise initial position. The hardware components of our system can be expanded to perform other tasks that benefit from a 3D point cloud and multi-camera system. For example, the inertial sensor can be used in multiple types of applications that require accurate inertial measurements, such as in inertial odometry, inertial SLAM, and IMU control. The sensor data can also be used for state estimation for UAV navigation. Conclusions {#sec4} =========== We presented a new method for 3D sensor calibration that can be used for mapping, georegistration, and navigation tasks that are both accurate and convenient. Our results demonstrate that our method can be used in indoor and outdoor environments and in various types of environments. We also presented the concept of using an INS within the UAV to obtain high quality georegistration results. In conclusion, we demonstrated the simplicity and accuracy of our approach by acquiring high-quality results that are similar to results achieved by using GPS. The authors thank David Barajas for his assistance with the LiDAR data acquisition, as well as all of the other members of the Mapping and Georegistration Laboratory for their help with the data acquisition and processing. In addition, we thank the members of UCF BURGER project for their help with hardware and software development. Finally, we thank the funding agencies for their support. This research was supported in part by funding from the National Science Foundation (IIS-1251421, IIS-1563921) and the United States Department of Transportation (49 CFR Part 23). Methods {#sec5} ======= In this section, we discuss the algorithmic methods used for the different components of our system. We discuss the calibration of the camera sensors and the LiDAR sensors in [Section 5.1](#sec5.1){ref-type="other"} and [5.2](#sec5.2){ref-type="other"}, respectively. For our particular system, we used an autonomous driving computer to process the captured images and LiDAR data. This section provides a general introduction to the different processing steps of this autonomous driving computer. In [Section 5.3](#sec5.3){ref-type="other"}, we describe the algorithms used for the image processing and object recognition processes. For processing LiDAR data, we use methods that have been previously described.^[@ref18]^ 5.1. Calibration of the Camera Sensors {#sec5.1} -------------------------------------- In this section, we provide a description of how the data captured by the camera sensor is processed to obtain an absolute pose using a calibration object (see [Figure 1](#fig1){ref-type="fig"}). The calibration object is designed such that it has six markers on its surface that are placed at known locations in the real world. This allows for the determination of the cameras' absolute pose relative to the calibration object. The method used to obtain the camera pose is a modified version of the "pinhole camera model", also known as the "Nister method".^[@ref20]^ The basic steps to perform this are as follows:(1)Capture images of the calibration object (i.e., *S* and *R* in [Figure 1](#fig1){ref-type="fig"}) at high frequency.(2)Process the images to obtain the six-point calibration using the Nister method.(3)Identify the coordinates of the markers in the image of the calibration object (i.e., *S~c~* and *R~c~* in [Figure 1](#fig1){ref-type="fig"}) using the previously identified markers. The first step involves obtaining images using the camera in each pose. The obtained images can then be processed with a custom program written in MATLAB to estimate the relative pose of the camera. This program first estimates the pose of the camera by calculating the location of the three markers from the captured images. Then, the program obtains two relative poses between two images (usually one of these two images is obtained before the other). Using these two images and the two relative poses, the program uses a third point to obtain the absolute pose of the camera. Finally, the program estimates the relative pose between two images that have a different marker configuration, and the method iterates until it finds a fixed pose. The calibration of the camera poses, as well as the LiDAR sensors, can be adjusted for various scenarios. In our work, we used a custom calibration object to perform the first step and a second custom calibration object to perform the second step. This allowed for high accuracy for georegistration. However, we note that in some cases it is possible to remove step (1) of this method, which means that the camera pose will be based only on the relative pose between the markers in the environment and the images obtained from the camera sensor. This can be done by capturing images of the calibration object without using the LiDAR sensor, using only a monocular camera for navigation, and acquiring reference images of the environment using a LiDAR sensor. The benefit of this approach is that step (2) is not required, which results in a faster calibration process and a smaller error in the absolute pose calculation. 5.2.