UAV Simulator: How to Connect Your Drone Controller to Your PC or Mac

Top: Attribute distribution across UAV123 dataset and a comparison of key attributes with OTB100. Bottom: Synthetic dataset generation and online tracker evaluation using the proposed simulator. For a legend of abbreviations, refer to Table 2.

Visual tracking on UAVs is a very promising application, since the camera can follow the target based on visual feedback and actively change its orientation and position to optimize for tracking performance. This marks the defining difference compared to static tracking systems, which passively analyze a dynamic scene. Since current benchmarks are pre-recorded scenes, they cannot provide a quantifiable measure on how slower trackers would affect the performance of the UAV in shadowing the target. In this paper, we propose the use of a photo-realistic simulator to render real-world environments and a variety of life-like moving targets typically found in unmanned aerial recordings. The simulator uses the Unreal Engine 4 to directly feed image frames to trackers and retrieve tracking results to update UAV flight. Any tracker (e.g. written in Matlab or C++) can be tested on the simulator across a diverse set of photo-realistic simulated scenarios. Using this simulator enables the use of new quantitative methods for evaluating tracker performance in the aforementioned aerial feedback loop.

uav simulator

Contributions. The contributions of our work are threefold. (1) We compile a fully annotated high-resolution dataset of 123 aerial video sequences comprising more than 110 K frames. It is as large or larger than most recent, generic object tracking datasets. (2) We provide an extensive evaluation of many state-of-the-art trackers using multiple metrics [42]. By labeling the videos in the benchmark with various attributes, we can also evaluate each tracker in regards to specific aerial tracking nuisances (e.g. scale/aspect ratio change, camera motion, etc.). (3) We provide a novel approach to perform tracker evaluation by developing a high-fidelity real-time visual tracking simulator. We present first results on the performance of state-of-the-art trackers running within its environment. The combination of the simulator with an extensive aerial benchmark provides a more comprehensive evaluation toolbox for modern state-of-the-art trackers and opens new avenues for experimentation and analysis.

UAV Simulation. In recent years, several UAV simulators have been created to test hardware in the loop (HIL). However, the focus is on simulating the physics of the UAV in order to train pilots or improve/tune features of a flight controller (e.g. JMAVSim [40]). The visual rendering in these simulators is often primitive and relies on off-the-shelf simulators (e.g. Realflight, Flightgear, or XPlane). They do not support advanced shading and post-processing techniques, are limited in terms of available assets and textures, and do not support MOCAP or key-frame type animation to simulate natural movement of actors or vehicles. Although simulation is popularly used in machine learning [2] and animation and motion planning [12, 20], the use of synthetically generated video or simulation for tracker evaluation is a new field to explore. In computer vision, synthetic video is primarily used for training recognition systems (e.g. pedestrians [14], 3D scenes [31], and 2D/3D objects [15, 32]), where a high demand for annotated data exists. The Unreal Engine 4 (UE4) has recently become fully open-source and it seems very promising for simulated visual tracking due in part to its high-quality rendering engine and realistic physics library.

The UE4 based simulator allows real-time tracker evaluation with the ability to simulate the physics of aerial flight, produce realistic high-fidelity renderings (similar to if not better than professional rendering software, e.g. 3DSMax and Maya), and automatically generate precise ground truth annotation for offline or real-time use cases (see Fig. 1). The UAV is modeled after the DJI S1000+, which was used to capture the majority of the benchmark. An accurate 3D model (same geometry/weight and thrust vectors) is subjected to game physics (UE4) and real-world conditions (e.g. wind and gravity). The ground truth trajectory and orientation of the target and UAV are recorded at every frame. The PID controllers for stabilization and visual servoing (gimbal) mimic the Pixhawk FC. For further details on the implementation, see the simulator documentation.

uav simulator matlab

uav simulator software

uav simulator unreal engine

uav simulator online

uav simulator python

uav simulator download

uav simulator for pc

uav simulator free

uav simulator mac

uav simulator linux

uav simulator pixhawk

uav simulator ros

uav simulator unity

uav simulator windows

uav simulator android

uav simulator app

uav simulator arduino

uav simulator autodesk

uav simulator blender

uav simulator c++

uav simulator course

uav simulator drone

uav simulator education

uav simulator flightgear

uav simulator gazebo

uav simulator hardware in the loop

uav simulator java

uav simulator javascript

uav simulator kit

uav simulator labview

uav simulator matlab simulink

uav simulator open source

uav simulator project

uav simulator raspberry pi

uav simulator research

uav simulator sdk

uav simulator steam

uav simulator tutorial

uav simulator ue4

uav simulator vrep

uav simulation and control video series

uav simulation and testing environment (uaste)

uav simulation environment for engineering (usee)

uav simulation framework (usf)

uav simulation in matlab and simulink (usms)

uav simulation platform (usp)

uav simulation software comparison (ussc)

uav simulation tools for robotics system toolbox (ustrt)

uav simulation with airsim (uswa)

UE4 allows for a large variety of post-processing rendering steps to create realistic and challenging scene images that simulate real-world UAV data. Although not implemented for this work, motion blur, depth of field, over/under exposure, HDR and many more features can be enabled. UE4 post-processing rendering allows assignment of custom depth maps to any mesh in the engine. The depth maps allows extraction of segmented annotation of the tracked target as seen through the camera viewpoint. We simulate the movement of both a human character and a 4WD vehicle moving along set trajectories within a detailed off-road race track with palm trees, cacti, mountains, historical buildings, lakes, and sand dunes (see Fig. 3). This is one example of many photo-realistic UE4 worlds created by the developer community in which our UAV simulator can be used. The UAV simulator enables the integration of any tracker (MATLAB or C++) into the tracking-navigation loop; at every frame, the output bounding box of the tracker is read and used to correct the position of the UAV.

Four trackers are selected for evaluation, namely SRDCF, MEEM, SAMF, and STRUCK. The ground truth bounding box generated from the custom depth map of the target is called GT. We first optimize the UAV visual servoing using the GT tracker (see supplementary material on our visual servoing technique). Despite absolute accuracy of the GT, the flight mechanics of the UAV limit its ability to always keep the target centered, since it must compensate for gravity, air resistance, and inertia. After evaluating the performance of the UAV with the GT, each tracker is run multiple times within the simulator provided with the same starting initialization bounding box. The target follows a pre-defined path and speed profile. The UAV tracks and follows the target for 3.5 min (ca. 6000 frames at 30 FPS). The target speed varies but is limited to 6 m/s, the UAV speed is limited to 12 m/s (similar to the real UAV). For evaluation, we measure the distance between the trajectory of the target and the UAV.

Our proposed UAV simulator along with novel evaluation methods enables tracker testing in real-world scenarios with live feedback before deployment. We will make this simulator publicly available to support more progress in the realm of UAV tracking, as well as, other computer vision tasks including aerial Structure-from-Motion (SfM), aerial localization, dynamic scene monitoring, etc. The simulator is not limited to UAVs alone but can be easily extended to simulate autonomous vehicles and evaluate their performance with algorithms designed for navigation and pedestrian detection.

Some drone flight simulators allow you to customize them for specific flying scenarios, so you can prepare for the exact situations you might face in your drone pilot work. Customizations can include the option to change the drone simulation environment as well as to change the type of drone being used in the drone simulator.

*Note: Pricing for both the Enterprise Version and Energy Version of DJI drone flight simulator is not publicly listed on the DJI site, so we are providing our best educated guess about the price range here.

This means that it comes with built-in classroom management and student progress tracking tools, which allow educators to track the progress that their students make while they use the simulator. An instructor can leverage these tools to see how often a student crashes his or her drone, and to track student progress in order to know whether students have improved over time or if they need more instruction.

Note: The above list is only of those controllers that have already been tested but other controllers may also work. To learn more about controllers for the droneSimPro drone flight simulator visit this page on the droneSimPro website.

In creating this drone flight simulator DRL sought the advice of leading experts on drone flight and conducted exhaustive testing and research. Every battery, motor and prop combination available was tested in order to determine the exact power curves of any potential drone combination.

The VECTOR-SIL solution executes a VECTOR autopilot software version that incorporates a simulator embedded into the autopilot code. While this assures SIL will behave exactly like the real autopilot at the logic level, execution times and sensor behavior are not the same as in real operation.

While the aircraft is flying (or here while the simulator is integrating differential equations), you can move the waypoints on the GCS interface by cliking and dragging (with the left button). When the mouse button is released, a popup window allows you to change the altitude of the waypoint. After validation, the waypoint changes are sent to the autopilot and the followed track is changed accordingly.

The VECTOR-HIL is the most advanced training simulator developed by UAV Navigation. The HIL enables the simulation of realistic flight condition using a real VECTOR hardware & software (FCC) and a second computational unit simulating the environment and sensor input (SIM).

Abstract:With the increasing popularity of vertical take-off and landing unmanned aerial vehicles (VTOL UAVs), a new problem arises: pilot training. Most conventional pilot training simulators are designed for full-scale aircrafts, while most UAV simulators are just focused on conceptual testing and design validation. The X-Plane flight simulator was extended to include new functionalities such as complex wind dynamics, ground effect, and accurate real-time weather. A commercial HIL flight controller was coupled with a VTOL convertiplane UAV model to provide realistic flight control. A real flight case scenario was tested in simulation to show the importance of including an accurate wind model. The result is a complete simulation environment that has been successfully deployed for pilot training of the Marvin aircraft manufactured by FuVeX.Keywords: HIL; flight simulator; pilot; commercial; UAV; X-Plane