You Could be Piloting Your Drone with Your Eyes (or Other Body Parts)
Photo Credit: Wix Stock Library
Some people might say that drone piloting has a steep learning curve. Of course, it takes time and practice to get those ideal aerial shots without crash landing everywhere you go, but what if there was an easier way to pilot a drone? While pilots usually fly their drones with a controller, there may come a time when we're all maneuvering the latest Phantom with different body parts like our eyes, arms, or even hips.
Roboticists from the University of Pennsylvania, U.S. Army Research Laboratory, and New York University have been working on a special project that would allow someone to pilot a drone with their eyes. Liangzhe Yuan, Christopher Reardon, Garrett Warnell, and Giuseppe Loiannothat are working with the Tobii Pro Glasses 2 equipped with an IMU (Inertial Measurement Unit) and an HD camera. The Tobii Pro Glasses 2 is an already existing product designed to "to capture natural viewing behavior in any real-world environment." Researchers were able to ally these glasses with a quadcopter for gaze-controlled flight.
Video Credit: Tobii Pro
Using a deep neural network, the glasses can detect the drone's location, the user's location, and where the user is looking. This technology also helps the drone to fly according to the user's head orientation instead of its own, so when a pilot orders their drone to fly to the left, they'll always know which way it's going. In their demonstration video on YouTube, you can see someone testing out the glasses in a workspace. From the glasses point of view, you can see a green dot follow the user's gaze. Once the tester's gaze has settled, a hovering drone flies to that location in the room. Essentially, the project functions as an experiment with "human-guided autonomy" for non-invasive human-robot interactions. Digital Trends described the glasses as a method that's "easy to use" and a possible aid to inexperienced pilots.
Video Credit: ARPL NYU
Eyes aren't the only way technology experts are looking to streamline drone piloting. The Swiss Federal Institute of Technology in Lausanne (EPFL) is in the process of developing a wearable drone controller. This new technology was developed with assistance from 17 volunteers who were asked to watch simulated drone footage with VR goggles. Researchers also told the volunteers to copy the drone's movement with their bodies while they wore electrodes and 19 motion-capture markers across their upper bodies so that the researchers could identify recurring patterns in their movement. With the data they collected, the researchers were able to design a suit that turns the pilot's body into a drone controller.
Later, another 39 volunteers tested how the wearable controller compared to a regular joystick controller by completing an actual drone course. Their results showed that the controller suit had better steering abilities, and the volunteers learned how to use the suit faster than the joystick. More information on this research can be found on the Proceedings of the National Academy of Sciences of the United States of America website. There, you can see the project's data in detail under the project name: Data-driven body–machine interface for the accurate control of drones. Funded by the Swiss National Science Foundation, these researchers were able to develop an "intuitive, gesture-based control interface for real and simulated drones".
Video Credit: École polytechnique fédérale de Lausanne (EPFL)
One of the researchers and authors of the study's paper, Jenifer Miehlbradt spoke with Inverse, a website dedicated to exploring "the science of anything," about their project. In her interview, she talks about her team's methodology and conveys that the goal of their research was to improve the connection between user and drone, similar to the roboticists. It seems like some technology experts believe that the joysticks or controllers that are currently used to pilot drones create a distance between the pilot and their drone that impedes control and makes learning to fly more difficult. With piloting being facilitated by these "non-invasive" methods, the pilot is free to observe their surroundings without focusing too much on flying.
I am a writer and an artist based in Georgia. Specializing in illustration, graphic design, and video art, I love to explore the new ways technology intersects with art. I think drones have done amazing things for photography and video art, making what would previously be costly and difficult more accessible. As a complete novice, it was only recently that I saw what independent artists could do with their drones, and I continue to be impressed by the sights that drones are able to explore and the images they can capture. Instagram: @tyesha.ferron
