The Perfect Swarm: Drones Descend on Manufacturing

Imagine, if you will, a bustling ball bearing factory, one where forklifts scoot and sputter to and fro, as they rush parts from conveyors to carts, from warehouses to loading docks. It’s a muddled, messy place, sending workers through a labyrinth of time-consuming twists and turns to get the parts they need to do their jobs.

Then, suddenly, a CNC machine needs a new bit. Rather than navigating this traditional path to the parts depot, the machinist picks up his tablet instead. He swipes his finger across the screen, and a minute later a briskly moving object appears overhead and quickly descends onto a nearby platform safely with exactly the bit he needs. Looks like 20 minutes of valuable production time isn’t wasted, the factory hits its goals, and everyone gets a big fat bonus.

This sounds like fantasy—a sci-fi daydream of the factory of the future. But it’s just another routine day, here... in the Drone Zone.

OK, so this scenario was completely made up (and a blatant attempt to channel Rod Serling), but the real benefit of drones in manufacturing facilities is not.  From light material handling to visual inspection to damage control, unmanned aerial vehicles surpass what humans can do in less time and for less money.

Companies such as Con Edison in New York have already found that sending a drone to check a boiler pipe is more effective than building 10 stories of scaffolding. And this spring, when your facilities engineer surveys your plant’s façade for winter damage, would you rather have them work at precarious heights, or have a drone do a few flybys to capture the building’s health in high resolution?

Deciding what industrial applications to add drones to is a lot like wondering what you would do if you could fly. You may respond, “What wouldn’t I do?” Now extrapolate that to your plant ops. The question we hope you’ll start asking with regards to drones is, “What could I do?”

We will soon find out.

Drones are expected to make a huge leap this year. The FAA predicts 4.8 million drone sales in 2017, nearly double 2016’s numbers. Most striking is that sales of commercial drones, which require FAA registration, will increase to 2.5 million. That’s a 416% increase. This steep incline plateaus over the next three years, so expect this to be the year when companies, most likely your competitors, blitzkrieg their applications with drone. If done right, the new air support should provide unprecedented speed, efficiency, adaptability to your production, and, most important, foster a safer environment.

First off, let’s agree that drones aren’t toys; they’re legitimate tools that every major supply chain in America is rushing to adopt.

Hell, 7-Eleven is already using Flirtey drones to deliver Big Bites and Slurpees in Reno, Nev., the first such delivery to customer homes.

And last fall, UPS teamed with drone maker CyPhy Works to successfully transport a life-saving medical device (an asthma inhaler) to an island three miles off the Massachusetts coast. You’ve also probably heard about Google, Walmart, and Amazon’s massive drone delivery projects that could span the nation in a complicated network of trucks and charging stations and maybe even a zeppelin. Yes, a frickin’ zeppelin.

Ironically, manufacturing is an area that rarely uses drones, but desperately needs them.

Currently, the top commercial uses are photography (42.9%) and real estate (20.7%), according to a Business Insider report, while industrial uses—construction, utilities, agriculture, and manufacturing—comprise about one-quarter.

For its part, manufacturing accounts for a paltry 1.5%. Uses include asset monitoring, inventorying by scanning RFID chips and bar codes, and visual inspection.

There’s an obvious reason why realtors and shutterbugs have been the only ones to fully adopt the tech until now. It doesn’t take the mad flying skills of a Chelsey “Sully” Sullenberger III to hover around a house or along a beachfront, especially with built-in autopilot features such as “return home.” In a high-speed automotive assembly plant, drones have a lot more to contend with, including robots that shoot welding arcs, and there’s always the danger of losing communication, which would give you a rogue missile that could kamikaze into turbine or conveyor or a coworker’s head.

Drones just haven’t been smart or safe enough to trust in a major operation.

Until now.

Brain Gain

As Richard Wagner’s “Ride of the Valkyries” wafts through the Beverly Wilshire Hotel conference hall during Qualcomm Technologies’ Abundance360 presentation in January, all eyes are on the three pigeon-sized quadcopters weaving around two stacks of steel drums on the stage.

At first glance, these lightweight drones—3D-printed by Qualcomm for the exclusive, swanky summit on disruptive tech—don’t appear imposing enough to warrant the sweeping orchestral arrangement meant to evoke winged horseriding battle maidens, or more recently, napalm-dropping attack choppers in Apocalypse Now.  But make no mistake. These are truly fierce flyers, and the music is absolutely justified.

Sure, they won’t strike terror like the U.S. Air Force’s missile-wielding Predator or Reaper UAVs, but they are scary smart. The quadcopters whiz around the barrels at increasing speeds, one after another, intuitively avoiding each other and the objects. The last in the line streams video to a large monitor. On the screen, red and blue boxes surround the various objects, the drone’s way of categorizing the different obstacles in its environment via machine learning.

You’d expect an actual Air Force ace or a supercomputer at the controls, allowing for the effortless maneuvering. But with these, what you see is what you get.

“Everything you see is running onboard the drone; there’s no external infrastructure,” reveals Matt Grob, Qualcomm Technologies CTO and EVP, to the audience. “And all of that is running on a board that weighs just 12 grams.”

The board is part of the hardware/software Qualcomm Snapdragon Flight platform, which began taking flight last year in six drone models, with more to come. Along with proprietary flight control software, there’s that 12-g mother board the size of a credit card, comprising a 2.26Ghz quad-core cpu, gpu, dsp, a 4K camera, comprehensive wireless connectivity and “loaded with sensors,” Grob says, from inertial to positional.

These include the accelerometers, gyros and cameras that will keep the drone aloft and able to fulfill jobs ranging from photography to visual inspection, with more uses being thought up all the time at Qualcomm’s drone innovation lab, nicknamed “The Aviary.”

Using the onboard cameras and sensors, the drones are able to autonomously determine their own flight paths, finding the most efficient routes, while avoiding obstacles, maintaining elevation, and countering adverse winds or loss of GPS signals.

The Snapdragon 801 system-on-chip (SoC) processor was originally developed for smartphones such as LG and Samsung, as well as long tail applications, including in the automotive industry. The low-power processors are also ideal for conserving battery life.

Now it’s been called upon to lift a nascent drone industry to greater heights, and to previously unreachable spaces in manufacturing.

Internet of Wings

“Snapdragon Flight is meant to help accelerate the drone industry,” explains Jim Merrick, director of marketing for Qualcomm’s Internet of Things business, of which drones are a part.

And that makes complete sense, because drones such as the ones using Snapdragon Flight are essentially autonomous self-contained flying IoT systems.

“We wanted to raise the bar on what are the features set for the minimum standard,” Merrick says. “You need 4K, you need to have the ability to track somebody. If I want my drone to be the autonomous selfie stick everyone envisions it to be, or my personal ESPN as it follows me down the mountainside as I’m mountain biking, it needs to have the computational horsepower and sophistication to track my activities, and at the same time avoid crashing into the tree.”

Here's a video of the Hover Camera, which uses the Snapdragon Flight platform to become an autonomous flying camera anyone can use:

For those photographers, realtors, and wannabe X-Gamers, that the camera makes them or their subjects look awesome is enough. But for brass tacks industrial types, your question should be “How can this tool make my job safer and easier?”

That computational power comes from the 2.26Ghz Quad core Krait 400 CPU, which facilitates sharp image capture, faster processing, the latest in connectivity, and better battery life. Commercial drones’ flights last around 8-20 min., while industrial drones can often stay aloft for up to 40 min.

First off, any IoT device worthy of the name has to be able to communicate. The Snapdragon 801 processor has 4G LTE, 802.11n for ultra-fast Wi-Fi and connecting multiple devices at once, USB 3.0, and Bluetooth 4.0.

That Dual-band 2x2 802.11n Wi-Fi chip allows for glorious 4K video streaming from the drone with no visual hiccups, something my PS4 can’t always do with Netflix. 

Here’s what Raj Talluri, the Senior VP of Product Management at Qualcomm, had to say to open the Abundance360 summit: “4K screen resolution is close to the point that the brain is unable to notice pixels. As such, somewhere between 4K and 8K, virtual reality becomes visually equal to visual reality.”

And Talluri says once your eyes feast upon the deliciously rich resolution, lesser quality will never satiate your visual appetite.

“If you buy a 4K TV and watch 4K content, it’s very hard to go back to 1080p,” he says. “It almost feels like you were watching a VHS tape when DVDs came out.”

Qualcomm has shipped as many as 250 million 4k-capable processors so far, according to Talluri. It appears that it won’t be long before the HD TVs we thought couldn’t look any better will be outdated. Along with a good excuse to buy a new TV, this upgrade allows operators to capture the crispest representation of whatever they’re filming. (We’ll talk about what those applications might be in the next section.)

Now to seamlessly handle all that unbridled video data, Qualcomm uses its Adreno 330 GPU.

“It’s more power efficient and weight efficient than discrete processors to do that work,” Merrick says.

The powerful GPU also allows for the platform’s most innovative feature for industrial applications: Visual Inertial Odometry (VIO). This allows the drone to navigate by the forward-facing camera rather than the GPS sensor, using high contrast areas such as doorways as landmarks. This is crucial in large facilities and plants where GPS is unreliable.

And while flying, a downward-facing camera sensor, called the Optical Flow, senses any unintended image changes due to a breeze or gust and compensates by leaning into the wind.  Merrick says it can remain stable even in 20 mph winds.

For typical flight stability, a Snapdragon-embedded drone can rely on the Bosch BMP280 barometric pressure sensor and Invensense MPU-9250 9-Axis Sensor.

Embedded with this suite of hardware and software, a drone transforms from novel consumer gizmo to a highly advanced hub of sensors and cameras that any and every industrial facility can find a dozen uses for.

Eye in the Sky

Last summer in São Carlos, Brazil, a technocentric city between São Paolo and Brasilia, Qualcomm partnered with Embrapa, the Brazilian Agricultural Research Corporation, to find out what impact drones can have at local farms. The goals were to reduce environmental impact while increasing crop yields.

Family farms produce about 70% of the South American superpower’s staple foods, from coffee to sugarcane to beef, all the things that make waking up tolerable. As vast as Brazil is, land is always at a major premium as expansion means cutting down precious rain forests.

Allowing these farms to obtain aerial photography of their acreage in real time, sent to an LTE-connected smart device, gives them the ability to detect crops stresses including infestations, over-fertilization, and water stress, and nutrient deficiency. Getting ahead of these problems ensures less waste and less harm to the environment.

These farmers cannot typically afford the technology that would allow them to get more out of less land. Snapdragon Flight, which costs about $675, can be equipped to a drone that costs less than $1,000.

“It’s affordable, even to the point it can be shared amongst a co-op of farmers, much like a combine would be shared in larger farms,” Merrick says.

If humble Brazilian farmers can benefit from smarter, more connected drones, America’s industrial plants and facilities really have no excuse for not investigating how a drone or two can keep profits up and accidents down.

Merrick points out how crude our way of aerial inspection is right now.

“With the high tensile power lines that traverse the nation today, ridiculously, the way they do it is two people in a helicopter, the pilot and one with binoculars,” Merrick explains.  “And they fly along and look at the wires along their entire length. First, that’s a mind-numbing job, and ridiculously expensive considering the flight time for helicopters.”

Then there’s the crumbling infrastructure in need $1 trillion fix.

“Most major bridges in our interstate highway system are in various states of disrepair,” Merrick says. The reason is the human cost; it’s dangerous work. These are things that could be done continuously with drones.”

And this type of inspection will detect safety and security hazards in and around any manufacturing facility. They’re not to the point of having Tasers attached, as seen in the show “APB,” but they could conceivably have robot arms attached to carry loads across a plant.

In a warehouse, having drones fly up on towering shelves to scan bar codes would be faster and easier than sending a person.

But why stop at regular cameras? Some drone companies, such as 3DR, have already started adding more sensors, such as a FLIR IR camera, for firefighting. Pairing IR, which can see through smoke, with a mobile drone that can sense and avoid obstacles on its own, is an intriguing use that could save lives in the case of a plant fire.

“Having a mobile sensor on a drone, if you have a factory floor and an acetone spill, a drone can get there very quickly with an air quality sensor and diagnose a problem, before you send in a human,” Merrick says.

The EuropeanMIRPHAB Project plans to equip miniature spectroscopes to drones for chemical sensing in processing applications and for hazard detection.

By far the coolest future use leverages the VIO algorithm to create virtual reality with six degrees of freedom (6DoF). Samsung Gear uses three, while the HTC Vibe has six, but requires beacons to send out the structured light grid to help the phone position itself.

Instead of beacons, Merrick says a Snapdragon Flight –enabled drone could achieve the same effect. (Qualcomm already has the Snapdragon VR820 platform to create immersive experiences.)

“You don’t want to calibrate each room,” Merrick says. “Instead, you can use the Snapdragon camera and VIO to track the position of VR headset. It’s that precise, just looking at what it sees in the room, and know if you’re moving forward or back, up, down, left or right.”

In that respect, drones will truly be able to transport us to a different reality. While that may be years away, we are certain this is the year drones will deliver more than just Slurpees; they’ll deliver results.