Exclusive: Your delivery robot will now offer the blind real-time, on-the-ground eyes around sidewalk hazards

The delivery robots rolling down your sidewalk have cameras, sensors, and a constant need to dodge whatever is in their path. Think fallen e-scooters, construction zones, and tricky curbs. That data gets stored so that other robots know what lies ahead of them—and it’s now going to the world’s most widely used GPS app for the blind so they can better navigate city streets.

Recommended Video

Coco Robotics, the Los Angeles-based startup operating roughly 10,000 delivery bots across the United States and Europe, is partnering with BlindSquare to remit real-time sidewalk hazard data directly to visually impaired pedestrians. The partnership, announced today, will go live across all six of Coco’s operating markets: Los Angeles, Miami, Chicago, and Jersey City in the U.S. and Helsinki and Turku in Finland.

As Coco’s robots make food deliveries for local restaurants, they continuously log every obstacle they encounter. That data feeds into Coco’s sidewalk map, updated to the minute, and under the new partnership, it will also flow to BlindSquare. The self-voicing app converts the information into spoken alerts delivered in 26 languages, warning users roughly 10 meters before they reach a hazard. In effect, thousands of delivery robots become on-the-ground eyes for people who cannot see what’s ahead.

Issues like bad curb cuts and obstacles like tipped-over scooters have posed significant hazards for the blind.Jonathan Wiggs/The Boston Globe via Getty Images

Boots on the ground, with wheels

The partnership grew out of a European Union grant funding Coco’s operations in Helsinki, where the city’s innovation arm, Forum Virium Helsinki, connected the two companies. Ilkka Pirttimaa, the Finnish developer who built BlindSquare 14 years ago and has watched it grow to roughly 90,000 downloads across 190 countries, was already part of the Helsinki grant consortium alongside Swarco, the traffic-signal manufacturer.

He told Fortune “I didn’t even know any blind persons” when he built BlindSquare. Instead, as someone who loved open data and looking at city maps, he followed blind users on Twitter and read their blog posts about daily obstacles, from wrong trams and unmarked intersections to missing audio cues and downright broken sidewalks. From there, he began assembling an app that could describe a surrounding environment entirely through sound.

The Coco partnership addresses a problem Pirttimaa said has worsened. “Sidewalks, they are a space where blind people sometimes are afraid to go because of e-scooters,” the founder said, adding both Bolt and Voi operate in Finland where he lives. “They are silent. They can go really fast. They can be parked incorrectly.”

But rather than calling for bans on them, Pirttimaa sees a technological fix: “If blind people would know about those e-scooters that are incorrectly parked, it would be beneficial. Robots, they are sharing the same space, and they encounter the same problems. But if that is shared to BlindSquare, then I can notify a blind user that, hey, there is an e-scooter on your way.”

A living map no city has built

The core value proposition is data that municipalities simply do not collect. Carl Hansen, Coco’s vice president of government relations, said the company has discovered that even cities with existing sidewalk data are working off stale information.

“Often when we first go to cities, we ask, what mapping data do you have?” he told Fortune. “Maps that haven’t been updated in a long, long time.”

The data points collected by Coco robots differ from that. “This is fresh to the day, to the hour, to the minute.”

The mapping system works on tiered persistence. When a robot encounters an obstacle, the system categorizes it and assigns a duration. A toppled e-scooter might stay in the map for six hours; active construction could remain for a week.

“The next Coco that comes along checks if it’s there again, and if it’s still there, maybe it gets added for another longer period,” Hansen explained, while structural issues get logged permanently, until the city fixes them.

The companies are also building a two-way exchange. BlindSquare users who pass a previously flagged location can report that an obstacle has been cleared, which in turn updates Coco’s internal routing maps. “There’s a kind of feedback loop making this better for all users,” Hansen said.

Coco CEO Zach Rash framed the partnership as the natural extension of infrastructure the company built for its own survival. “One of the first things we had to build as a company was turn-by-turn directions that are distinct for a robot, and that’s different than car directions. That’s also different than walking directions,” Rash said. “As a byproduct of that, that’s probably the best way for most people to walk through the city. But particularly if you’re blind or in a wheelchair, you’re just rolling the dice if you try to take the straightest path in some of these cities.”

Zach Rash, the CEO of Coco robots.G