The robots are here! (And they brought snacks!) 

Relay Robotics’ hotel concierge bots are delivering toothbrushes to guests’ rooms. Fetch Robotics’ warehouse robots are shuttling packages between stations. And Serve Robotics’ sidewalk delivery robots are bringing people lunch. 

You will never see us turning down robots bearing gifts, but all these new robots have one big shortcoming: they are not ready to talk to us––or each other. 

In this post we will address:

• Why robots can’t communicate 
• Challenges for interoperable communications
• Lessons we can learn from IoT and the cloud
• Next steps for those seeking interoperability today

Why can’t robots communicate with each other?

On a construction site some carpenters prefer DeWalt tools and others prefer Milwaukee, and (while this may limit who can share batteries) most of these tools will drive the same standard sizes of nails and screws. Standards for nails and screws help to make DeWalt and Milwaukee tools interoperable. We don’t have accepted standards for how to communicate with construction (or other service) robots yet––but at Fresh we are actively involved in developing messaging standards to support robotic systems integration.

Despite having advanced steering mechanisms like swerve drives, the standards for communication in fleets is lacking. And without standards, this current generation of service robots is not able to communicate outside of the closed vertical stacks implemented by their manufacturers. For example Relay’s hotel delivery robots are integrated with a (vertical) traffic control system that calls elevators for the Relay bots and sends them to the correct floor. But if a Serve sidewalk delivery robot is bringing your lunch from down the street it will be stuck at the front door of your hotel––Relay’s traffic management system is not able to communicate with Serve’s robots.

To enable service robots to talk to each other (and to us) we need better communication infrastructure––we need interoperable messaging standards––and we need (cloud) platforms capable of managing heterogeneous fleets of robots using these standards. This is why we are building Harmony, an open technology stack that supports:

People and robots getting work done together.

Challenges for Interoperable Communication with Robots

To support being good workplace partners with people we expect robots will become proficient in a wide range of modes of communication, from surpassing Siri at conversation to peer to peer networking with nearby robots. However, as a starting place we expect service robots to, at a minimum, be able to connect to WiFi and communicate over the network to services that manage robot traffic, building services such as doors and elevators, and task management.

While some of these fleet management systems are closed verticals (such as Relay’s) other groups are working to build interoperable standards. MassRobotics has worked with several AMR vendors to propose an Autonomous Mobile Robot Interoperability Standard (AMR Interop) that defines a message format for robots to report their current location and status to a server that then provides a map of robot traffic in the area. The Open Robotics Middleware Framework (OpenRMF) was initially developed to help hospital robots from different vendors avoid traffic jams.

OpenRMF is an extension of the Robot Operating System (ROS), which uses the Data Distribution Service (DDS) as communications middleware to send messages between services. DDS was developed to keep the networked systems on Navy warships communicating even when parts of the network are literally on fire.

One of the advantages of DDS is it uses a publish/subscribe (Pub/Sub) messaging model. In Pub/Sub a group of clients can both publish and subscribe to message topics and when one client publishes to a topic, all of the clients subscribing to that topic receive the message. A big advantage of this is that it decouples the two sides of an application from each other––rather than addressing a message to a particular service each client just publishes on a topic and then any other service interested in that topic can subscribe to it. DDS also provides message schemas to define the messages that will be sent on each topic, which helps to further decouple the sending and receiving services.

While DDS generally does a great job of managing communications inside a single robot, OpenRMF extends this DDS network across the edge network covering an entire building to (attempt to) communicate between robots and fleet management services. Unfortunately DDS (by default) relies on UDP multicast to maintain connections across a network and UDP multicast does not work well on WiFi networks. Additionally, in ROS the DDS security layer is disabled by default, and enabling it is still experimental (and involved). Without security enabled DDS messages can be intercepted (or spoofed) by anyone connected to the building WiFi. Even less exotic messaging systems such as the RESTful HTTP API provided by the AMR Interop server can accept messages from robots at the edge but cannot reliably initiate messages in the other direction (from the server to the robot) because of the need to punch a hole through any firewalls and NAT layers.

Lessons in Interoperability from the IoT and the Cloud

While we applaud the spirit of OpenRMF and other early attempts at interoperability, communications with robots (and other edge devices) tend to suffer from a variety of indignities unique to edge networking. 

Robots could benefit from borrowing some of the open source tooling already being used in the Internet of Things (IoT) and the Cloud to reliably send and receive messages. One IoT tool with a distinguished pedigree in supporting edge communications is MQTT, a communications protocol developed by IBM to gather telemetry data from pipeline sensors over a satellite link. MQTT uses a Pub/Sub messaging model similar to DDS, except that with MQTT clients publish messages to brokers and brokers forward messages on to clients subscribing to a given topic. As long as the broker has a public IP address, clients can all communicate with each other, even from behind firewalls and NAT’d routers. rob

While MQTT does not provide message schemas there are a variety of other open source schema tools commonly used in IoT and Cloud Computing. The OpenAPI Initiative uses JSON Schema to define the messages returned by RESTful APIs along with tooling to automatically generate API documentation. The CloudEvents Specification gives a structured way to form a message (describing an event) using both an encoding schema (such as JSON and a transport protocol (such as MQTT). CloudEvents can even be transmitted across transport protocols, for example from the edge to a cloud broker over MQTT and then between cloud services over Kafka, a cloud-scale event streaming platform.

Next steps for seamless communication with heterogeneous fleets of Robots

If you’re dealing with challenges building and managing your own fleet of robots (including systems integration, cloud architecture, or any of the other topics mentioned in this post) we’d love to connect.

In the meantime, stay tuned for our next post in this series on Communicating with Robots, where we’ll look at what robots are talking about using these emerging standards!