We’re all excited for the Jetson-like world with robotics in every aspect of life, solving a variety of challenging problems. But most are failing to leave safe, controlled lab spaces. Why?

The Imperfect Information Problem: Operating in the real-world requires robust solutions that can readily manage the chaos of dynamic environments and imperfect data. In the case of robotics, this is the deciding factor between a commercially-successful robot and a failure to launch.

Why Marine Vehicles: Coordinating marine vehicles is an ideal example of automation in a complex, dynamic environment. How do you ensure that your vehicles can safely track and operate around other ships, make the most of the variety of potentially-disagreeing sensors, and robustly handle the busy environment while completing critical tasks?

The NavAbility Case Study: At NavAbility we’re using data from MIT SeaGrant‘s REx/Philos vehicles to demonstrate how any robot can extract map information from multiple sensors, identify and track dynamic objects like ships, and use prior information to navigate effectively in a dynamic environment.

Left: The MIT SeaGrant vehicle path in Boston harbor, position derived solely from this radar data. Right: The MIT SeaGrant REx vehicle.

The Next Generation of Marine Robotics

Marine surface vehicles benefit from GPS information. However true automation requires complete situational awareness by integrating all the sensor data – such as radar and camera information – into a comprehensive map of the current environment. 

At NavAbility, we flipped the problem around:

    • Can we produce a complete map using just radar data? Why? This is the “hard” problem and we wanted to demonstrate this first. It’s also relevant to subsurface vehicle applications as well as high-resolution surface vehicles positioning. GPS can easily be added if available, improving the solution’s accuracy.
    • Can we utilize multiple data sources (such as camera feeds and radar)? A robust object identification and tracking system will make use of the map and all accompanying sensor data for the best solution.
    • Can we share this map between vehicles for coordinated swarming? A shared map is significantly more useful than isolated maps on each vehicle.

Imagine if the various sensors are combined into a single source of truth that:

    • Robustly ignores imperfect data by considering all hypotheses (important with radar data)
    • Includes both the vehicle’s location as well as position+velocity estimates of all nearby obstacles
    • Streams the data into a shared map that can be consumed by all vehicles in the environment
    • Continuously improves as more data is incorporated into the map
    • Publishes updated estimates of all objects in the environment
    • Provides complete situational awareness for all path planning systems

Announcing the Marine Application Example

This example is now available on our Marine Application page. Source code and data is also publicly available!

We will include sections of this example in our tutorial at ICRA2022, so please join us in Philadelphia on May 27.


This case study provides a complete mapping system that can easily be extended to comprehensive obstacle detection.

A powerful feature of the NavAbility platform is that it automatically provides an index of all your robotics data, and it can be queried both by time and position.  The indexed radar and camera data can easily be used to build a obstacle tracking and path planning system that is significantly more accurate than existing systems because:

      • It uses both radar and camera data as well as all the path history of the vehicle(s)
      • It integrates all data from all vehicles
      • The NavAbility algorithm robustly manages conflicting information (mis-classifications and incorrect measurements)
Conceptual overview demonstrating obstacle tracking by combining camera and radar data. Dynamic information - like object velocity and path - can be seamlessly extracted from the processed data to produce actionable maps of the dynamic environment. This information is available to all automated vehicles for collision avoidance and cooperative planning.

More Information

We are looking for hardware partners to integrate this into a commercial product. Reach out to us at info@navability.io!

Additional resources: