Two types of streams are ingested by the Formant platform.
- Telemetry Streams, used for observability
- Teleoperation Streams, used for real-time control and command of the robot
A telemetry stream is a typed, named collection of datapoints. A telemetry stream may contain text data, numerical data, perception data, navigation data, log files, or any other of Formant’s supported stream types.
Telemetry streams are served with low latency to the frontend and persisted to long-term storage in the cloud. Long-term persistence allows one to visualize the state of an application at any point in the past, or power other workflows such as analytics, machine learning, and compliance.
A datapoint is the atomic unit of telemetry data. Datapoints belong to a Telemetry Stream.
Each datapoint contains a timestamp, a payload, and an optional set of tags. For numeric or text datapoints, the payload is a value. For rich-media datapoints like images or point clouds, the payload is a reference to an asset.
ROS type streams
The Formant Agent may be configured to ingest ROS topics as telemetry streams. The agent has built-in support for a number of ROS data types, including, but not limited to:
- Numeric (all standard float and int message types)
- Image (from both sensor_msgs/CompressedImage and sensor_msgs/Image message types)
- Point clouds (from sensor_msgs/PointCloud2 message type)
- Localization (from nav_msgs/OccupancyGrid and nav_msgs/Odometry)
If a ROS message cannot be interpreted as one of the supported ROS data types, the agent will attempt to ingest it as a JSON datapoint.
An optional configuration parameter allows you to specify (using dot notation) a path to a subfield of the ROS message you wish to extract.
You may ingest data using the agent’s gRPC or HTTP API.
Examples of integrating with these endpoints in several languages are available on Github.
The agent exposes its gRPC API on port 5501 by default.
If you are unfamiliar with gRPC you should start by reading about it here.
After you set up your gRPC development environment, you can use the provided protocol buffers to generate language-specific code stubs.
The agent exposes its HTTP API on port 5502 by default.
Files & Directories
The Formant Agent may be configured to ingest new lines in a file or new files in a directory as telemetry streams. Please see the Files & Directories section of the agent configuration in the Formant web app for more details.
Stream Visualization Types
Telemetry Streams can be visualized on the Formant Web Application as one of the following types.
|bitset||An array of bits (with only two possible values: 0 or 1)
||A file such as a ROS bag or other binary log
|health||Displays agent health|
||An (optionally annotated) image
||A set of odometry, map, point cloud, path and goal represented in a map
|metrics||Numeric data represented as a number|
|numeric||Numeric data represented as a graph|
||A point cloud representation
|transform tree||Visualize full tree of coordinate transforms in 3D|
The full set of supported stream types is accessible in the datapoint.proto definition.
A teleoperation stream is a logical stream of data between a user and a device during a Teleoperation Session.
Teleoperation streams are specified as part of a device’s configuration.
Teleoperation streams may be configured as command streams (for user-to-device communication) or observe streams (for device-to-user communication).
Each teleoperation stream will add a UI element to the device’s Teleoperation Context.