Getting Started

Everything you need to know to get started with Formant.

    Streams

    Two types of streams are ingested by the Formant platform.

    1. Telemetry Streams, used for observability
    2. Teleoperation Streams, used for real-time control and command of the robot

    Telemetry streams

    A telemetry stream is a typed, named collection of datapoints. A telemetry stream may contain text data, numerical data, perception data, navigation data, log files, or any other of Formant’s supported stream types.

    Telemetry streams are served with low latency to the frontend and persisted to long-term storage in the cloud. Long-term persistence allows one to visualize the state of an application at any point in the past, or power other workflows such as analytics, machine learning, and compliance.

    Datapoint

    A datapoint is the atomic unit of telemetry data. Datapoints belong to a Telemetry Stream.

    Each datapoint contains a timestamp, a payload, and an optional set of tags. For numeric or text datapoints, the payload is a value. For rich-media datapoints like images or point clouds, the payload is a reference to an asset.

    ROS type streams

    The Formant Agent may be configured to ingest ROS topics as telemetry streams. The agent has built-in support for a number of ROS data types, including, but not limited to:

    1. Text
    2. Numeric (all standard float and int message types)
    3. Image (from both sensor_msgs/CompressedImage and sensor_msgs/Image message types)
    4. Point clouds (from sensor_msgs/PointCloud2 message type)
    5. Location
    6. Localization (from nav_msgs/OccupancyGrid and nav_msgs/Odometry)
    7. Transform

    If a ROS message cannot be interpreted as one of the supported ROS data types, the agent will attempt to ingest it as a JSON datapoint.

    An optional configuration parameter allows you to specify (using dot notation) a path to a subfield of the ROS message you wish to extract.

    non-ROS streams

    You may ingest data using the agent’s gRPC or HTTP API.

    Examples of integrating with these endpoints in several languages are available on Github.

    gRPC

    The agent exposes its gRPC API on port 5501 by default.

    If you are unfamiliar with gRPC you should start by reading about it here.

    After you set up your gRPC development environment, you can use the provided protocol buffers to generate language-specific code stubs.

    HTTP

    The agent exposes its HTTP API on port 5502 by default.

    Files & Directories

    The Formant Agent may be configured to ingest new lines in a file or new files in a directory as telemetry streams. Please see the Files & Directories section of the agent configuration in the Formant web app for more details.

    Stream Visualization Types

    Telemetry Streams can be visualized on the Formant Web Application as one of the following types.

    Type Description
    bitset An array of bits (with only two possible values: 0 or 1)
    file
    A file such as a ROS bag or other binary log
    health Displays agent health
    image
    An (optionally annotated) image
    location
    A geolocation
    localization
    A set of odometry, map, point cloud, path and goal represented in a map
    metrics Numeric data represented as a number
    numeric Numeric data represented as a graph
    point cloud
    A point cloud representation
    text
    A string
    transform tree Visualize full tree of coordinate transforms in 3D

    The full set of supported stream types is accessible in the datapoint.proto definition.

    Teleoperation Streams

    A teleoperation stream is a logical stream of data between a user and a device during a Teleoperation Session.

    Teleoperation streams are specified as part of a device’s configuration.

    Teleoperation streams may be configured as command streams (for user-to-device communication) or observe streams (for device-to-user communication).

    Each teleoperation stream will add a UI element to the device’s Teleoperation Context.

    © 2020 Formant • 1999 Bryant St · San Francisco, CA 94110