Deep dive

From remote debugging to data discovery, learn how Formant works from top to bottom.

    Configure Teleoperation

    Administrators can configure teleoperation for ROS based robots from the Formant Web Application. If your robot is not ROS based, you can use the Formant APIs to transmit teleoperation streams and then configure it using the following method. 

    There are primarily 3 parts to configuring teleoperation:

    1. Essentials: Configure the camera/image streams that you want to see and the joystick you will be using
    2. Advanced: Configure navigation topic, buttons (commands)  and status topics that you want to use
    3. View: Configure how you want these  to appear on the screen

    Essential configuration

    The following topics form the essentials of teleoperation. 

    Camera/Image

    Click on IMAGE  to add new image streams. You can add up to two images.

    If compatible image topics are detected on your robot, they are displayed as minicards on the popup. Click on the minicard to add that as the teleoperation image.

    ROS based image topics

    Image topics with type sensor_msgs/Image will be streamed with H264 video compression. The Formant agent will accept the following values for the "encoding" property of Image messages:

     rgb8 rgba8 bgr8 bgra8 mono8 mono16 yuv422

    If your "encoding" value is unsupported, you may be able to adjust the "encoding" property of the image message, without changing the underlying data. In some cases, you will need to transform both the data and the "encoding" property to supported values. Please contact us if you run into these issues.

    Audio

    Click on AUDIO  to add new audio stream. You can only add one audio stream.

    You can add audio from ROS or directly from hardware. If compatible audio topic or hardware is detected on your robot, they are displayed as minicards on the popup. Click on the minicard to add that as the teleoperation audio. 

    If ROS audio_capture node: use format=wave and add the ROS topic to teleop stream configuration.

    Joystick

    Click on JOYSTICK  to add new joysticks. You can add up to two joysticks.

    If compatible joystick topics are detected on your robot, they are displayed as minicards on the popup. Click on the minicard to add that as the joystick. 

    Once you have added a joystick, you can configure it by clicking on the  edit button.

    Joystick configuration

    Any joystick, including gamepads (e.g. DualShock 4 or Xbox One controllers) that is detected by your machine's OS and supported by the HTML gamepad API (basically anything) can be used for teleoperating. Joysticks are detected automatically as soon as any button is pressed. 

    You can also use the following keyboard shortcuts to joystick your robot. 

    1. WASD control left joystick's up, left, down, right 
    2. Arrow keys control the right joystick

    Position: The joystick can be configured to be positioned on the right side or left side.

    Twist dimension: The twist messages sent by teleop can be configured to map each joystick axis to a value of the twist. One or two joysticks can be configured. 

    Twist messages will be sent to the device when clicking, dragging, and unclicking the software joysticks.

    Gamepad axis: Index of the input axis on a connected gamepad, as detected by the browser. (eg. 0 == Left thumbstick horizontal axis )>

    Scale: Multiplication factor applied to the joystick input. To invert, use a negative value.

    Expo: Exponent applied to the joystick input before the scale factor. For linear output, set this value to 1.

    The formula for a joystick's output value is:

    Abs(JoystickInput ^ Expo) * Scale * Sign(JoystickInput).

    Advanced configuration

    Localization/Navigation

    Specify the topics that will provide the robot’s navigation with respect to its environment. The following topics are ingested as a single telemetry stream. All but odometry topic is optional.

    Telemetry stream name

    Specify a friendly name for the ROS topic that you want to ingest. This name will be displayed on all interfaces that your users and operators will be using. 

    Odometry topic 

    Specify the odometry topic that estimates the position and velocity of the robot (e.g. nav_msgs/Odometry).

    Map topic 

    Specify the map topic (e.g. /map)

    Point cloud or laser scan topics

    Specify the point cloud or laser scan topics (separated by comma) to be visualized as part of the localization stream. We use PCD formant for point clouds.

    Path topic 

    Specify the ROS topic that indicates the path (e.g. nav_msgs/Path) of the robot.

    Goal topic

    Specify the goal topic to display the goal of the robot.

    World reference frame

    Specify the world reference frame. This is used to visualize one's localization state without ingesting the entire /tf topic. Must be the name of a frame published on /tf that is a parent frame of both map and odometry topics.

    Buttons

    Buttons publish boolean messages to the robot. This is useful for setting up real-time commands to the robot. You can configure up to 5 buttons for display.

    Status

    Status subscribes to boolean messages from the robot. This is typically used to convey the state of robot in real-time.

    View configuration

    There are 3 streams that can be viewed while teleoperating a robot. The view configuration is used to configure the view that appears under each box

    Primary view: This is the primary view that consumes the entire view area (black color above). It can be one of real-time image stream, telemetry image stream or localization.

    Sidebar view 1:  It can be one of real-time image stream, telemetry image stream or localization.

    Sidebar view 2:  It can be one of real-time image stream, telemetry image stream or localization.




    © 2020 Formant • 1999 Bryant St · San Francisco, CA 94110