toolkit

About · Data · UI · Universe · FAQ · Partners

A collection of open source libraries to help build your custom robot platform on top of Formant APIs.

About

With this repo’s tools you can make custom web and 3D experiences on top of Formant using ThreeJS, get information about your fleet of devices, create realtime visualizations of your robots, and more.

Embed your own application in Formant
Create your own application that uses Formant’s APIs and RealtimeSDK for data
Create WebView’s for mobile applications that show 3D visualization of your fleet

Documentation

How does it work?

We understand you want complete control over your user experience, so Formant provides:

A Data SDK to get realtime and historical data
Simple 3D elements that you can import into your ThreeJS scene
UI custom elements to make your application visually look similar to Formant’s ecosystem

Data

Formant Dataflow

FAQ

How do I get data?

This depends on the type of app you’re making:

I’m making an application outside of formant

Using user login credentials or service accounts, you can use the Data SDK to get access to device data.

import { Authentication, Fleet } from "@formant/data-sdk";

await Authentication.login("sam@robot.xyz", "12345");

// Get all devices
const allDevices = await Fleet.getDevices();

// find the device(s) you want
const device = allDevices.find((d) => d.name === "spot");

// Get data ...
const data = await device.getLatestTelemetry();

I’m making an application within Formant as a custom view

The Data SDK will be able to figure out from url the authentication needed to access the current viewing device.

import { Authentication, Fleet } from "@formant/data-sdk";

await Authentication.waitTilAuthenticated();

// Get the context of a device is passed along as a query string
const device = await Fleet.getCurrentDevice();

// Get data ...
const data = await device.getLatestTelemetry();

How do I run an example

install NodeJS
cd examples/teleop
npm install
npm run dev
Add the url indicated in terminal as a custom view with the right query parameters ( most likely “?device={device_id}&auth={auth}”)
Go the device page you want to see the custom view on
Select the custom view from the views dropdown

For more detailed instruction on custom views, see our custom view readme.

What do you have for ThreeJS?

three-formant-urdf

This ThreeJS element will show a URDF of a robot using the zipped URDF uploaded for formant and control it’s joint states in realtime.

three-formant-video

This ThreeJS element will show a realtime video stream of your device as a plane.

I don’t want to use these libraries, how do I use the HTTP API?

If your just interested in using our APIs, there’s two main steps

1) Get an authorization token

await fetch("https://api.formant.io/v1/admin/auth/login", {
  method: "POST",
  body: JSON.stringify(),
  headers: {
    "Content-Type": "application/json"
  }
});

curl -X POST "https://api.formant.io/v1/admin/auth/login" \
 -H "Accept: application/json" \
 -H "Content-Type: application/json" \
 -d '{"email":"sam@robot.xyz","password":"12345"}'

This will return a JWT token.

{
   "authentication":{
      "accessToken": "abc......xyz",
      ...
    },
    ...
}

2) Call an API with the token

// the token from above
cons token = "abc......xyz"

await fetch("https://api.formant.io/v1/admin/device-details/query", {
  method: "POST",
  body: JSON.stringify(),
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer "+token
  }
});

curl -X POST "https://api.formant.io/v1/admin/device-details/query" \
 -H "Accept: application/json" \
 -H "Content-Type: application/json" \
 -H "Authorization: Bearer abc......xyz" \
 -d '{"enabled":true,"type":"default"}'

I don’t want to use WebPack/Vite, can I just use plain JavaScript?

Yes, you can do it in two ways, first as an ES6 module:

<script type="module">
  import {
    Authentication,
    Fleet,
  } from "https://cdn.jsdelivr.net/npm/@formant/data-sdk/dist/data-sdk.es6.js";
  await Authentication.waitTilAuthenticated();
  const devices = await Fleet.getDevices();
  window.document.body.innerHTML = devices.map((_) => _.name).join("<br>");
</script>

or as a non-module:

<script src="https://cdn.jsdelivr.net/npm/@formant/data-sdk/dist/data-sdk.umd.js"></script>
<script type="module">
  const { Authentication, Fleet } = window.FormantDataSDK;
  await Authentication.waitTilAuthenticated();
  const device = await Fleet.getCurrentDevice();
  window.document.body.innerHTML = JSON.stringify(
    await device.getLatestTelemetry()
  );
</script>

How do I take a URDF and convert it into a zip for Formant?

Formant URDF

To prepare a URDF for the web, you must collect the .urdf file from ROS. You will be consructing a zip that contains the:

urdf in the root folder
meshes referenced by the urdf

Currently formant only supports .dae models. You can easily convert models using ctmconv.

Install using ubuntu:

apt-get install openctm-tools

ctmconv leg.stl leg.dae

An example zip layout will look like

my_arm.zip
- my_arm.urdf
- meshes/
  - arm1.dae
  - arm2.dae

It’s important that your meshes be referenced by relative links. Remove absolute paths that start with “package://”.

<mesh filename="meshes/base_link.dae" />

Can I make an electron app?

Yep, checkout the example.

Formant Electron example

Can I make a custom realtime data channel to my device?

await device.startRealtimeConnection();
const j = await device.createCustomDataChannel("string_command");
await j.waitTilReady();
j.send("konami");
j.addListener((message) => {
 console.log(message);
});

To receive this custom realtime data channel on your device:

def string_command_callback(message):
    print(str(message))

if __name__ == "__main__":
    c = FormantClient()

    c.register_custom_data_channel_message_callback(
        string_command_callback, channel_name_filter=["string_command"])

Can I get high performance data channel connection options?

You can change your connection network reliability settings

const options = { ... };
const j = await device.createCustomDataChannel("joystick", options);

Here’s some combinations we find useful

Reliable TCP-like: `{ ordered: true }`
- This is the default option `createCustomDataChannel` uses

High Speed UDP-like: `{ ordered: false, maxRetransmits: 0 }`
- Good for non-critical one shot messages

Time Constrained UDP-like: `{ ordered: false, maxPacketLifeTime: 300 }`
- Good for things like joysticks

Is there a simple way to get a Formant look and feel?

When building an embedded app, you might want a quick way to get a website that looks like Formant. This css augments class-less semantic HTML making it incredibly easy to use.

Formant Look and Feel

<link
  rel="stylesheet"
  href="https://cdn.jsdelivr.net/gh/formantio/toolkit/css/formant.css"
/>

See an example here.

Can I build a robotics app on my own domain?

Yes, we even provide a simple component ui-sdk-login to handle authorization for you if you just want to focus on user experience.

Formant Login Component

See an example here

Partners

Our partners building on Formant.

This site is open source. Improve this page.