Why I Made This
An early version of the LivelyStudio Interface used a version of the now-standard ros3d and roslib library to communicate between the interface and the backend. This library is great at producing a design equivalent in functionality to the native desktop rviz tool. While the design worked fine at a functional level, the implementation of these two libraries required a couple sacrifices. First, since the functionality of rviz and ros3d is meant to be handled in the backend, this meant that all interactions with the 3D interface had to be routed first to the backend, and then back to the front-end. Any interface updates — however large or small — likewise had to be routed through the backend, even if they could have theoretically been done entirely locally. This introduced inherent latency, making the system less usable. On top of this issue was a more general interaction challenge, which is that while rviz and ros3d worked well for debugging interfaces and system feedback, they didn’t provide the type of interactivity needed (by the users and us) in this more design-focused type of interface. Following an evaluation that highlighted these issues, I ended up following a different approach, using the same underlying technology (three.js) as ros3d, but with an architecture more focused on configurability and interactivity. Part of this meant separating the visualization from the inherent ROS implementation and communication that these other tools used. The philosophy was to use data representations analogous to systems like ROS, but not explicitly dependent on it. In that way I could use whichever communication approach I wanted (I could create my own socket or use the publishers/subscribers from the roslib library), or I could make the interface entirely client-side, if ROS communication wasn’t always required. That endeavor resulted in robot-scene. Like OpenVP, this ended up being useful in a variety of projects, including Coframe and the final version of LivelyStudio.