Is it possible to use ROS controllers in Unity

edit

Brief Description

My main objective is to simulate a robotic arm with gripper on a mobile platform using Unity. I want the robot to be capable of picking up and placing objects in the environment using physics and friction, not by making the object a child of the gripper or fixing it to the gripper. I will go into further detail of the issue I'm having below, but basically I cannot figure out an easy way to use ros_control with Unity. Specifically, I want to use a joint_trajectory_controller to control a robot arm in Unity so that it will be able to follow position, velocity, and acceleration trajectories to pick up an object.

I am using two computers:

• Windows 10 64bit desktop running Unity 2019.3 with ROS-sharp
• Ubuntu 16.04 64bit, Linux kernel 4.15.0-106-generic with ROS kinetic and Gazebo 7

Detailed Description

To accomplish my goal listed above, I have been following various tutorials on how to achieve the same thing in Gazebo. Initially I was following a Husky navigation tutorial since Husky will be the physical robot I will have access to eventually, and being able to control a mobile robot with waypoint markers seemed like a good place to start. Getting the demo to work in Gazebo was pretty straight forward, so I started modifying the launch files to launch the ROS-sharp rosbridge_websocket instead of Gazebo. However, I eventually realized that the cmd_vel commands sent to Gazebo in the demo could not be translated by ROS-sharp in Unity. I believe this is because a diff_drive_controller, which accepts these commands, is used to control Husky in the demo through some plugin adapter in Gazebo. My understanding is that no such "plugin" exists in ROS-sharp for Unity to use ros_control. Fortunately, the diff_drive_controller was not too hard to understand, and someone had already written a script to subscribe to cmd_vel and translate to motor velocities here in appendix C. After a few more tweaks and scripting, I had the Husky navigation tutorial running in Unity.

I was pretty excited about getting that to work, but my elation was short lived after I realized how much more difficult it would be to accomplish the same task for an arm and gripper. I could not find a straight forward tutorial that would allow me to simulate a Husky with an arm and gripper in Gazebo, so eventually I settled on simulating the TIAGo robot since it has all of the capabilities I need, good documentation, and a demo that demonstrates everything I'm looking for. After finding the joint_trajectory_controller which I believe is the controller I would need to rewrite in ROS-sharp, I realized that reverse engineering this controller would be much more difficult. I am unfamiliar with how to create quintic splines from position, velocity, and acceleration, and am not even sure what the arm controller is doing considering it is a position_controller yet PID gains are specified and the goal trajectory is given in terms of ... (more)