Robotic arm object tracking

edit

Hi @Abdu, so you essentially have the answer in the previous comments. i just try to summarize steps here:

1. Use an object detector that provides 3D pose of the object you want to track. find_object_2d looks like a good option, though I use OKR
2. Use MoveIt! to reach the object pose: you can request this throw one of the several interfaces. For example in Python you will call 'set_pose_target' and then 'go' on a MoveGroupCommander object.

As @l4ncelot said, you must be careful with the object's pose frame. But you can provide a stamped pose to MoveIt!, so unless you make a mistake, it will be able to resolve the frame to arm reference.

All this is to send the arm to current object pose; if what you want is to move the arm to a position next to the object, then you must do some TF operation to calculate this pose. For example, once you have the object pose, you can transform to the arm base reference frame and calculate a pose closer to the arm but with the same heading

Anyway, if you provide some more details of what are you trying to achieve, would help us to provide a better answer. But I hope this is enough to start hacking!

UPDATE 16 May 2019 To pick an object w/ MoveIt!, to my understanding, u have 2 main options: 1. add a collision object to the planning scene where u have detected one (as I do here). Then you can use the high-level commands pick and place 2. send the arm to the estimated pose and control the gripper yourself, as done here