Recording large high speed data sets in ROS

edit

I think you know already the answer. Because each node only communicate between each other, there is no central data storage.

So you have to communicate to each other, to get the messege, or record all traffic.

So you may use the network traffic recording and may this plugin/filter for Wireshark help you to filter the message. https://www.wireshark.org/docs/dfref/... You look for. As you know it afterwards.

As a robot consists of drifferent parts, you normally look only at one part and try to fix it and then look at the next part. And one part generate normal not mutch data(as log it is not video streams or velodymes) but you also know which data you need befor the test.

My suggestion isn't a fix so much as a suggestion that your hardware architecture may not be well suited to your application. There is an old joke, a guy meets with his doctor, he says, "doctor it hurts when I go like this" and the guy proceeds to twists is arm halfway around and starts swinging his forearm around violently. The doctor responds, "The solution is simple, don't do that." You're probably better off trying a different way than squeezing your hardware as hard as you can.

I would suggest not pushing your hardware beyond about 80% of its capacity. I would first look at your topics, do you really need all of them? Can things like images have their resolution lowered, preferably in hardware. If you're still stuck perhaps you can split up your bags a bit. You can run rosbag on multiple hosts each with their own physical disks. I would suggest looking at this tutorial on how to accomplish that.