ROS2: Fast publisher, slow subscriber. Is it possible to slow down the publisher?
Suppose a scenario where a publisher is set to send data at a specific rate, e.g. 10Hz. The subscriber, however, is a slow processor; he might need 200ms to process the published data. It is necessary for the application that the subscriber is processing all messages from the publisher.
The default behaviour of ROS2 seems to be to loose messages, the subscriber will only get each 2nd message.
Is it possible at all to slow down the publisher to ensure that the subscriber will receive all published messages while not buffering messages infinitely? I'd like to set a queue size for the connection and the publisher should block if the queue is full. I played around with various QOS settings, but I did not succeed. It seems that KEEP_LAST is an indication that messages can be dropped. KEEP_ALL, however, buffers all the messages until at some point of time the publisher raises an exception.
I do not understand your use-case. If you have a publisher that is faster than the subscriber can process then either you drop messages or you get a queue that fills up infinitely. What data is being published? Why do you need it to slow down? Do you have access to the source code? Why don't you just make the publisher publish at a rate the subscriber can handle? Why is it not ok to just apply a message filter that gets every 2nd, 4th or 10th message?
The most prominent example is the resimulation of a system in offline mode with an "as fast as can" strategy. Here, the data usually comes from disk and you do not want to loose data while you also don't want to slow down the computation more than necessary. Often, you want to try out new algorithms in rapid prototyping, and these algorithms might be slower than realtime, because they are not (yet) optimized. Clearly, you want to see the performance of the algorithms with respect to all input messages stored on disk to keep the output comparable.
There are also examples in online processing, but they are more complicated to explain here.
This should be solvable if both publisher and subscriber have properly separated computation from coordination. 'Scheduling' of when computation should happen can be separated from the computation itself by using an Executor in ROS. If your nodes are all composable, then using a "pipeline" or "fifo" based scheduler should allow you to run things the way you want them: a subscriber cannot run before the publisher in the pipeline has finished, neither will run faster than the slowest one and the entire graph should exhibit the desired behaviour.
For 'normal' use of the nodes there would be less/no need to run them in a pipeline fashion and regular executors and/or individual executors/processes could be used instead.
If either publisher or subscriber have mixed coordination with computation, this would not be possible, but that would be a problem with the publisher and subscriber in that case.
Note that ...(more)
Note: I write should be solvable.
Separating computation from coordination (or any of the other Cs) is not something typically done in ROS 1 or ROS 2 nodes, although ROS 2 makes it easier / forces people to do it more.
Fiddling around with DDS QoS settings is what you don't want to do here. It would not scale, nor be deterministic.
Is it possible to do fifo at the moment in ROS2? To my understanding the only information you get from DDS through the wait-set is whether or not something is ready to be executed. I know for a fact that currently people from Bosch are looking at extending the rmw layer to expose things like timestamps from DDS up to the ROS2 layers to allow for more deterministic scheduling. Writing your own executor is definitely doable, but it will likely require changes at multiple abstraction levels.
Getting pipeline execution behaviour would require writing a custom executor at this point, yes.
My comment (note: not an answer) was not meant as an immediately applicable solution, but more an observation on the general problem of "how could I structure/influence computation in my node graph?".
I see many users introducing boolean flags, additional topics, changes to QoS settings, etc. This made me remark about the separation of computation and coordination in my comment.
this does not seem to be necessary for what the OP wants: (s)he appears to be interested in having separate nodes that process messages "as fast as possible" without losing messages. The best way to do that would be ...(more)
.. I describe above could be used. It would (should?) not need any changes in either publisher or subscriber, run everything as fast as possible and result in completely deterministic behaviour.
Provided both publisher and subscriber "behave" of course.
Thanks for the interesting discussion. The solution with custom executors was not obvious to me before. Indeed it might be a solution for the offline processing problem.
However - if I understand this correctly - this effectively prevents nodes from being executed in different processes, as the executor handles only in-process tasks? To take advantage of the pipeline/FIFO pattern described by @gvdhoorn, the complete graph must be executed in the same process? Note: my ROS experience is limited, so I might be completely wrong with above assumptions.