Collecting data from multiple topics
I'm designing a node in my system that needs to gather data from two topics (A and B), process it, and publish the results to another topic. I've considered a few implementation alternatives considering migrating to services, using fixed-rate loops, and retaining the data that a topic handler last received. Here are a few alternatives:
- Create a handler for each topic. In a loop running at a fixed rate, publish the results using the data that was last received for each topic.
- Create a handler for each topic. In handler A, publish results using the data from received in that handler with the data that was last received from handler B.
- Migrate topic A to be a getter service instead. In the handler for topic B, publish results using the data received in that handler and the data received from a call to the new service.
Which method is best? Is there a better way to do this?
The best way to do this will depend on your requirements for the data - if you get three messages on topic B and one on A for a given time frame, do you want to publish a single message with the most recent from each, or 3 messages, one for each B? Does the output need to come at a given rate?