ROS1 vs ROS2 serialization
Hello,
I am serializing the following msg
:
string a
uint8 b
uint8[] c
I am comparing the serialization between ROS1
and ROS2
and I don't know what are the "extra" bytes I get with ROS2
:
Serialized ROS1 message: 11 00 00 00 03 00 00 00 41 42 43 0A 05 00 00 00 0B 0C 0D 0E 0F
Serialized ROS2 message: 19 00 00 00 00 01 00 00 04 00 00 00 41 42 43 00 0A 7f 00 00 05 00 00 00 0B 0C 0D 0E 0E
I am also printing the length of the messages (first 4 bytes). Some differences I see:
ROS2
adds00 01 00 00
at the beginning of the message. What are these 4 bytes represent? Are they related to the message?- For mytype.a it ends with 00 in
ROS2
, is it something related to it or I made a mistake somewhere? - What are the 3 bytes before the length of mytype.c ? (
7f 00 00
)?
Here is the code for ROS1
:
#include "ros/ros.h"
#include "serialization/mytypes.h"
#include <sstream>
#include <iostream>
#include <fstream>
#include <boost/format.hpp>
namespace ser = ros::serialization;
using namespace std;
serialization::mytypes create_serialization_mytypes();
serialization::mytypes create_serialization_mytypes() {
serialization::mytypes result;
result.a = "ABC";
result.b = 10;
for(int i=0; i<5; i++){
result.c.push_back(11+i);
}
return result;
}
int main(int argc, char **argv)
{
ros::init(argc, argv, "node_publisher");
ros::NodeHandle n;
serialization::mytypes mytypes;
mytypes=create_serialization_mytypes();
uint32_t serial_size = ros::serialization::serializationLength(mytypes);
boost::shared_array<uint8_t> buffer(new uint8_t[serial_size]);
ser::OStream stream(buffer.get(), serial_size);
ser::serialize(stream, mytypes);
std::cout << "Serialized ROS1 message: ";
std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size&0xFF) << " ";
std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size>>8&0xFF) << " ";
std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size>>16&0xFF) << " ";
std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(serial_size>>24&0xFF) << " ";
for(int i=0; i<serial_size; i++){
std::cout << std::uppercase << setfill('0') << setw(2) << hex << +(buffer[i]) << " ";
}
std::cout << endl;
return 0;
}
And the code for ROS2
:
#include <chrono>
#include <memory>
#include "rclcpp/rclcpp.hpp"
#include "rclcpp/serialization.hpp"
#include "serialization/msg/mytypes.hpp"
using namespace std::chrono_literals;
class MinimalPublisher : public rclcpp::Node
{
public:
serialization::msg::Mytypes create_serialization_mytypes() {
serialization::msg::Mytypes result;
result.a = "ABC";
result.b = 10;
for(int i=0; i<5; i++){
result.c.push_back(11+i);
}
return result;
}
MinimalPublisher()
: Node("minimal_publisher"), count_(0)
{
publisher_ = this->create_publisher<serialization::msg::Mytypes>("topic", 10); // CHANGE
// printf("Finished\n");
timer_ = this->create_wall_timer(
500ms, std::bind(&MinimalPublisher::timer_callback, this));
}
private:
void timer_callback()
{
auto message = serialization::msg::Mytypes();
message = create_serialization_mytypes();
auto message_header_length = 8u;
auto message_payload_length = static_cast<size_t>(sizeof(message));
serialized_msg.reserve(message_header_length + message_payload_length);
static rclcpp::Serialization<serialization::msg::Mytypes> serializer;
//serializer.serialize_message(message.get(), &serialized_msg);
serializer.serialize_message(&message, &serialized_msg);
printf("Serialized ROS2 message: ");
printf("%02x ", (serialized_msg.size()&0xFF));
printf("%02x ", (serialized_msg.size()>>8&0xFF));
printf ...
As discussed in #q371410, ROS 2 uses DDS as the default middleware. DDS implementations typically use CDR (Common Data Representation) as the serialisation format.
You probably want to look into the relevant standards. The OMG standardises just about everything to do with DDS.
For ROS 2 specifically, in Foxy, the default middleware is Fast-DDS, which uses Fast-CDR.
Wireshark should be able to dissect the common RTPS implementations as well.
It is possible to dissect it with wireshark. However, I haven't seen something like http://wiki.ros.org/ROS/Connection%20Header with a similar, simple and straight forward explanation so far
Could you describe what it is you're trying to do? Are you writing a custom client library and want/need to deserialise message payload yourself, or is this purely academic interest? If the former, reusing an existing implementation may be more efficient. If the latter, you may want to refer to the standards surrounding CDR, XCDR(2) and dynamic datatypes et al.
that's probably because it's not so simple and it's not ROS 2 specific any more, in contrast to the ROS 1 implementations. Part of the reason to rely on a standardised middleware is to be able to delegate the work of implementing, maintaining and documenting everything yourself.
Perhaps eProsima/Micro-CDR could help as well. It's a fairly low-level and ...(more)
Pedantic, but:
actually, that's not "ROS 2", but whichever middleware you have configured for your particular installation.
by default, that'll be Fast-DDS with Fast-RTPS and their implementation of CDR.
But it could be Intel's DPS, or Continental's eCAL.
Yes, it is a sort of custom client library and I need to serialize/deseralize so that is why I am using the provided functions. It is not a completely client library. I am only concern about the serialization/deserialization. So, how it is being done already as the code I have it is perfect. My issue is to understand the "extra" bytes generated by the "ROS2" code vs the ROS1 one.
then I would suggest using an existing CDR implementation instead of trying to reverse engineer it yourself.
well, you seem to be going in the direction of implementing a CDR (de)serialisation library yourself. I wouldn't call that perfect. Personally I'd reuse an existing implementation.
I also don't believe you can assume DDS (with CDR) will be used necessarily.
I haven't checked, but it could well be that other middlewares will use a different serialisation scheme altogether.
See #q369220 for a Q&A with links to a few different RMW implementations. As you can see there are quite a few.
And see design/ROS 2 middleware interface for information on how the RMW abstraction layer works.