Error when using external odometry for RTAB-Map SLAM

edit

Hello,

I have a differential-drive robot that is equipped with a realsense D435 RGBD camera and IMU. I wanted to fuse the sensor readings from the stereo odometry produced from the stereo_odometry node in the rtabmap_ros package with the IMU that I have using an extended kalman filter using the robot_localization package. When I was able to implement this, I placed the produced odometry topic, /odometry/filtered, in the standard rtabmap launch file rtabmap.launch by replacing the remaped odom topic in the rtabmap SLAM node rtabmap (line 410) with the filtered odometry topic:

<remap from="odom"                   to="/odometry/filtered"/>

When doing this, the filtered odometry is produced fine, however, it seems as though the SLAM node cannot receive neither stereo camera images nor the produced external odometry because no point cloud nor occupancy grid data appears and I consistently receive this error message:

[ WARN] [1679352722.546661048]: /rtabmap/rtabmap: Did not receive data since 5 seconds! Make sure the input topics are published ("$ rostopic hz my_topic") and the timestamps in their header are set. If topics are coming from different computers, make sure the clocks of the computers are synchronized ("ntpdate"). Parameter "approx_sync" is false, which means that input topics should have all the exact timestamp for the callback to be called. /rtabmap/rtabmap subscribed to (exact sync):

/odometry/filtered *

/camera/infra1/image_rect_raw_relay *

/camera/infra2/image_rect_raw_relay *

/camera/infra1/camera_info *

/camera/infra2/camera_info *

/rtabmap/odom_info *

When I run rostopic echo on all of the above topics, they all output some values consistently, so I am not sure why the SLAM node is not receiving the messages. Below, I will display the launch file that I am using to initialize the rtabmap.launch file, the IMU, and the ekf node:

<launch>

<!-- This launch assumes that you have already
   started you preferred RGB-D sensor and your IMU.
   TF between frame_id and the sensors should already be set too. -->

<arg name="rviz"                    default="true" />
<arg name="frame_id"                default="base_link"/>
<arg name="vis_odom_topic"          default="/rtabmap/odom"/>
<arg name="imu_topic"               default="/imu/data" />
<arg name="imu_remove_gravitational_acceleration" default="true" />
<arg name="localization"            default="false"/>

<include file="$(find rtabmap_ros)/launch/rtabmap.launch">
    <arg unless="$(arg localization)" name="rtabmap_args" value="--delete_db_on_start"/>
    <arg name="frame_id" value="$(arg frame_id)"/>
    <arg name="stereo"         value="true"/>
    <arg name="subscribe_rgbd" value="false"/>
    <arg name="subscribe_scan" value="true"/>

    <!-- stereo related topics -->
    <arg name="left_image_topic"        default="/camera/infra1/image_rect_raw" />
    <arg name="right_image_topic"       default="/camera/infra2/image_rect_raw" />      <!-- using grayscale image for efficiency -->
    <arg name="left_camera_info_topic"  default="/camera/infra1/camera_info" />
    <arg name="right_camera_info_topic" default="/camera/infra2/camera_info" />

    <arg name="rgbd_topic"  value="/rgbd_image"/>

    <arg name="visual_odometry" value="true"/>
    <arg name="queue_size"  value="30"/>
    <arg name="compressed"  value="true"/>
    <arg name="rviz"        value="$(arg rviz)"/>
    <arg if="$(arg rviz)" name="rtabmapviz"  value="false"/>  

</include>

<!--Launch static transform between base_link and camera_link-->
<node name="base_to_camera_static_publisher" pkg="tf2_ros" type="static_transform_publisher" args ...