Multimaster_discovery (multimaster_fkie package) not recognizing other master nodes in multi-roscore system
Hi everyone, I have been struggling to set up the system consisting of several running roscores (multiple ROS master nodes). Just for a context sake, I want to implement multi-platform system where robots will have their own onboard master running and will communicate with other masters in the network. The idea was to test the behavior firstly in simple scenario where two masters are running on the same machine like in tutorials provided by: ROS.org and this document. I am running ROS kinetic, version 1.12.6 on my Linux Ubuntu 16.04 desktop machine.
However, despite checking all the necessary steps, the master_discovery nodes cannot recognise any other masters and I can't find the problem. I would appreciate any insight! :)
For trouble-shooting purposes I explain in short the procedure I used.
-enabled broadcasting : sudo sh -c "echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts"
so that cat /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts returns 0.
-Configured hostnames and IP addresses in /etc/hosts, where I am using comp_host as ROS_HOSTNAME:
127.0.0.1 localhost
127.0.1.1 host_name
192.xxx.xxx.xxx comp_host
-To find out multi cast IP, run netstat -g which returns (among other):
Interface RefCnt Group
--------------- ------ ---------------------
wlo1 3 224.0.0.251
wlo1 1 all-systems.mcast.net
...
-I opened two terminals with multiple windows. In each window of the first terminal I exported ROS environment variables (; is here just to mark cmmd in a new line) export ROS_HOSTNAME=comp_host; export ROS_IP=192.xxx.xxx.xxx; export ROS_MASTER_URI=http://comp_host:11311. While in the second terminal, i changed the port and exported following in every window: export ROS_HOSTNAME=comp_host; export ROS_IP=192.xxx.xxx.xxx; export ROS_MASTER_URI=http://comp_host:11312. In the first terminal I run roscore, rosrun master_discovery_fkie master_discovery _mcast_group:=224.0.0.251 and rosrun master_sync_fkie master_sync. I also publish dummy text in a topic for testing purposes. In second: roscore --port 11312, rosrun master_discovery_fkie master_discovery _mcast_group:=224.0.0.251 and rosrun master_sync_fkie master_sync.
I do not get any warning or error, but the master_discovery nodes cannot find other master. I can't read the topic from another ROS system too. 224.0.0.251 is pinged without problems and all the nodes are run on appropriate ROS_MASTER_URIs.
Some of the terminal outputs: roscore on terminal 1:
started roslaunch server http://comp_host:46207/
ros_comm version 1.12.6
SUMMARY
========
PARAMETERS
* /rosdistro: kinetic
* /rosversion: 1.12.6
NODES
auto-starting new master
process[master]: started with pid [4871]
ROS_MASTER_URI=http://comp_host:11311/
setting /run_id to 71d29aca-de21-11e6-a42d-a01d48b95782
process[rosout-1]: started with pid [4884]
started core service [/rosout]
master_discovery and master_sync on terminal1:
rosrun master_discovery_fkie master_discovery _mcast_group:=224.0.0.251
[INFO] [1484814933.038655]: ROS Master URI: http://comp_host:11311
[INFO] [1484814933.048587]: Check the ROS Master[Hz]: 1
[INFO] [1484814933.048817]: Heart beat [Hz]: 0.02
[INFO] [1484814933.048988]: Active request after [sec]: 60
[INFO] [1484814933.049146]: Remove after [sec]: 300
[INFO] [1484814933.049301]: Robot hosts: []
[INFO] [1484814933.049455]: Approx. mininum avg. network load: 1 ...
Hi, did you tested it without setting ROS_HOSTNAME and ROS_IP?
I tried originally only with ROS_MASTER_URI, but roscore would overwrite my selected hostname and replace it with localhost which I didn't want. I would export ROS_MASTER_URI like http://hostname:port and the output of roscore would be http://localhost:port . ROS_HOSTNAME and IP solved that part.
something is going wrong with multicast communication on the selected interface. Normally the master_discovery should receive own hearbeat messages and add itself to the list. This does not happen. This is the reason, why I asked to try it with localhost.
can you test master_discovery with default values and if it works, try to change ROS_HOSTNAME and ROS_IP.
Hi, I run with 127.0.1.1 hostname which was default. The list of Robot hosts is still empty. I tried with localhost too, but again it is empty. Multicast address was 226.0.0.0 on both nodes, which is default, and multicast is enabled.
you can try to run "rosrun master_discovery_fkie master_discovery _log_level:=DEBUG". Run twice to see more output. You can also try with "_send_mcast:=False" to see at least itself, only with current version from github. Or try with "_robot_hosts:=[comp_host]"