RGBD based human detection on a merged pointcloud
I am trying to use RGBD based people detection on a pointcloud merged from two individual real sense d415 cameras mounted vertically, image below.
The merge is performed by transforming the individual pointclouds to the base_link (centre of the robot) and adding them using PCL + operator.The resultant pointcloud is as below.
As prerequisite for the people detection, a ground plane needs to be estimated, but as it is seen in the pointcloud, due to the mounting positions o f the cameras not a lot of ground can be seen. As a temporary solution, we have added an aritfical ground plane(a pointcloud matching what would have been the actual ground plane) as seen below.
The person detection algorithm works through detection of people in RGB images obtained from the pointcloud data. Upon passing the ground planes obtained from the artifical plane and the merged pointcloud, the algorithm seems to run fine (By that I mean there are necessary variables being calculated) but there are no detections.
The same setup seems to work if the cameras are positioned horizontally, without the angle orientation and since a lot of the ground is visible then, we don't have to pass the artificial ground plane. The system also seems to work with one camera mounted vertically.
I am wondering if this could be an issue with the camera intrinsic matrix we are passing that is needed for the pcl people detection algorithm, as it does not really match with the merged pointcloud we are providing as an input for the detection.Has anyone tried something similar before in the sense, passing a merged pointcloud to people detection?
Since I'm not an PCL expert and dont have an indepth understanding of I'm unable to understand the problem here. Perhaps it could be a a very benign oversight from our end ?
Please feel free to ask questions to gain a better understanding of the situation and guide us in the right direction.
UPDATE: Wanted to provide a small update in case anyone is interested. The algorithm is able to determine clusters, but the issue is that I get a lot of false detection and people being detected are not consistent as well. If anyone is interested, I can provide more data and the pcd files as well, if needed, for analysis.
Thanks
