colcon test "Processes under test stopped before tests completed"

edit

This issue only shows up on our slow CI runners. It doesn't show up locally.

I have a launch_testing test that passes but doesn't generate a result. The test itself is minimal; it just tests this:

EXPECT_TRUE(true);

These were the commands to run it:

colcon test --packages-select task_planning --> finished, looks good

colcon test-result --verbose --> 12 tests, 0 errors, 0 failures, 0 skipped --> all good

The test log in colcon_ws/log/latest_test/task_planning looks fine. It says the test has passed. CI prints this:

 [basic_task_planning_test-1] [  PASSED  ] 1 test.
    [basic_task_planning_test-1] Done shutting down
    [INFO] [basic_task_planning_test-1]: process has finished cleanly [pid 43801]
    ok

    ----------------------------------------------------------------------
    Ran 1 test in 2.376s

    Processes under test stopped before tests completed
    -- run_test.py: return code 1
    -- run_test.py: verify result file '/home/runner/work/harvester/harvester/.work/target_ws/build/task_planning/test_results/task_planning/test_launch_basic_task_planning_test.test.py.xunit.xml'
  >>>
build/task_planning/test_results/task_planning/test_launch_basic_task_planning_test.test.py.xunit.xml: 2 tests, 0 errors, 2 failures, 0 skipped
- task_planning.TestGTestWaitForCompletion test_gtest_run_complete
  <<< failure message
    Exception: Launch stopped before the active tests finished.

And the latest test result in colcon_ws/log/latest_test_result/logger_all.log says this:

20220410-0125/Test.xml': the root tag is neither 'testsuite' nor 'testsuites'

My test launch file:

    # -*- coding: utf-8 -*-
from ament_index_python.packages import get_package_share_directory
import launch
from launch.actions import TimerAction
from launch_ros.actions import Node
import launch_testing
import os
import sys
import unittest


def generate_test_description():

this_package_path = get_package_share_directory("task_planning")

sys.path.append(this_package_path + "/launch/")
from mtc_planning_launch_common import generate_common_launch_description

common_config = generate_common_launch_description()

basic_task_planning_gtest = Node(
    executable=launch.substitutions.PathJoinSubstitution(
        [
            launch.substitutions.LaunchConfiguration("test_binary_dir"),
            "basic_task_planning_test",
        ]
    ),
    output="screen",
    parameters=common_config,
)

return launch.LaunchDescription(
    [
        launch.actions.DeclareLaunchArgument(
            name="test_binary_dir",
            description="Binary directory of package "
            "containing test executables",
        ),
        basic_task_planning_gtest,
        launch_testing.actions.ReadyToTest(),
    ]
), {
    "basic_task_planning_gtest": basic_task_planning_gtest,
}


class TestGTestWaitForCompletion(unittest.TestCase):
    # Waits for test to complete, then waits a bit to make sure result files are generated
    def test_gtest_run_complete(self, basic_task_planning_gtest):
        self.proc_info.assertWaitForShutdown(basic_task_planning_gtest, timeout=4000.0)


@launch_testing.post_shutdown_test()
class TestGTestProcessPostShutdown(unittest.TestCase):
    # Checks if the test has been completed with acceptable exit codes
    def test_gtest_pass(self, proc_info, basic_task_planning_gtest):
        launch_testing.asserts.assertExitCodes(
            proc_info, process=basic_task_planning_gtest
        )