How can I visualize Dijkstra and A* steps in rviz?

edit

If you use the same notion of distance in both algorithms, then you should indeed expect to get the same shortest path results, except you may have multiple paths with the same smallest length, where you will get one of them (eg in a rectangular grid, going 1 diagonal, and one straight versus one straight and then one diagonal has equal length, but they are different paths as you visit different nodes).

To visualize the explored nodes, the last step in both algorithms is to trace the path from the end-point back to the starting-point, as the sequence of nodes to walk. In addition, display all nodes in the 'closed list' of A* (and the equivalent thing in the Dijkstra algorithm).

Note that an A* heuristic of "the constant 0" will degenerate A* to Dijkstra, in which case the set explored nodes will also be the same with both algorithms.