Revision history [back]

I have looked into the code in rl texplore a little bit. According to what you said, I assume that you are using a model-based agent. As far as I know, although continuous regression methods like m5tree is available as modelling technique, you still have to discretize your state space in planning/learning (according to my memory they did not implement value function approximation algorithms), so an infinitely large state space may be problematic.

I do not know whether you are doing planning or learning. Planning should be OK with infinitely large state space. For learning the problem is that you cannot keep an infinitely large table to save the q-values (I think their code asks for parameters to limit the range of each attribute in the state vector), and you can implement your own function approximation algorithm to solve this.