Abstract : Balancing exploratory and exploitative behavior is an essential dilemma faced by adaptive agents. The problem of finding a good trade-off between exploration (learn new things) and exploitation (act optimally based on the current knowledge) has been largely studied for Markov Decision Processes (MDP), but it is relatively new for Factored MDPs. In this presentation we present a strategy to solve the exploration-exploitation dilemma coupled with a learning mechanism designed to learn the FMDP parameters. The solution consists in explicitly creating two different policies, each one designed for exploring or exploiting.