Exploration Of Encoding And Decoding Methods For Spiking Neural Networks On The Cart Pole And Lunar Lander Problems Using Evolutionary Training
Abstract:
Spiking Neural Networks are increasingly drawing interest due to their potential for large efficiency gains when used with neuromorphic computers. However, when attempting to replicate the successes of Artificial Neural Networks, challenges are faced due to their vastly different architectures and therefore differing methods for training and optimisation. There has been minimal analysis of the differences between encoding and decoding methods and the effect of state space exposure periods on the performance of these networks. The core contribution of this paper is the detailed analysis of decoding methods, state exposure periods, and a learned input encoding method of an evolved Spiking Neural Network within the Reinforcement Learning context. This is demonstrated using the Cart Pole and Lunar Lander Reinforcement Learning problems. The paper discovers a negative correlation between the generation to reach the goal and the state space exposure period over all decoding methods tested. The state exposure period is also found to influence the number of random actions taken due to the decoding methods being unable to select an action. This paper explores the differences in temporal and rate-based decoding as well as identifying benefits in resetting networks to their default states between episode steps. Additionally, the novel input encoder, is effective at pre-processing state information using the same evolutionary algorithm as the rest of the network.