Unconventional robot chassis remain an under-explored venue of research that poses numerous hard problems in engineering, control and locomotion. In particular, cube-shaped robots have appeared in various architectural concepts and sketches, as well as robotics labs and proposals on robot swarms. Their advantages are inherent stability in the absence of power input, lack of external moving parts, which makes them useful in liquid or hazardous environment, and natural affinity towards stacking and forming semi-permanent structures. In this article we explore the possibility of applying reinforced learning techniques to the problem of controlling a cube-shaped robot. Several existing algorithms are tried on a simpler cart-pole system. Finally, we elaborate on an idea of abandoning a black-box approach and using automated differentiation to implement a numerical rigid-body simulator that can provide gradients for optimization.