A problem of constructing stable recursive algorithms to be used within a broad class of identification and learning problems is considered. An approach is presented leading to obtaining strongly consistent algorithms. Both cases of multi and single input/multi and single output (MIMO, MISO, SISO) linear stochastic dynamic systems are involved. Thus obtained, the recursive algorithms do not involve inversion of the performance index Hessian and are stable to sampled data, in contrast to conventional recursive schemes. Simulation examples are presented, which confirm practical efficiency of the approach.