In this study, one of the main problems of practical implementation of adaptive control and identification systems is considered. This is the problem of choice of an adaptation rate value to provide the required convergence rate of linear regression parameters. The most widely applied method to identify such regression parameter values is the gradient descent (GD) one. But the adaptation rate is usually chosen manually for it and kept constant in the course of the plant functioning. The recursive least-squares (RLS) method can be used instead of GD as it provides the developer with the equation to adjust the adaptation rate automatically, taking into account the current value of the regressor. Therefore, the aim of this research is to compare the known properties of GD and RLS methods and demonstrate the capability of the RLS to provide a constant convergence rate under the condition of the time-varying regressor.