A new method for entropy-randomized machine learning is proposed based on empirical risk minimization instead of the exact fulfillment of empirical balance conditions. The corresponding machine learning algorithm is shown to generate a family of exponential distributions, and their structure is found.