Large Scale Machine Learning (Big Data): Difference between revisions

If your training set is small enough that it can be loaded entirely in memory, but you are experiencing Out-Of-Memory (OOM) errors during training, the culprit is likely a memory-intensive solver. Many common Machine Learning methods in <code>scikit-learn</code> have variations of [https://en.wikipedia.org/wiki/Stochastic_gradient_descent Stochastic Gradient Descent (SGD)] available as an option and replacing the default solver by an SGD-based one is often a straightforward solution to OOM errors.

The following example compares a [https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html Ridge Regression] performed using the default solver and a SGD-based one. You can monitor memory usage by running the command <code>htop</code> on the terminal while the Python code runs.