Steps to take to begin experimenting with deep learning.
Following the advice of Julia Evans, I'm going to write about things that I wish I knew a year ago. Machine Learning research sometimes feels like an activity reserved for the intellectually superior, while us mere mortals enjoy their trimmings when they publish and the open-source community implements it.
This myth was completely debunked for me when I took the fast.ai course, Practical Deep Learning for Coders, v3. So my first tip, do it. Also, this post by Rachel Thomas gives some great advice to those thinking about grad school.
If some of the stuff below seems like gibberish, fast.ai is a good place to go.
Find a dataset that interests you. Here's a good post about where to find it. Try not to get analysis paralysis, just choose one. You're not married to the dataset you choose, but get to know it.
Here are plenty of resources to learn to visualize and analyze a dataset.
If you want free access to a GPU, here you go. If you want to know why or setting up a GPU is becoming frustrating.
Rather than wasting your time loading all of the data each time you want to check for bugs, create a tiny dataset with just one instance of your dataset and overfit it with a very simple model.
Once you can overfit your model on one instance, begin using more of your data and try to overfit it by adding more layers. Don't use any regularization for the moment (e.g., Dropout, L1/ L2 regularization), this is another sanity preserving tip so you know that your model is learning.
Once you add all of your data, if training is taking too long, leave it aside as an experiment and continue your work on the 10% of data. If you're overfitting when you add more layers and all of your data, here's five steps to reduce overfitting.