Want to become a better tennis player? If you repeatedly practice serving in the same spot, you will master the serve in that spot. exact location, if conditions remain similar. Practicing your serve in various spots will take a lot longer to master, but in the end, you’ll be a better tennis player and much more capable of taking on a fierce opponent.
The reason for this is a matter of variability: the more we are exposed, the better our neural networks are able to generalize and calculate which information is important for the task and which is not. It also helps us learn and make decisions in new contexts.
From fox to dogs
This principle of generalization can apply to many things, including learning languages or recognizing dog breeds. For example, a child will have difficulty learning what a “dog” is if exposed only to Chihuahuas instead of many dog breeds (Chihuahuas, Beagles, Bulldogs, etc.), which show the true variation of Canis lupus familiaris. Including information about what is do not in the category of dogs – for example foxes – also helps us build generalizations, which helps us weed out irrelevant information.
“Learning from less variable inputs is often rapid, but may not generalize to new stimuli,” says lead researcher Dr. Limor Raviv from the Max Planck Institute (Germany). “But this important information has not been unified into a single theoretical framework, which has obscured the big picture.”
To better understand the patterns behind this generalization framework and how variability affects the learning process of humans and computers, Raviv’s research team explored more than 150 studies of variability and generalization in the fields of learning. computing, linguistics, motor learning, visual and formal perception. education.
The researchers found that there are at least four types of variability, including:
- Numeracy (set size), which is the number of different examples; like the number of places on the tennis court where a served ball could land
- Heterogeneity (differences between examples); serving in the same place versus serving in different places
- Situational (context) diversity; face the same opponent on the same terrain or a different element on a different terrain
- Planning (interlacing, spacing); how often do you practice and in what order do you practice the components of a task
“These four types of variability have never been directly compared, which means we currently don’t know which is more efficient for learning,” Raviv says.
According to the “Mr Miyagi principle”, inspired by the 1984 film The Karate Kidpracticing unrelated skills — like waxing cars or painting fences — might actually benefit learning other skills: in the film’s case, martial arts.
Lemon or lime?
So why does including variability in training slow things down? One theory is that there are always exceptions to the rules, which makes learning and generalizing more difficult.
For example, while color is important in distinguishing lemons from limes, it would not be useful in distinguishing cars and trucks. Then there are atypical examples – like a Chihuahua that doesn’t look like a dog and a fox that looks like it, but isn’t.
So, in addition to learning a rule for creating neural shortcuts, we also have to learn exceptions to those rules, which makes learning slower and more complicated. This means that when training is variable, learners must actively reconstruct memories, which takes more effort.
Put a face to a name
So how do we train ourselves and computers to recognize faces? The illustration below is an example of variations of a fox for machine learning. Providing multiple variations – including image rotation, color, and partial masking – improves the machine’s ability to generalize (in this case, identify a fox). This data augmentation technique is an efficient way to increase the amount of data available by providing variations of the same data point, but it slows down the learning speed.
Humans are the same: the more variables we are presented with, the harder it is for us to learn – but this eventually pays off in a greater ability to generalize knowledge in new contexts.
“Understanding the impact of variability is important to literally every aspect of our daily lives. In addition to affecting the way we learn language, motor skills and categories, it even impacts our social lives. Ravi explains. “For example, facial recognition is affected by whether people grew up in a small community (less than 1,000 people) or in a larger community (more than 30,000 people). Exposure to fewer faces during childhood is associated with decreased facial memory.
The learning message for humans and AI is clear: variation is key. Change up your tennis serve, play with many different dogs, and practice the language with a variety of speakers. Your brain (or algorithm) will thank you for it…eventually.