Association Rule Learning
"Following patterns in order streams" is used today for:
- Sales and discount prediction
- Analysis of purchased goods
- Product placement on shelves
- Analyzing website browsing patterns
Popular algorithms: Apriori, Euclat, FP-growth
These algorithms encompass all methods for analyzing shopping baskets, automating marketing strategies, and other event-related tasks. When you have a sequence of something and want to find a pattern in it - try these.
There’s no understanding why association rule learning seems to be the least discussed topic in the machine learning category. Classic methods are based on an overview of all purchased goods using trees or sets. Algorithms can only search for patterns, but they cannot generalize or reproduce new samples.
In the real world, every large retailer builds its own proprietary solution, so subtle revolutions are here for you.
- Reinforcement Learning
"Throw a robot into a maze and let it find an exit"
Used today for:
Self-driving cars
Robot vacuums
Games
Automated trading
Enterprise resource management
Popular algorithms: Q-Learning, SARSA, DQN, A3C, Genetic Algorithm
Finally, we get to something resembling real artificial intelligence. In many articles, reinforcement learning is placed somewhere between supervised and unsupervised learning. They have nothing in common! Is it because of the name?
Reinforcement learning is used in cases where your problem is not at all related to data, but you have an environment to live in. Like the world of video games or a city for a car to drive in.
Knowing all the traffic rules in the world doesn't teach an autopilot how to drive on roads. Regardless of how much data is collected, we still can't predict all possible situations. That's why the goal is to minimize errors, not predict all movements.
The main idea for reinforcement learning is survival in an environment. Throw the poor little robot into real life, punish it for mistakes, and reward it for correct actions. The same way we teach our children, right?
A more effective approach here - to build a virtual city and let the driving car learn all its tricks there first. This is exactly how we train autopilots now. Based on a real map, create a virtual city, populate it with pedestrians, and let the car learn to kill as few people as possible. When the robot is reasonably confident in this artificial GTA, it is released for testing on real streets. It’s fun.
There may be two different approaches - model-based and model-free.
Model-based means the machine needs to memorize a map or parts of it. This is a very old approach since an autopilot car can’t memorize the entire planet.
In model-free learning, the machine doesn't remember every move but tries to generalize situations and behave rationally while earning maximum reward.
Remember the news about AI beating a top player in the Go game. Despite it being proven a while ago that the number of combinations in this game exceeds the number of atoms in the universe.
This means the machine couldn’t memorize all combinations and thus won Go (as it did chess). At each turn, it simply chose the best move for each situation, and did it so well as to outsmart a human mind.
This approach is a core concept behind Q-learning and its derivatives (SARSA & DQN). The “Q” in the abbreviation stands for “Quality” as the robot learns to perform the “most practical” action in each situation, and all situations are memorized as a simple Markov process.
Such a machine can test billions of situations in a virtual environment, remembering which solutions lead to greater rewards. But how can it distinguish previous situations from a completely new one? If a self-driving car is at a crosswalk and the traffic light is green - does it mean it can go now? What happens if an ambulance is approaching with sirens blaring nearby?
The answer today is “nobody knows”. There’s no easy answer. Researchers are constantly searching for it but finding only workarounds. Some manually code all situations which allows them to resolve exceptional cases, like the trolley problem. Others will go deep and let neural networks do the work of figuring things out. This leads us to the evolution of Q-learning called Deep Q-Network (DQN). But they are not a silver bullet either.
Reinforcement learning training seems like real AI to an average person. Because it makes you think, this machine makes decisions in real-life situations! This topic is currently very transparent, advancing at an incredible speed, and merging with a neural network to clean your floor more precisely. The amazing world of technologies!
Humanity has yet to find a task where model-based methods are more effective than model-free ones. But they are great for student experiments and allow individuals to excite their university supervisors with “AI” without much effort.
