View profile

Autoencoders and rotten bananas

Autoencoders and rotten bananas
By Santiago • Issue #1 • View online
I’ve wanted to write underfitted.io since email was invented.
Okay, maybe I’m exaggerating slightly, but it really doesn’t matter: today is the first day!
underfitted.io is our trip together to the world of machine learning. We’ll put on our hard-nosed goggles and take a plane ride through an industry that leans heavily towards academics and theoretical conversations. Simply put, undefitted.io is laser-focused on getting to the practicality of every situation.

A long time ago, I fell in love with autoencoders
I wanted to start the newsletter by talking about autoencoders because they made me fall in love with machine learning early on.
Hey, if they got me, maybe this is the gateway that will make you more involved in the field! Let’s take a look.
Autoencoders are data compression algorithms built using neural networks. A network encodes (compresses) the original input into an intermediate representation, and another network reverses the process to get the same input back.
The encoding process generalizes the input data. I like to think of it as the Summarizer in Chief of the network: its entire job is to represent the entire dataset as compactly as possible, so the decoder can do a decent job at reproducing the original data back.
This encoding and decoding process is lossy, which means we will lose some details from the original input.
My attempt at drawing the structure of a basic autoencoder.
My attempt at drawing the structure of a basic autoencoder.
But how are these autoencoders useful?
Any uncommon characteristics in the input data will never make it past the autoencoder’s bottleneck. The encoder is very picky about the features it encodes, so it’ll never select anything that’s not representative of the whole dataset. This is a great property that we can use to detect anomalies in the data!
For example, imagine that we train an autoencoder with thousands of pictures of bananas. The autoencoder will get really good at reproducing the original picture back because it’s learned how to represent them.
A banana. A ripe banana 🍌, to be more precise, courtesy of unsplash.com.
A banana. A ripe banana 🍌, to be more precise, courtesy of unsplash.com.
What would happen if we show the autoencoder a rotten banana instead? Well, our autoencoder didn’t learn to represent rotten bananas, so the decoder will never be able to reproduce it back correctly. The result will look like a ripe banana because that’s all the information the decoder had 🤷‍♂️.
If we take the original image and compute the difference with the output, we’ll see a large divergence. If they both were ripe bananas, the divergence would be small, so we immediately know there’s a problem with the input image.
One more interesting application
We can also do something different. What would happen if we teach the autoencoder to transform the input image into something else?
For example, we could teach an autoencoder to remove noise from pictures by showing it images with noise and expecting back their corresponding clean version.
Input dataset is full of noisy pictures while the target are clean images.
Input dataset is full of noisy pictures while the target are clean images.
In this case, we teach the encoder to develop an intermediate representation that helps the decoder produce clean pictures. After we train this autoencoder, we can remove the noise from any picture similar to the input dataset!
Too long; didn’t read.
Autoencoders are cool.
If you are hot about them and want to learn more, check Jeremy Jordan’s Introduction to autoencoders. It’s a comprehensive and excellent article (and it has a second part, which is a bonus!)
And if you want to play with some code, I dusted an old example from François Chollet, made some adjustments to it, and published it on the official keras.io site: Convolutional Autoencoder For Image Denoising. You should be able to run it in Google Colab (make sure you turn on a GPU.)
A little bit more about the newsletter
Just because this is the first issue, let me tell you what my plans are—and they will undoubtedly change at some point, but that’ll be another story.
With underfitted.io, I have a single goal in mind: I want to write a story every week that teaches you something about machine learning. It won’t be two, three, or four things. It will be one story every issue that hopefully make you feel a little bit smarter than before.
And I’ll try to make you talk. I’ll ask you to reply to these emails. I’ll try for us to interact. I want to hear from you, the things you are building and how underfitted.io can help you. A conversation is a thousand times better than anything I could ever write.
Today's sponsor is awesome!
I’ve always wanted to write this — especially because today’s sponsor is me 🙃, coming with a 30% off both of my courses:
Just click on the links, and the discount will be automatically applied.
And if you don’t care and don’t click, it doesn’t matter. I still love you, and I hope to see you next week!
Did you enjoy this issue?
Santiago

Every week, I’ll teach you something new about machine learning.

Underfitted is for people looking for a bit less theory and a bit more practicality. There's enough mathematical complexity out there already, and you won't find any here.

Come for a journey as I navigate a space that's becoming more popular than Apollo 13, Area 51, and the lousy sequel of Star Wars combined.

If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue