Deep Learning 11 : What is Dropout?

 What is Dropout?

• Dropout is a regularization technique used in deep learning to reduce overfitting.
• During training, it randomly “drops” (sets to 0) a fraction of neurons in a layer.
• This forces the network to learn more robust patterns instead of relying too heavily on specific neurons.

⚙️ How It Works

1. Training phase (forward pass):
• Each time the model processes a batch, dropout randomly deactivates some neurons.
• Example:
• Pass 1 → neurons n1, n3, n4 dropped.
• Pass 2 → neurons n2, n5 dropped.
• The pattern changes every batch, so the model can’t depend on fixed neurons.
2. Testing/Inference phase:
• Dropout is disabled.
• All neurons are active, but their outputs are scaled to account for dropout during training.

📌 Why Use Dropout?

• Prevents overfitting (memorizing training data instead of generalizing).
• Encourages redundancy in feature learning.
• Improves generalization to unseen data.
• Simple and effective — often used with rates like 0.2 (20%) or 0.5 (50%).

✅ Example in Keras

python

from tensorflow.keras.models import Sequential

from tensorflow.keras.layers import Dense, Dropout

 

model = Sequential([

    Dense(128, activation='relu', input_shape=(784,)),

    Dropout(0.5),  # randomly drop 50% of neurons

    Dense(64, activation='relu'),

    Dropout(0.2),  # randomly drop 20% of neurons

    Dense(10, activation='softmax')

])

In short: Dropout is like making your model “forget” parts of itself during training so it learns to be flexible and generalize better.