Transfer Learning with VGG16 for Land Use Classification

Training deep neural networks from scratch requires massive datasets and significant computational resources. Transfer learning offers a pragmatic alternative: leverage the features learned by a model pre-trained on a large-scale dataset (like ImageNet) and adapt them to a new, potentially much smaller, target domain. This approach consistently outperforms training from scratch, especially when labeled data is scarce.

In this post, we apply transfer learning using VGG16 pre-trained on ImageNet to classify aerial images from the UC Merced Land Use dataset. We compare a frozen feature extractor against a fine-tuned model, and demonstrate how TensorBoard can be used to monitor training in real time.

1. The Dataset: UC Merced Land Use

We continue working with the UC Merced dataset: 2,100 aerial images (256×256 pixels, RGB) across 21 land use categories, split into 60% training, 20% validation, and 20% test using stratified sampling.

With only ~60 training images per class, training a deep network from scratch is challenging. This is exactly the scenario where transfer learning shines.

2. VGG16 Architecture

VGG16 is a 16-layer deep convolutional network proposed by the Visual Geometry Group at Oxford. It was trained on ImageNet (1.2 million images, 1000 classes) and achieved 92.7% top-5 accuracy. The architecture follows a simple, elegant pattern: stacks of 3×3 convolutions with increasing filter depths [64, 128, 256, 512], interleaved with max-pooling layers.

The full VGG16 has 138.4 million parameters, of which the convolutional feature extractor accounts for 14.7M and the three dense classification layers account for 123.6M. For transfer learning, we remove the dense layers (include_top=False) and attach a custom classification head:

# Full VGG16 with classification head: 138,357,544 parameters
VGG16(weights="imagenet", include_top=True, input_shape=(224,224,3))

# Feature extractor only: 14,714,688 parameters
VGG16(weights="imagenet", include_top=False, input_shape=(256,256,3))

3. Transfer Learning Strategies

We build a model that places VGG16's convolutional base as a feature extractor, followed by a custom classification head for our 21 classes:

def get_vgg_model(n_classes, fine_tuning=True):
    base_model = VGG16(weights="imagenet",
                       include_top=False,
                       input_shape=(256,256,3))

    model = Sequential([
        base_model,
        Flatten(),
        Dense(100, activation="relu"),
        Dense(50, activation="relu"),
        Dense(n_classes, activation="softmax")
    ])

    model.compile(optimizer=SGD(learning_rate=0.001),
                  loss="categorical_crossentropy",
                  metrics=['accuracy'])

    if fine_tuning:
        base_model.trainable = False  # Freeze VGG16 weights
    else:
        model.trainable = False       # Freeze everything

    return model

We test two configurations:

Strategy A: Frozen (No Fine-Tuning)

The entire model is frozen — both the VGG16 backbone and the classification head. This means no weights are updated during training. The model can only use the ImageNet features as-is, with no adaptation to the target domain. This serves as a baseline to show that a randomly initialized classification head with frozen features cannot learn.

• Total parameters: 17,997,709
• Trainable parameters: 0

Strategy B: Fine-Tuning

The VGG16 backbone is frozen, but the classification head is trainable. The pre-trained convolutional filters extract general visual features (edges, textures, shapes), and the dense layers learn to map these features to our 21 land use classes.

• Total parameters: 17,997,709
• Trainable parameters: 3,283,021 (classification head only)

Preprocessing for Pre-trained Models

A critical but often overlooked detail: pre-trained models expect inputs preprocessed in the same way as their training data. VGG16 was trained with BGR channel ordering and channel-wise mean subtraction. Keras provides preprocess_input for this:

from tensorflow.keras.applications.vgg16 import preprocess_input

# In the DataGenerator:
if self.preprocess_input is not None:
    x_sample = self.preprocess_input(x_sample)

Skipping this preprocessing step would create a distribution mismatch between the training data and what the network expects, significantly degrading performance.

4. Training and TensorBoard Monitoring

We train the fine-tuned model for up to 200 epochs with SGD (lr=0.001), early stopping (patience=20), and TensorBoard logging:

callbacks = [
    ModelCheckpoint("model_vgg16_fine_tune.h5",
                    mode="max", save_best_only=True,
                    monitor="val_accuracy", verbose=1),
    EarlyStopping(patience=20, verbose=1, mode='auto'),
    TensorBoard(logdir, histogram_freq=1)
]

The fine-tuned model converges remarkably fast. Key milestones from the training log:

• Epoch 1: 12.9% accuracy → Epoch 5: 63.7% accuracy (rapid feature adaptation)
• Epoch 8: 94.4% training accuracy, 63.5% validation (overfitting begins)
• Epoch 10: 100% training accuracy reached with only 10 epochs
• Epoch 29: Early stopping triggered after 20 epochs without validation improvement

TensorBoard provides real-time visualization of these training dynamics, including scalar metrics, model graph structure, weight distributions, and histograms. Results are shareable via TensorBoard.dev.

5. Results

Model	Train Loss	Val Loss	Test Loss	Train Acc	Val Acc	Test Acc
VGG16 Frozen	12.242	11.472	12.210	4.93%	5.47%	2.34%
VGG16 Fine-Tuned	0.001	1.359	0.999	100.00%	71.88%	75.78%

The results are striking:

• The frozen model performs at random chance (~5% for 21 classes). Without any trainable parameters, it cannot adapt to the target domain at all. This confirms that even powerful features need an adaptable classification head.
• The fine-tuned model reaches 75.78% test accuracy, a dramatic improvement. Compare this to our best result from training from scratch in the previous post (68.03% with Adam, or 63.22% with augmentation) — transfer learning provides a +7.75 to +12.56 percentage point gain.
• The model achieves 100% training accuracy by epoch 10, confirming that the pre-trained features are highly informative even for aerial imagery (despite ImageNet being composed of natural photographs).

Features learned on ImageNet transfer remarkably well to aerial/satellite imagery. The low-level features (edges, textures, color gradients) are universal across visual domains.

6. Key Takeaways

1. Transfer learning dramatically outperforms training from scratch on small datasets. The 75.78% test accuracy with VGG16 fine-tuning exceeds all from-scratch experiments (best: 68.03%).
2. The classification head must be trainable. Freezing the entire network (backbone + head) results in random chance performance. At minimum, the final dense layers must be learnable.
3. Pre-trained features are domain-transferable. Features learned from natural photographs (ImageNet) transfer effectively to aerial imagery, confirming that low-level visual features are universal.
4. Preprocessing must match the pre-trained model's expectations. VGG16 requires BGR channel ordering and specific mean subtraction — ignoring this creates a distribution mismatch that degrades performance.
5. Convergence is significantly faster. The fine-tuned model reaches 100% training accuracy in just 10 epochs, compared to 100+ epochs needed when training from scratch.
6. TensorBoard is essential for experiment tracking. Real-time monitoring of scalars, weight distributions, and model graphs makes it easier to diagnose issues and compare runs. TensorBoard.dev enables sharing results with collaborators.

Further improvements could include unfreezing the top convolutional blocks of VGG16 for deeper fine-tuning, using learning rate scheduling, or combining transfer learning with data augmentation.