CP3501: Deep Learning

Week 1 - Introduction to Deep Learning

James Cook University

Trimester 1, 2026

Your Instructor

SV

Stephen Vu

  • PhD in Recommender Systems, Queensland University of Technology (QUT)
  • AI Specialist at Telstra
  • 4 years teaching experience across QUT, Kaplan, and Central Queensland University
  • Research focus: Machine Learning, Recommender Systems, Deep Learning applications

What is Deep Learning?

Deep Learning is a subset of machine learning that uses artificial neural networks to learn from data, enabling computers to perform tasks that typically require human intelligence.

Artificial Intelligence
→
Machine Learning
→
Deep Learning
Key Insight: Deep learning models automatically discover patterns in data without being explicitly programmed with rules.

Deep Learning vs Traditional Programming

Traditional Programming

  • Explicit rules defined by programmer
  • If-then-else logic
  • Hard to adapt to new situations
  • Example: "If pixel is red, classify as apple"

Deep Learning

  • Model learns patterns from examples
  • Automatic feature discovery
  • Adapts to new data
  • Example: "Here are 1000 apple images, learn what makes an apple"

Why Deep Learning Now?

100x
More computational power than 2010
1000x
More available data than 2000
90%
Of world's data created in last 2 years

Three Key Enablers:

  • Big Data: Massive datasets for training
  • Computing Power: GPUs and cloud computing
  • Algorithm Advances: Better architectures and training methods

Quick Check: Understanding Deep Learning

What is the main difference between traditional programming and deep learning?

A) Deep learning is faster than traditional programming
B) Deep learning learns patterns from data automatically rather than following explicit rules
C) Deep learning requires less data than traditional programming
D) Deep learning cannot adapt to new situations

Real-World Applications

Computer Vision

  • Medical diagnosis from X-rays and MRI scans
  • Autonomous vehicle navigation
  • Face recognition systems
  • Quality control in manufacturing

Natural Language Processing

  • Language translation (e.g., Google Translate)
  • Chatbots and virtual assistants
  • Sentiment analysis for customer feedback
  • Document summarization

Other Applications

  • Recommendation systems (Netflix, Spotify, Amazon)
  • Fraud detection in financial transactions
  • Drug discovery and protein folding
  • Climate modeling and weather forecasting

CP3501 Course Structure

Weekly Format:

  • 2 hours lecture: Theory and concepts
  • 2 hours practical workshop: Hands-on coding
Week Topic
1-5 Foundations: Neural networks, CNNs, Transfer learning
6 Mid-term Examination (30%)
7-10 Advanced topics: NLP, Transformers, Ethics
10 Group Project Due (30%)
Exam Period Final Examination (40%)

Assessment Breakdown

40%
Final Examination
(Centrally administered)
30%
Mid-term Exam
(Week 6, Online)
30%
Group Project
(Due Week 10)

Important Notes:

  • Exams: Closed book, no GenAI tools permitted
  • Project: GenAI tools allowed with proper declaration
  • Pass requirement: Minimum 50% overall + reasonable attempt at all assessments
  • Individual contributions: Group members may receive different marks based on contribution

Subject Learning Outcomes

By the end of this subject, you will be able to:

SLO Description
SLO1 Explain and apply core deep learning concepts including tensors, loss functions, optimization, and transfer learning
SLO2 Build, train, and tune models through dataset preparation, metric selection, and iterative experimentation
SLO3 Design end-to-end solutions for image, text, or tabular data with reproducible code
SLO4 Identify and address data ethics issues including bias, fairness, privacy, and model limitations

Quick Check: Course Structure

Which assessment allows the use of Generative AI tools?

A) Final Examination
B) Mid-term Examination
C) Group Project (with proper declaration)
D) None of the assessments

Debunking Common Myths

Myth Reality
You need advanced mathematics High school mathematics is sufficient to get started
You need massive datasets Transfer learning allows excellent results with <50 examples
You need expensive computers Free cloud platforms (Google Colab, Kaggle) provide powerful GPUs
You need a PhD Practical skills matter more than advanced degrees
You must understand theory first Code-first approach builds intuition before deep theory

Mathematics: What You Actually Need

Essential (You likely know this)

  • Basic arithmetic operations
  • Understanding of graphs and coordinates
  • Concept of functions (f(x) = y)
  • Simple probability (0 to 1 scale)

Helpful (We'll explain as needed)

  • Matrix multiplication (arrays of numbers)
  • Derivatives (rate of change)
  • Exponential functions
  • Statistical mean and variance
Our Approach: We introduce mathematical concepts when needed in context, not upfront. You'll build intuition through coding before seeing formal mathematics.

How Much Data Do You Really Need?

5-10
Examples per category
(with strong transfer learning)
50-100
Examples per category
(good practical results)
1000+
Examples per category
(state-of-the-art performance)

Transfer Learning: The Secret Weapon

By starting with a model pre-trained on millions of images, you can achieve excellent results with small datasets. The model has already learned to recognize basic patterns like edges, textures, and shapes—you just teach it to recognize your specific categories.

Your Development Environment

What We'll Use

  • Jupyter Notebooks: Interactive coding environment
  • Python: Programming language
  • PyTorch: Deep learning framework
  • fast.ai: High-level library built on PyTorch

Computing Options

  • Google Colab: Free cloud GPUs
  • Kaggle Notebooks: Free cloud environment
  • JCU Resources: On-campus computing labs
  • Personal Computer: Optional, not required
Today's Workshop: We'll set up your environment and train your first model within 30 minutes!

Quick Check: Getting Started

What is transfer learning?

A) Transferring data between computers
B) Using a pre-trained model and adapting it to a new task with less data
C) Moving code from one programming language to another
D) Sharing models between team members

The Core Training Loop

Inputs Architecture Predictions Loss Labels Update Parameters Parameters

This loop repeats thousands of times, gradually improving the model's predictions

Understanding Each Component

Inputs

The data you want to make predictions from (e.g., images, text, sensor readings)

Architecture

The structure of the neural network (e.g., ResNet, Transformer). Think of this as the "recipe" for processing inputs.

Parameters (Weights)

Numerical values that the model learns and adjusts during training. A typical model has millions of these.

Predictions

The model's output (e.g., "this image is a cat" or "sentiment is positive")

Loss Function and Optimization

Loss Function

A mathematical function that measures how wrong the model's predictions are compared to the actual labels.

Goal: Minimize the loss
Lower loss = Better predictions

Optimization

The process of adjusting parameters to reduce loss. Most commonly uses gradient descent or variants like Adam.

Analogy: Like walking downhill in fog—you can only sense the slope beneath your feet, so you take small steps in the steepest downward direction.

Essential Vocabulary

Term Simple Definition
Epoch One complete pass through the entire training dataset
Batch A small subset of data processed together (e.g., 32 images at once)
Learning Rate How big a step to take when updating parameters (too big = unstable, too small = slow)
Overfitting When model memorizes training data but fails on new data (like a student who memorizes answers but doesn't understand)
Validation Set Data held aside to check if model works on unseen examples

Quick Check: Core Concepts

What is overfitting?

A) When the model trains too quickly
B) When the model memorizes training data but performs poorly on new data
C) When the model has too few parameters
D) When the training loss is too high

Why Split Your Data?

All Available Data Training Set (80%) Used to learn patterns Validation Set (20%) Check performance
Critical Principle: Never test your model on data it has seen during training. This is like giving students the exact test questions before the exam—it doesn't measure true understanding.

Transfer Learning: Standing on Giants' Shoulders

Traditional Approach

  • Start with random parameters
  • Learn everything from scratch
  • Requires millions of examples
  • Takes days or weeks to train
  • Needs massive computing resources

Transfer Learning

  • Start with pre-trained model
  • Fine-tune on your specific task
  • Works with small datasets (<100 examples)
  • Trains in minutes or hours
  • Works on standard laptops
Example: A model trained on ImageNet (14 million images) has learned to recognize edges, textures, patterns, and object parts. You can adapt this knowledge to recognize medical images, Australian wildlife, or manufacturing defects with just dozens of examples.

Your First Model: Image Classifier

What We'll Build Today

A classifier that can distinguish between different categories of images (e.g., birds, cars, or any categories you choose).

~10
Lines of code
~5
Minutes to train
90%+
Accuracy achievable

The Code: That's It!

# Import the fast.ai vision library from fastai.vision.all import * # Load your data dls = ImageDataLoaders.from_folder( 'data/birds', # Your image folder train='train', # Training subfolder valid='valid', # Validation subfolder item_tfms=Resize(224) # Resize images ) # Create and train the model learn = vision_learner(dls, resnet34, metrics=error_rate) learn.fine_tune(3) # Train for 3 epochs # Make a prediction learn.predict('new_bird_image.jpg')

This code uses transfer learning with a ResNet34 architecture pre-trained on ImageNet

Today's Hands-On Workshop

Time Activity
0:00 - 0:20 Set up Google Colab environment and access course notebooks
0:20 - 0:40 Walk through first image classification example (pre-built dataset)
0:40 - 1:00 Train your first model and evaluate results
1:00 - 1:30 Collect your own image dataset and train a custom classifier
1:30 - 2:00 Experiment with different datasets, troubleshoot, and share results

What Students Have Built

Week 1 Projects

  • Zanzibar Building Classifier: Classified building conditions from drone imagery (4 categories, 90% accuracy)
  • Panama Transport Identifier: Distinguished different types of buses and transit vehicles
  • Medical Image Analysis: Identified tumor vs normal tissue in pathology slides
  • Environmental Sound Classification: Achieved 80% accuracy on urban sound dataset

Advanced Projects (Later Weeks)

  • Fraud detection system using mouse movement biometrics
  • Assistive app for visually impaired users (computer vision + text-to-speech)
  • Real-time quality control for manufacturing defects

Deep Learning in Industry

$15.7T
Projected AI contribution to global economy by 2030
97M
New AI-related jobs expected by 2025
75%
Of enterprises using AI/ML by 2024

Industry Applications

  • Healthcare: Diagnostic accuracy exceeding human experts in radiology
  • Finance: Real-time fraud detection saving billions annually
  • Manufacturing: Quality control systems reducing defects by 90%
  • Agriculture: Precision farming increasing yields by 30%
  • Retail: Personalization driving 15-20% revenue increases

Quick Check: Practical Understanding

Why should you split your data into training and validation sets?

A) To make training faster
B) To reduce the amount of data you need to collect
C) To evaluate model performance on unseen data and detect overfitting
D) To satisfy course requirements

Getting Help and Resources

JCU Resources

  • LearnJCU: Course materials, assessments, announcements
  • Consultation Hours: Check LearnJCU for schedule
  • Study Groups: Form groups via discussion board
  • Computing Labs: On-campus access with GPUs

External Resources

  • fast.ai Course: Free additional videos and materials
  • fast.ai Forums: Active community for technical questions
  • PyTorch Documentation: Official framework docs
  • Papers with Code: Latest research implementations
Best Practice: When stuck, first check the course materials and forums. If still unclear, attend consultation hours or post on the discussion board. Learning to debug and problem-solve is part of the learning process!

How to Succeed in This Course

1. Active Participation

Attend lectures and workshops. Deep learning is highly practical—hands-on experience is essential.

2. Practice Regularly

Code between classes. Try modifying examples, experiment with different datasets, and learn from failures.

3. Iterate and Experiment

Deep learning is about experimentation. Don't expect perfect results first try—iteration is the path to improvement.

4. Engage with Community

Share your work, ask questions, help classmates. The best learning happens through discussion and collaboration.

Looking Ahead: Week 2

Next Week's Topics

  • Understanding how neural networks actually learn (backpropagation)
  • Data augmentation techniques to improve model robustness
  • Choosing appropriate metrics for different problems
  • Debugging common training issues
Preparation: Complete today's workshop exercises and experiment with your own dataset. Bring questions about anything that confused you.

Week 1 Key Takeaways

Concept
1 Deep learning enables computers to learn patterns from data automatically
2 Transfer learning allows excellent results with small datasets
3 You don't need advanced math, massive data, or expensive hardware to get started
4 The training loop: inputs → architecture → predictions → loss → update parameters
5 Always validate on unseen data to detect overfitting
6 Code-first approach: build intuition before deep theory

Ready to Get Started?

Let's move to the practical workshop and train your first model!

Dr. Stephen Vu
stephen.vu@jcu.edu.au

Use arrow keys or navigation buttons to move between slides

Slide 1 of 34