Learning Path Open Access

The Complete Computer Vision Roadmap 2025–2026

Complete computer vision roadmap from image fundamentals to CNNs, object detection, Vision Transformers, and multimodal vision-language models

Abstract

Computer vision has quietly become one of the most consequential branches of AI - powering everything from medical diagnosis and autonomous vehicles to industrial inspection and the image-generation tools reshaping creative work. Yet learning resources are unevenly split: many jump straight into fine-tuning a pretrained detector without ever explaining convolution or camera geometry, while academic courses stay theoretical and never touch a deployment pipeline. This roadmap bridges that gap. It moves from image fundamentals and classical computer vision through CNN architectures, object detection, segmentation, Vision Transformers, 3D geometry, video understanding, and the multimodal vision-language models now defining the field's frontier - closing with the efficient deployment skills that get a model out of a notebook and onto a device. It is realistic about timeline (12–18 months of sustained effort), organised into thirteen phases, and implementation-first throughout.

Why This Roadmap?

Computer vision sits behind an unusually wide range of technology people interact with daily - face unlock on a phone, medical imaging diagnosis support, autonomous vehicle perception, factory-floor defect detection, and the text-to-image generators reshaping creative software. Despite this breadth, learning resources are oddly polarised: many tutorials fine-tune a pretrained YOLO model in an afternoon without ever explaining what a convolution actually computes, while university courses stay rigorously theoretical and rarely touch deployment.

This roadmap is built to avoid both failure modes. It is geometrically grounded, starting with classical image processing and camera geometry before deep learning, because the intuition transfers and shows up repeatedly in interviews and real systems. It is architecturally complete, covering the full progression from CNNs through Vision Transformers and multimodal vision-language models. And it is deployment-aware from the start, because a model that only works in a Colab notebook has not solved a computer vision problem yet.

Who is this for? Learners with neural network fundamentals who want to specialise in vision. Software engineers moving into perception roles for robotics, AR/VR, or autonomous systems. Data scientists and ML engineers whose work increasingly touches image or video data. Researchers preparing for graduate study or vision-focused research scientist roles. If neural network basics - backpropagation, training dynamics, a first CNN - are unfamiliar, our Deep Learning Roadmap covers that ground before you arrive here. Once you have worked through this roadmap, our ML Interview Guide is the natural next step toward an offer.

Phase 0: Prerequisites & Assessment (2–3 weeks)

Computer vision draws on linear algebra, basic geometry, and neural network fundamentals more heavily than most other ML specialisations. This phase confirms the groundwork is solid before the architectures start compounding.

What You Should Already Know

Comfortable Python, NumPy array manipulation (indexing, broadcasting, reshaping - image tensors are just multi-dimensional arrays), and neural network fundamentals: what a gradient is, how backpropagation works, and ideally a first exposure to a basic CNN. If these are unfamiliar, our Deep Learning Roadmap covers Phases 0 through 3 before you need them here.

Mathematics Refresher for Vision

  • Matrix operations and linear transformations - images are matrices, and transformations (rotation, scaling, perspective) are matrix operations applied to pixel coordinates
  • Basic trigonometry and coordinate geometry - needed for camera projection and geometric transformations in Phase 6
  • Convolution as an operation, conceptually, before the deep-learning version arrives in Phase 2
  • Probability basics - needed for later evaluation metrics and generative vision models

Development Environment

  • Google Colab - Free GPU access, the standard starting point for vision experiments
  • OpenCV - Install and confirm you can load, display, and manipulate an image
  • PyTorch and torchvision - The primary framework for the rest of this roadmap
Checkpoint: You should be able to: (1) Load an image as a NumPy array and reason about its shape (height, width, channels). (2) Apply a matrix transformation to a set of 2D coordinates by hand. (3) Explain backpropagation and gradient descent without notes. (4) Run a PyTorch tensor operation on a Colab GPU.

Phase 1: Image Fundamentals & Classical Computer Vision (4–6 weeks)

Before deep learning automated feature extraction, computer vision was built on hand-engineered techniques grounded in geometry and signal processing. This phase is not historical trivia - these techniques remain the standard in real-time, embedded, and geometry-heavy systems, and understanding them makes every later deep learning architecture more intuitive.

Digital Image Representation

Pixels, colour spaces (RGB, HSV, grayscale, and why HSV is often preferred for colour-based segmentation), image resolution and channels, and bit depth. Understand image formats (JPEG's lossy compression artefacts, PNG's lossless compression) and why format choice matters for training data quality.

Image Processing Fundamentals

  • Filtering: Convolution as a sliding-window operation (the same mathematical operation CNNs later learn automatically); Gaussian blur for noise reduction; sharpening filters
  • Edge Detection: Sobel and Prewitt operators (gradient-based), Canny edge detector (the classical gold standard, combining gradient computation, non-maximum suppression, and hysteresis thresholding)
  • Corner & Keypoint Detection: Harris corner detector, SIFT (Scale-Invariant Feature Transform) and ORB (a fast, patent-free alternative) for finding distinctive, matchable points across images
  • Morphological Operations: Erosion, dilation, opening, closing - for cleaning up binary masks

Geometric Transformations

Translation, rotation, scaling, and affine transformations as matrix operations on pixel coordinates; homography and perspective transforms for correcting viewpoint distortion; image warping and interpolation (nearest-neighbour, bilinear, bicubic) and their quality/speed tradeoffs.

Feature Matching & Classical Recognition

Matching keypoints between images (brute-force and FLANN-based matchers) - the classical basis for panorama stitching, object tracking, and visual odometry before deep learning. Histogram of Oriented Gradients (HOG) combined with a classical classifier (SVM) was the standard pedestrian and object detector before CNNs, and understanding it clarifies exactly what CNNs later made obsolete and why.

Critical: Implement the Canny edge detector and a basic image convolution operation from scratch using only NumPy - no OpenCV built-ins for the core operation. This forces you to understand what a "filter" and a "feature map" mean mechanically, which is exactly what a CNN's convolutional layer automates and learns in Phase 2.
Figure 1. Interactive visualisation of the convolution/filtering operation underlying both classical image processing and CNN feature extraction. Embedded from GeoGebra, a free maths visualisation platform.
Checkpoint: You should be able to: (1) Implement convolution and edge detection from scratch. (2) Explain the difference between SIFT/ORB keypoints and CNN-learned features. (3) Apply a homography to correct perspective distortion in an image. (4) Explain why HOG+SVM was replaced by CNNs for object detection.

Phase 2: Convolutional Neural Networks for Vision (5–7 weeks)

CNNs replaced hand-engineered features with learned ones, and remain the backbone of a large share of production computer vision systems. This phase builds architectural fluency, not just the ability to call a pretrained model.

Convolution as a Learnable Operation

Filters/kernels as learned parameters rather than hand-designed operators; padding (valid vs. same) and stride and their effect on output spatial dimensions; pooling (max and average) for spatial downsampling and a degree of translation invariance; the parameter-sharing property that makes convolution vastly more sample-efficient than a fully-connected layer for image data - the central inductive bias this whole phase is built on.

Classic CNN Architectures - In Historical & Conceptual Order

ArchitectureKey Innovation
LeNet-5 (1998)First successful CNN, digit recognition - established the conv-pool-FC pattern
AlexNet (2012)ReLU, dropout, GPU training - sparked the deep learning resurgence in vision
VGGNet (2014)Depth via stacked small 3×3 filters - showed depth matters more than filter size
GoogLeNet / Inception (2014)Multi-scale filters in parallel ("Inception module") for computational efficiency
ResNet (2015)Residual/skip connections - solved the degradation problem, enabled 100+ layer networks
DenseNet (2017)Dense connections between all layers - maximal feature reuse, parameter-efficient
EfficientNet (2019)Compound scaling of depth/width/resolution for optimal accuracy-per-FLOP
ConvNeXt (2022)Modernised CNN design using transformer-era training recipes - closes much of the ViT performance gap
Critical: Implement a ResNet-style residual block from scratch in PyTorch and explain, in your own words, why skip connections solve the degradation problem - deep plain CNNs degrade in accuracy even with proper normalisation, which is distinct from (and easily confused with) the vanishing gradient problem. This distinction is one of the most frequently probed architecture questions in CV interviews.

Transfer Learning & Fine-Tuning

Using pretrained CNN backbones (via torchvision.models) and fine-tuning on a target task. Feature extraction (freezing the backbone, training only a new head) vs. full fine-tuning, and how to decide based on target dataset size and similarity to the pretraining domain (typically ImageNet).

Best Courses for CNN Fundamentals

Checkpoint: You should be able to: (1) Build a CNN from scratch in PyTorch and explain every layer's purpose. (2) Implement and explain residual connections. (3) Fine-tune a pretrained backbone on a new image classification task. (4) Explain the architectural progression from LeNet to ConvNeXt and what problem each innovation solved.

Phase 3: Training Vision Models & Data Augmentation (3–4 weeks)

Vision models are unusually sensitive to data quality, augmentation strategy, and training discipline. This phase covers the practical craft specific to getting vision models to train reliably and generalise.

Data Augmentation

Geometric augmentations (random crop, flip, rotation, scaling) and photometric augmentations (colour jitter, brightness/contrast adjustment, Gaussian noise). Modern, more aggressive techniques: CutMix and MixUp (blending two training images and their labels), RandAugment and AutoAugment (automatically searched augmentation policies). Understand augmentation as a form of regularisation and domain-knowledge injection - augmentations should reflect real variation the deployed model will face, not be applied indiscriminately.

Handling Class Imbalance

Common in real vision datasets (medical imaging, defect detection, rare-species classification). Techniques: weighted loss functions, oversampling minority classes, focal loss (down-weighting easy examples so training focuses on hard, often minority-class, examples - originally developed for object detection but broadly applicable).

Evaluation Metrics Specific to Vision

Top-1 and Top-5 accuracy for classification; precision, recall, and F1 per class for imbalanced classification; confusion matrices for multi-class error analysis. (Detection- and segmentation-specific metrics - IoU, mAP, Dice coefficient - are covered in Phase 4 where they are used.)

Diagnosing Vision-Specific Training Problems

Distinguishing a data problem (mislabelled images, corrupted files, class imbalance) from an architecture or hyperparameter problem before assuming the model is at fault. Visualising a batch of augmented training images before training - a five-minute check that catches an enormous share of preprocessing bugs. Using Grad-CAM or similar saliency techniques to sanity-check that a model is attending to the right image regions rather than exploiting spurious correlations (background, watermarks, image borders).

Checkpoint: You should be able to: (1) Design an augmentation strategy appropriate to a target deployment domain. (2) Diagnose class imbalance and apply an appropriate correction. (3) Visualise and interpret a Grad-CAM saliency map. (4) Explain why "the model is confused" is rarely the correct first diagnosis for a vision training failure.

Phase 4: Object Detection & Segmentation (6–8 weeks)

Classification answers "what is in this image"; detection and segmentation answer "where," at increasing levels of precision. This is the phase most production computer vision roles are built around.

Object Detection: Two-Stage Detectors

Faster R-CNN (Ren et al., 2015) - a Region Proposal Network first suggests candidate object regions, then a second stage classifies and refines each region's bounding box. Understand anchor boxes (pre-defined reference boxes of various scales/aspect ratios that proposals are regressed relative to) and why two-stage detectors trade speed for accuracy compared to single-stage alternatives.

Object Detection: Single-Stage Detectors

YOLO (You Only Look Once, Redmon et al., 2016) reframes detection as a single regression problem over a grid, predicting bounding boxes and class probabilities directly in one forward pass - dramatically faster, making real-time detection practical. SSD (Single Shot Detector) applies a similar single-stage philosophy with multi-scale feature maps. Understand the speed/accuracy tradeoff between YOLO-family and Faster R-CNN-family detectors, and why YOLO's successive versions (up through modern Ultralytics releases) remain the default choice for real-time and edge deployment.

Detection Mechanics Every CV Engineer Must Know

  • Intersection over Union (IoU): The core metric for how well a predicted box matches a ground-truth box
  • Non-Maximum Suppression (NMS): Removing duplicate, overlapping detections of the same object, keeping only the highest-confidence box
  • Mean Average Precision (mAP): The standard detection benchmark metric, averaging precision-recall performance across classes and IoU thresholds
Critical: Implement IoU computation and non-maximum suppression from scratch in NumPy or PyTorch - no library calls. These two functions are short but conceptually dense, and being unable to write them from memory is one of the most common gaps exposed in CV technical interviews.

Semantic, Instance & Panoptic Segmentation

Semantic segmentation labels every pixel with a class without distinguishing individual object instances. Fully Convolutional Networks (FCNs) replaced fully-connected classification heads with convolutional ones to produce dense, per-pixel predictions. U-Net (Ronneberger et al., 2015) - an encoder-decoder architecture with skip connections between corresponding encoder and decoder layers - remains the dominant architecture for segmentation, especially in medical imaging, due to its strong performance on relatively small datasets.

Instance segmentation additionally separates distinct objects of the same class. Mask R-CNN (He et al., 2017) extends Faster R-CNN with a parallel branch predicting a segmentation mask for each detected object. Panoptic segmentation unifies semantic and instance segmentation, assigning every pixel both a class and, for "thing" classes, an instance ID.

Segmentation Evaluation

Dice coefficient and IoU (Jaccard index) at the pixel level, both especially standard in medical imaging where class imbalance (small tumour vs. large background) makes raw pixel accuracy misleading.

Checkpoint: You should be able to: (1) Implement IoU and NMS from scratch. (2) Explain the architectural difference between one-stage and two-stage detectors and their tradeoffs. (3) Train a U-Net or Mask R-CNN on a segmentation dataset and evaluate with Dice/IoU. (4) Choose and justify a detection or segmentation architecture given latency and accuracy constraints.

Phase 5: Vision Transformers & Attention in Vision (4–5 weeks)

The transformer architecture, originally built for language, now underlies much of state-of-the-art computer vision. This phase covers how attention-based models process images and where they outperform, or lag behind, CNNs.

From Sequences to Patches: The Vision Transformer

Dosovitskiy et al.'s ViT (2021) splits an image into fixed-size patches (e.g. 16×16 pixels), linearly embeds each patch, adds positional embeddings, and feeds the resulting sequence through a standard transformer encoder - the same architecture used for text. Understand why ViT lacks CNNs' built-in translation-invariance bias, requiring substantially more training data (or strong pretraining) to match CNN performance, and why at sufficient scale it matches or exceeds CNNs.

Self-Attention for Global Context

Unlike a convolution's local receptive field (which grows only through depth or dilation), self-attention lets every patch directly attend to every other patch from the first layer - giving ViTs an immediate global receptive field. This is especially valuable for tasks requiring long-range spatial reasoning (scene understanding, relationships between distant objects).

Hybrid & Efficient Vision Transformer Variants

  • Swin Transformer: Computes attention within local windows that shift between layers, reintroducing a hierarchical, CNN-like structure while retaining transformer benefits - more efficient for dense prediction tasks like detection and segmentation
  • DETR (Detection Transformer): Reframes object detection as a direct set-prediction problem using a transformer encoder-decoder, eliminating the need for hand-designed anchor boxes and NMS
  • Hybrid CNN-Transformer backbones: Using a CNN for early, local feature extraction and a transformer for later, global reasoning - a common practical compromise

ViT vs. CNN: Practical Decision-Making

ViTs generally need more data or stronger pretraining to match CNN accuracy on small datasets, but scale more gracefully with data and compute. CNNs remain more parameter-efficient and often faster at inference for a given accuracy level, particularly relevant for edge deployment (covered in Phase 11). ConvNeXt (Phase 2) narrows this gap further by borrowing training recipes from the transformer literature. A working computer vision engineer needs to make this choice deliberately, not by default.

Checkpoint: You should be able to: (1) Explain how ViT converts an image into a transformer-compatible sequence. (2) Explain why ViTs need more data than CNNs to reach comparable accuracy from scratch. (3) Fine-tune a pretrained ViT on a classification task. (4) Justify a CNN vs. ViT vs. hybrid choice given a specific deployment constraint.

Phase 6: 3D Vision & Geometric Deep Learning (4–6 weeks)

Most computer vision so far in this roadmap treats images as flat 2D grids. This phase covers reconstructing and reasoning about the 3D world the camera actually observed - essential for robotics, autonomous vehicles, and AR/VR, and a strong differentiator even for engineers not targeting those specific domains.

Camera Geometry & Calibration

The pinhole camera model; intrinsic parameters (focal length, principal point) and extrinsic parameters (camera position and orientation); lens distortion and its correction; camera calibration using checkerboard patterns, a standard practical step before any metric 3D reconstruction task.

Stereo Vision & Depth Estimation

Recovering depth from two cameras with a known baseline, using disparity (the pixel offset of the same point between the two views) and triangulation. Modern learned monocular depth estimation (predicting depth from a single image using a trained network) as a complementary, hardware-simpler alternative where stereo rigs are impractical.

Structure from Motion & SLAM

Structure from Motion (SfM) reconstructs 3D scene geometry and camera poses from multiple 2D images of the same scene, typically using matched keypoints (Phase 1) and bundle adjustment. Simultaneous Localisation and Mapping (SLAM) solves the related, real-time problem of a moving camera or robot building a map while simultaneously tracking its own position within it - core to robotics and AR.

Point Clouds & 3D Deep Learning

Representing 3D data as point clouds (unordered sets of 3D points, often from LiDAR) rather than grids requires permutation-invariant architectures - PointNet and its successors process point clouds directly without voxelising them into a grid first. Voxel-based and mesh-based representations as alternatives, each with different memory and accuracy tradeoffs.

Neural Radiance Fields & 3D Gaussian Splatting

NeRF (Mildenhall et al., 2020) represents a 3D scene implicitly as a neural network mapping 5D coordinates (3D position + 2D viewing direction) to colour and density, enabling photorealistic novel-view synthesis from a sparse set of input images. 3D Gaussian Splatting achieves similar novel-view synthesis quality with an explicit, much faster-to-render representation, and has rapidly become the practical standard for real-time applications since its introduction.

Checkpoint: You should be able to: (1) Calibrate a camera and correct lens distortion. (2) Explain how disparity relates to depth in stereo vision. (3) Explain the difference between SfM and SLAM. (4) Describe at a conceptual level how NeRF or 3D Gaussian Splatting represents a scene.

Phase 7: Video Understanding & Multimodal Vision-Language Models (4–5 weeks)

This phase extends vision along two frontiers: adding the time dimension (video) and connecting vision to natural language - currently the fastest-moving research area in the entire field.

Video Understanding

Video adds a temporal dimension that single-image architectures do not model. 3D convolutions (extending 2D convolution to also convolve across time) capture short-range motion patterns directly. Two-stream networks process spatial (RGB frames) and temporal (optical flow) information in parallel branches. Video transformers extend ViT-style patch attention across both space and time, at significant compute cost. Core tasks: action recognition, temporal action localisation, and video object tracking (maintaining object identity across frames, distinct from per-frame detection).

Vision-Language Contrastive Pretraining: CLIP

CLIP (Radford et al., 2021) jointly trains an image encoder and a text encoder to embed matched image-caption pairs close together and mismatched pairs far apart in a shared representation space, at internet scale. This enables zero-shot image classification - classifying images using natural-language class descriptions with no task-specific training data at all - and forms the backbone of most modern text-to-image generation and image search systems.

Vision-Language Models for Reasoning & Generation

LLaVA-style architectures attach a vision encoder (often a pretrained ViT or CLIP image encoder) to a large language model via a lightweight projection layer, enabling image-conditioned text generation: visual question answering, detailed captioning, and multi-step visual reasoning in natural language. Understand the practical training recipe - a vision-language alignment stage followed by instruction tuning - and current limitations (hallucinated details, weak fine-grained counting and spatial reasoning).

Grounding & Open-Vocabulary Detection

Grounding models connect natural-language phrases to specific regions in an image, extending detection beyond a fixed, pre-defined label set. Open-vocabulary detection and segmentation models (built on CLIP-style joint embeddings) can detect or segment object categories described in free text at inference time, without retraining - a significant practical advance over the fixed-class-list detectors covered in Phase 4.

Checkpoint: You should be able to: (1) Explain how temporal information is incorporated into a video model architecture. (2) Explain how CLIP's contrastive objective enables zero-shot classification. (3) Describe the architecture of a vision-language model like LLaVA at a component level. (4) Use an open-vocabulary detector for a category not in a fixed training label set.

Phase 8: Self-Supervised Learning for Vision (3–4 weeks)

Labelled image data is expensive to produce at scale. Self-supervised learning extracts training signal from the structure of unlabelled images themselves, and is how most modern vision backbones are pretrained before any task-specific fine-tuning.

Contrastive Self-Supervised Learning

SimCLR (Chen et al., 2020) trains representations by generating two randomly augmented views of the same image, pulling their embeddings together, and pushing embeddings of different images apart - requiring no labels at all. Understand the critical role of strong augmentation and large negative-sample batches (or memory banks/momentum encoders, as in MoCo) in making the contrastive objective effective.

Masked Image Modelling

MAE (Masked Autoencoders, He et al., 2022) applies the BERT-style masking idea to images: masking a large fraction of image patches (typically 75%) and training a model to reconstruct the missing pixels from the remaining visible patches. This non-contrastive objective is simpler to implement and scales efficiently, and has become a standard pretraining strategy for ViT backbones.

Why This Matters in Practice

Self-supervised pretraining lets you obtain a strong, general-purpose visual backbone from large amounts of unlabelled domain-specific data (e.g. unlabelled medical scans, unlabelled satellite imagery) before fine-tuning on a smaller labelled subset - often the difference between a viable and non-viable project in data-scarce specialised domains.

Checkpoint: You should be able to: (1) Explain how contrastive and masked self-supervised objectives differ mechanically. (2) Pretrain or fine-tune from a self-supervised checkpoint on a downstream task. (3) Explain when self-supervised pretraining is worth the added complexity versus standard supervised transfer learning.

Phase 9: Generative Vision Models (4–5 weeks)

Generative vision - producing new images rather than analysing existing ones - has become one of the field's most visible applications, from creative tools to synthetic training data generation.

Generative Adversarial Networks (GANs) for Vision

A generator network competes against a discriminator in a minimax game, the generator learning to produce images indistinguishable from real training data. DCGAN established convolutional architectures suited to image generation; StyleGAN introduced style-based generation offering fine-grained, disentangled control over generated image attributes (pose, texture, colour) and remains widely used for face and texture synthesis where fast, single-step generation matters.

Diffusion Models for Image Generation

Denoising Diffusion Probabilistic Models (Ho et al., 2020) generate images by learning to reverse a gradual noising process, step by step turning random noise into a coherent image. Latent diffusion (running this process in a compressed latent space rather than pixel space) made high-resolution diffusion models computationally practical and underlies Stable Diffusion and most modern text-to-image systems. Classifier-free guidance steers generation toward a text prompt; ControlNet adds precise structural conditioning (edge maps, pose skeletons, depth maps) on top of a pretrained diffusion model without retraining it from scratch.

Image-to-Image & Conditional Generation

Style transfer (rendering one image's content in another's artistic style); image inpainting (filling masked or missing regions plausibly); super-resolution (upscaling low-resolution images with learned, perceptually-aware detail); image editing guided by text prompts, now largely built on conditioned diffusion models rather than the GAN-based pix2pix-style approaches that preceded them.

Checkpoint: You should be able to: (1) Explain the GAN adversarial training dynamic and its instability. (2) Explain the forward and reverse diffusion process for image generation. (3) Condition a pretrained diffusion model on a custom structural input (e.g. via ControlNet). (4) Choose an appropriate generative approach given fidelity, control, and inference-speed requirements.

Phase 10: Domain Specialisations (Ongoing, Choose 1–2)

With the core architectures covered, depth in a chosen application domain is what makes a portfolio and interview performance genuinely competitive.

SpecialisationKey TopicsBest Resource
Medical ImagingSegmentation (tumours, organs), classification from CT/MRI/microscopy, uncertainty estimation, regulatory validationU-Net paper, domain journals
Autonomous Vehicles & Robotics3D perception, sensor fusion (camera + LiDAR), real-time detection, SLAMHartley & Zisserman, robotics-specific CV courses
Manufacturing & Industrial InspectionDefect detection, anomaly detection, real-time constraints, classical CV hybridsPyImageSearch practical guides
Satellite & Aerial ImageryMulti-spectral data, large-image tiling strategies, change detection, crop/land-use classificationKaggle satellite datasets
AR/VR & Generative Vision3D reconstruction, NeRF/Gaussian Splatting, real-time diffusion, pose estimationHF Diffusion Course
Checkpoint: You should be able to: (1) Read and implement recent papers in your chosen domain. (2) Build an end-to-end system credible for a portfolio in that specialisation. (3) Explain the domain's distinctive data, evaluation, or deployment constraints - not just its architectures.

Phase 11: Efficient Computer Vision & Deployment (4–5 weeks)

Vision models are frequently deployed under tight latency, memory, or power constraints - on mobile devices, embedded cameras, or real-time video streams - that make efficient inference a first-class concern, not an afterthought.

Model Compression for Vision

Quantisation (FP32 → FP16/INT8) to shrink model size and speed up inference, with an accuracy tradeoff that must be measured, not assumed; structured pruning of channels or entire layers for hardware-friendly speedups; knowledge distillation, training a small, fast student model to match a larger teacher's outputs - common for deploying detection or segmentation models on edge hardware.

Efficient Architectures for Edge Deployment

MobileNet's depthwise separable convolutions dramatically reduce computation compared to standard convolutions with modest accuracy cost; SqueezeNet and other parameter-efficient designs built explicitly for constrained hardware. Understand FLOPs and parameter count as proxy efficiency metrics, but always validate against real on-device latency, since these proxies do not always correlate cleanly with actual hardware performance.

Deployment Formats & Runtimes

  • ONNX - Framework-agnostic model interchange format for portable deployment
  • ONNX Runtime - Optimised cross-platform inference engine
  • TensorFlow Lite - Standard for Android and embedded deployment
  • Core ML - Apple's on-device inference framework for iOS/macOS
  • NVIDIA TensorRT - High-performance GPU inference optimisation

Real-Time Video Pipeline Considerations

Frame-rate vs. accuracy tradeoffs; batching strategies for throughput vs. single-frame latency for responsiveness; hardware acceleration (GPU, mobile NPU) and its effect on architecture choice; the practical difference between a model that works in a benchmark and one that sustains real-time performance on production camera hardware.

Serving & MLOps for Vision Systems

  • Docker for reproducible deployment
  • FastAPI for building inference APIs
  • Monitoring for data drift specific to vision - camera hardware changes, lighting condition shifts, and seasonal variation are common, under-monitored failure sources
Checkpoint: You should be able to: (1) Quantise a vision model and benchmark its accuracy/latency tradeoff. (2) Export a model to ONNX and run it with an optimised runtime. (3) Explain the design considerations for a real-time video inference pipeline. (4) Identify vision-specific data drift risks in a deployed system.

Phase 12: Portfolio Projects & Career Positioning (Ongoing)

A strong computer vision portfolio needs to demonstrate the full pipeline - data handling, architecture justification, rigorous evaluation, and often deployment - not just a high accuracy number on a familiar benchmark.

What Makes a Strong Computer Vision Project?

  • Uses real or realistically messy data - Not another clean, pre-cropped classification dataset; something with class imbalance, noisy labels, or unusual imaging conditions
  • Justifies its architecture choice - Explains why a CNN, ViT, or hybrid was chosen given the data scale and deployment constraints, with an ablation if possible
  • Uses task-appropriate evaluation - mAP for detection, Dice/IoU for segmentation, not just accuracy for every task regardless of fit
  • Addresses deployment where relevant - Even a simple exported ONNX model with a measured latency benchmark signals production awareness
  • Includes error analysis - Showing failure cases and a hypothesis for why they occur, not just aggregate metrics

Project Ideas by Domain

DomainProject Ideas
Classification & Fine-Grained RecognitionFine-grained species/product classification with a fine-tuned CNN or ViT, comparing architecture choices with ablations
Detection & SegmentationCustom-dataset object detector (fine-tuned YOLO) for a niche use case, medical image segmentation with U-Net
3D & Geometric VisionCamera calibration and 3D reconstruction pipeline from a personal image set, a small NeRF or Gaussian Splatting scene
Generative VisionFine-tuned diffusion model with LoRA or ControlNet for a specific style or structural conditioning task
Multimodal VisionZero-shot classification pipeline built on CLIP, a small visual question-answering demo using an open vision-language model
Efficient/Edge DeploymentQuantised, ONNX-exported real-time detector benchmarked on CPU vs. GPU vs. mobile hardware

Publishing & Recognition

  • Kaggle Competitions: Vision-heavy competitions (classification, detection, segmentation) - kaggle.com/competitions
  • Research Papers: Submit to arXiv, workshop tracks at CVPR, ICCV, ECCV - vision-specific workshops are a realistic first publication target
  • Open Source: Contribute to torchvision, Ultralytics, Detectron2
  • Model Sharing: Publishing well-documented fine-tuned models and demos on Hugging Face Hub is a strong, visible portfolio signal

Career Pathways

RoleFocusTypical Path
Computer Vision EngineerApplied detection/segmentation, production integration, deploymentPortfolio + strong PyTorch and deployment fluency
Perception Engineer (Robotics/AV)3D vision, sensor fusion, real-time constraintsPortfolio + Phase 6 (3D vision) depth + robotics-adjacent projects
Applied/Research Scientist - VisionNovel architectures, generative or multimodal vision researchPhD or strong publication record + first-author workshop/conference papers
ML Engineer - Edge/Embedded VisionModel compression, on-device inference, hardware-aware optimisationStrong SWE background + this roadmap's Phase 11 depth
Job Search: Target companies and labs whose published work or engineering blogs match your specialisation - a robotics company's perception team looks for different signals than a healthcare AI startup's imaging team. Network via workshop papers, open-source contributions, and Kaggle competition results as much as LinkedIn. Prepare deliberately for technical interviews - our ML Interview Guide covers coding rounds, ML/CV theory questions, and system design in depth, and pairs directly with the architectural knowledge built across this roadmap.

Timeline & Realistic Pace

PeriodPhaseWeekly HoursFocus
Months 0–1Phase 0–1: Prerequisites & Classical CV20 hrs/weekMath refresher, OpenCV setup, from-scratch convolution and edge detection.
Months 1–2.5Phase 2: CNNs for Vision28 hrs/weekClassic architectures, ResNet from scratch, transfer learning.
Months 2.5–3.5Phase 3: Training & Augmentation25 hrs/weekAugmentation strategy, class imbalance, Grad-CAM diagnostics.
Months 3.5–5.5Phase 4: Detection & Segmentation30 hrs/weekYOLO, Faster R-CNN, U-Net, Mask R-CNN, IoU/NMS from scratch.
Months 5.5–7Phase 5: Vision Transformers28 hrs/weekViT, Swin, DETR, CNN vs. transformer tradeoffs.
Months 7–8.5Phase 6: 3D Vision & Geometry25 hrs/weekCamera calibration, stereo depth, SfM/SLAM, NeRF fundamentals.
Months 8.5–10Phase 7: Video & Multimodal Vision28 hrs/weekVideo architectures, CLIP, vision-language models, open-vocabulary detection.
Months 10–11.5Phase 8–9: Self-Supervised & Generative Vision25 hrs/weekSimCLR, MAE, GANs, diffusion models, ControlNet.
Months 11.5–14Phase 10: Domain Specialisation28 hrs/weekChoose 1–2 tracks and go deep - papers, projects, benchmarks.
Months 14–16Phase 11: Efficient CV & Deployment25 hrs/weekQuantisation, distillation, ONNX export, real-time pipeline design.
Months 16–18+Phase 12: Portfolio & Career20 hrs/week (ongoing)Refine portfolio, publish, contribute to open source. Pair with our ML Interview Guide to accelerate your job search.

Total commitment: ~12–18 months at 20–30 hours/week for intensive learning, then ongoing for career growth and staying current. This assumes neural network fundamentals are already in place (see the Deep Learning Roadmap if not). Engineers already comfortable with CNNs and transformers can compress Phases 2 and 5; those targeting only 2D application domains (skipping 3D vision entirely) can reasonably shorten the total timeline by one to two months.

Common Mistakes When Learning Computer Vision

  • Skipping classical CV to reach deep learning faster - Camera geometry, convolution mechanics, and keypoint matching are not historical trivia; they resurface constantly in interviews, 3D vision, and hybrid production pipelines.
  • Treating accuracy as the only metric that matters - Using plain classification accuracy to evaluate a detector or segmentation model, instead of mAP or Dice/IoU, produces misleading conclusions about model quality.
  • Not visualising data and predictions before trusting metrics - A five-minute visual check of augmented batches and prediction overlays catches an enormous share of silent data and preprocessing bugs that a loss curve alone will not reveal.
  • Ignoring class imbalance until deployment - Especially common in medical imaging and defect detection, where the "interesting" class is rare; a model can score high accuracy while missing nearly every positive case.
  • Choosing an architecture by popularity rather than constraint fit - Defaulting to the largest available ViT or the newest paper's architecture without considering data scale, latency budget, or deployment hardware.
  • Never validating that a model attends to the right image regions - Skipping Grad-CAM or similar saliency checks allows models exploiting spurious correlations (backgrounds, watermarks, borders) to go undetected until they fail in production.
  • Underestimating deployment constraints until the end of a project - Discovering a model is too slow or too large for its target device only after training is complete wastes significant time; treat latency and memory budgets as design constraints from the start.

Frequently Asked Questions

  • You need neural network fundamentals - MLPs, backpropagation, training dynamics, and ideally a first pass through CNNs - before this roadmap will make full sense, since modern computer vision is built almost entirely on deep learning. This roadmap does cover CNN fundamentals in depth in Phase 2 for completeness, but it moves quickly. If backpropagation, gradient descent, and basic CNN operations are unfamiliar, spend two to three weeks on Phases 1–3 of our Deep Learning Roadmap first. Classical CV (Phase 1 here) does not require deep learning at all and is a reasonable place to start regardless of your background.
  • Yes, though its role has narrowed. Classical techniques - edge detection, corner detection, optical flow, homography estimation, camera calibration - remain the standard in real-time embedded systems, robotics, augmented reality, and any setting where a deep model is too slow or too data-hungry to justify. They are also frequently the first stage of a hybrid pipeline (classical preprocessing feeding a learned model) and are common interview material because they test geometric intuition that deep learning can obscure. Do not skip Phase 1 to reach CNNs faster; understanding what convolution and feature extraction meant before learning made them automatic makes the deep learning material far more intuitive.
  • Learn PyTorch first. The overwhelming majority of computer vision research code, pretrained model releases (via torchvision and Hugging Face), and detection/segmentation frameworks (Detectron2, MMDetection, Ultralytics YOLO) are PyTorch-first or PyTorch-only as of 2025–2026. TensorFlow (via Keras 3) remains relevant for certain mobile and edge deployment pipelines through TensorFlow Lite, so light exposure later is worthwhile, but it should not be your primary framework for learning.
  • Classification assigns a single label to an entire image ("this image contains a cat"). Object detection localises and classifies multiple objects, producing bounding boxes with labels and confidence scores ("cat at these coordinates, dog at those coordinates"). Segmentation goes further: semantic segmentation labels every pixel with a class (all "cat" pixels, all "background" pixels) without distinguishing individual instances; instance segmentation additionally separates distinct objects of the same class (this cat vs. that cat); panoptic segmentation unifies both, labelling every pixel with both a class and, where applicable, an instance ID. Each task has distinct architectures, loss functions, and evaluation metrics, all covered in Phase 4 of this roadmap.
  • No. Vision Transformers (ViTs) match or exceed CNN performance at sufficient data and compute scale, and have become the backbone of choice for many large-scale and multimodal vision systems. But CNNs remain highly competitive, more sample-efficient on smaller datasets due to their built-in translation-invariance bias, and often faster and cheaper for edge/mobile deployment. Modern hybrid architectures (ConvNeXt, which modernises CNN design using transformer-era training recipes, and hybrid CNN-transformer backbones) blur the line further. A competent computer vision engineer in 2026 needs to understand both families and the tradeoffs between them, not treat one as universally superior.
  • It depends heavily on your target domain. For roles in autonomous vehicles, robotics, AR/VR, and industrial/manufacturing inspection, 3D vision - camera calibration, stereo depth, structure from motion, point cloud processing, NeRFs and 3D Gaussian Splatting - is core, non-optional material. For roles centred on 2D image understanding (content moderation, medical imaging, general-purpose visual search, most consumer product vision features), 3D vision is a valuable differentiator but not strictly required to be competitive. If you are unsure which direction you want, cover Phase 6 (3D Vision) at a conceptual level regardless, and go deep only if a specific project or role pulls you toward it.
  • They represent the field's current frontier and are increasingly what "state of the art" means in applied computer vision. CLIP jointly embeds images and text so that vision tasks can be posed with natural-language descriptions rather than fixed label sets, enabling zero-shot classification and forming the backbone of most modern text-to-image and image-search systems. LLaVA-style vision-language models attach a vision encoder to a large language model, enabling image-conditioned reasoning, captioning, and visual question answering in natural language. As of 2025–2026, familiarity with these architectures is expected for most computer vision roles beyond narrowly classical detection/segmentation work, and this roadmap's Phase 7 covers them in depth.
  • Start with MNIST and CIFAR-10/100 to validate that your from-scratch implementations work correctly - these are intentionally small and fast to iterate on. Move to a subset of ImageNet or Tiny-ImageNet to understand training at greater scale and diversity. For object detection and segmentation, COCO (Common Objects in Context) is the standard benchmark and a strong portfolio dataset. For a domain-specific or novel portfolio project, Kaggle's computer vision datasets (kaggle.com/datasets) span medical imaging, satellite imagery, wildlife monitoring, and manufacturing defect detection - pick one aligned with your target specialisation rather than defaulting to the most popular one.
  • Extensively, and these are often less saturated, high-impact career paths than general-purpose consumer vision. In healthcare, CV models assist radiologists with tumour detection and organ segmentation in CT/MRI scans, and pathologists with cell classification in microscopy slides - though regulatory approval (FDA, CE marking) and rigorous validation are core parts of the job, not an afterthought. In manufacturing, vision systems perform automated defect detection and quality control on production lines, often under tight latency and real-time constraints that favour classical CV or lightweight CNNs over large transformers. In agriculture, satellite and drone imagery combined with CV enables crop health monitoring and yield prediction. In retail, vision powers automated checkout and inventory tracking. Each of these domains has distinct data constraints, evaluation standards, and deployment realities worth researching before committing to a specialisation.
  • For someone with solid neural network fundamentals (MLPs, backpropagation, basic CNNs) already in place, 12–18 months of consistent, focused effort at 20–30 hours/week is realistic to reach a genuinely competitive level for computer vision engineer or applied scientist roles. This assumes you follow the full progression - classical CV, CNN architectures, detection, segmentation, transformers, and at least one specialisation with a portfolio-quality project - rather than only fine-tuning pretrained models. If you are also building deep learning fundamentals from scratch alongside this roadmap, expect the timeline to extend by three to five months; our Deep Learning Roadmap is the recommended parallel or prior track for that foundational work.
  • Computer vision interviews test a distinct mix of skills: expect questions on convolution arithmetic (output size given kernel/stride/padding), architecture tradeoffs (why ResNet over VGG, when to choose a ViT over a CNN), detection and segmentation metrics (IoU, mAP, Dice coefficient) and their failure modes, and live coding of image preprocessing or a small CNN. Be ready to discuss a project end-to-end - data collection or sourcing, augmentation strategy, architecture choice and justification, evaluation, and deployment constraints. Practise explaining non-maximum suppression and anchor boxes without notes, since object detection concepts are commonly probed. Our ML Interview Guide covers the broader interview process - coding rounds, ML system design, and behavioural rounds - that pairs with the vision-specific depth built in this roadmap.

Comprehensive Resource List

Free Courses & Learning Platforms

ResourceWhat It Covers
Stanford CS231N: CNNs for Visual RecognitionThe definitive course for this entire roadmap - CNNs, detection, segmentation, and more
Coursera - Andrew Ng's Deep Learning SpecializationNeural network and CNN fundamentals (audit free)
Fast.ai - Practical Deep Learning for CodersTop-down, intuitive, code-first approach with strong computer vision coverage
Hugging Face Computer Vision CourseVision Transformers, CLIP, vision-language models, and the modern Hugging Face vision ecosystem
Hugging Face Diffusion Models CourseDiffusion model theory and implementation for generative vision
PyImageSearchPractical tutorials bridging classical and deep-learning computer vision
OpenCV Official TutorialsClassical image processing, geometric transforms, and 3D vision fundamentals
Ultralytics YOLO DocumentationModern real-time object detection training and deployment

Recommended Books

BookWhy It Matters
Multiple View Geometry in Computer Vision - Hartley & ZissermanThe canonical rigorous reference for camera geometry, stereo vision, and 3D reconstruction.
Deep Learning - Goodfellow, Bengio, Courville (free)The definitive rigorous reference for the neural network theory underlying every architecture in this roadmap.
Hands-On Machine Learning - Aurélien Géron (GitHub companion)Excellent practical bridge from ML fundamentals into applied deep learning with code.
Dive into Deep Learning (free, interactive)Runnable code alongside every mathematical concept, including a strong computer vision chapter set.
Computer Vision: Algorithms and Applications - Szeliski (free PDF)Comprehensive survey spanning classical and modern computer vision techniques.

Key Websites, Tools & Libraries

ResourceUse For
Papers With Code - Computer VisionState-of-the-art vision results with linked implementations
COCO DatasetStandard benchmark for detection, segmentation, and keypoint estimation
Hugging Face HubPretrained vision and vision-language models, datasets, Diffusers/Transformers libraries
Detectron2 (Meta AI)Production-grade detection and segmentation research framework
AlbumentationsFast, comprehensive image augmentation library
Kaggle Computer Vision DatasetsPractice datasets and competitions across every vision sub-domain
Distill.pubBeautiful, interactive visual explanations of vision and representation-learning concepts
Weights & BiasesExperiment tracking, model versioning, and visualising training runs

Conclusion: Your Computer Vision Journey Starts Now

Computer vision is one of the most tangible, visually satisfying branches of AI to learn - you can watch your model draw bounding boxes, segment an image, or generate a photorealistic picture, and immediately see whether it worked. This roadmap provides a structured, geometrically grounded path from image fundamentals through CNNs, detection, segmentation, transformers, 3D vision, and generative models. But no roadmap replaces action.

Start today. Pick Phase 0, confirm your fundamentals, then move into Phase 1 and implement edge detection by hand before any deep learning library gets involved. Write code before reading more theory. Build projects on real, imperfect data. Share your work - a visual result is one of the easiest kinds of project to make compelling to a non-technical audience, which matters for building a following and a network.

The timeline is 12–18 months. This reflects the field's genuine breadth - classical CV, CNN architectures, detection, segmentation, transformers, 3D geometry, and a specialisation each demand real understanding, not surface familiarity. Skipping the from-scratch exercises (convolution, IoU, NMS, a residual block) to move faster is the single most common way learners end up unable to explain their own project's architecture under interview pressure.

Finally: computer vision rewards visual intuition as much as mathematical rigour. Look at your data. Look at your model's predictions. Look at its failures. The architectures are elegant, but the field ultimately advances by people who look closely and notice what the numbers alone would not have shown them. And when you are ready to turn this knowledge into a job offer, our ML Interview Guide is the natural next step.

Next Steps

  • Confirm neural network fundamentals are solid - Deep Learning Roadmap if not
  • Pick a start date and commit to a schedule (20–30 hours/week)
  • Install OpenCV and PyTorch/torchvision today, and load your first image as a NumPy array
  • Join a learning community (Discord servers, Hugging Face forums, local computer vision meetups)
  • Build in public - share your from-scratch implementations, training runs, and visual results openly
  • Never stop looking closely at your data and your model's outputs - it is the habit that separates good CV engineers from great ones