Computer Vision with Ultralytics YOLO: Detection, Classification & Segmentation — Complete Beginner's Guide

Imagine a camera that can identify every car on a highway, detect a tumour in an X-ray in milliseconds, or recognise a defective part on a factory line — all without a human looking at each frame. That is Computer Vision. And with Ultralytics YOLO11, you can build systems like this in a few lines of Python.

This guide takes you from zero — installing Python — all the way to training your own YOLO model on a custom dataset. Every code block here is verified and runs. No skipping steps, no assuming you already know things.

What Is Computer Vision?

Computer Vision (CV) is the field of AI that enables machines to interpret and understand visual information — images, videos, and live camera streams. Just as humans use their eyes and brain together to understand a scene, CV uses cameras and deep learning models to do the same, often faster and with greater consistency than a human expert.

Some things CV can do right now, in production:

• Detect objects in video at 100+ frames per second (YOLO11n on GPU)
• Identify plant diseases in drone footage over entire farms
• Count people in a crowd from CCTV footage
• Read license plates on a moving vehicle
• Detect cancer in radiology scans with accuracy matching specialist doctors
• Guide a robot arm to pick and place objects it has never seen before

Computer Vision Frameworks: Quick Overview

Before diving into Ultralytics, here is a quick map of the landscape so you know what exists and when to use each:

Why Ultralytics YOLO?

YOLO stands for You Only Look Once. Unlike older detection systems that scanned an image multiple times, YOLO processes the entire image in a single forward pass through the network — making it fast enough to run in real time.

Ultralytics develops and maintains YOLO and has built it into a complete, beginner-friendly platform. The numbers speak for themselves:

Trusted by Duolingo, Shell, Siemens, Renault, Philips, Intel and thousands of other companies. YOLO11 is their latest generation — faster, more accurate, and supports 5 vision tasks from a single install.

Complete Environment Setup

We will set up a clean, professional Python environment for this project. Follow every step exactly — no skipping.

Open a terminal (Command Prompt on Windows, Terminal on Mac/Linux) and check if Python is installed:

python --version
# or try:
python3 --version

If you see something like Python 3.10.12 or higher — you are good. If you get an error, install Python:

A clean folder for your Computer Vision work. Run these commands in your terminal:

# Create the project directory
mkdir learning_computer_vision

# Enter the folder
cd learning_computer_vision

A virtual environment (venv) is an isolated Python installation for this project. It keeps your project's packages separate from other Python projects — a professional best practice.

# Create the virtual environment (a folder named 'venv' will appear)
python -m venv venv

Activating "switches" your terminal into the isolated environment. You must do this every time you open a new terminal for this project.

venv\Scripts\activate

source venv/bin/activate

After activating, your terminal prompt will show (venv) at the start — that is how you know it is active:

With the venv active, install the required packages:

# Install Ultralytics (includes PyTorch, OpenCV, and all dependencies)
pip install ultralytics

# Install Jupyter for interactive notebooks
pip install jupyter

Verify the installation worked:

python -c "import ultralytics; ultralytics.checks()"

You should see output like this (verified on our machine):

Launch Jupyter in your project folder:

jupyter notebook

Your browser will open at http://localhost:8888. Click New → Python 3 (ipykernel) to create a new notebook. Rename it to cv_yolo_demo.ipynb.

The 3 Core CV Tasks at a Glance

Ultralytics YOLO11 supports 5 vision tasks. Here we focus on the 3 most important ones for beginners:

A simple rule of thumb: if a bounding box is enough, use detection. If you only need one label per image, use classification. If you need the exact shape of each object, use segmentation.

Task 1: Object Detection

What it does

Object detection finds all instances of known objects in an image and draws a rectangular bounding box around each one, with a label and confidence score. For example, given a photo of a street, it might find: car (0.97), person (0.89), traffic light (0.82).

Running detection with a pretrained model (5 lines of code)

In your Jupyter notebook, create a new cell and type:

import ultralytics
ultralytics.checks()
# Expected: "Setup complete ✅"

from ultralytics import YOLO

# Load YOLO11 nano — pretrained on COCO (80 object classes)
# First run downloads the model (~6 MB) automatically
model = YOLO("yolo11n.pt")

# Run detection on a sample image
results = model("https://ultralytics.com/images/bus.jpg")

# Print what was detected
for r in results:
    for box in r.boxes:
        cls_name = model.names[int(box.cls[0])]
        conf = float(box.conf[0])
        print(f"Detected: {cls_name} ({conf:.0%} confidence)")

Verified output from our machine (Ultralytics 8.3.241):

# Save image with boxes drawn on it
results[0].save("detection_result.jpg")
print("Saved! Open detection_result.jpg to see the boxes.")

# OR display it directly in the notebook
results[0].show()

Understanding the result

The results[0] object contains everything about the detection:

• results[0].boxes — list of all detected bounding boxes
• box.cls — class ID (integer). Use model.names[int(box.cls[0])] to get the name
• box.conf — confidence score (0.0 to 1.0). 0.94 means 94% sure
• box.xyxy — box coordinates as [x1, y1, x2, y2] in pixels
• results[0].orig_shape — original image size (height, width)

Training YOLO detection on your own dataset

Using a pretrained model on your own data is called fine-tuning. You take the model that already knows about 80 classes from COCO, and teach it your specific classes — like "phone" and "laptop" on a desk, or "crack" and "healthy" on a wall.

Step A — Dataset folder structure

YOLO expects this exact structure:

Step B — Label file format

Each .txt label file has one line per object. Each line:

• class_id — integer starting from 0 (0=cat, 1=dog, etc.)
• center_x, center_y — centre of the box (0.5 = middle of image)
• width, height — size of the box (1.0 = full image width/height)
• If an image has no objects, the label file should exist but be empty

Step C — The data.yaml file

Place this file inside your dataset folder. It tells YOLO where the data is and what classes exist:

# Where your dataset lives (absolute or relative path)
path: my_dataset

# Paths to images (relative to 'path')
train: images/train
val: images/val
test: images/test  # optional

# Number of classes
nc: 2

# Class names — index 0 must match label id 0
names:
  0: cat
  1: dog

Step D — Train the model

from ultralytics import YOLO

# Start from a pretrained YOLO11 nano model
model = YOLO("yolo11n.pt")

# Train on your custom dataset
results = model.train(
    data="my_dataset/data.yaml",
    epochs=50,          # number of training passes over data
    imgsz=640,          # image size (resize all images to 640×640)
    batch=16,           # images processed at once (lower if RAM is limited)
    device="0",         # use "0" for GPU, "cpu" for CPU-only
    project="runs/train",
    name="my_experiment"
)

After training, your best model weights are saved at runs/train/my_experiment/weights/best.pt. Load and use it like this:

best_model = YOLO("runs/train/my_experiment/weights/best.pt")
results = best_model("path/to/new_image.jpg")
results[0].show()

Task 2: Image Classification

What it does

Classification assigns a single label to the entire image. There are no boxes — the model simply answers "what is this image of?" with a probability for each class. Use it when you don't need to know where something is, only what it is.

Real examples: grading fruit quality (fresh vs rotten), classifying X-rays as normal or pneumonia, distinguishing between plant species.

from ultralytics import YOLO

# Load classification model (pretrained on ImageNet — 1000 classes)
model_cls = YOLO("yolo11n-cls.pt")

# Classify an image
results = model_cls("https://ultralytics.com/images/bus.jpg")

# Print top-5 predictions
for r in results:
    top5_idx = r.probs.top5           # list of top 5 class indices
    top5_conf = r.probs.top5conf       # their confidence scores
    print("Top 5 predictions:")
    for idx, conf in zip(top5_idx, top5_conf):
        print(f"  {r.names[idx]}: {float(conf):.1%}")

Classification dataset structure (different from detection!)

Classification uses a folder-per-class structure — the folder name is the label. No label files needed.

Training a classification model

from ultralytics import YOLO

model = YOLO("yolo11n-cls.pt")  # pretrained on ImageNet

results = model.train(
    data="my_clf_dataset/",  # path to folder with train/ and val/ subfolders
    epochs=30,
    imgsz=224,             # classification typically uses smaller images
    batch=32
)

Task 3: Instance Segmentation

What it does

Segmentation goes further than detection: instead of a rectangular box, it draws a pixel-perfect mask around each instance of an object. If you have two overlapping cars, you get two separate masks — one for each car. This is called instance segmentation (as opposed to semantic segmentation which doesn't distinguish instances).

This requires more detailed annotations and more compute, but gives much richer output — critical for robotics, medical imaging, and precise manufacturing inspection.

from ultralytics import YOLO

# YOLO11 nano segmentation model
model_seg = YOLO("yolo11n-seg.pt")

results = model_seg("https://ultralytics.com/images/bus.jpg")

for r in results:
    print(f"Objects with masks: {len(r.masks.data) if r.masks else 0}")
    for box in r.boxes:
        name = model_seg.names[int(box.cls[0])]
        conf = float(box.conf[0])
        print(f"  {name}: {conf:.0%}")

# Save with masks drawn
results[0].save("segmentation_result.jpg")

Segmentation dataset structure

Same folder structure as detection, but the label format is different — instead of 4 box coordinates, you provide a polygon traced around the object:

Real training example: Car Parts Segmentation

Here is a real training run from the official Ultralytics notebook on the Car Parts Segmentation dataset (23 classes, 3,156 training images). This is what your terminal will look like when training runs:

from ultralytics import YOLO

# Start from pretrained YOLO11 nano segmentation model
model = YOLO("yolo11n-seg.pt")

results = model.train(
    data="my_dataset/data.yaml",  # same YAML as detection
    epochs=50,
    imgsz=640,
    batch=16,
    project="runs/segment",
    name="carparts_experiment"
)

# Run inference with the best trained model
best = YOLO("runs/segment/carparts_experiment/weights/best.pt")
results = best("test_car.jpg", save=True)

from ultralytics import YOLO

model = YOLO("runs/train/my_experiment/weights/best.pt")

# Export to ONNX — works everywhere, up to 3x faster on CPU
model.export(format="onnx")

# Export to TensorRT — up to 5x faster on NVIDIA GPU
model.export(format="engine")

# Export to TFLite — for Android/embedded devices
model.export(format="tflite")