Digital Image Processing and Pattern Recognition: Important Topics Deep Notes in Easy English (Units 1-3)
Unit 1: Introduction to Image and Pattern Recognition
1.1 Pixel
A pixel (picture element) is the smallest unit of a digital image - one tiny square holding a single intensity or color value. An image that is 1024 x 768 is a grid of 1024 columns and 768 rows of pixels. In a grayscale image each pixel is one number, usually 0 (black) to 255 (white). In a color image each pixel holds three numbers - red, green and blue. Everything else in image processing is built on reading and changing these numbers.
1.2 Pattern Recognition
Pattern recognition is the task of teaching a computer to assign an observation to a category: is this handwritten mark a 3 or an 8, is this X-ray normal or abnormal, is this email spam? The machine never sees the world directly - it sees measurements (features), and it learns a rule that maps measurements to labels. The field combines statistics, linear algebra and learning algorithms, and it sits underneath face unlock, voice assistants and medical diagnosis tools.
1.3 Feature Vector
A feature vector is the list of measurements that describes one object as a point in space. Suppose we classify fruit using weight and redness: an apple becomes the vector (150, 0.8), a cherry (10, 0.9). With two features the point lives in 2-D; with 100 features it lives in 100-D space. Classification then becomes geometry: points of the same class should sit close together, far from other classes. Choosing good features is half the work of pattern recognition.
1.4 Training Set
The training set is the collection of labeled examples the system learns from - photos of cats and dogs with the correct label attached to each. The learning algorithm adjusts its internal rule until it labels most training examples correctly. Two rules of thumb: the training set must look like the real data the system will face, and it must be separate from the test set, otherwise you are grading the student on questions they already memorized.
1.5 Template Matching
Template matching finds an object by sliding a small reference image (the template) across a bigger image and scoring how similar each position is. Imagine searching a page of text for a specific letter shape: you overlay the letter's picture at every spot and record the match score; the highest score is your detection. It is simple and needs no training, but it breaks when the object changes size, rotation or lighting, and it is slow because every position must be checked.
1.6 Proximity Measure
A proximity measure is a number that says how close two feature vectors are - either as distance (small = similar) or as similarity (large = similar). The everyday choice is Euclidean distance: d(x, y) = √Σ(xi - yi)², the straight-line distance between two points. Others: Manhattan distance (sum of absolute differences, like walking city blocks), cosine similarity (angle between vectors, popular for text), and Hamming distance for binary strings. The classifier's behavior depends heavily on this choice, so pick the measure that matches what "similar" means for your data.
1.7 Metric and Non-Metric Proximity Measures, with Examples
A metric is a distance function that obeys four rules: (1) non-negative, (2) zero only when comparing an object with itself, (3) symmetric - d(x,y) = d(y,x), and (4) the triangle inequality - going directly from x to y is never longer than going through a third point z: d(x,y) ≤ d(x,z) + d(z,y). Euclidean and Manhattan distances are metrics. A non-metric measure breaks at least one rule. Example: the squared Euclidean distance breaks the triangle inequality; human similarity judgments can be non-symmetric ("North Korea is more similar to China than China is to North Korea"). Metrics are nice because many algorithms and proofs rely on those four rules; non-metrics sometimes fit real perception better.
1.8 Covariance
Covariance measures how two features move together. Positive covariance: when one is above its mean, the other tends to be above its mean too (height and weight). Negative: one rises while the other falls (speed and travel time). Near zero: no linear relationship. The formula: cov(X,Y) = (1/n)Σ(xi - x̄)(yi - ȳ). Sign tells the direction of the relationship, and after dividing by both standard deviations you get correlation, which also tells the strength (-1 to +1).
1.9 Chi-Squared Test
The chi-squared test checks whether what you observed matches what a theory predicts. Compute χ² = Σ(O - E)²/E over categories, where O is the observed count and E the expected count. Small χ² means observations fit the expectation; large χ² means something systematic is going on. Example: a die rolled 120 times should show each face about 20 times - if six appears 40 times, χ² grows large and we suspect the die is loaded. In pattern recognition it tests goodness of fit (does this feature follow a normal distribution?) and independence (is this feature actually related to the class?).
1.10 Coordinate Convention Used in Digital Images
Images do not use the math-class coordinate system. The origin (0, 0) is the TOP-left pixel; x grows to the right and y grows DOWNWARD. So the bottom-right pixel of a 640 x 480 image is (639, 479). Two consequences: formulas from textbooks sometimes flip sign on y, and many libraries index pixels as (row, column) = (y, x), not (x, y). Most bugs in image code come from mixing these two orders.
1.11 Design Pipeline of a Pattern Recognition System
A face-unlock system follows exactly this path: capture the frame, align and brighten the face, measure facial landmarks, compare them with the enrolled template, then unlock or reject.
1.12 Supervised vs Unsupervised Pattern Recognition
| Point | Supervised | Unsupervised |
|---|---|---|
| Data | Labeled examples (input + correct answer) | Unlabeled data only |
| Goal | Learn a mapping to predict labels | Discover structure: clusters, groups |
| Examples | Spam filter, digit recognition | Customer segmentation, topic grouping |
| Evaluation | Accuracy against known labels | Harder - human judgment of clusters |
Supervised learning is learning with an answer key; unsupervised is sorting a mixed pile with no key at all.
1.13 Role of Learning and Adaptation in Pattern Recognition
Learning means the classification rule is derived from examples rather than hand-coded: the system estimates the best decision boundary from training data. Adaptation means the rule keeps updating when conditions change - a spam filter must adapt as spammers change their tricks, and a speech system adapts to its owner's accent. Without learning, every rule would need a human to write it; without adaptation, yesterday's rule silently rots as the world shifts.
1.14 Importance of Covariance in a Pattern Dataset
Covariance tells you whether features carry duplicate information. If two features are strongly correlated (say, height in cm and height in inches), one of them adds nothing - dropping it makes the model simpler and often better. Covariance also shapes class geometry: a class whose features covary forms an elongated elliptical cloud, and good classifiers (like the full Bayesian classifier of Unit 2) tilt their decision boundaries to match that elongation. PCA, from the dimension-reduction topic, is literally the eigen-analysis of the covariance matrix.
1.15 Probability, Expectation, Mean and Covariance - and Why They Matter in Pattern Recognition
- Probability quantifies uncertainty: P(class | features) is the whole game of classification.
- Expectation E[X] = Σ x P(x) is the long-run average of a random quantity.
- Mean is the sample estimate of that average - the center of a class's point cloud.
- Covariance captures spread and feature relationships around the mean.
Every classifier in this course is built from these four pieces: the Bayes classifier compares probabilities, and Gaussian classifiers describe each class entirely by its mean vector and covariance matrix. Master these and Unit 2 becomes straightforward.
Unit 2: Statistical Pattern Recognition
2.1 Bayesian Decision Theory
Bayesian decision theory is the framework for making the best possible classification decision under uncertainty. "Best" means lowest probability of error (or lowest cost). The idea: for each class, compute how probable it is given what you measured, then pick the class with the highest probability. If mistakes have different costs - missing a disease is worse than a false alarm - the framework weighs those costs too. It is the theoretical ceiling of Unit 2: every other classifier is an approximation of it.
2.2 Prior Probability, Likelihood and Posterior Probability
- Prior P(ω): how common a class is before you measure anything. If 1% of emails are spam, the prior for spam is 0.01.
- Likelihood p(x|ω): how probable the measurement x is, assuming the object belongs to class ω. If spam emails contain the word "prize" 30% of the time, that is a likelihood.
- Posterior P(ω|x): the updated probability of the class AFTER seeing the measurement - the number the classifier actually needs.
Think of it as updating belief: start from the prior, let the evidence (likelihood) speak, arrive at the posterior.
2.3 Bayes' Theorem
The rule connecting the three quantities above:
P(ω|x) = p(x|ω) P(ω) / p(x)
Worked example, step by step: a disease affects 1% of people (prior 0.01). A test detects it 99% of the time when present (likelihood 0.99) and gives 5% false positives on healthy people. For a positive test: Step 1 - multiply likelihood x prior for each class: diseased gives 0.99 x 0.01 = 0.0099; healthy gives 0.05 x 0.99 = 0.0495. Step 2 - the evidence p(x) is the sum over all classes: 0.0099 + 0.0495 = 0.0594. Step 3 - normalize to get the posterior: P(disease|positive) = 0.0099 / 0.0594 ≈ 0.167 - only about 17%. The rare prior dominates: most positives are false alarms. This is why priors can never be ignored.
2.4 Classifier and Its Main Steps
A classifier is the function that takes a feature vector and outputs a class label. Building one has four main steps: (1) collect and label training data; (2) choose a model - a form for the decision rule, such as Gaussian classes; (3) train - estimate the model's parameters (means, covariances) from the data; (4) evaluate on separate test data and deploy if good enough. At run time the classifier simply computes a score per class and outputs the winner.
2.5 Discriminant Function
A discriminant function gi(x) scores how well point x fits class ωi. The classifier computes one score per class and assigns the class with the largest score: decide ωi if gi(x) > gj(x) for all j ≠ i. Any monotone transform of the score works - taking a logarithm changes nothing about the winner, which is why Gaussian classifiers are always written in log form. For the Bayes classifier the discriminant is (the log of) the posterior, which Bayes' rule expands to gi(x) = ln p(x|ωi) + ln P(ωi) - the shared denominator p(x) is dropped because it is the same for every class and cannot change the winner. Read the two terms as: the first scores how well x matches the class's typical measurements, the second adds how common the class is to start with. Plugging in a Gaussian likelihood turns this into the concrete linear and quadratic forms of topic 2.11.
2.6 Decision Boundary
The decision boundary is the surface where two classes tie: gi(x) = gj(x). On one side the classifier says class i, on the other side class j. With two Gaussians of equal covariance the boundary is a straight line; with different covariances it bends into a curve (hyperquadric). The boundary is where all the action is: points near it are classified with low confidence, and moving or tilting it is exactly what learning does.
2.7 Maximum Likelihood Estimation (MLE)
MLE answers: given data and a model form, which parameter values make the observed data most probable? Assumption: the n samples x1, x2, ..., xn are independent and identically distributed - all drawn from the same Gaussian with unknown mean μ and variance σ².
Step 1 - write the likelihood. Independence means the joint probability of the whole dataset is the product of the single-point densities: L(μ, σ²) = Π (1/√(2πσ²)) exp(-(xi - μ)²/(2σ²)), the product running over all n samples.
Step 2 - take the log. A product of exponentials is painful to differentiate, so work with the log-likelihood (a monotone transform - the maximum stays at the same parameter values): ln L = -(n/2) ln(2π) - (n/2) ln(σ²) - (1/(2σ²)) Σ(xi - μ)².
Step 3 - differentiate, set to zero, solve. For the mean: ∂(ln L)/∂μ = (1/σ²) Σ(xi - μ) = 0. The sum of (xi - μ) is zero exactly when μ equals the average, so μ-hat = (1/n) Σ xi - the sample mean. For the variance: ∂(ln L)/∂σ² = -n/(2σ²) + (1/(2σ⁴)) Σ(xi - μ)² = 0, which rearranges to σ²-hat = (1/n) Σ (xi - μ-hat)² - the average squared deviation from the mean.
Worked numbers. Samples 4, 6, 5: μ-hat = (4+6+5)/3 = 5. Then σ²-hat = ((4-5)² + (6-5)² + (5-5)²)/3 = (1+1+0)/3 = 2/3. MLE is the standard way classifiers learn their numbers from a training set: the Gaussian parameters of every class in Unit 2 are estimated exactly this way.
2.8 Normal Density Estimation
Normal (Gaussian) density estimation assumes each class's features follow a bell curve and estimates its parameters from samples. In one dimension: p(x) = (1/√(2π)σ) exp(-(x - μ)²/(2σ²)) - estimate μ as the sample mean and σ² as the sample variance. In d dimensions the same idea uses a mean vector and covariance matrix. Once the density is estimated, classifying a new point is just comparing densities: which class's bell curve is taller at x?
2.9 Gaussian (Normal) Distribution
The bell-shaped distribution that appears everywhere in nature because sums of many small random effects converge to it (the central limit theorem). It is fully described by two numbers: mean μ (the center) and variance σ² (the spread). About 68% of samples lie within one σ of the mean, 95% within two, 99.7% within three. Pattern recognition uses it heavily for two practical reasons: many real measurements are roughly bell-shaped, so a Gaussian is a good first model for a class, and its density is one short formula - once mu and sigma^2 are estimated, plugging in any x gives p(x|class) directly, which is exactly the likelihood number Bayes' rule needs.
2.10 Covariance Matrix
The covariance matrix Σ collects all pairwise covariances of a feature vector: entry (i, j) is cov(xi, xj), and the diagonal holds each feature's variance. It describes the shape of a class's point cloud: a diagonal matrix means axis-aligned spread (features uncorrelated - and fully independent only when the data is jointly Gaussian); large off-diagonal values mean the cloud tilts. In the Gaussian density it controls the ellipsoids of constant probability, and its inverse (the Mahalanobis distance) lets the classifier measure distance while respecting each feature's scale and correlation.
2.11 Linear and Non-Linear Discriminant Functions and Their Roles
A linear discriminant has the form g(x) = wᵘx + w0 - a weighted sum of features plus a bias. Its decision boundary is a straight line (or flat hyperplane); it is simple, fast and hard to overfit, and it is exactly optimal when classes are Gaussians sharing one covariance matrix. The weights w point in the direction that best separates classes; w0 shifts the boundary. A non-linear discriminant allows curved boundaries - quadratic terms xᵘAx + wᵘx + c, neural networks, kernels. It is needed when classes interleave (the XOR pattern is the classic failure of linear rules) and arises naturally for Gaussians with different covariance matrices. Role summary: start linear; go non-linear only when the data's geometry demands it, paying with more data and more overfitting risk.
Unit 3: Image Preliminaries and Processing (selected topics)
3.1 Image Acquisition
Image acquisition is the whole journey from real-world light to a grid of numbers. Three ingredients are always present: an energy source that illuminates the scene (sunlight, a camera flash, X-rays), a scene that reflects or transmits that energy, and a sensor system that collects the energy and converts it into electrical signals and then numbers. The quality of every later step - enhancement, segmentation, classification - is limited by what happens here.
3.2 Image Sensor
The image sensor is the chip that turns incoming light into electricity. It is a grid of millions of light-sensitive sites; photons landing on each site free electrons, and the accumulated charge at every site becomes one pixel's value. The two mainstream sensor technologies, CCD and CMOS, do this same job with different internal plumbing (see 3.7).
3.3 Photosite
A photosite is one light-collecting well on the sensor - the hardware version of a pixel. It is a small photodiode region that converts photons into charge during the exposure. The bigger the photosite, the more light it collects, which is why large sensors perform better in the dark. One photosite measures brightness only; color comes from the color filters placed in front of it (see 3.13).
3.4 Electromagnetic Radiation
Light is electromagnetic radiation - waves of electric and magnetic fields characterized by wavelength. The visible band is roughly 400-700 nm (violet to red), but imaging uses much more: X-rays in medicine, infrared in night vision, radio waves in MRI. The wavelength you image with decides what you can see: bone shows up in X-rays, heat in infrared. A digital image is fundamentally a record of how a scene interacts with the chosen band of this spectrum.
3.5 Optical System
The optical system - the lens assembly - collects light from the scene and projects a sharp image onto the sensor. Its jobs: gather as much light as possible, focus rays from each scene point onto one sensor point, and control depth of field and magnification. A poor optical system blurs or distorts the image before the sensor ever sees it, and no amount of later processing can fully undo that.
3.6 Function of the Lens in a Camera
- Focusing: bends light rays so each scene point lands as a sharp point on the sensor - moving the lens elements focuses on near or far objects.
- Aperture control: the adjustable opening decides how much light passes (f-number) and how much of the scene is in focus (depth of field).
- Framing: the focal length sets the field of view - short is wide-angle, long is telephoto.
3.7 CCD and CMOS
| Point | CCD | CMOS |
|---|---|---|
| Readout | Charge is shifted bucket-brigade style to one output amplifier | Each photosite has its own amplifier, read in parallel |
| Quality tradition | Historically cleaner, lower noise | Caught up; modern CMOS matches CCD |
| Power and speed | Higher power, slower | Low power, very fast readout |
| Cost and integration | Specialized, costlier | Cheap, can embed processing on-chip - used in phones |
3.8 ADC (Analog-to-Digital Converter)
The ADC converts each photosite's analog voltage into a binary number. Its bit depth sets the number of possible levels: an 8-bit ADC gives 256 levels, a 12-bit ADC gives 4096. More bits mean finer brightness steps and smoother shadows. Together with the sensor, the ADC decides the camera's intensity resolution - the fineness of the brightness scale recorded for every pixel.
3.9 Sampling and Quantization
Digitizing an image happens in two steps. Sampling is chopping space: the continuous scene is measured only at discrete grid points - the number of samples per unit distance is the spatial resolution. Too few samples and fine detail aliases into fake patterns (moire). Quantization is chopping brightness: each measured value is rounded to one of L levels - 256 levels for 8-bit images. Too few levels create false contours in smooth regions like skies. A digital image = sampling in space + quantization in intensity.
3.10 Feature Extraction and Feature Selection
Feature extraction computes useful measurements from raw data: from an image region we might extract area, edge strength, texture scores or a HOG descriptor - compressing thousands of pixels into a short vector. Feature selection then keeps only the features that actually separate the classes and discards redundant or noisy ones. Extraction builds the vocabulary; selection edits it. Good features make a simple classifier accurate; bad features defeat the fanciest one.
3.11 Grayscale Images
A grayscale image stores one intensity per pixel - no color. Typically 8 bits: 0 is black, 255 is white, and the 254 values between are grays. Grayscale is enough for many tasks (documents, X-rays, industrial inspection), and it is the standard starting point for algorithms because the math handles one channel instead of three. Color images are often converted to grayscale first by a weighted sum: Y ≈ 0.299R + 0.587G + 0.114B, matching the eye's sensitivity.
3.12 Reflection, Absorption and Transmission of Light During Image Acquisition
When illumination strikes a surface, three things share the energy. Reflection: part bounces back - this is what ordinary cameras capture; a white shirt reflects most wavelengths, a black one few. Absorption: part is soaked up and turned into heat - the colors an object absorbs are exactly the ones it does NOT show. Transmission: part passes through - glass transmits most light, which is why X-ray and backlit imaging see inside objects. Every pixel's value is the net result of this three-way split at that scene point.
3.13 Color Filter Array (CFA)
Each photosite measures only brightness, so color cameras place a tiny colored filter over each one - a mosaic called the color filter array. The standard Bayer pattern tiles 2x2 blocks of one red, one blue and two green filters (two greens because the eye is most sensitive to green). Each pixel therefore records only one color; the missing two are estimated from neighbors in a step called demosaicing. So most of a JPEG's color values are clever interpolation, not direct measurement.
3.14 Spatial Resolution and Intensity Resolution
- Spatial resolution: how fine the sampling grid is - pixels per unit distance. Higher spatial resolution resolves smaller detail; too low and thin lines or textures break into blocks and moire.
- Intensity resolution: how many brightness levels are quantized - bits per pixel. Too few levels produce banding (false contours) in smooth gradients.
A useful rule: halving spatial resolution costs detail; halving intensity levels costs smoothness. Good imaging balances both for the task.
3.15 Wrapper Methods (Feature Selection)
Wrapper methods choose features by actually training the classifier: wrap the selection process around the model. Try a subset, train, measure accuracy; keep the subset that performs best. Strategies include forward selection (start empty, add the most helpful feature one at a time), backward elimination (start full, remove the least useful), and exhaustive search (try everything - only feasible for few features). Contrast with filter methods, which score features by statistics alone (like the chi-squared test) without training. Wrappers find subsets tuned to the real classifier but cost heavy computation and can overfit small datasets.
Comments
Post a Comment