Last updated Mer 4 Mar 2024 12:42:01 EST
In this article we explore seam carving, a content-aware image resizing technique that enables images to be resized or reshaped without distorting the most important content within the image.
Unlike traditional scaling methods that uniformly scale down or crop images, seam carving works by identifying and removing or duplicating paths of pixels called "seams" that traverse the image from one edge to another. These seams are selected based on an energy map of the image, where lower energy values are assigned to less important areas of the image, such as empty skies or uniform backgrounds, and higher energy values to areas with more detail or importance. By iteratively removing or duplicating these least noticeable paths, seam carving allows for reducing or expanding the size of an image in a way that preserves the visual integrity of its key features, making it especially useful for adapting images to fit different screen sizes or aspect ratios without noticeable distortion.
The process begins with the computation of an "energy map" of the image. This map is a representation of the image where each pixel's value corresponds to its importance or saliency. The energy of a pixel is typically calculated based on its contrast with neighboring pixels, using gradients or edge detection algorithms like the Sobel operator. High-energy pixels are those that are part of significant features in the image, such as edges, textures, or objects, while low-energy pixels belong to less important areas, like homogeneous backgrounds.
function energy = computeEnergy(image)
% Convert image to grayscale
grayImage = rgb2gray(image);
% Apply Gaussian filter
sigma = 1.5; % You can adjust the value based on your preference
kernelSize = 7; % You can adjust the size based on your preference
gaussFilter = fspecial('gaussian', [kernelSize, kernelSize], sigma);
smoothedImage = conv2(double(grayImage), gaussFilter, 'same');
% Compute gradients using imgradientxy
[gradX, gradY] = imgradientxy(smoothedImage);
% Compute energy as the magnitude of the gradients
energy = sqrt(gradX.^2 + gradY.^2);
end
A seam is defined as a continuous path of pixels running from one edge of the image to the opposite edge (top to bottom or left to right) that minimizes the total energy. The path of a seam is constrained so that it moves in a straight or diagonal direction from one pixel to the next, ensuring the seam is connected and does not skip any part of the image. The identification of the optimal seam is typically achieved through dynamic programming, which efficiently searches for the path with the lowest cumulative energy.
function [optimalSeam, seamEnergy] = findOptimalSeam(energy)
[rows, cols] = size(energy);
% Initialize seam energy matrix and backtracking matrix
seamEnergy = zeros(rows, cols);
% Copy the first row from the energy matrix
seamEnergy(1, :) = energy(1, :);
% Compute cumulative energy and backtracking matrix
for i = 2:rows
for j = 1:cols
if j == 1
% Handle leftmost edge case
[minVal, index] = min([seamEnergy(i-1, j), seamEnergy(i-1, j+1)]);
seamEnergy(i, j) = energy(i, j) + minVal;
elseif j == cols
% Handle rightmost edge case
[minVal, index] = min([seamEnergy(i-1, j-1), seamEnergy(i-1, j)]);
seamEnergy(i, j) = energy(i, j) + minVal;
else
% General case
[minVal, index] = min([seamEnergy(i-1, j-1), seamEnergy(i-1, j), seamEnergy(i-1, j+1)]);
seamEnergy(i, j) = energy(i, j) + minVal;
end
end
end
% Find the starting index of the optimal seam
[~, startCol] = min(seamEnergy(rows, :));
% Backtrack to find the optimal seam
optimalSeam = zeros(rows, 1);
optimalSeam(rows) = startCol;
for i = rows-1:-1:1
if startCol == 1
[~, index] = min([seamEnergy(i, startCol), seamEnergy(i, startCol+1)]);
startCol = startCol + index - 1;
elseif startCol == cols
[~, index] = min([seamEnergy(i, startCol-1), seamEnergy(i, startCol)]);
startCol = startCol - 2 + index;
else
[~, index] = min([seamEnergy(i, startCol-1), seamEnergy(i, startCol), seamEnergy(i, startCol+1)]);
startCol = startCol - 2 + index;
end
optimalSeam(i) = startCol;
end
end
Once the seam with the lowest energy is identified, the process continues with either the removal or duplication of this seam. Removing a seam narrows the image by one pixel in width or height (depending on the seam's orientation), effectively reducing the size of the image while attempting to preserve its most important content. Conversely, duplicating a seam increases the size of the image by adding a line of pixels adjacent to the low-energy seam, which is less likely to disrupt the visual coherence of the image.
function updatedImage = removeSeam(image, seam)
[height, width, channels] = size(image);
updatedImage = zeros(height, width - 1, channels, 'uint8');
for i = 1:height
colToRemove = seam(i);
updatedImage(i, :, :) = [image(i, 1:colToRemove-1, :), image(i, colToRemove+1:end, :)];
end
end
Seam carving is typically an iterative process. To achieve the desired image size, seams are removed or added one at a time, with the energy map and seams recalculated after each modification. This iterative approach allows the algorithm to adapt dynamically to the changing content of the image, ensuring that the most visually significant features are preserved even as the image's dimensions are altered.
function createSeamCarvingVideo(inputImagePath, outputVideoPath)
% Load the original color image
originalImage = imread(inputImagePath);
[height, width, col] = size(originalImage);fprintf("h:%d w:%d \n\nProgress: ",height,width)
copyImage = originalImage; %% useless: could use original
maxFrames = width - 1; % Number of frames needed to reduce the width to 1
outputVideo = VideoWriter(outputVideoPath, 'MPEG-4');
open(outputVideo);
for frame = 1:maxFrames
% Compute energy of the original image
energyImage = computeEnergy(copyImage);
% Find and visualize the optimal seam on the original image
[optimalSeam, ~] = findOptimalSeam(energyImage);
for i = 1:size(optimalSeam, 1)
copyImage(i, optimalSeam(i), 1) = 255; % Set red channel to maximum
copyImage(i, optimalSeam(i), 2) = 0; % Set green channel to minimum
copyImage(i, optimalSeam(i), 3) = 0; % Set blue channel to minimum
end
%imwrite(copyImage,sprintf("out1_%dpre.png", frame ));
paddedImage = padarray( copyImage , [0,frame-1], 0,'post');
prog = (100*(frame/maxFrames));
fprintf(1,'\b\b\b\b%3.0f%%',prog);
writeVideo(outputVideo, paddedImage);
copyImage = removeSeam(copyImage, optimalSeam);
%imwrite(copyImage,sprintf("out2_%d.png", frame ));
end
% Create a VideoWriter object
close (outputVideo);
end
To execute, simply add the input image to the inputImagePath variable. Add the path for the output video in the outputVideoPath variable. Call createSeamCarvingVideo with inputImagePath and outputVideoPath and wait!
inputImagePath = 'path/to/your/image.jpeg';
outputVideoPath = 'path/to/output/video.mp4';
createSeamCarvingVideo(inputImagePath, outputVideoPath);
Try it out and let me know!