Perception and color in images

Rémanence visuelle et opposition de couleurs. Fixer le point noir central de l'image de gauche pendant 30s, puis déplacer son regard vers le point noir de droite. Observer que les couleurs perçues dans cette nouvelle image sont les complémentaires de celle du carré original. Comment l'expliquez-vous ?

Adaptation chromatique locale. Fixer le point noir au centre de l'image du haut (entre les deux zones uniformes) pendant 30s environ, puis déplacer le regard vers le point blanc de l'image du bas. Observer que l'image du bas apparait beaucoup plus uniforme après cette adaptation.

Contraste simultané Dans l'image suivante, les rectangles centraux ont tous le même niveau de gris. Comment expliquez-vous le fait qu'on les perçoivent différemment ?

For the following exercices, you need either Matlab or Scilab.

You need first to download the file TP.zip, to unzip it so that you have a directory named TP containing two subdirectories scilab_src (which contains a few necessary scilab commands for loading and displaying images) and im (which contains images needed in the TP).

If you use Scilab you should add scilab_src to the path by typing (replacing PATH by the correct path…) :

getd('/PATH/TP/scilab_src');

The comments are indicated by '%'in Matlab and by '//' in Scilab.

Load and display a color image. A color image \(u\) is made of three channels : red, green and blue. In Matlab and Scilab, a color image in \(\mathbb{R}^{N\times M}\) is stored as a \(N\times M\times 3\) matrix.

Scilab

imrgb = read_bmp_bg('im/parrot.bmp');  // load an image
[nrow,ncol,nchan]=size(imrgb);               
ccview(imrgb); //display the color image
R = u(:,:,1); G = u(:,:,2); B = u(:,:,3);
fview([R,G,B]); // display the three channels

Matlab

imrgb = imread('im/parrot.bmp');  % load an image
[nrow,ncol,nchan] = size(imrgb);  % image size
figure;
subplot(2,2,1); imshow(imrgb,[]), title('color image')
subplot(2,2,2), imshow(imrgb(:,:,1),[]), title('red channel');
subplot(2,2,3), imshow(imrgb(:,:,2),[]), title('green channel');
subplot(2,2,4), imshow(imrgb(:,:,3),[]), title('blue channel');

It might be useful to convert the color image to gray level. This can be done by averaging the three channels, or by computing another well chosen linear combination of the coordinates R, G and B. The rgb2gray function in the image processing toolbox computes \(0.2989 * R + 0.5870 * G + 0.1140 * B\).

imgray = sum(imrgb,3)/3;  

Before most computations, it is important to convert an image to double !!!

imrgb = double(imrgb);  

Be careful with the function imshow(). If u is an image encoded in double, imshow(u) will display all values above 1 in white and all values below 0 in black. If the image u is encoded on 8 bits though, imshow(u) will display 0 in black and 255 in white. In order to scale u and use the full colormap, use the command imshow(u,[]).

imhsv = rgb2hsv(imrgb); % rgb2hsv2 in Scilab 

close all; clear all;
u=[0:1:1000];
r=exp(-(u-440).^2/(2*40^2));
g=exp(-(u-540).^2/(2*40^2));
b=exp(-(u-570).^2/(2*40^2));
figure; plot3(r(380:700),g(380:700),b(380:700),'kd');

u=[0:1:1000];
r=exp(-(u-440).^2/(2*40^2));
g=exp(-(u-540).^2/(2*40^2));
b=exp(-(u-570).^2/(2*40^2));
figure; param3d1(r(380:700),g(380:700),list(b(380:700),-1));

Display the color distribution of an RGB image as a 3D point cloud.

Matlab

u = double(imread('simpson512.png'));
[nr,nc,nch] = size(u);
rgb = reshape(u,nr*nc,nch);
plot3(rgb(:,1),rgb(:,2),rgb(:,3),'.');

Scilab

u = double(imread('simpson512.png'));
[nr,nc,nch] = size(u);
rgb = matrix(u,nr*nc,nch);
figure; param3d1(rgb(:,1),rgb(:,2),list(rgb(:,3),-1));

The point cloud is too dense for vizualization. A solution is to perform clustering on the point cloud before displaying it. The command 'scatter' permits to display the point cloud with colors based on the weights of the points.

Matlab

K = 100;
[IDX,C] = kmeans(rgb,K,'Start','sample');
h=zeros(K,1);
for i=1:K
    t = find(IDX==i);
    h(i) = length(t);
end
scatter3(C(:,1),C(:,2),C(:,3),[],h);

Color clustering can be used for color image quantization (\(K\)-means boils down to Lloyd-Max quantization in 1D).

v = C(IDX,:);
uquantif = reshape(v,nr,nc,3);
imshow(uquantif,[]);

• Add a gaussian white noise to \(u\) before the quantization to perform color dithering.

The following exercice can be done with parrot.bmp or lighthouse.bmp. We first create a RAW image from the image imrgb. A each pixel, only one channel is known (red, green or blue), follwing the Bayer CFA. Remind that the bayer CFA has the following structure :

R		G		R		G
G		B		G		B
R		G		R		G
G		B		G		B

bayer = zeros(nrow,ncol,nchan);
raw(1:2:nrow,1:2:ncol,2)   = imrgb(1:2:nrow,1:2:ncol,2);
raw(2:2:nrow,2:2:ncol,2)   = imrgb(2:2:nrow,2:2:ncol,2);
raw(1:2:nrow,2:2:ncol,1)   = imrgb(1:2:nrow,2:2:ncol,1);
raw(2:2:nrow,1:2:ncol,3)   = imrgb(2:2:nrow,1:2:ncol,3);  

Display the resulting image.

imshow(raw,[]); %ccview(raw); in Scilab

Now, in order to retrieve the original image, we can try to interpolate each channel, for instance by bilinear interpolation.

R = raw(:,:,1); G = raw(:,:,2); B = raw(:,:,3);
G = G+conv2(G, [0 1 0; 1 0 1; 0 1 0]/4, 'same'); % Interpolation of the green channel at the missing points
B = B+conv2(B,[1 0 1; 0 0 0; 1 0 1]/4, 'same');
B = B+conv2(B,[0 1 0; 1 0 1; 0 1 0]/4, 'same'); % Interpolation of the blue channel at the missing points
R = R+conv2(R,[1 0 1; 0 0 0; 1 0 1]/4, 'same');
R = R+conv2(R,[0 1 0; 1 0 1; 0 1 0]/4, 'same'); % Interpolation of the red channel at the missing points
output(:,:,1)=R; output(:,:,2)=G; output(:,:,3)=B;

Display the result

imshow(output,[]); % ccview(output) in Scilab

Zoom on the metal grid and observe the mosaic artifacts. How can you explain them ? In order to understand these artifacts, create a white image with a black square in the first top quarter and apply the same process (RAW image creation and demosaicing by bilinear interpolation of each channel). Comment the result.