Image Segmentation Chin-Ya Huang, Mon-Ju Wu ECE 533 Final Project, Fall 2006 University of Wisconsin- Madison PDF created with pdfFactory Pro trial version www.pdffactory.com INTRODUCTION Image segmentation is an important technology for image processing. There are many applications whether on synthesis of the objects or computer graphic images require precise segmentation. With the consideration of the characteristics of each object composing images in MPEG4, object-based segmentation cannot be ignored. Nowadays, sports programs are among the most popular programs, and there is no doubt that viewers’ interest is concentrated on the athletes. Therefore, demand for image segmentation of sport scenes is very high in terms of both visual compression and image handling using extracted athletes. In this project, we introduce a basic idea about color information and edge extraction to achieve the image segmentation. The color information helps obtain the texture information of the target image while the edge extraction detects the boundary of the target image. By combining these, the target image can be correctly segmented and represent. Besides, because color information and edge extraction can use basic image processing methods, they can not only demonstrate what textbook claims but also make us realize their function works. We expect that we can extract most part of the target. PDF created with pdfFactory Pro trial version www.pdffactory.com LITERATURE REVIEW There are many algorithms used for image segmentation, and some of them segmented an image based on the object while some can segment automatically. Nowadays, no one can point out which the optimal solution is due to different constraints. In [1], a similarity close measure was used to classify the belonging of the pixels, and then used region growing to get the object. Unfortunately, it required a set of markers, and if there is an unknown image, it is hard to differentiate which part should be segmented. Linking the area information and the color histogram were considered for building video databases based on objects [2]. However, the color information has to be given first, and it is not useful for the life application. A genetic algorithm adapted the segmentation process to changes in image characteristics caused by variable environmental conditions [3], but it took time learning. In [4], a two-step approach to image segmentation is reported. It was a fully automated model-based image segmentation, and improved active shape models, line-lanes and live-wires, intelligent scissors, core-atoms, active appearance models. However, there were still two problems left. It is strong dependency on a close-to-target initialization, and necessary for manual redesign of segmentation criteria whenever new segmentation problem is encountered. The authors in [5] proposed a graph-based method, the cut ratio is defined following the idea of NP-hard as the ratio of the corresponding sums of two different weights of edges along the cut boundary and models the mean affinity between the segments separated by the boundary per unit boundary length. It allows efficient iterated region-based segmentation as well as pixel-based segmentation. Moreover, in order to understand an image and recognize the represented objects, it is necessary to locate in the image where the objects are [6]. The homogeneity between two pixels and the distance function are included to measure the segmented results |),(),(| wvIyxII -=D [7]. In [6], a confidence level PDF created with pdfFactory Pro trial version www.pdffactory.com .,...,1,),(min NjippL jiji =-= is used as a new performance measure to evaluate the accuracy segmentation algorithm. In [8], the minimizing function including the approximation mean square error RMSE and the number of distinct region tried to achieve a good segmentation result. In [9], edge detection is a fundamental concept for segmentation because it is easy not only to understand its principle but also to implement. Comparing the basic edge detection method, which is get from Matlab commend, with our algorithm, there are some results as follows: Figure 1: (a) Matlab “edge” command (left) (b) Proposed algorithm (right) PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 2: (a) Matlab “edge” command (left) (b) Proposed algorithm (right) Figure 1 (a) and Figure 2 (a) show the segmented result by Matlab “edge” command, and Figure 1 (b) and Figure 2 (b) show our proposed algorithm. It is pointed out that although the proposed algorithm extracts some undesired parts into the segmentation result, it represents almost the entire target image while the Matlab “edge” command seems to show the boundary of the target image only. It is because the proposed algorithm considers the color information at the same time in order to recover some disconnected parts while the Matlab “edge” command actually has problem identifying them as part of the target image. Figure 3: (a) Matlab “edge” command (left) (b) Proposed algorithm (right) PDF created with pdfFactory Pro trial version www.pdffactory.com In Figure 3, there are the segmentation results by performing the Matlab “edge” command (a), and the proposed algorithm (b). It can be shown that the Matlab “edge” command works better than the proposed algorithm because it can correctly extract the whole boundary. However, the algorithm of Matlab can only extract the image with clear vision. In other words, it has problem extracting the boundaries with blurred image and cannot pass the pixel information of those area. On the other hand, the algorithm we proposed can connect some of the disconnected boundary line segments by performing dilation, so our algorithm contains more pixel information than the algorithm of Matlab. Basically, the proposed algorithm has better performance than the basic edge extraction command due to the consideration of connect the discontinuous points automatically. The more detail about the proposed algorithm will be present in the “Discussion” section. PDF created with pdfFactory Pro trial version www.pdffactory.com APPROACH In our algorithms, there are some criteria. First of all, we need to be aware of the target image which we would like to segment out. Second, the background image has to be blurred and the color of the target image should be different to that of background image as much as possible. Moreover, we expect the appendages of the target image to cross over each other as least as possible. Our approach is to obtain color information of the target image and boundary extraction separately and simultaneously. We apply the character of HSI to acquire the information of the pixels of the target image. In the mean time, we use the Matlab “edge” and “imfill” command to extract the boundary and fill the image region whose boundaries make a closure. Afterwards, we combine them by getting the union of the two results. Finally, we perform and final modification and remove the noise. The whole algorithm is as follows: 1) Firstly, acquire the color information and the edge information separately. 2) Use the hue, saturation and intensity to get color information. 3) Use the Matlab “edge” command to extract the image boundary. 4) Combine the above results by getting the union of (2) and (3). 5) Final modification (noise removal). PDF created with pdfFactory Pro trial version www.pdffactory.com Work Performed The morphology and color based image segmentation method is proposed. By setting a threshold based on the pixel value of the hue, saturation, and intensity (H, S, I) separately, these color information of the object can represent the parts with the image close to these color information. The character of HSI is used to analyze color because they are the three components of the original color. Sine the hue, saturation, and intensity are independent of one another, we can use them to process the image separately without worrying the correlation of them. On the other hand, if the character of RGB is used instead, the color of the segmented results will change correspondingly when a few pixel values are changed. Afterwards, to compensate the lack of the boundary when segmenting the image by the character of HSI, an algorithm is used to extract the boundaries of the target image. Therefore, we use the Matlab “edge” command to extract the approximate boundaries. After getting the approximate boundaries, we want to get the information of the pixels which reside in the extracted boundaries. However, the Matlab “edge” command has problem full extracting all the boundaries and making all the boundaries a closure. Therefore, when performing the Matlab “infill” command, only the image region whose surrounding boundaries make a closure would be filled. In order to fix it, the dilation is performed to connect the disconnected boundaries. After the dilation, performing “imfill” would fill the image regions more effectively. Then we combine the images segmented out from the above two methods by getting the union of the two images so both the boundary information and color information could be obtained. However, getting the union of them won’t increase the noise significantly because when we perform the Matlab “edge” on the image whose background image is blurred, it eliminates most of the background image and only keeps the boundaries with the clear image region. PDF created with pdfFactory Pro trial version www.pdffactory.com Finally, we need to perform final modification by removing the noise. To eliminate the noise, we connect the disconnect boundaries by performing dilation on the boundaries of the target image, and then we intersect the combines image with the dilated boundaries to eliminate the trivial noise that doesn’t reside in the target image. The result of the intersection is the final image. The flowchart in Figure 5 and Figure 6 shows the detail of the “Texture Extraction” and “Shape Extraction” block. Figure 5: Flowchart of the proposed algorithm Figure 6: Function diagram of “Texture Extraction” and “Edge Extraction” PDF created with pdfFactory Pro trial version www.pdffactory.com RESULT After picking a numbers of images which fit the criteria from the website, 16 images are selected and partially segmented from the original image. The original images and processed images are posted as follows: Figure 7: The original (upper) and processed (lower) image of “Image 1”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 8: The original (left) and processed (right) image of “Image 2”. Figure 9: The original (left) and processed (right) image of “Image 3”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 10: The original (left) and processed (right) image of “Image 4”. Figure 11: The original (left) and processed (right) image of “Image 5”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 12: The original (upper) and processed (lower) image of “Image 6”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 13: The original (upper) and processed (lower) image of “Image 7”. Figure 14: The original (upper) and processed (lower) image of “Image 8”. Figure 15: The original (upper) and processed (lower) image of “Image 9”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 16: The original (upper) and processed (lower) image of “Image 10”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 17: The original (upper) and processed (lower) image of “Image 11”. Figure 18: The original (upper) and processed (lower) image of “Image 12”. Figure 19: The original (left) and processed (right) image of “Image 13”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 20: The original (left) and processed (right) image of “Image 14”. Figure 21: The original (upper) and processed (lower) image of “Image 15”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 22: The original (upper) and processed (lower) image of “Image 16”. From the images above, it is obvious that for those images whose background images are blurred can be segmented with nice quality. It is because when the Matlab “edge” command is applied to extract the boundary of the image, only the partial images with clear boundary would be extracted. Therefore, by performing the Matlab PDF created with pdfFactory Pro trial version www.pdffactory.com “edge” command, the blurred background images would be eliminated, and only the clear foreground images would be left. After getting the approximate boundaries, we want to get the information of the pixels which reside in the extracted boundaries. However, the Matlab “edge” command has problem full extracting all the boundaries and making all the boundaries a closure. Therefore, when performing the Matlab “infill” command, only the image region whose surrounding boundaries make a closure would be filled. In order to fix it, the dilation is performed to connect the disconnected boundaries. After the dilation, performing “imfill” would fill the image regions more effectively. Other than using the existent Matlab code, the characters of HSI (hue, saturation, and intensity) is also applied to segment the image. Only the information of the pixels with similar HSI character to the foreground image would be kept. In order to apply the characters of HSI to segment the image, partial images of the foreground image is cut to calculate the average value of the hue, saturation, and intensity, and then the deviation of the HSI value is set so create different ranges of the hue, saturation, and intensity. Afterwards, the HSI value of every pixels of the image is examined. Only the information of the pixel whose HSI value falls in the range will stay. Otherwise, its information will be set to zero. By applying the character of HSI to image segmentation, we can obtain all the information of the pixel whose HSI value is similar to that in foreground images. After performing two methods mentioned above, the approximate color information of pixels and the information of boundaries are acquired. To combine both of the information, we get the union of the results from the above two steps so the result from different methods would be combined. The reason why we get the union of the results instead of the intersection is we only got partial images from the two different methods mentioned above. So only by getting the union of them can we PDF created with pdfFactory Pro trial version www.pdffactory.com get an image that is more close to the target image which we want to segment out from the entire image. However, getting the union of them won’t increase the noise significantly because when we perform the Matlab “edge” on the image whose background image is blurred, it eliminates most of the background image and only keeps the boundaries with the clear image region. The image of the union of results of the two methods above contains most of information of the target image and some noise. In order to eliminate the noise, we try to connect the disconnect boundaries by dilating the boundaries a few times, and then we intersect the combines image with the dilated boundaries to eliminate the trivial noise that doesn’t reside in the target image. The result of the intersection is the final image. PDF created with pdfFactory Pro trial version www.pdffactory.com DISCUSSION Our algorithm is the best algorithm we could come up with during the limited time. However, it still has some flaws and limitations that need to be fixed and improved so the algorithm could be more solid. First of all, the target image must be known. Since the method which applies HSI to image segmentation is to cut the partial images of HSI and calculate the HSI values, so the target image must be known. Second, the target image must have clear vision instead of blurred image. Other than applying the character of HSI to the image segmentation, another method applied to segment the image is to use the Matlab “edge” command to extract the approximate boundaries. Therefore, when the background image is clear, this method will also extract the boundaries of background image instead of extracting the boundaries of the foreground image mostly. Since the Matlab “imfill” command is afterwards performed to fill the image region whose boundaries make a closure, the image with clear boundaries will be extracted, including both the foreground and the background images. Figure 23: The original (left) and processed (right) image of “Image 17”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 24: The original (upper) and processed (lower) image of “Image 18”. Figure 23 and Figure 24 are two example of the failure of extracting the target image. Due to the clearness of both the foreground and the background images, we have problem extracting only the boundaries of the foreground image. Another limitation of our algorithm is that the appendages of the target image cannot cross over each other. When the appendages of the target image cross over each other, the boundaries of the appendages would make a closure. In other words, PDF created with pdfFactory Pro trial version www.pdffactory.com when we perform the Matlab “edge” and “imfill” commands later, the background image which resides within the closure of the boundaries of the appendages would be considered as part of the foreground image so their pixel information would be passed to the segmented image. Figure 25: The original (upper) and processed (lower) image of “Image 19”. PDF created with pdfFactory Pro trial version www.pdffactory.com Figure 26: The original (left) and processed (right) image of “Image 20”. Figure 25 and Figure 26 are two example of the failure of extracting the target image. Due to the cross over of appendages, part of the background image is considered as the target image. Take Figure 25 for instrance, the turf is part of the background image and should be eliminated. However, since it is surrounded by the appendages which cross over each other, it is considered as part of the target image and segmented out mistakenly. Same thing happens to Figure 26. Since the waist of the two female athletes makes a closure, the partial water image surrounded by the waist of two female athletes is considered as part of the target image and segmented out mistakenly. PDF created with pdfFactory Pro trial version www.pdffactory.com FUTURE WORK First of all, an interpolation algorithm needed to be developed. Since the Matlab “edge” command can only extract the approximate boundaries, the interpolation algorithm is necessary to connect the disconnected boundary line segments. Take Figure 27 for instance, the edge (Figure 27 left) along of the left leg of the soccer player in the image is disconnected. After performing the Matlab “imfill” command, not all the image region of the target image is filled (Figure 27 right). Therefore, if an interpolation algorithm is developed, the disconnected boundary line segment could be connected so that the Matlab “imfill” command would fill all the target images. We tried to develop a powerful interpolation algorithm. However, we cannot create a solid interpolation within the limited time. Figure 27: The edge of “Image 2” by performing the Matlab “edge” command Moreover, other than using the character of HSI to extract the image, it will be helpful to use other characters to extract the image so the segmented pixel information of the target image will be more solid. Even though the character of HSI extracts most PDF created with pdfFactory Pro trial version www.pdffactory.com of the color information of the target image, it also extracts the pixels in the background image whose HSI value fall in the setting range. Therefore, if we can come up with other character to segment the image, we should be able to remove part of the noise but keep the completeness of the target image in the mean time. In addition, a more powerful algorithm of noise removal needs to be developed. With our current algorithm of noise removal, it does remove most of the noise from the background image. However, it has its own limitation and flaws. Since the dilation is performed in order to connect the disconnected boundary line segments, so the background image surrounding the target image will also be extracted while they actually should be eliminated. This noise relates to the disconnected boundary line segments. Therefore, if we can develop a more powerful interpolation algorithm, we would not have to perform dilation to connect the disconnected boundary line segments and the background image surrounding the target image will not be extracted. Furthermore, the Matlab “edge” command will also extract edge the background image which has clear vision and cannot be eliminated, so a more powerful algorithm of noise removal would improve the accuracy of image segmentation. PDF created with pdfFactory Pro trial version www.pdffactory.com REFERENCE [1] F. Meyer, “Color image segmentation,” in Proc. Int. Conf. Image Processing, Maastricht, The Netherlands, 1992 [2] M. Naemura, A. Fukuda, Y. Mizutani, Y. Izumi, Y. Tanaka, and K. Enami, “Morphological Segmentation of Sport Scenes using Color Information, ” IEEE Transactions on broadcasting, vol. 46, no. 3, Sep. 2000. [3] B. Bhanu, S. Lee, and J. Ming. ‘Adaptive image segmentation using a genetic algorithm,’ IEEE Transactions on systems, man, and cybernetics, vol. 25, No. 12, Dec. 1995. [4] M. Brejl, and M. Sonka, “Object localization and border detection criteria design in edge-based image segmentation: automated learning from examples” IEEE Transactions on medical image, vol. 19, No. 10, Oct. 2000. [5] S. Wang, and J. M. Siskind, “Image segmentation with ratio cut,” IEEE Transaction on pattern analysis and machine intelligence, vol. 25, No. 6, Jun. 2003. [6] Y. Ding, G. J. Vachtsevanos, A. J. Y. Jr, Y. Zhang, and Y. Wardi, “A Recursive Segmentation and Classification Scheme for Improving Segmentation Accuracy and Detection Rate in Real-time Machine Vision”, IEEE DSP [7] M. Tabb and N. Ahuja, “Multiscale Image Segmentation by Integrated Edge and Region Detection, ” IEEE Transactions on image processing, vol. 6, no. 5, May. 1997. [8] W. Vanzella and V. Torre, “A Versatile Segmentation Procedure”, IEEE Trans on systems, man and cybernetics, vol. 36, no. 2, pp. 366-378, Apr. 2006. [9] R. C. Gonzalez, R. E. Woods, “Digital Image Processing”, Prentice Hall. [10] Y. H. Hu, “ECE 533 Classnotes”, 2006 PDF created with pdfFactory Pro trial version www.pdffactory.com Appendix: Matlab Source code PDF created with pdfFactory Pro trial version www.pdffactory.com % ECE533 Project: Image Segmentation % Written by: Mon-Ju Wu, Chin-Ya Huang %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Segment the "uniform" from the image. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Read in the whole image and get its size % RGB: the image in RGB format RGB = imread('picture1.jpg'); HSI = rgb2hsv(RGB); H = HSI(:,:,1); S = HSI(:,:,2); I = HSI(:,:,3); [row, col, layer] = size(RGB); % Call the function "EdgeDetection" to get the edge of the image. edge = EdgeDetection(RGB); % fImage: the matrix used to store the information extracted by using % "Hue" and "Saturation" as threshold. Only the pixel value falling % withing the range of "Hue" and "Saturation" value will change the % correpsonding pixel value in "fImage" to 1. fImage = zeros(row, col); % Read in the HUE of certain part of the image rgb = imread('uniform.jpg'); % Call the "Hue" function to obtain the range of the hue value and pass the % high and low end values back to the program. % hLow: lower end of the range of the hue. % hHigh: higher end of the range of the hue. [hLow, hHigh] = Hue(rgb, 0.05); [sLow, sHigh] = Saturation(rgb, 0.2); [iLow, iHigh] = Intensity(rgb, 0.2); PDF created with pdfFactory Pro trial version www.pdffactory.com % Call the "Segment" function to obtain the partially segmented image. % tempImage: the temperary image passed back from "Segment" function. tempImage = Segment(RGB, hLow, hHigh, sLow, sHigh, iLow, iHigh); % Screen through every pixel in the "tempImage" matrix. If the pixel value % is NOT zero, then pass the pixel value to the corresponding location of % the "sImage" matrix. Otherwise, don't pass any value. for i=1:row for j=1:col if tempImage(i,j)>0 fImage(i,j) = tempImage(i,j); end end end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Segment the "leg" from the image. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Read in the HUE of certain part of the image rgb = imread('leg.jpg'); % Call the "Hue" function to obtain the range of the hue value and pass the % high and low end values back to the program. % hLow: lower end of the range of the hue. % hHigh: higher end of the range of the hue. [hLow, hHigh] = Hue(rgb, 0.02); [sLow, sHigh] = Saturation(rgb, 0.2); [iLow, iHigh] = Intensity(rgb, 0.1); % Call the "Segment" function to obtain the partially segmented image. % tempImage: the temperary image passed back from "Segment" function. PDF created with pdfFactory Pro trial version www.pdffactory.com tempImage = Segment(RGB, hLow, hHigh, sLow, sHigh, iLow, iHigh); % Screen through every pixel in the "tempImage" matrix. If the pixel value % is NOT zero, then pass the pixel value to the corresponding location of % the "sImage" matrix. Otherwise, don't pass any value. for i=1:row for j=1:col if tempImage(i,j)>0 fImage(i,j) = tempImage(i,j); % Pass the "leg" information to "fImage" end end end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Segment the "face" from the image. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Read in the HUE of certain part of the image rgb = imread('face.jpg'); % Call the "Hue" function to obtain the range of the hue value and pass the % high and low end values back to the program. % hLow: lower end of the range of the hue. % hHigh: higher end of the range of the hue. [hLow, hHigh] = Hue(rgb, 0.02); [sLow, sHigh] = Saturation(rgb, 0.2); [iLow, iHigh] = Intensity(rgb, 0.1); % Call the "Segment" function to obtain the partially segmented image. % tempImage: the temperary image passed back from "Segment" function. tempImage = Segment(RGB, hLow, hHigh, sLow, sHigh, iLow, iHigh); % Screen through every pixel in the "tempImage" matrix. If the pixel value PDF created with pdfFactory Pro trial version www.pdffactory.com % is NOT zero, then pass the pixel value to the corresponding location of % the "sImage" matrix. Otherwise, don't pass any value. for i=1:row for j=1:col if tempImage(i,j)>0 fImage(i,j) = tempImage(i,j); % Pass the "face" information to "fImage" end end end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Segment the "socks" from the image. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Read in the HUE of certain part of the image rgb = imread('socks.jpg'); % Call the "Hue" function to obtain the range of the hue value and pass the % high and low end values back to the program. % hLow: lower end of the range of the hue. % hHigh: higher end of the range of the hue. [hLow, hHigh] = Hue(rgb, 0.05); [sLow, sHigh] = Saturation(rgb, 0.2); [iLow, iHigh] = Intensity(rgb, 0.1); % Call the "Segment" function to obtain the partially segmented image. % tempImage: the temperary image passed back from "Segment" function. tempImage = Segment(RGB, hLow, hHigh, sLow, sHigh, iLow, iHigh); % Screen through every pixel in the "tempImage" matrix. If the pixel value % is NOT zero, then pass the pixel value to the corresponding location of % the "sImage" matrix. Otherwise, don't pass any value. for i=1:row PDF created with pdfFactory Pro trial version www.pdffactory.com for j=1:col if tempImage(i,j)>0 fImage(i,j) = tempImage(i,j); % Pass the "uniform" information to "fImage" end end end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Combine the segmented image information. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Call the "Combine" function of combine the "fImage" matrix from % "Segmentation.m" and "edge" matrix from "EdgeDetection.m". As long as the % pixel value in any of these two matrix 1, then assign the pixel value in % "combine" matrix with the value 1. Otherwise, its pixel value stays 0. combine = Combine(RGB, fImage, edge); % sImage: the matrix used to store the segmented image after combining the % result from "EdgeDetection.m" and "Segmentation.m" sImage = zeros(row, col, layer); % Screen through every pixel in the "combine" matrix. If its pixel value is % 1, then the pixel information in the original image will be passed to the % new matrix, "sImage". Otherwise, the pixel vakue in that corresponding % location of sImage stays 0. for i=1:row for j=1:col for k=1:layer if combine(i,j)==1 sImage(i,j,1) = H(i,j); sImage(i,j,2) = S(i,j); sImage(i,j,3) = I(i,j); end PDF created with pdfFactory Pro trial version www.pdffactory.com end end end %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Convert the image back to RGB format and do the final modification. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Convert the image of "sImage" matrix from HSI format back to RGB format. % sRGB: the image converted from HSI to RGB format sRGB = hsv2rgb(sImage); % Create a 7x7 structure element to dilate the "edge" matrix (from % "EdgeDetection.m") to fill in some of the bounaries that were not % connected. Perform dilation 5 times, and then apply "imfill". % d: the filled-in boundary after dilating the edges. block=ones(5,5); d=imdilate(edge,block); for i=1:2 d = imdilate(d,block); end d = imfill(d,'holes'); % After intersection, the target image will be captured. However, part of % the unwanted image around the boundary are also captured. Therefore, we % intersect image with the segmented image, "sRGB", to filter out the % unwanted part. % finalImage: the intersection of "d" and "sRGB". It is the final image. finalImage = zeros(row,col,layer); for i=1:row for j=1:col if d(i,j)>0 finalImage(i,j,1) = H(i,j); finalImage(i,j,2) = S(i,j); PDF created with pdfFactory Pro trial version www.pdffactory.com finalImage(i,j,3) = I(i,j); end end end % Convert the image of "finalImage" matrix from HSI format back to RGB format. % finalImageRGB: the image converted from HSI to RGB format sRGB = hsv2rgb(sImage); finalImageRGB = hsv2rgb(finalImage); % Show the image on the screen. figure, imshow(RGB), hold on imshow(finalImageRGB) PDF created with pdfFactory Pro trial version www.pdffactory.com function [hLow, hHigh]=Hue(hue, deviation) % ECE533 Project: Image Segmentation % Written by: Mon-Ju Wu, Chin-Ya Huang % This is a function to obtain the range of the hue value and pass the high % and low end values back to program "ECE533". % Convert the hue image from RGB into HSI format % hueHSI: a matrix with the information of Hue, Saturation, and Intensity. % hueH: a MxN matrix which only carries the information of Hue. hueHSI = rgb2hsv(hue); hueH = hueHSI(:,:,1); % Get the size of hueH % hueRow: the number of rows of hueH. % hueCol: the number of columns of hueH. [hueRow, hueCol] = size(hueH); % Calculate the averaged hue value of the selected area hTotal = 0; for i=1:hueRow for j=1:hueCol hTotal = hTotal + hueH(i,j); end end hAverage = hTotal/(hueRow*hueCol); % Create the two boundary of the certain hue value. % hLow: the low end of the hue range % hHigh: the high end of the hue range. hLow = hAverage - deviation; hHigh = hAverage + deviation; PDF created with pdfFactory Pro trial version www.pdffactory.com function [sLow, sHigh]=Saturation(saturation, deviation) % ECE533 Project: Image Segmentation % Written by: Mon-Ju Wu, Chin-Ya Huang % This is a function to obtain the range of the saturation value and pass % the high and low end values back to program "ECE533". % Convert the saturation image from RGB into HSI format % hueHSI: a matrix with the information of Hue, Saturation, and Intensity. % saturationS: a MxN matrix which only carries the information of Saturation. saturationHSI = rgb2hsv(saturation); saturationS = saturationHSI(:,:,2); % Get the size of saturationS % saturationRow: the number of rows of saturationS. % saturationCol: the number of columns of saturationS. [saturationRow, saturationCol] = size(saturationS); % Calculate the averaged saturation value of the selected area sTotal = 0; for i=1:saturationRow for j=1:saturationCol sTotal = sTotal + saturationS(i,j); end end sAverage = sTotal/(saturationRow*saturationCol); % Create the two boundary of the certain saturation value. % sLow: the low end of the saturation range % sHigh: the high end of the saturation range. sLow = sAverage - deviation; sHigh = sAverage + deviation; if sHigh>1 sHigh = 1; end if sLow<0 sLow = 0; end PDF created with pdfFactory Pro trial version www.pdffactory.com function [iLow, iHigh]=Intensity(intensity, deviation) % ECE533 Project: Image Segmentation % Written by: Mon-Ju Wu, Chin-Ya Huang % This is a function to obtain the range of the intensity value and pass % the high and low end values back to program "ECE533". % Convert the intensity image from RGB into HSI format % intensityHSI: a matrix of the information of Hue, Saturation, and Intensity. % intensityI: a MxN matrix which only carries the information of Intensity. intensityHSI = rgb2hsv(intensity); intensityI = intensityHSI(:,:,3); % Get the size of intensityI % intensityRow: the number of rows of intensityI. % intensityCol: the number of columns of intensityI. [intensityRow, intensityCol] = size(intensityI); % Calculate the averaged intensity value of the selected area iTotal = 0; for i=1:intensityRow for j=1:intensityCol iTotal = iTotal + intensityI(i,j); end end iAverage = iTotal/(intensityRow*intensityCol); % Create the two boundary of the certain intensity value. % iLow: the low end of the intensity range % iHigh: the high end of the intensity range. iLow = iAverage - deviation; iHigh = iAverage + deviation; if iHigh>1 iHigh = 1; end if iLow<0 iLow = 0; end PDF created with pdfFactory Pro trial version www.pdffactory.com function HSfill=Segment(RGB, hLow, hHigh, sLow, sHigh, iLow, iHigh) % ECE533 Project: Image Segmentation % Written by: Mon-Ju Wu, Chin-Ya Huang % This is a function to filter the image and find out the pixels whose hue % and saturation are within the threshhold value. It will screen through % segment image according to the range of the hue. and % Convert the RGB image into HSI % HSI: a matrix with the information of Hue, Saturation, and Intensity. % H: a MxN matrix which only carries the information of Hue. % S: a MxN matrix which only carries the information of Saturation. % I: a MxN matrix which only carries the information of Intensity. HSI = rgb2hsv(RGB); H = HSI(:,:,1); S = HSI(:,:,2); I = HSI(:,:,3); % Get the size of H % Hrow: the number of rows of H. % Hcol: the number of columns of H. [Hrow, Hcol] = size(H); % Perform "closing" on Hue matrix % Screen through the Hue matrix (H), and the Saturation matrix (S). If its % value falls within the range of the Hue and Saturation, then assign its % value to 1. Otherwise, let its value stay zero. Afterwards, we perform % "closing" to fill in the gap, and use "imfill" to fill in the pixels. % HSclosing: the Hue matrix after closing. % HSfill: the result after applying "imfill" function to "Hclosing" matrix. % B: a 3*3 structure element HSclosing = zeros(Hrow, Hcol); B = ones(3,3); for i=1:Hrow for j=1:Hcol if (H(i,j)>=hLow) && (H(i,j)<=hHigh) && (S(i,j)>=sLow) && (S(i,j)<=sHigh) && (I(i,j)>=iLow) && (I(i,j)<=iHigh) HSclosing(i,j) = 1; PDF created with pdfFactory Pro trial version www.pdffactory.com end end end HSclosing = imclose(HSclosing, B); HSfill = imfill(HSclosing); % So now we obtain the value of the Hue, Saturation, and Intensity of % certain area within the certain range of the hue. Now we pass these % values of "fImage" back to the main program, "ECE533". PDF created with pdfFactory Pro trial version www.pdffactory.com function result=EdgeDetection(RGB) % using matlab toolbox, edge, imfill, imclose(morphorogy), rgb2gray, % rgb2BW, % R,G,B processing seperate % Read in the image A=RGB; % Convert the image into gray scale, R, G, and B. Perform the "edge" % function to get the edge in different format. % E_W: whole edge information without segregating the image. % E_R: edge information of the Red. % E_G: edge information of the Green. % E_B: edge information of the Blue. B=rgb2gray(A); E_W=edge(B,'prewitt',0.1); E_R=edge((A(:,:,1))); E_G=edge((A(:,:,2))); E_B=edge((A(:,:,3))); % E: the combination of the edge information of R, G, and B. E=E_R+E_G+E_B; LPF=ones(5,5); [row col]=size(E); result=zeros(row, col); for i=3:row-2 for j=3:col-2 result(i,j)=sum(sum(LPF.*E(i-2:i+2,j-2:j+2))); if (result(i,j)>0) result(i,j)=1; end end end PDF created with pdfFactory Pro trial version www.pdffactory.com function cImage=Combine(RGB, fImage, edge) % Combine the information of the "fImage" matrix and the "edge" matrix. HSI = rgb2hsv(RGB); H = HSI(:,:,1); S = HSI(:,:,2); I = HSI(:,:,3); edge = edge; [Hrow, Hcol] = size(H); % Now caompare the "fImage" matrix with the "edge" matrix from % "EdgeDetection.m", and then perform and "imfill" again. Aftwerwards, % return the combined and filled image. % Create a matrix "cImage" to combine the pixel information of the "fImage" % matrix and the "edge" matrix. If the pixel vale in any of these two % matrices is greater than 0, then we assgin the pixel valeu in the % "cImage" matrix to 1. Otherwise, it stays 0. cImage = zeros(Hrow, Hcol); % Create a matrix "fImage" to restore the filtered image. % fImage: the filtered image %fImage = zeros(Hrow, Hcol); for i=1:Hrow for j=1:Hcol if (fImage(i,j)>0) || (edge(i,j)>0) cImage(i,j) = 1; end end end cImage = imfill(cImage); PDF created with pdfFactory Pro trial version www.pdffactory.com TASK ASSIGNMENT Project: Image Segmentation Team: Chin-Ya Huang, Mon-Ju Wu Task Chin-Ya Huang Mon-Ju Wu Project Proposal 50% 50% Algorithm Development 50% 50% Matlab code programming 50% 50% Data (Image) Processing 50% 50% Presentation preparation 50% 50% Project Report 50% 50% TOTAL 50% 50% ___________________________ ___________________________ Chin-Ya Huang Mon-Ju Wu ___________________________ Date PDF created with pdfFactory Pro trial version www.pdffactory.com