Assignment 2 COMP9519 Multimedia Systems CSE/UNSW Assignment 2 Version 1.1, 16 Sep 2010 Deadline of Submission: 4:30 PM, 22 October 2010 1. Project description You are required to write a program in either C/C++ or Java to process color histogram based image retrieval, then to carry out experiments using your software code, and finally to write a report. The assignment folder is http://www.cse.unsw.edu.au/~cs9519/assig-2_10/ All database and query images are in http://www.cse.unsw.edu.au/~cs9519/assig-2_10/cs9519_images.tar.gz . The list of all images in the database is in file http://www.cse.unsw.edu.au/~cs9519/assig-2_10/test_list.txt. The list of all query images is in file http://www.cse.unsw.edu.au/~cs9519/assig-2_10/test_in.txt. Please note that in these two text files the last two parameters in each row are the width and height of the image respectively. You may also want to change the path of the image files depending on you directory structure. All images are given in both raw YUV 420 (band interleaved) and JPEG formats. YUV is the suggested format to use. All images are 378*252 in their resolution. All images are stored in different sub-directories. The name of each sub-directory denotes the manually generated categorization of images in that sub-directory. This should be regarded as the ground truth of categorization when evaluating retrieval accuracy. The code you write should be able to perform image retrieval based on color histogram distances between images. It should support at least the following parameters: Distance metric: L1, L2 and Chi-square Color space: RGB Color space partitioning: 16 bins (R:G:B=2:4:2, i.e. R, G, and B are partitioned into 2, 4 and 2 sections, respectively) and 128 bins (R:G:B=4:8:4) Therefore, totally there are 6 combinations as shown in the table below. RGB 16 bins 128 bins L1 x x L2 x x Chi-Squared x x You then are required to perform the following tasks: Task 1 - 3 (85 %). Task 1. For a query image, you are required to use your software to calculate the color histogram distances between the query image and all images in the database. The returned color histogram distances should be ordered into a list in terms of increasing distance (i.e. from the smallest distance to the largest distance). Then you are required to use your software to calculate the precision among the top 30 returned results (i.e., the first 30 returned images with the smallest distances). Assuming among the 30 images, the number of images belonging to the same category as the query image is N_correct, then precision=N_correct/30. Task 2. Using the precision numbers calculated in Task 1, calculate the average precision over all query images in the same category. For instance, calculate the average precision over all Horse query images. Task 3. Using the precision numbers calculated in Task 1, calculate the average precision over all query images. You are also required to carry out the following experiments using your software: For every query image, run Tasks 1, 2 and 3 for all 3 distance metric and color space partitioning combinations mentioned earlier. When running the experiments, pay attention to: 1. The effect of different distance metric and color space partitioning combinations on retrieval accuracy. 2. The types of retrieval errors. 3. The factors influencing retrieval speed. See the deliverables session below for the requirements on the result files and the report. Task 4 (15 %). Experiment on different color space or texture feature and repeat task 3. Comment on the performance of your selected feature. Compare the performance of your selected feature with RGB color space. 2. Deliverables and Submission procedure 2.1. Development environment When developing your code under Linux, you are responsible for ensuring that it could be compiled under standard CSE Linux system. However, when under Windows, the suggested develop environment is Microsoft Visual Studio 2008. 2.2. Deliverables You should submit: 1. The software code you develop that performs the tasks above (including source code, make file and any project file/configuration file that you used). If you used any scripts to generate the results, you should also include the scripts. Include a readme file to describe the platform you used, compiling steps and how to use your software. Any other people should be able to compile the code and repeat your experiments by following your readme file. All source code should be adequately commented. For every distance metric (L1, L2 or Chi-square) and number of bins combination, you should submit: 2. (Task 1) For every query image, a result file that lists the sorted distances between the query image and all images in the database. The result file should be named as res_QUERYIMAGENAME.txt where QUREYIMAGENAME is to be replaced with proper image name, eg, res_arborgreens_Image01.txt. The result files should be in plain text format, with each line containing a distance value and the corresponding database image name. The line numbers should reflect the distance values, that is, the distance value of at line x should be smaller than the distance value at any line x+k where k>0. 3. (Tasks 2 and 3) A text file listing a) for each query image category, the average precision and b) average precision over all query images. You should finally draft a report in PS or PDF format. In your report, you should describe the experiments that you carried out, analyze the results and summaries your findings (Please answer the following questions separately in your report): 1) The effect of the distance metric on the accuracy of image retrieval 2) The effect of the number of histogram bins on the accuracy of image retrieval 3) Analyze the retrieval error types and suggest ways to improve the retrieval accuracy 4) Analyze and suggest ways to improve the retrieval speed (assuming we only need to return 30 images for each query image) 5) The effect of different feature types on the accuracy of image retrieval (If Task 4 is attempted) 2.3. Directory structure You should follow the following directory structure when saving your result files for submission: YourStudentID --> Code |-> L1_RGB_242 (Results) |-> L1_RGB_484 |-> L2_RGB_242 |-> L2_RGB_484 |-> Chi_RGB_242 |-> Chi_RGB_484 |-> Report The code part should include, 1. All produced source code in a package together with 2. A Makefile if using Linux, or 3. A project file within the package if using Microsoft Visual Studio 2008. Make sure that your code could be run once it is installed. 2.4. Submission Only electronic submission is accepted. Zip your submission into one file. Make sure before submission that your zipped file could be unzipped correctly in CSE linux machines. Upload your submission by using give command in CSE computer system. How to zip in linux? :/ zip r YourStudentID.zip YourStudentID/. How to unzip in Linux :/ unzip YourStudentID.zip How to submit by give command :/ give cs9519 ass2 YourStudentID.zip 2.5. Deadline The deadline to submit your results is 4:30PM, 22 Oct 2010. 2.1. Late penalties The assignment is worth 30 marks. You are strongly advised to start early. You will lose 2 marks for each day that the assignment is late. Extensions will not be granted unless you have legitimate reasons and have let the LIC know ASAP, preferably one week before its due data. 2.2. Questions For questions regarding Assignment 2, please email Weihong Wang (weihongw@cse.unsw.edu.au) 3. Warning The work you submit must be your own. Submission of work partially or completely derived from any other person or jointly written with any other person is not permitted. Assignments will be checked. The penalties for copying range from receiving no marks for the assignment to complete failure of the subject. Do not provide or show your assignment work to any other person apart from teaching staff of COMP9519. If you knowingly provide or show your assignment work to another person for any reason, and work derived from it is submitted by the other person, you may be penalized, even if the work was submitted without your knowledge or consent. This may apply even if your work is submitted by a third party unknown to you. Acknowledgement: Thanks to Dr. Jack Yu and Dr. Jing Chen for providing the original paperwork.
