COMP9018 Assignment 1: OpenGL Performance COMP9018 Assignment 1: OpenGL Performance Introduction In many applications of Computer Graphics, performance is critical, while others it is not. In such situations, a scientific approach and appropriate cost-benefit analysis is important. For example, if you are building software to run on a single machine, and you want 10 per cent better performance, is it cheaper to buy a faster graphics card that costs $30 more, or is it better to spend four days of a consultant's time at $80 an hour optimising the code? On the other hand, if you are developing a flight simulation app that's going to be run on $300,000 hardware, and the military demands a frame rate of no less than 60 frames per second, and you need to achieve it in order to meet contract spec, is it worth it? To carefully consider these issues, it is important to learn how to benchmark code; in other words: write several implementations, evaluate their performance accurately, and then decide given the costs and benefits whether it is worth it. Our objective for this assignment are: To teach you about different optimisation schemes in OpenGL. To be able to analyse a problem and evaluate the costs and benefits of different approaches to a problem. To get some data and discuss in class. Hopefully we'll be able to get a good sample across hardware for comparison purposes. The assignment requires to implement a simple 3D viewer that reads data in a very simple format (OFF); and renders a simple animation of it. You will compare different OpenGL approaches for rendering the times and produce a 5 to 8 page report on the outcome. Specification Your program may be written in either Java or C. It must compile on the CSE machines, and you must include a makefile for it; such that calling "make" causes it to compile the appropriate binaries. If it is in C, it must be called offview, and in Java it must be called OffView. Your program must read files in a simplified version of OFF - the Object File Format, which contains a description of a set of polygons. The OFF files you will be given will be (a) guaranteed to have vertex normals supplied (b) fit inside the unit sphere (c) be in the NOFF format (d) colours are expressed between 0 and 1. Your program reads in the .off file, and the other options outlined below and generates an animation for a certain number of frames before shutting down and exiting (with the exception of the trackball option below). Your program must accept the following options: -r [x|y|z]: Rotate about the x, y (the default) or z axis. -f n: Number of frames to run the animation for (default=360). 0 means repeat forever. -a n: Angle to rotate by for each frame (default=1). -t: Trackball mode. In this mode, it is like imagining the object is embeded in a marble that you can roll using the mouse. The application should not stop until the user presses 'q'. -o [n|d|v]: The optimisation mode. 'n' is normal rendering: The object is displayed by sending the data in immediate mode to the renderer repeatedly. 'd' is display list mode: A display list is created and each frame is rendered by calling the display list. 'v' is vertex array mode: A vertex array is created and rendered using immediate mode calls. Default is normal. -b: use back face culling. Default is off. -w n: Width of the window in pixels. It is assumed to be square. Default is 400. For example, to show an animation of the file mushroom.off rotating around the y axis by 1 degree at a time for 1000 frames, and a window size of 300 that would be called as follows:
% offview -r y -a 1 -f 1000 -w 300 mushroom.off
You should position the eye at (0,0,5), looking at the origin with a vertical field of view of 60 degrees. There is a single light source at (5,5,5), with full intensity. There is no ambient light, there is no specular light, and the background colour is black. Report You must also submit a report (reasonable size fonts please) that discusses the following: Major implementation details and issues. Details of how much time you spent implementing the features; in particular: back face culling, display lists and vertex arrays. Description of the platform you used for testing (stuff like operating system, language, OpenGL implementation, processor, memory, graphics hardware etc). Details of the testing methodology you used. Results of your benchmarks. An evaluation/summary of the results discussing which techniques are worth the effort and whether they represent significant gains or not. The report will be worth half the marks, and your code will be worth half the marks. Your code WILL be marked for documentation and code quality as well as functionality. Marks will NOT be allocated on performance on test data; unless we find a big discrepancy between how fast you say your program is and how fast we see it as; then you'll lose marks. Your performance will not directly be compared with those of others. Measuring performance There are several ways of measuring performance, all of which are more or less acceptable. The recommended way for Linux users (in my opinion, you don't have to do it this way) is to use the /usr/bin/time command. For example, if I do:
% /usr/bin/time geomview teapot.off
7.35user 0.07system 0:21.76elapsed 34%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1619major+627minor)pagefaults 0swaps
The above means the program ran for a total of 21.76 seconds (the elapsed aka wall clock time), but it only used 7.35 seconds of processor time -- this is the user time. The system time is for stuff that the kernel did like reading files, scheduling etc and is not usually considered in FPS calculations. The usual measure of graphics performance is frames per second (fps, not first person shooter). So, for example, if in the above program I had generated 150 frames, then the frame rate is 150/7.35=20fps. Note that you divide by user time and not elapsed time. This is a little bit naughty, since there is a "start up" time required to load code and data into memory and get things like the display going that still takes part of the user time. But if the animation is sufficiently long (say more than 20 to 30 seconds), then this does not have a significant effect on performance. There are other ways to measure fps. For example, Magician comes with a class for measuring FPS. The important thing is to standardise and ensure that you measure all things the same way. Another possible approach is to measure the time within your program. This is even more accurate. Under Java, you can use System.currentTimeMillis() at the beginning and end of your display() functions (not including the repost). With C under Linux, you can use getrusage() to get how much CPU time you've been using. With C under Windows, it's a little trickier to get CPU usage information, but have a look at PSAPI.DLL, available from Microsoft. Or, just use straight time function. If you do not have a way to work out the process' CPU time and can only get real (wall clock) time measurements, then make sure your system is as unloaded as possible when doing the tests. Disable as many CPU eaters as possible, e.g. virus checkers, ICQ, MSN messenger, etc etc. Benchmarking Benchmarking is a complicated issue which we don't have time to cover deeply in advanced graphics. However, a brief discussion follows. In order to benchmark, it is important to decide what questions you want answered. For this assignment, some of the main questions of interest (there may be others) are: How do different techniques, such as back face culling, display lists and vertex arrays affect performance? How do the above parameters change with different complexity models? How does it affect an object with 500 polygons against a scene with 50,000 polygons? What effect does the size of the window have? Does it make a difference if the window is 100x100 or 1000x1000? Once you decide on the questions you want answered, you then develop a testing methodology. The basic principle of testing is ceteris paribus - "all other things being equal". At any one point in time, you only want to look at a single factor and, as much as possible, leave everything else the same. This means, for example, that you should check that the machine you are running the tests on is not heavily loaded (ideally it should have 0 load), that you are comparing performance on the same size data files etc. Ideally, you should perform all tests on identical hardware. You should not, for example, be compiling stuff in the background while running your tests. It is usual in testing to repeat experiments more than once and average the results. A technique that's sometimes used, for example, is to run a test ten times, throw out the highest and the lowest, and take the average of the remaining values. On computers, the first run of a test is usually discarded anyway, as it is assumed that there is some overhead getting the program "into cache". Usually you want to do one test at a time. Also, you want to run the program for as many frames as possible, so that the "startup time" (which is especially high in the case of Java) is amortised over several frames. Provisions You will be allowed to use code to implement the trackball-style rotation functionality using quaternions. The following code sample is from SGI and could prove useful in your implementation. For languages other than C, most languages have some kind of trackball functionality available for them. Here is some code from SGI for trackball-style rotation, and some demonstration code on how it is used. trackball.h trackball.c dinospin.c (demonstrates how to use the trackball). Other than code for the trackball, the code must be entirely your own work. Files in OFF format can be viewed using geomview. You will be provided with a variety of objects in the simplified OFF format. Geomview also comes with some convertors from other formats; I have also provided a script called "normalise.sh" which should generate the simplified OFF format data from any OFF file. normalise.sh ensures that normals are created for each vertex and that the object fits (exactly) in the unit sphere. Geomview is installed at CSE in ~cs9018/geomview/. You can find some pre-simplified OFF files here. There are also a number of tools to help you make OFF files. These include anytooff, which will flatten any OOGL file to an OFF file. Once converted into an OFF file, you can use normalise.sh to simplify it to our format. Some modellers directly export OOGL; if not, then generate VRML and then convert to OOGL using vrml2oogl. The OFF format The OFF format is relatively simple. For the purposes of this assignment, we are going to use a simplified subset. It consists of the following: A header: For the simplified case this will be NOFF (normals per vertex, colours per polygon. This is followed by a newline character. A line that contains three numbers: vertices faces edges. The edges number is ignored, so you can just leave it at 0. There is a newline character and then a blank line. A list of how ever many vertices follows. Each line consists of six values: position (x,y,z) and normal (x,y,z). There is a blank line. A list of faces. This consists of the number n of vertices in the polygon, followed by the indices of n vertices. This is the index of the vertex given in the first part of the file, starting from 0. Anything after this is a colour description. Colours may be described as follows: blank (in which case, assume (r,g,b) = (0.6,0.6,0.6)) A single floating point number, in which case, this is taken to be equally for red, green, and blue. three or four floating point numbers for red, green and blue (the fourth is alpha, but ignore the last number for this assignment). The full OFF format is described here; and a quick tour of OOGL is here. Here is a very simple example of a cube expressed in NOFF format (available as facecube.off):
NOFF
8 6 12
0.57735 0.57735 0.57735 0.57735 0.57735 0.57735
0.57735 0.57735 -0.57735 0.57735 0.57735 -0.57735
0.57735 -0.57735 0.57735 0.57735 -0.57735 0.57735
0.57735 -0.57735 -0.57735 0.57735 -0.57735 -0.57735
-0.57735 0.57735 0.57735 -0.57735 0.57735 0.57735
-0.57735 0.57735 -0.57735 -0.57735 0.57735 -0.57735
-0.57735 -0.57735 0.57735 -0.57735 -0.57735 0.57735
-0.57735 -0.57735 -0.57735 -0.57735 -0.57735 -0.57735
4 0 2 3 1 0.05 0.8 0.1 0.75
4 4 5 7 6 0.2 0.05 0.8 0.75
4 0 4 6 2 0.9 0.9 0.02 0.75
4 1 3 7 5 0 0.7 0.4 0.75
4 0 1 5 4 0.1 0.4 0.7 0.75
4 2 6 7 3 0.7 0.7 0 0.75
Hints Implement a feature at a time. Start with the loading, normal display, then rotation, then trackball, then backface culling (a one line thing), then display lists and vertex arrays. Don't use fixed size anything in your code. I may be tempted to load objects with 500,000 polygons, so don't be stupid and assume there's an upper limit of 65,000 or something. You might want to look at sites like AnandTech, ZDNet and Sharky Extreme for some demonstrations of how people benchmark things in the real world. Extensions Extensions can get you up to another 4 marks (i.e. 20 per cent) in this assignment. Some possible extensions include: implementing the full OFF format (including textures) and comparing the results; looking at different ways of setting up data in memory so that the vertex arrays are indexed differently; comparing C and Java implementations; comparing performance on accelerated and non-accelerated hardware of similar types; using OpenGL extensions for even more optimisation (e.g. compiled vertex arrays, or the GL_NV_FENCE extensions etc). Submissions Electronic submissions are due at midnight 29 August 2003. Reports are due at the 4 September 2003 lecture. For the electronic submission, include all .c,.cpp,.h and .java files, as well as a makefile. Additionally, you may submit interesting .off files that you used. You can submit using the give command as follows:
% give cs9018 ass1 your_files
Waleed Kadous Last modified: Wed Aug 20 12:17:06 EST 2003
