How do I run glean?

Note: These instructions assume you're running glean on a GNU/Linux system. We hope the translation to running glean on a Windows system is straightforward. If not, we're always happy to receive additional documentation for the distribution. :-)

glean runs in two modes: Gathering information about your OpenGL implementation (run mode) and comparing results of two previous runs (compare mode). After using glean in either mode, you may also find it helpful to evaluate results with other tools in the suite.

To see a list of all available Glean tests:

    $GLEAN_ROOT/bin/glean --listtests

For a description of each test, run with -v (verbose):

    $GLEAN_ROOT/bin/glean -v --listtests

Generating test results

To analyze the OpenGL implementation on your machine, first choose a directory name and then run glean, instructing it to store results in that directory:

    $GLEAN_ROOT/bin/glean -r myogl

Unless the -o or --overwrite option is specified, glean will not allow writing to an already existing results directory.

As glean runs, it produces a log containing information like the following:

blendFunc:  FAIL rgb8, db, z16, s8, accrgba16, win+pmap, id 34
	source factor = GL_SRC_ALPHA, dest factor = GL_ONE_MINUS_SRC_ALPHA
	Readback had 1.00002 bits in error; blending had 1.43898 bits in error.

texBindPerf:  PASS rgb8, db, z16, s8, accrgba16, win+pmap, id 34
	Approximate texture binding time = 10.8911 microseconds.
	Range of valid measurements = [9.34022, 16.2041]

getString:  PASS rgb8, db, z16, s8, accrgba16, win+pmap, id 34

Each test produces output including its name, a description of the format of the window in which it is run, and a brief summary of its results. Each summary always includes one of the words PASS, FAIL, or NOTE; the meaning of the first two is obvious, and NOTE simply means that something is worthy of the reader's attention. (Some tests don't explicitly pass or fail; these generate only NOTEs.)

A more verbose log, including descriptions of each test, will be generated if you supply the -v option on the glean command line.

To run just a particular set of tests, or to exclude some tests from the usual run of all tests, you can use the -t option on the command line:

    $GLEAN_ROOT/bin/glean -r mysubset -t "basic+getString"
    $GLEAN_ROOT/bin/glean -r mysubset -t "-blendFunc-texBindPerf"

Some of the tests in glean (such as the texBindPerf test shown in the example above) involve benchmarking OpenGL operations. To obtain the most accurate, repeatable results for these tests, there should be no other processes running on your machine, and you should run glean at higher than normal priority. However, glean makes multiple sample runs and attempts to stabilize the system before taking measurements, in order to get sensible results even if you don't take those precautions.

Comparing test results

To compare two previous runs, invoke glean with the -c (compare) option:

    $GLEAN_ROOT/bin/glean -c myogl anotherogl

    texBindPerf:  DIFF rgb8, db, z16, s8, accrgba16, win+pmap, id 34
	    myogl appears to have higher performance.

    getString:  DIFF rgb8, db, z16, s8, accrgba16, win+pmap, id 34
	    Extensions in anotherogl but not in myogl:
		    GL_ARB_multitexture
	    Extensions in myogl but not in anotherogl:
		    GL_EXT_shared_texture_palette
	    Extensions in both myogl and in anotherogl:
		    GL_EXT_abgr
		    GL_EXT_blend_color
		    GL_EXT_blend_logic_op
		    GL_EXT_blend_minmax
		    GL_EXT_blend_subtract
		    GL_EXT_multitexture
		    GL_EXT_paletted_texture
		    GL_EXT_point_parameters
		    GL_EXT_polygon_offset
		    GL_EXT_rescale_normal
		    GL_EXT_texture3D
		    GL_EXT_texture_object
		    GL_EXT_vertex_array
		    GL_MESA_resize_buffers
		    GL_MESA_window_pos
		    GL_SGIS_multitexture
		    GL_SGIS_texture_edge_clamp

glean will note any significant differences between the two runs, displaying the test name, the keyword DIFF, the format of the window used for the test, and an explanation of the differences. Again, you can generate a more extensive log by specifying the -v option on the command line. In addition to extra explanatory information, glean will also list tests without significant differences, marking them with the notation "SAME".

Looking at test results in detail

When you run glean with the -r option, glean uses the name of each test to create a subdirectory within the main test result directory you specified on the command line.

Within these subdirectories you'll find a text file named results containing the machine-readable form of the test results. The precise format of the data in the results file varies from test to test; you'll need to look at the source code to understand it fully.

One special case is that of TIFF images generated by a test. These also appear in the test's subdirectory, and have names of the form iNNN.tif, where NNN represents a three-digit number. The glean log will explain which test result is associated with which image number.

You can use the showtiff tool to display an image:

    $GLEAN_ROOT/bin/showtiff myogl/rgbTriStrip/i001.tif

Use the right-hand mouse button to pop up a menu of available commands.

You can also use the difftiff tool to compare two images:

    $GLEAN_ROOT/bin/difftiff myogl/rgbTriStrip/i001.tif \
        anotherogl/rgbTriStrip/i001.tif

Again, pressing the right-hand mouse button will pop up a command menu. difftiff allows you to display either image, a difference image (using selectable difference threshold and color), or a difference image overlayed on one of the two base images. This makes it easier to spot subtle differences between images generated by two different OpenGL implementations.

Drawing surface configurations and showvis

In current OpenGL implementations, one creates a drawing surface (usually a window) by selecting a configuration that describes the depth of the color buffers, whether a Z buffer is present, etc. from those configurations that the implementation supports. You can list the drawing surface configurations that are available on your system with showvis:

$GLEAN_ROOT/bin/showvis

rgb8, db, z16, s8, accrgba16, win+pmap, id 34

In this particular case, there is only one supported configuration. The color portion of the framebuffer is 24 bits deep (r, g, and b each have 8 bits) and it's double-buffered. There is a 16-bit depth (Z) buffer, an 8-bit stencil buffer, and an RGBA accumulation buffer with 16 bits per color component. The configuration can be used for creating windows and pixmaps. It has an X11 Visual ID of 34.

By default, showvis lists all the configurations your system supports. However, it's possible to ask showvis about configurations with specific characteristics. You do this by providing command-line options that are logical expressions using C syntax and involving variables that are related to attributes of drawing surface configurations. You may also request that the configurations be listed in increasing or decreasing order of any particular attribute. Some examples:

    $GLEAN_ROOT/bin/showvis db

Lists all drawing surface configurations that support double buffering.

    $GLEAN_ROOT/bin/showvis "window && rgb && r == g && g == b, min z"

Lists all drawing surface configurations for RGB windows where the three color channels have equal size, sorted with smallest Z first.

You may also use these filter expressions to restrict glean to a particular set of drawing surface configurations. For example,

    $GLEAN_ROOT/bin/glean -r rgbWindows --visuals "rgb && window"

would run only those glean tests that can be run on RGB windows. Color index windows, tests that only run on pixmaps, etc., would be omitted.

For more information on filter expressions, try executing:

    $GLEAN_ROOT/bin/showvis --help