isfast and pdb - An OpenGL Performance Database
Contents
Download:
When you're writing an OpenGL application, how do you know whether a
particular feature (like depth buffering or texture mapping) is fast
enough to be useful?
If you want your application to run fast on a variety of machines,
while taking advantage of as many hardware features as possible, you
need to write code that makes configuration decisions at runtime.
In OpenGL all the core features are provided, even when there is no
hardware support for them and they must be implemented completely in
software. There is no OpenGL routine that reports whether a feature is
implemented partially or completely in hardware.
Even if such a routine existed, knowing whether a feature is
implemented in hardware isn't sufficient to know whether it's fast
enough for your application. For example, some graphics accelerators
actually render very small triangles more slowly than a software
renderer on the same machine; the accelerators are faster only when the
number of pixels per triangle exceeds some threshold.
Furthermore, features interact in essentially unpredictable ways. For
example, a machine might have hardware support for texture mapping, but
only for some texture sizes. Or depth buffering might be fast only as
long as sufficient memory is available after loading textures. And so
on.
The bottom line is that a routine that reports whether a given feature
is supported in hardware is actually a lot more complicated and less
useful than you'd expect!
So how do you decide whether a given OpenGL feature is fast? The answer
is "Measure it." Since the performance of a section of
graphics code is dependent on dozens of pieces of information from the
runtime environment, no other method is as well-defined and reliable.
Performance measurement can be tricky. You need to flush the graphics
pipeline properly, and account for timer granularity and measurement
overhead.
Measuring all the features needed by your application might take a
while -- probably too long to make your users wait for the results each
time your application starts. Therefore you'll want to save performance
measurements and reuse them whenever possible.
And you might want to measure things other than graphics: Disk and
network throughput, processing time for a particular set of data,
performance on uniprocessor and multiprocessor systems.
This document describes two libraries that can help with all of the
tasks just mentioned.
libpdb - "Performance DataBase" routines for measuring
execution rates and maintaining a simple database.
libisfast - A set of routines demonstrating libpdb that answer
common questions about the performance of OpenGL features (using
reasonable but subjective criteria).
These libraries aren't a substitute for comprehensive benchmarking and
performance analysis, but they can handle simple tasks easily.
libpdb provides five routines:
-
pdbOpen() opens a performance database.
-
pdbReadRate() reads the execution rate for a given benchmark
(identified by a machine name, application name, benchmark name, and
version string) from the database.
-
pdbMeasureRate() measures the execution rate for a given
operation.
-
pdbWriteRate() writes the execution rate for a given
benchmark into the database.
-
pdbClose() closes the performance database and writes it
back to disk if necessary.
All libpdb routines return a value of type pdbStatusT, which is a
bitmask of error conditions. If the value is zero (PDB_NO_ERROR),
then the call completed successfully. If the value is nonzero, then it
is a combination of one or more of the following conditions:
-
PDB_OUT_OF_MEMORY - An attempt to allocate memory failed.
-
PDB_SYNTAX_ERROR - The database contains one or more records
that could not be parsed.
-
PDB_NOT_FOUND - The database does not contain the record
requested by the application.
-
PDB_CANT_WRITE - The database file could not be updated.
-
PDB_NOT_OPEN - pdbOpen() was not invoked before
calling one of the other libpdb routines.
-
PDB_ALREADY_OPEN - pdbOpen() was called while the
database is still open (e.g., before pdbClose() is invoked).
Every program must call pdbOpen() before using a database, and pdbClose()
when the database is no longer needed. pdbOpen() opens a
database file and reads all the performance measurements into main
memory. pdbClose() releases all memory used by the library, and
writes the database back to its file if any changes have been made by
invoking pdbWriteRate().
Synopsis
pdbStatusT pdbOpen(char *filename);
pdbStatusT pdbClose(void);
pdbOpen() returns PDB_NO_ERROR on success, PDB_OUT_OF_MEMORY
if there was insufficient main memory to store the entire database, PDB_SYNTAX_ERROR
if the contents of the database could not be parsed or seemed
implausible (e.g. a nonpositive performance measurement), or PDB_ALREADY_OPEN
if the database has been opened by a previous call to pdbOpen()
and not closed by a call to pdbClose(). The argument 'filename'
is the pathname of the database file. If you specify NULL for the
pathname, it uses the default (pdb.dat on Windows systems).
pdbClose() returns PDB_NO_ERROR on success, PDB_CANT_WRITE
if the database file is unwritable for any reason, or PDB_NOT_OPEN
if the database is not open.
Normally applications should look for the performance data they need
before going to the trouble of taking measurements. pdbReadRate()
is used for this.
Synopsis
pdbStatusT pdbReadRate ( const char* machineName,
const char* applicationName,
const char* benchmarkName,
const char* versionString,
double* rate );
Example
GetPerformanceData() {
double rate;
pdbOpen(NULL);
if (pdbReadRate(NULL, "myApp", "triangles", glGetString(GL_VERSION), &rate) == PDB_NO_ERROR)
printf("%g triangle calls per second\n", rate);
pdbClose();
}
The first argument is a zero-terminated string giving the name of the
machine for which the measurement is sought. If NULL, the default
machine name is used. (In Windows environments, the host name is an
appropriate choice, and the default machine name is the host name
returned by GetMachineName().)
The second argument is the name of the application. This is used as an
additional database key to reduce accidental collisions between
benchmark names.
The third argument is the name of the benchmark.
The fourth argument is a string identifying the desired version of the
benchmark. For OpenGL performance measurements, the string returned by glGetString(GL_VERSION)
is a good value for this argument. Other applications might use the
version number of the benchmark, rather than the version number of the
system under test.
The fifth argument is a pointer to a double-precision floating-point
variable which receives the performance measurement (the
"rate") from the database. The rate indicates the number of
benchmark operations per second that were measured on a previous run.
if pdbReadRate() returns zero, then it completed successfully
and the rate is returned in the last argument. If the requested
benchmark is not present in the database, it returns PDB_NOT_FOUND.
Finally, if pdbReadRate() is called when the database has not
been opened by pdbOpen(), it returns PDB_NOT_OPEN.
When the application is run for the first time, or when the performance
database file has been removed (perhaps to allow a fresh start after a
hardware upgrade), pdbReadRate() will not be able to find the desired
benchmark. If this happens, the application should use pdbMeasureRate()
to make a measurement.
Synopsis
pdbStatusT pdbMeasureRate ( pdbCallbackT initialize,
pdbCallbackT operation,
pdbCallbackT finalize,
int calibrate,
double* rate );
Example
void SetupOpenGLState(void) {
/* set all OpenGL state to desired values */
}
void DrawTriangles(void) {
glBegin(GL_TRIANGLE_STRIP);
/* specify some vertices... */
glEnd();
}
GetPerformanceData() {
double rate;
pdbOpen(NULL);
if (pdbReadRate(NULL, "myApp", "triangles", glGetString(GL_VERSION), &rate) != PDB_NO_ERROR) {
SetupOpenGLState();
pdbMeasureRate(glFinish, DrawTriangles, glFinish, 1, &rate);
} printf("%g triangle calls per second\n", rate);
pdbClose();
}
The first argument is a pointer to the initialization function. The
initialization function is run before each set of operations. For
OpenGL performance measurement, it's appropriate to use glFinish()
for initialization, to make sure that the graphics pipe is quiet.
However, for other performance measurements, the initialization
function could be used to create test data, preload caches, etc. It may
be NULL, in which case no initialization is performed.
The second argument is a pointer to the operation function. This
function performs the operations that are to be measured. Usually
you'll want to make sure that any global state needed by the operation
is set up before calling the operation function, so that you don't
include the cost of the setup operations in the measurement.
The third argument is a pointer to a finalization function. This is run
once, after all the calls to the operation function are complete. In
the example above, we used glFinish() again to ensure that the
graphics pipeline is idle. It may be NULL, in which case no
finalization is performed.
The finalization function must be "calibrated" so that the
overhead of calling it may be subtracted from the time used by the
operation function. If the fourth argument is nonzero, then pdbMeasureRate()
calibrates the finalization function. If the fourth argument is zero,
then pdbMeasureRate() uses the results of the previous
calibration. Recalibrating each measurement is the safest approach, but
it roughly doubles the amount of time needed for a measurement. For
OpenGL, it should be OK to calibrate once and recalibrate only when
using a different pixel format or video driver.
The final argument is a pointer to a double-precision floating-point
variable which receives the execution rate. This rate is the number of
times the operation function was called per second.
pdbMeasureRate() attempts to compute a number of repetitions
that results in a run time of about one second. (Calibration requires
an additional second.) It's reasonably careful about timekeeping on
systems with low-resolution clocks.
pdbMeasureRate() always returns PDB_NO_ERROR.
Once a rate has been measured, it should be stored in the database by
calling pdbWriteRate().
Synopsis
pdbStatusT pdbWriteRate ( const char* machineName,
const char* applicationName,
const char* benchmarkName,
const char* versionString,
double rate );
Example
GetPerformanceData() {
double rate;
pdbOpen(NULL);
if (pdbReadRate(NULL, "myApp", "triangles", glGetString(GL_VERSION), &rate) != PDB_NO_ERROR) {
SetupOpenGL();
pdbMeasureRate(glFinish, DrawTriangles, glFinish, 1, &rate);
pdbWriteRate(NULL, "myApp", "triangles", glGetString(GL_VERSION), rate);
}
printf("%g triangle calls per second\n", rate);
pdbClose();
}
The first four arguments of pdbWriteRate() match the first four
arguments of pdbReadRate().
The final argument is the performance measurement to be saved in the
database.
pdbWriteRate() will return PDB_NO_ERROR if the
performance measurement was added to the in-memory copy of the
database, PDB_OUT_OF_MEMORY if there was insufficient main
memory to do so, or PDB_NOT_OPEN if the database is not open.
When pdbWriteRate() is called, the in-memory copy of the
performance database is marked "dirty." pdbClose()
takes note of this and writes the database back to disk.
libisfast is a set of demonstration routines that show how libpdb can
be used to measure and maintain OpenGL performance data. libisfast is
based on purely subjective performance criteria. If they're appropriate
for your application, please feel free to use them. If not, please copy
the source code and modify it accordingly.
In all cases that follow, the term "triangles" refers to a
triangle strip with 37 vertices. The triangles are drawn with
perspective projection, lighting, and smooth (Gouraud) shading. Unless
otherwise stated, display-list-mode drawing is used. (This makes isfast
yield more useful results when the target machine is being accessed
over a network.)
The app must initialize isfast before making any performance
measurements, and clean up after the measurements are finished. On
Windows systems these tasks are accomplished by calling
int IsFastOpenDisplay(const char* displayName);
and
void IsFastCloseDisplay(void);
respectively. IsFastOpenDisplay() returns zero if the display
could not be opened, and nonzero if the display was opened
successfully.
DepthBufferingIsFast() returns nonzero if depth buffered
triangles can be drawn at least one-half as fast as triangles without
depth buffering:
int DepthBufferingIsFast(void);
ImmediateModeIsFast() returns nonzero if immediate-mode
triangles can be drawn at least one-half as fast as display-listed
triangles:
int ImmediateModeIsFast(void);
LineAAIsFast() returns nonzero if blended, antialiased lines can
be drawn at least one-half as fast as ordinary aliased lines:
int LineAAIsFast(void);
Keep in mind that LineAAIsFast() uses just one of several
possible algorithms for line antialiasing. You might need to use one of
the others (e.g., with depth buffering) for best-quality results or
best performance in your application.
StencillingIsFast() returns nonzero if stencilled triangles can
be drawn at least one-half as fast as non-stencilled triangles:
int StencillingIsFast(void);
TextureMappingIsFast() returns nonzero if texture-mapped
triangles can be drawn at least one-half as fast as non-texture-mapped
triangles:
int TextureMappingIsFast(void);
Although the routines in libisfast will be useful for a number of
applications, we suggest that you study them and modify them for your
own use. That way you'll explore the particular performance
characteristics of your machines: their sensitivity to triangle size,
triangle strip length, culling, stencil function, texture map type,
texture coordinate generation method, etc.
Keep in mind that while the results of the libisfast routines are
interesting, they apply only to very limited special cases.
The source directory has four subdirectories:
-
demo - Contains a trivial main program to call the routines in
libisfast.
-
examples - Contains a more sophisticated example based on the
techniques used in libisfast. (Windows version only)
-
isfast - Source code for libisfast.
-
pdb - Source code for libpdb.
Each subdirectory has its own makefile.