Frafs Bench Viewer - tool for viewing Fraps benchmark files

raffriff

Moderator
Staff member
Site Contributor
Frafs Bench Viewer Version 0.3.0.1 (download)
  • bugfixes
  • preferences dialog and other UI improvements; see below
  • faster file loading (with default file parsing only)
  • improved file statistics
  • small & large standard size selector (lower right corner)
Using Frafs Bench Viewer, Part 1 - Creating the benchmark files:

Go to the Fraps FPS tab:
  1. set up an output folder
  2. set up a hotkey
  3. Benchmark settings:
    • Frametimes = ON
    • others = off (don't care)
Test Without Capturing
  1. Start the game or application to be benchmarked.
  2. Choose a scene that is repeatable.
  3. Play through the scene without recording.
  4. Hit the benchmark hotkey.
  5. The FPS overlay goes away; this is normal.
  6. Hit the benchmark hotkey again to stop.
  7. A file named will be created named
    "<application> <date> <time> frametimes.csv"
Test While Capturing

Now go back to the same scene, but record (Fraps) it this time. Start benchmarking after
recording starts (FPS overlay is hidden). Stop benchmarking before recording stops (FPS
overlay is restored). Finally, stop recording.

Again, choose a scene that is repeatable; take care to start & stop benchmarking at the
same place each time.​

Using Frafs Bench Viewer, Part 2 - Viewing the benchmark files:

The files created by Fraps are "CSV" or "comma separated values," which can be easily
opened in any spreadsheet program. Fraps Bench Viewer performs some basic analysis and
visual presentation for ease of use.

To view a Fraps "* frametimes.csv" file in Fraps Bench Viewer, drag the file onto the
open program window or onto the application icon.

After a few moments of processing, you will see a chart. You have a choice of three ways
to view the data:
  • Time: frame render times
  • FPS: calculated instantaneous FPS, i.e. (1 / frame time)
  • Rank: all frame times sorted from shortest to longest
Overlaid on the chart you will see a basic summary:
  • "Avg. FPS" = calculated average FPS
  • "Avg. Time (ms)" = calculated average time between frames
  • "1% Time (ms)" = difference time for the worst 1% (best 99%) frames.
  • "0.1% Time (ms)" = difference time for the worst 0.1% (best 99.9%) frames
There are reference lines on the chart for
  • 33.3 ms (or 30 fps)
  • 16.7 ms (or 60 fps)
  • average frame time or FPS
The charts are resizable. Mouse wheel zooms, mouse drag pans. You can also resize the
vertical axis by dragging the mouse or by entering a maximum "Y" value. To do this you
move the mouse to the left margin, or press [V], and a dialog will pop up.

Once you zoom in past a certain point, a tooltip will be shown with extra information on
individual frames.

Using Frafs Bench Viewer, Part 3 - Guide to the program interface:



  • Preferences dialog, General tab
    • enable/disable auto opening of Fraps benchmark folder
    • select from five plot colors
    • label the horizontal axis as frame numbers or time-of-day (as guessed by the program)
    • change file parsing (in a limited way) to accept non-Fraps data (experimental)
      .
  • Preferences dialog, Text tab
    • show/hide file name
    • show/hide file statistics
    • show stats for entire file or (if zoomed in) only displayed events
    • show/hide user notes
      .
  • Toolbar:
    • browse for any folder & remember it
    • save current image
    • copy current image
    • edit/adjust vertical scale
    • edit/adjust horizontal scale
    • open Preferences dialog
      .
  • Right-click menu:
    • save current image
    • copy current image
    • open Preferences dialog
      .
  • Size menu (lower right corner)
    • small & large standard size selector
      .
  • Mouse actions:
    • right click = menu
    • move mouse in left margin area = edit/adjust vertical scale
    • move mouse in lower-left or lower-right margin area = edit/adjust horizontal scale
    • left click = hide edit/adjust dialogs
    • drag with left button down = panning (view earlier/later events)
    • scroll bar = panning, alternate method
    • mouse wheel = zooming (view events with more or less detail)
      Bug: there is a sticky mouse wheel issue sometimes - I can't find the cause at the moment. If it happens, you can still zoom with the plus/minus keys; it is sometimes un-stuck by opening & closing the Preferences dialog, etc.
      .
  • Hotkeys:
    • (B) = open Fraps benchmark folder
      Bug:
      currently broken
      ...
    • (+) = zoom in
    • (-) = zoom out
    • (left arrow) or (page up) = move the chart to show earlier events if zoomed in
    • (right arrow) or (page down) = move the chart to show later events if zoomed in
    • (V) = edit/adjust vertical scale
    • (H) = edit/adjust horizontal scale
    • (P) = open Preferences dialog
    • (Ctrl+Z) = undo previous zoom or pan (multi-level)
    • (F5) = Time chart
    • (F6) = FPS chart
    • (F7) = Rank chart
-----------------------------------------------------------------------
This post was updated 7 Oct 2013 - here's the original:
Ever since I saw this article: Inside the second: A new look at game benchmarking, I wanted to make an app to analyze & display Fraps benchmark files - and here it is:



Update Dec 2012 - now open source - download FRAFS Bench Viewer on SourceForge

This way of looking at performance *may* help us get to the bottom of the intermittent lagging problem some people have.
Basically, average FPS means very little if enough frames are way below average.

Anyone interested is invited to check this tool out. This is something that will probably become useful only after a lot of people submit reports, and we compare benchmark data to real world performance.
 
Last edited:

raffriff

Moderator
Staff member
Site Contributor
More on microstutter and why we need to look beyond average frames-per-second as a measure of performance...and why we need a tool like the Fraps Frametime Checker.

Why FPS fails

As you no doubt know, nearly all video game benchmarks are based on a single unit of measure, the ubiquitous FPS, or frames per second. FPS is a nice instant summary of performance, expressed in terms that are relatively easy to understand...

...the usual average FPS score has come under fire repeatedly over the years for being too broad a measure. We've been persuaded by those arguments, so for quite a while now, we have provided average and low FPS rates from our benchmarking runs and, when possible, graphs of frame rates over time...

Still, even that approach has some obvious weaknesses. We've noticed them at times when results from our FRAPS-based testing didn't seem to square with our seat-of-the-pants experience. The fundamental problem is that, in terms of both computer time and human visual perception, one second is a very long time...

FPS isn't always bad as a summary of performance. One way to overcome this weakness is to look "inside the second." Game developers have done it for years...delving into how much GPU time each API call occupies when producing a single frame.
From Inside the second: Gaming performance with today's CPUs

Check these two charts. Average FPS looks about the same (FPS = 1/frame time) but there are more "late" frames with the i5 processors compared to the i7s:

Intel core i5


Intel core i7


This is really bad for Fraps. Those occasional "late" frames cause further cascading delays, especially when capturing video, I think. If you are capturing at 60 fps (frametime = 1/60 second or 16.7 ms) any frame that takes longer to be rendered - and written to disk - than 16.7 ms has "missed the bus," and must wait for the next 1/60 second time slot.

This techreport article is a follow-up to their excellent report on multiple GPUs and microstutter, Inside the second: A new look at game benchmarking
 
Hello raffriff,

I browsed this site quite a few times before but registered on the forum today only to ask some help about this tool, looks promising and very helpful but for some reasons it doesn't work with me, i tried it on 2 different PC's with the same results.

I download it, launch the exe to extract the files, then i launch the "FrametimeChecker.exe" i got the program but without the chart tab like on your screenies, well this is what i have when i launch it and feed it with a small csv file made for testing it with Fraps 3.5.9 registered version ( tested different csv's from different games, same results beside the numbers changing ) :


Nothing appears in "Diff"or "Graph" and the rest is not correct like average FPS etc pr missing apparently.

Tested on Windows seven 64 bits
I got net framework 4 installed on both PC's ( i believe .4 cover the 3.5 needed no ? )

If you need more infos about my system or else just ask, maybe i miss something on how this tool works.

Thx a lot.
 

ETAdmin

Administrator
Staff member
Very nice, though I see a problem with longer benchmarks. I did a 240 second one and my browser crashed when trying to paste (which makes sense, it was 14k lines of text)

Here is the output minus the frame-by-frame data:

Code:
Frametime Checker Report:
C:\Fraps\Benchmarks\Gw2 2012-09-16 18-51-35-13 frametimes.csv
 
Avg. FPS............. :    59
Avg. Time (ms)....... :    16.8666
1% Time (ms)......... :    41.4500
0.1% Time (ms)....... :    64.4800
> 16.7 ms (60 fps)... :    44.1%
The Chart tab doesn't like pasting into the forum, I had to paste it into an editing program and save it:

benchmark.jpg
 

raffriff

Moderator
Staff member
Site Contributor
Yanak,
Thank you for looking at this. I will make a new version with a setup program and more error-checking. :)
 

raffriff

Moderator
Staff member
Site Contributor
admin - wow, I'm glad it worked for you. I thought Yanak had found a fatal flaw. Anyway I will update with a more robust version in an hour or two. I'll eventually add more export options; a short report like the one you pasted will be a high priority.
 

raffriff

Moderator
Staff member
Site Contributor
Bump for updates!
  • short form report
  • report overlaid on chart
  • save chart to disk
  • stops analyzing after 30k frames (that's 8.33 minutes @ 60 fps)
  • reliability enhancements & better error messages
  • bugfix (thanks for the report in your PM, Yanak)
Download, Install, Run
(see next post)
 

raffriff

Moderator
Staff member
Site Contributor
Bump for updates! "FFTCheck (Fraps Frametime Checker)," version 0.2.2
  • .NET requirements downgraded from 3.5 to .2.0 for easier installation
  • much improved setup program
  • much faster processing
  • multiple chart types
    • frame times
    • calculated FPS
    • frames sorted by render time
  • slightly better looking charts; resizable window
  • Right-click menu
    • Copy/Save >
      • Save image
      • Copy image
      • Copy text summary info
      • Copy system report
    • Explore >
      • Fraps benchmark folder
      • Fraps movie folder
      • FFTCheck folder
  • Multiple language support (volunteer translators needed)
[EDIT -there's a slight bug in text display, will fix asap]

Download, Install, Run
  • Update Dec 2012 - now open source - download FRAFS Bench Viewer 0.2.6
  • .NET 2.0 is required; Windows XP and above should have it already.
    • the setup program will NOT prompt you to install .NET if needed.
      If Bench Viewer doesn't start, go to Microsoft at the above link and get .NET 2.0.
  • When Bench Viewer runs, you see a blank window; right click it and select Explore,
    Fraps benchmark folder. Drag any "#### frametimes.csv" file onto the program window.
 

ETAdmin

Administrator
Staff member
This is a really cool utility you're working on. If I ever get into a situation where I upgrade my CPU and nothing else I would be very curious to see the impact on these graphs.

Some suggestions:
  • Save-as image could output to a single image file for all 3 tabs, or do a prefix/suffix to the filename to avoid naming conflicts
  • Add the bbcode block for 'code' to the pasted text since a forum post is probably going to be where it ends up
 

Rensje

Well-Known Member
Site Contributor
Here's the results of a brief 60-second benchmark that I ran on Torchlight 2.

Time:
ts2_time.png

FPS:
ts2_fps.png

Rank:
ts2_rank.png

Raffriff, can you maybe explain how I should be looking at these graphs and what I can learn from them? I am very interested in learning more about how Fraps works, and how different components of modern PCs interact with one another, but to be brutally honest, these images do not really mean much to me at the moment.

What I will say is that the installer ran beautifully, the software is easy to use and I have no bugs or problems to report!
 

raffriff

Moderator
Staff member
Site Contributor
If I ever get into a situation where I upgrade my CPU and nothing else I would be very curious to see the impact on these graphs.
Did you notice any lagging etc when making that test? Your average FPS was 60 with a lot of frames going longer than that. Were you recording at 30 FPS in unlocked mode by any chance?

Raffriff, can you maybe explain how I should be looking at these graphs and what I can learn from them?
We'll have to learn to read the charts as we gain experience - but I think I can make some general observations....

For example, looking at the Rank chart for your benchmark run, we see most of the frames took exactly 16.7 ms, or 1/60 second, which seems ideal. One percent of the frames took 19.457 ms, or 51 FPS, which means 99% of the frames were 51 FPS or better. Likewise, 99.9% of your frames were 38 FPS or better, which seems very good to me. Compare to the run Admin posted - the many "long" frames there give the Time chart a comb-like appearance, and his 99.9% mark was only 16 FPS. Yet both runs had an average frame rate of 60 FPS. Do the different-appearing charts reflect a difference in the real world performance?
 

ETAdmin

Administrator
Staff member
Did you notice any lagging etc when making that test? Your average FPS was 60 with a lot of frames going longer than that. Were you recording at 30 FPS in unlocked mode by any chance?
Yes, I was intentionally pushing the game's graphics to see what my system could pull. Here are 3 benchmarks ran at 3 different resolutions, 1280x720 > 1920x1080 > 1920x1200
 

Attachments

i effing love techreport's frame render time article (and continued use in benchmarks). that metric can at least show if a videocard or videocard line will have issues with a certain game. kinda like how my GTX 560 has massive random momentary stutters opposed to my HD5870 having a similarly low framerates, but overall being smooth and consistent in BF3.

also, here's another bit shamelessly stolen from the techreport article for use with the measurements taken above:

Code:
Frame
time
in ms  FPS rate
 
8.3    120
16.7    60
20      50
25      40
33.3    30
50      20
66.7    15
100     10
 
i haven't tried capping vs not capping video, but would it be easy/hard/a PITA to add in an option to cap the top of the graph at 50 or 66.7 or 100 (or custom)?

epsilon-final 2012-10-08 19-14-24-70-Time.png fltatz_uncovering_static_fin 2012-10-08 19-07-38-12-Time.png agenda 2012-10-08 18-56-34-05-Time_.png
 

raffriff

Moderator
Staff member
Site Contributor
do you think you could add in an option to cap the top of the graph
I haven't mentioned it, but you can do this now in version 0.2.5 -
  • Move mouse cursor to left or bottom edge of chart; shaded rectangles appear.
  • When mouse cursor is inside one of the rectangles, cursor changes to scissors.
  • Clicking with scissors cursor in the left rectangle makes the click point the new maximum, trimming off the higher values and expanding the chart vertically.
  • Trim the left and right extremes in the same way.
  • Ctl+Z to undo.
Also, mouse wheel zooms horizontally (Time & FPS tabs only) and mouse drag pans.
When you change the horizontal axis on the Time tab (with zoom, trim or pan), the FPS tab changes to match, and vice versa, to make it easy to flip between the two views.
When you zoom in far enough, a tool tip appears showing individual frame info.
 
TIL (and today i realized i'd be doing such a thing) that you can capture video AND frametimes at the same time!

here's a quick and unscientific test. looks like framerates fluctuate pretty rapidly without FRAPS capturing

bl2-1a.png bl2-1b.png bl2-1c.png

Yes, I was intentionally pushing the game's graphics to see what my system could pull. Here are 3 benchmarks ran at 3 different resolutions, 1280x720 > 1920x1080 > 1920x1200
yeah, i hear GW2 is downright punishing on CPUs/GPUs, afaik in highly populated areas/raids
 
Very nice program, thanks for making this. Is there any way to manually set the Y-axis limits? I know you can cut them down, but sometimes they'll start at 50ms and sometimes at 500ms. I'd like to make all my charts use 60 (or 120)ms as the limit of the Y-axis, but when it starts with less than that I don't know how to increase it.
 
Top