Frafs Bench Viewer - tool for viewing Fraps benchmark files

Discussion in 'FRAPS General Discussion, Guides, and Tutorials' started by raffriff, Sep 15, 2012.

  1. raffriff

    raffriff Moderator Staff Member Site Contributor

    Frafs Bench Viewer Version 0.3.0.1 (download)
    • bugfixes
    • preferences dialog and other UI improvements; see below
    • faster file loading (with default file parsing only)
    • improved file statistics
    • small & large standard size selector (lower right corner)
    Using Frafs Bench Viewer, Part 1 - Creating the benchmark files:

    Go to the Fraps FPS tab:
    1. set up an output folder
    2. set up a hotkey
    3. Benchmark settings:
      • Frametimes = ON
      • others = off (don't care)
    Test Without Capturing
    1. Start the game or application to be benchmarked.
    2. Choose a scene that is repeatable.
    3. Play through the scene without recording.
    4. Hit the benchmark hotkey.
    5. The FPS overlay goes away; this is normal.
    6. Hit the benchmark hotkey again to stop.
    7. A file named will be created named
      "<application> <date> <time> frametimes.csv"
    Test While Capturing

    Now go back to the same scene, but record (Fraps) it this time. Start benchmarking after
    recording starts (FPS overlay is hidden). Stop benchmarking before recording stops (FPS
    overlay is restored). Finally, stop recording.

    Again, choose a scene that is repeatable; take care to start & stop benchmarking at the
    same place each time.​

    Using Frafs Bench Viewer, Part 2 - Viewing the benchmark files:

    The files created by Fraps are "CSV" or "comma separated values," which can be easily
    opened in any spreadsheet program. Fraps Bench Viewer performs some basic analysis and
    visual presentation for ease of use.

    To view a Fraps "* frametimes.csv" file in Fraps Bench Viewer, drag the file onto the
    open program window or onto the application icon.

    After a few moments of processing, you will see a chart. You have a choice of three ways
    to view the data:
    • Time: frame render times
    • FPS: calculated instantaneous FPS, i.e. (1 / frame time)
    • Rank: all frame times sorted from shortest to longest
    Overlaid on the chart you will see a basic summary:
    • "Avg. FPS" = calculated average FPS
    • "Avg. Time (ms)" = calculated average time between frames
    • "1% Time (ms)" = difference time for the worst 1% (best 99%) frames.
    • "0.1% Time (ms)" = difference time for the worst 0.1% (best 99.9%) frames
    There are reference lines on the chart for
    • 33.3 ms (or 30 fps)
    • 16.7 ms (or 60 fps)
    • average frame time or FPS
    The charts are resizable. Mouse wheel zooms, mouse drag pans. You can also resize the
    vertical axis by dragging the mouse or by entering a maximum "Y" value. To do this you
    move the mouse to the left margin, or press [V], and a dialog will pop up.

    Once you zoom in past a certain point, a tooltip will be shown with extra information on
    individual frames.

    Using Frafs Bench Viewer, Part 3 - Guide to the program interface:

    [​IMG]
    [​IMG]
    • Preferences dialog, General tab
      • enable/disable auto opening of Fraps benchmark folder
      • select from five plot colors
      • label the horizontal axis as frame numbers or time-of-day (as guessed by the program)
      • change file parsing (in a limited way) to accept non-Fraps data (experimental)
        .
    • Preferences dialog, Text tab
      • show/hide file name
      • show/hide file statistics
      • show stats for entire file or (if zoomed in) only displayed events
      • show/hide user notes
        .
    • Toolbar:
      • browse for any folder & remember it
      • save current image
      • copy current image
      • edit/adjust vertical scale
      • edit/adjust horizontal scale
      • open Preferences dialog
        .
    • Right-click menu:
      • save current image
      • copy current image
      • open Preferences dialog
        .
    • Size menu (lower right corner)
      • small & large standard size selector
        .
    • Mouse actions:
      • right click = menu
      • move mouse in left margin area = edit/adjust vertical scale
      • move mouse in lower-left or lower-right margin area = edit/adjust horizontal scale
      • left click = hide edit/adjust dialogs
      • drag with left button down = panning (view earlier/later events)
      • scroll bar = panning, alternate method
      • mouse wheel = zooming (view events with more or less detail)
        Bug: there is a sticky mouse wheel issue sometimes - I can't find the cause at the moment. If it happens, you can still zoom with the plus/minus keys; it is sometimes un-stuck by opening & closing the Preferences dialog, etc.
        .
    • Hotkeys:
      • (B) = open Fraps benchmark folder
        Bug:
        currently broken
        ...
      • (+) = zoom in
      • (-) = zoom out
      • (left arrow) or (page up) = move the chart to show earlier events if zoomed in
      • (right arrow) or (page down) = move the chart to show later events if zoomed in
      • (V) = edit/adjust vertical scale
      • (H) = edit/adjust horizontal scale
      • (P) = open Preferences dialog
      • (Ctrl+Z) = undo previous zoom or pan (multi-level)
      • (F5) = Time chart
      • (F6) = FPS chart
      • (F7) = Rank chart
    -----------------------------------------------------------------------
    This post was updated 7 Oct 2013 - here's the original:
     
    Last edited: Jan 3, 2014
    billman, Rensje and ETAdmin like this.
  2. raffriff

    raffriff Moderator Staff Member Site Contributor

    More on microstutter and why we need to look beyond average frames-per-second as a measure of performance...and why we need a tool like the Fraps Frametime Checker.

    From Inside the second: Gaming performance with today's CPUs

    Check these two charts. Average FPS looks about the same (FPS = 1/frame time) but there are more "late" frames with the i5 processors compared to the i7s:

    Intel core i5
    [​IMG]

    Intel core i7
    [​IMG]

    This is really bad for Fraps. Those occasional "late" frames cause further cascading delays, especially when capturing video, I think. If you are capturing at 60 fps (frametime = 1/60 second or 16.7 ms) any frame that takes longer to be rendered - and written to disk - than 16.7 ms has "missed the bus," and must wait for the next 1/60 second time slot.

    This techreport article is a follow-up to their excellent report on multiple GPUs and microstutter, Inside the second: A new look at game benchmarking
     
  3. Hello raffriff,

    I browsed this site quite a few times before but registered on the forum today only to ask some help about this tool, looks promising and very helpful but for some reasons it doesn't work with me, i tried it on 2 different PC's with the same results.

    I download it, launch the exe to extract the files, then i launch the "FrametimeChecker.exe" i got the program but without the chart tab like on your screenies, well this is what i have when i launch it and feed it with a small csv file made for testing it with Fraps 3.5.9 registered version ( tested different csv's from different games, same results beside the numbers changing ) :

    [​IMG]
    Nothing appears in "Diff"or "Graph" and the rest is not correct like average FPS etc pr missing apparently.

    Tested on Windows seven 64 bits
    I got net framework 4 installed on both PC's ( i believe .4 cover the 3.5 needed no ? )

    If you need more infos about my system or else just ask, maybe i miss something on how this tool works.

    Thx a lot.
     
  4. ETAdmin

    ETAdmin Administrator Staff Member

    Very nice, though I see a problem with longer benchmarks. I did a 240 second one and my browser crashed when trying to paste (which makes sense, it was 14k lines of text)

    Here is the output minus the frame-by-frame data:

    Code:
    Frametime Checker Report:
    C:\Fraps\Benchmarks\Gw2 2012-09-16 18-51-35-13 frametimes.csv
     
    Avg. FPS............. :    59
    Avg. Time (ms)....... :    16.8666
    1% Time (ms)......... :    41.4500
    0.1% Time (ms)....... :    64.4800
    > 16.7 ms (60 fps)... :    44.1%
    The Chart tab doesn't like pasting into the forum, I had to paste it into an editing program and save it:

    benchmark.jpg
     
  5. raffriff

    raffriff Moderator Staff Member Site Contributor

    Yanak,
    Thank you for looking at this. I will make a new version with a setup program and more error-checking. :)
     
  6. raffriff

    raffriff Moderator Staff Member Site Contributor

    admin - wow, I'm glad it worked for you. I thought Yanak had found a fatal flaw. Anyway I will update with a more robust version in an hour or two. I'll eventually add more export options; a short report like the one you pasted will be a high priority.
     
  7. raffriff

    raffriff Moderator Staff Member Site Contributor

    Bump for updates!
    • short form report
    • report overlaid on chart
    • save chart to disk
    • stops analyzing after 30k frames (that's 8.33 minutes @ 60 fps)
    • reliability enhancements & better error messages
    • bugfix (thanks for the report in your PM, Yanak)
    Download, Install, Run
    (see next post)
     
  8. raffriff

    raffriff Moderator Staff Member Site Contributor

    Bump for updates! "FFTCheck (Fraps Frametime Checker)," version 0.2.2
    • .NET requirements downgraded from 3.5 to .2.0 for easier installation
    • much improved setup program
    • much faster processing
    • multiple chart types
      • frame times
      • calculated FPS
      • frames sorted by render time
    • slightly better looking charts; resizable window
    • Right-click menu
      • Copy/Save >
        • Save image
        • Copy image
        • Copy text summary info
        • Copy system report
      • Explore >
        • Fraps benchmark folder
        • Fraps movie folder
        • FFTCheck folder
    • Multiple language support (volunteer translators needed)
    [EDIT -there's a slight bug in text display, will fix asap]

    Download, Install, Run
    • Update Dec 2012 - now open source - download FRAFS Bench Viewer 0.2.6
    • .NET 2.0 is required; Windows XP and above should have it already.
      • the setup program will NOT prompt you to install .NET if needed.
        If Bench Viewer doesn't start, go to Microsoft at the above link and get .NET 2.0.
    • When Bench Viewer runs, you see a blank window; right click it and select Explore,
      Fraps benchmark folder. Drag any "#### frametimes.csv" file onto the program window.
     
  9. ETAdmin

    ETAdmin Administrator Staff Member

    Time:
    Gw2 2012-09-16 18-51-35-13-Time.png


    FPS:
    Gw2 2012-09-16 18-51-35-13-FPS.png


    Rank:
    Gw2 2012-09-16 18-51-35-13-rank.png
     
  10. ETAdmin

    ETAdmin Administrator Staff Member

    This is a really cool utility you're working on. If I ever get into a situation where I upgrade my CPU and nothing else I would be very curious to see the impact on these graphs.

    Some suggestions:
    • Save-as image could output to a single image file for all 3 tabs, or do a prefix/suffix to the filename to avoid naming conflicts
    • Add the bbcode block for 'code' to the pasted text since a forum post is probably going to be where it ends up
     
  11. raffriff

    raffriff Moderator Staff Member Site Contributor

    I'm working on a rapid update and will include those suggestions. Thanks!
     
    Rensje likes this.
  12. Rensje

    Rensje Site Contributor Well-Known Member

    Here's the results of a brief 60-second benchmark that I ran on Torchlight 2.

    Time:
    ts2_time.png

    FPS:
    ts2_fps.png

    Rank:
    ts2_rank.png

    Raffriff, can you maybe explain how I should be looking at these graphs and what I can learn from them? I am very interested in learning more about how Fraps works, and how different components of modern PCs interact with one another, but to be brutally honest, these images do not really mean much to me at the moment.

    What I will say is that the installer ran beautifully, the software is easy to use and I have no bugs or problems to report!
     
  13. raffriff

    raffriff Moderator Staff Member Site Contributor

    Did you notice any lagging etc when making that test? Your average FPS was 60 with a lot of frames going longer than that. Were you recording at 30 FPS in unlocked mode by any chance?

    We'll have to learn to read the charts as we gain experience - but I think I can make some general observations....

    For example, looking at the Rank chart for your benchmark run, we see most of the frames took exactly 16.7 ms, or 1/60 second, which seems ideal. One percent of the frames took 19.457 ms, or 51 FPS, which means 99% of the frames were 51 FPS or better. Likewise, 99.9% of your frames were 38 FPS or better, which seems very good to me. Compare to the run Admin posted - the many "long" frames there give the Time chart a comb-like appearance, and his 99.9% mark was only 16 FPS. Yet both runs had an average frame rate of 60 FPS. Do the different-appearing charts reflect a difference in the real world performance?
     
  14. ETAdmin

    ETAdmin Administrator Staff Member

    Yes, I was intentionally pushing the game's graphics to see what my system could pull. Here are 3 benchmarks ran at 3 different resolutions, 1280x720 > 1920x1080 > 1920x1200
     

    Attached Files:

  15. i effing love techreport's frame render time article (and continued use in benchmarks). that metric can at least show if a videocard or videocard line will have issues with a certain game. kinda like how my GTX 560 has massive random momentary stutters opposed to my HD5870 having a similarly low framerates, but overall being smooth and consistent in BF3.

    also, here's another bit shamelessly stolen from the techreport article for use with the measurements taken above:

    Code:
    Frame
    time
    in ms  FPS rate
     
    8.3    120
    16.7    60
    20      50
    25      40
    33.3    30
    50      20
    66.7    15
    100     10
     
  16. i haven't tried capping vs not capping video, but would it be easy/hard/a PITA to add in an option to cap the top of the graph at 50 or 66.7 or 100 (or custom)?

    epsilon-final 2012-10-08 19-14-24-70-Time.png fltatz_uncovering_static_fin 2012-10-08 19-07-38-12-Time.png agenda 2012-10-08 18-56-34-05-Time_.png
     
  17. raffriff

    raffriff Moderator Staff Member Site Contributor

    I haven't mentioned it, but you can do this now in version 0.2.5 -
    • Move mouse cursor to left or bottom edge of chart; shaded rectangles appear.
    • When mouse cursor is inside one of the rectangles, cursor changes to scissors.
    • Clicking with scissors cursor in the left rectangle makes the click point the new maximum, trimming off the higher values and expanding the chart vertically.
    • Trim the left and right extremes in the same way.
    • Ctl+Z to undo.
    Also, mouse wheel zooms horizontally (Time & FPS tabs only) and mouse drag pans.
    When you change the horizontal axis on the Time tab (with zoom, trim or pan), the FPS tab changes to match, and vice versa, to make it easy to flip between the two views.
    When you zoom in far enough, a tool tip appears showing individual frame info.
     
  18. TIL (and today i realized i'd be doing such a thing) that you can capture video AND frametimes at the same time!

    here's a quick and unscientific test. looks like framerates fluctuate pretty rapidly without FRAPS capturing

    bl2-1a.png bl2-1b.png bl2-1c.png

    yeah, i hear GW2 is downright punishing on CPUs/GPUs, afaik in highly populated areas/raids
     
  19. raffriff

    raffriff Moderator Staff Member Site Contributor

    Yanak likes this.
  20. Very nice program, thanks for making this. Is there any way to manually set the Y-axis limits? I know you can cut them down, but sometimes they'll start at 50ms and sometimes at 500ms. I'd like to make all my charts use 60 (or 120)ms as the limit of the Y-axis, but when it starts with less than that I don't know how to increase it.
     

Share This Page