A Full-Screen Exclusive Mode (FSEM) Worm

Full-screen exclusive mode (FSEM) suspends most of Java's windowing environment, bypassing the Swing and AWT graphics layers to offer almost direct access to the screen. It allows graphics card features, such as page flipping and stereo buffering, to be exploited and permits the screen's resolution and bit depth to be adjusted.

The graphics hardware acceleration used by FSEM has a disadvantage: it utilizes video memory (VRAM), which may be grabbed back by the OS when, for example, it needs to draw another window, display a screensaver, or change the screen's resolution. The application's image buffer, which is stored in the VRAM, will have to be reconstructed from scratch. A related issue is that VRAM is a finite resource, and placing too many images there may cause the OS to start swapping them in and out of memory, causing a slowdown in the rendering.

Aside from FSEM, J2SE 1.4 includes a VolatileImage class to allow images to take advantage of VRAM. Only opaque images and those with transparent areas are accelerated; translucent images can be accelerated as well but only in J2SE 5.0. Many forms of image manipulation can cause the acceleration to be lost.

In practice, direct use of VolatileImage is often not required since most graphical applications, such as those written with Swing, attempt to employ hardware acceleration implicitly. For instance, Swing uses VolatileImage for its double buffering and visuals loaded with getImage( ) are accelerated if possible, as are images used by the Java 2D API (e.g., those built using createImage( )). However, more complex rendering features, such as diagonal lines, curved shapes, and anti-aliasing utilize software rendering at the JVM level.

Another issue with hardware acceleration is that it is principally a Windows feature since DirectDraw is employed by the JVM to access the VRAM. Neither Solaris nor Linux provide a way to directly contact the VRAM.

A Sun tutorial for FSEM is at http://java.sun.com/docs/books/tutorial/extra/fullscreen/, and the rationale behind the VolatileImage class is described at http://java.sun.com/j2se/1.4/pdf/VolatileImage.pdf.

Figure 4-9 shows a screenshot of the FSEM version of WormChase, which is identical to the UFS interface in Figure 4-5.

Figure 4-9. The FSEM WormChase

Class diagrams showing the public methods for this version of WormChase are shown in Figure 4-10.

The WormChase and WormPanel classes have been combined into a single WormChase class; it now contains the animation loop, which explains its use of the Runnable interface. This approach could be employed in the UFS version of WormChase. The Worm and Obstacles classes are unchanged.

Figure 4-10. Class diagrams for the FSEM version of WormChase

The code for the FSEM WormChase can be found in the Worm/WormFSEM/ directory.

The FSEM WormChase Class

The constructor for WormChase is similar to the constructors for the WormPanel classes of previous sections:

     public WormChase(long period)     {       super("Worm Chase");       this.period = period;       initFullScreen( );   // switch to FSEM       readyForTermination( );       // create game components       obs = new Obstacles(this);       fred = new Worm(pWidth, pHeight, obs);       addMouseListener( new MouseAdapter( ) {         public void mousePressed(MouseEvent e)         { testPress(e.getX( ), e.getY( )); }       });           addMouseMotionListener( new MouseMotionAdapter( ) {         public void mouseMoved(MouseEvent e)         { testMove(e.getX( ), e.getY( )); }       });       // set up message font       font = new Font("SansSerif", Font.BOLD, 24);       metrics = this.getFontMetrics(font);       // specify screen areas for the buttons       pauseArea = new Rectangle(pWidth-100, pHeight-45, 70, 15);       quitArea = new Rectangle(pWidth-100, pHeight-20, 70, 15);       // initialise timing elements       fpsStore = new double[NUM_FPS];       upsStore = new double[NUM_FPS];       for (int i=0; i < NUM_FPS; i++) {         fpsStore[i] = 0.0;         upsStore[i] = 0.0;       }       gameStart( );  // replaces addNotify( )     }  // end of WormChase( )

WormChase( ) ends with a call to gameStart( ), which contains the code formerly in the addNotify( ) method. As you may recall, addNotify( ) is called automatically as its component (e.g., a JPanel) and is added to its container (e.g., a JFrame). Since I'm no longer using a JPanel, the game is started directly from WormChase's constructor.

Setting Up Full-Screen Exclusive Mode

The steps necessary to switch the JFrame to FSEM are contained in initFullScreen( ):

     // globals used for FSEM tasks     private GraphicsDevice gd;     private Graphics gScr;     private BufferStrategy bufferStrategy;     private void initFullScreen( )     {       GraphicsEnvironment ge =          GraphicsEnvironment.getLocalGraphicsEnvironment( );       gd = ge.getDefaultScreenDevice( );       setUndecorated(true);    // no menu bar, borders, etc.       setIgnoreRepaint(true);               // turn off paint events since doing active rendering       setResizable(false);       if (!gd.isFullScreenSupported( )) {         System.out.println("Full-screen exclusive mode not supported");         System.exit(0);       }       gd.setFullScreenWindow(this); // switch on FSEM           // I can now adjust the display modes, if I wish       showCurrentMode( );   // show the current display mode       // setDisplayMode(800, 600, 8);   // or try 8 bits       // setDisplayMode(1280, 1024, 32);       pWidth = getBounds( ).width;       pHeight = getBounds( ).height;       setBufferStrategy( );     }  // end of initFullScreen( )

The graphics card is accessible via a GraphicsDevice object, gd. It's tested with GraphicsDevice.isFullScreenSupported( ) to see if FSEM is available. Ideally, if the method returns false, the code should switch to using AFS or UFS, but I give up and keep things as simple as possible.

Once FSEM has been turned on by calling GraphicsDevice.setFullScreenWindow( ), modifying display parameters, such as screen resolution and bit depth, is possible. Details on how this can be done are explained below. In the current version of the program, WormChase only reports the current settings by calling my showCurrentMode( ); the call to my setDisplayMode( ) is commented out.

initFullScreen( ) switches off window decoration and resizing, which otherwise tend to interact badly with FSEM. Paint events are not required since I'm continuing to use active rendering, albeit a FSEM version (which I explain in the section "Rendering the Game").

After setting the display characteristics, the width and height of the drawing area are stored in pWidth and pHeight. Once in FSEM, a buffer strategy for updating the screen is specified by calling setBufferStrategy( ):

     private void setBufferStrategy( )     { try {         EventQueue.invokeAndWait( new Runnable( ) {           public void run( )           { createBufferStrategy(NUM_BUFFERS);  }         });       }       catch (Exception e) {         System.out.println("Error while creating buffer strategy");         System.exit(0);       }       try {  // sleep to give time for buffer strategy to be done         Thread.sleep(500);  // 0.5 sec       }       catch(InterruptedException ex){}       bufferStrategy = getBufferStrategy( );  // store for later     }

Window.createBufferStrategy( ) is called with a value of 2 (the NUM_BUFFERS value), so page flipping with a primary surface and one back buffer is utilized.

Page flipping is explained in detail in the next section.

EventQueue.invokeAndWait( ) is employed to avoid a possible deadlock between the createBufferStrategy( ) call and the event dispatcher thread, an issue that's been fixed in J2SE 5.0. The thread holding the createBufferStrategy( ) call is added to the dispatcher queue, and executed when earlier pending events have been processed. When createBufferStrategy( ) returns, so will invokeAndWait( ).

However, createBufferStrategy( ) is an asynchronous operation, so the sleep( ) call delays execution for a short time so the getBufferStrategy( ) call will get the correct details.

The asynchronous nature of many of the FSEM methods is a weakness of the API making it difficult to know when operations have been completed. Adding arbitrary sleep( ) calls is inelegant and may slow down execution unnecessarily.

Other asynchronous methods in GraphicsDevice include setDis-playMode( ) and setFullScreenWindow( ).

Double Buffering, Page Flipping, and More

All of my earlier versions of WormChase have drawn to an off-screen buffer (sometimes called a back buffer), which is copied to the screen by a call to drawImage( ). The idea is illustrated in Figure 4-11.

Figure 4-11. Double buffering rendering

The problem with this approach is that the amount of copying required to display one frame is substantial. For example, a display of 1,024x768 pixels, with 32-bit depth, will need a 3-MB copy (1024x768x4 bytes), occurring as often as 80 times per second. This is the principal reason for modifying the display mode: switching to 800x600 pixels and 16 bits reduces the copy size to about 940 KB (800x600x2).

Page flipping avoids these overheads by using a video pointer if one is available since a pointer may not be offered by older graphics hardware. The video pointer tells the graphics card where to look in VRAM for the image to be displayed during the next refresh. Page flipping involves two buffers, which are used alternatively as the primary surface for the screen. While the video pointer is pointing at one buffer, the other is updated. When the next refresh cycle comes around, the pointer is changed to refer to the second buffer and the first buffer is updated.

This approach is illustrated by Figures 4-12 and 4-13.

Figure 4-12. Page flipping (1); point to buffer 1; update buffer 2

The great advantage of this technique is that only pointer manipulation is required, with no need for copying.

I'll be using two buffers in my code, but it's possible to use more, creating a flip chain. The video pointer cycles through the buffers while rendering is carried out to the other buffers in the chain.

In initFullScreen( ), Window.createBufferStrategy( ) sets up the buffering for the window, based on the number specified (which should be two or more). The method tries a page flipping strategy with a video pointer first and then copies using hardware acceleration is used as a fallback. If both of these are unavailable, an unaccelerated copying strategy is used.

Figure 4-13. Page flipping (2); update buffer 1; point to buffer 2

Rendering the Game

The game update and rendering steps are at the core of run( ), represented by two method calls:

     public void run( )     {       // previously shown code       while(running) {         gameUpdate( );         screenUpdate( );         // sleep a while         // maybe do extra gameUpdate( )'s       }       // previously shown code     }

gameUpdate( ) is unchanged from before; it updates the worm's state. screenUpdate( ) still performs active rendering but with the FSEM buffer strategy created in initFullScreen( ):

     private void screenUpdate( )     { try {         gScr = bufferStrategy.getDrawGraphics( );         gameRender(gScr);         gScr.dispose( );         if (!bufferStrategy.contentsLost( ))           bufferStrategy.show( );         else           System.out.println("Contents Lost");         Toolkit.getDefaultToolkit( ).sync( );   // sync the display on some systems       }       catch (Exception e)       { e.printStackTrace( );         running = false;       }     }  // end of screenUpdate( )

screenUpdate( ) utilizes the bufferStrategy reference to get a graphics context (gScr) for drawing. The try-catch block around the rendering operations means their failure causes the running Boolean to be set to false, which will terminate the animation loop.

gameRender( ) writes to the graphic context in the same way that the gameRender( ) methods in earlier versions of WormChase write to their off-screen buffer:

     private void gameRender(Graphics gScr)     {       // clear the background       gScr.setColor(Color.white);       gScr.fillRect (0, 0, pWidth, pHeight);       gScr.setColor(Color.blue);       gScr.setFont(font);       // report frame count & average FPS and UPS at top left       // report time used and boxes used at bottom left       // draw the Pause and Quit buttons       // draw game elements: the obstacles and the worm       // game over stuff     } // end of gameRender( )

The only change is at the start of gameRender( ); there's no longer any need to create an off-screen buffer because initFullScreen( ) does it by calling createBufferStrategy( ).

Back in screenUpdate( ), BufferStrategy.contentsLost( ) returns true or false, depending on if the VRAM used by the buffer has been lost since the call to getdrawGraphics( ); buffer loss is caused by the OS taking back the memory.

Normally, the result will be false, and BufferStrategy.show( ) will then make the buffer visible on screen. This is achieved by changing the video pointer (flipping) or by copying (blitting).

If contentsLost( ) returns TRue, it means the entire image in the off-screen buffer must be redrawn. In my code, redrawing will happen anyway, during the next iteration of the animation loop, when screenUpdate( ) is called again.

Finishing Off

The finishOff( ) method is called in the same way as in the UFS version of WormChase: either at the end of run( ) as the animation loop is finishing or in response to a shutdown event:

     private void finishOff( )     {       if (!finishedOff) {         finishedOff = true;         printStats( );         restoreScreen( );         System.exit(0):       }     }     private void restoreScreen( )     { Window w = gd.getFullScreenWindow( );       if (w != null)         w.dispose( );       gd.setFullScreenWindow(null);     }

The call to restoreScreen( ) is the only addition to finishOff( ). It switches off FSEM by executing GraphicsDevice.setFullScreenWindow(null). This method also restores the display mode to its original state if it was previously changed with setDisplayMode( ).

Displaying the Display Mode

initFullScreen( ) calls methods for reading and changing the display mode (though the call to setDisplayMode( ) is commented out). The display mode can only be changed after the application is in full-screen exclusive mode:

     public void initFullScreen( )     {       // existing code       gd.setFullScreenWindow(this); // switch on FSEM       // I can now adjust the display modes, if I wish       showCurrentMode( );       // setDisplayMode(800, 600, 8);     // 800 by 600, 8 bits, or       // setDisplayMode(1280, 1024, 32);  // 1280 by 1024, 32 bits       // more previously existing code     }  // end of initFullScreen( )

showCurrentMode( ) prints the display mode details for the graphic card:

     private void showCurrentMode( )     {       DisplayMode dm = gd.getDisplayMode( );       System.out.println("Current Display Mode: (" +               dm.getWidth( ) + "," + dm.getHeight( ) + "," +               dm.getBitDepth( ) + "," + dm.getRefreshRate( ) + ")  " );     }

A display mode is composed of the width and height of the monitor (in pixels), bit depth (the number of bits per pixel), and refresh rate. DisplayMode.getBitDepth( ) returns the integer BIT_DEPTH_MULTI (-1) if multiple bit depths are allowed in this mode (unlikely on most monitors). DisplayMode.getRefreshRate( ) returns REFRESH_RATE_UNKNOWN (0) if no information is available on the refresh rate and this means the refresh rate cannot be changed.

The output from showCurrentMode( ) is shown below, with a screen resolution of 1,024x768, 32-bit depth and an unknown (unchangeable) refresh rate:

     >java WormChase 100     fps: 100; period: 10 ms     Current Display Mode: (1024,768,32,0)

Changing the Display Mode

A basic question is, "Why bother changing the display mode since the current setting is probably the most suitable one for the hardware?"

The answer is to increase performance. A smaller screen resolution and bit depth reduces the amount of data transferred when the back buffer is copied to the screen. However, this advantage is irrelevant if the rendering is carried out by page flipping with video pointer manipulation.

A game can run more quickly if its images share the same bit depth as the screen. This is easier to do if I fix the bit depth inside the application. A known screen size may make drawing operations simpler, especially for images that would normally have to be scaled to fit different display sizes.

My setDisplayMode( ) method is supplied with a width, height, and bit depth, and attempts to set the display mode accordingly:

     private void setDisplayMode(int width, int height, int bitDepth)     {       if (!gd.isDisplayChangeSupported( )) {         System.out.println("Display mode changing not supported");         return;       }           if (!isDisplayModeAvailable(width, height, bitDepth)) {         System.out.println("Display mode (" + width + "," +                       height + "," + bitDepth + ") not available");         return;       }       DisplayMode dm = new DisplayMode(width, height, bitDepth,                DisplayMode.REFRESH_RATE_UNKNOWN);   // any refresh rate       try {         gd.setDisplayMode(dm);         System.out.println("Display mode set to: (" +                  width + "," + height + "," + bitDepth + ")");       }       catch (IllegalArgumentException e)       {  System.out.println("Error setting Display mode (" +                 width + "," + height + "," + bitDepth + ")");  }       try {  // sleep to give time for the display to be changed         Thread.sleep(1000);  // 1 sec       }       catch(InterruptedException ex){}     }  // end of setDisplayMode( )

The method checks if display mode changing is supported (the application must be in FSEM for changes to go ahead) and if the given mode is available for this graphics device, via a call to my isDisplayModeAvailable( ) method.

isDisplayModeAvailable( ) retrieves an array of display modes usable by this device, and cycles through them to see if one matches the requested parameters:

     private boolean isDisplayModeAvailable(int width, int height, int bitDepth)     /* Check that a displayMode with this width, height, and        bit depth is available.        I don't care about the refresh rate, which is probably        REFRESH_RATE_UNKNOWN anyway.     */     { DisplayMode[] modes = gd.getDisplayModes( ); // modes list       showModes(modes);       for(int i = 0; i < modes.length; i++) {         if ( width == modes[i].getWidth( ) &&              height == modes[i].getHeight( ) &&              bitDepth == modes[i].getBitDepth( ) )           return true;       }       return false;     }  // end of isDisplayModeAvailable( )

showModes( ) is a pretty printer for the array of DisplayMode objects:

     private void showModes(DisplayMode[] modes)     {       System.out.println("Modes");       for(int i = 0; i < modes.length; i++) {         System.out.print("(" + modes[i].getWidth( ) + "," +                                modes[i].getHeight( ) + "," +                                modes[i].getBitDepth( ) + "," +                                modes[i].getRefreshRate( ) + ")  ");         if ((i+1)%4 == 0)           System.out.println( );       }       System.out.println( );     }

Back in my setDisplayMode( ), a new display mode object is created and set with GraphicDevice's setDisplayMode( ), which may raise an exception if any of its arguments are incorrect. GraphicDevice.setDisplayMode( ) is asynchronous, so the subsequent sleep( ) call delays execution a short time in the hope that the display will be changed before the method returns. Some programmers suggest a delay of two seconds.

The GraphicsDevice.setDisplayMode( ) method (different from my setDisplayMode( )) is known to have bugs. However, it has improved in recent versions of J2SE 1.4, and in J2SE 5.0. My tests across several versions of Windows, using J2SE 1.4.2, sometimes resulted in a JVM crash, occurring after the program had been run successfully a few times. This is one reason why the call to my setDisplayMode( ) is commented out in initFullScreen( ).

My setDisplayMode( ) can be employed to set the screen size to 800x600 with an 8-bit depth:

     setDisplayMode(800, 600, 8);

The resulting on-screen appearance is shown in Figure 4-14. The reduced screen resolution means that the various graphical elements (e.g., the text, circles, and boxes) are bigger. The reduced bit depth causes a reduction in the number of available colors, but the basic colors used here (blue, black, red, and green) are still present.

The output from WormChase lists the initial display mode, the range of possible modes, and the new mode:

     D>java WormChase 100     fps: 100; period: 10 ms     Current Display Mode: (1024,768,32,0)     Modes     (400,300,8,0)  (400,300,16,0)  (400,300,32,0)  (512,384,8,0)     (512,384,16,0)  (512,384,32,0)  (640,400,8,0)  (640,400,16,0)     (640,400,32,0)  (640,480,8,0)  (640,480,16,0)  (640,480,32,0)     (800,600,8,0)  (800,600,16,0)  (800,600,32,0)  (848,480,8,0)     (848,480,16,0)  (848,480,32,0)  (1024,768,8,0)  (1024,768,16,0)     (1024,768,32,0)  (1152,864,8,0)  (1152,864,16,0)  (1152,864,32,0)     (1280,768,8,0)  (1280,768,16,0)  (1280,768,32,0)  (1280,960,8,0)     (1280,960,16,0)  (1280,960,32,0)  (1280,1024,8,0)  (1280,1024,16,0)     (1280,1024,32,0)     Display mode set to: (800,600,8)

Figure 4-14. WormChase with a modified display mode

An essential task if the display mode is changed is to change it back to its original setting at the end of the application. WormChase does this by calling gd.setFullScreenWindow(null) in restoreScreen( ).

Timings for FSEM

Timing results for the FSEM WormChase are given in Table 4-3.

Table 4-3. Average FPS/UPS rates for the FSEM WormChase

Requested FPS	20	50	80	100
Windows 98	20/20	50/50	81/83	84/100
Windows 2000	20/20	50/50	60/83	60/100
Windows XP (1)	20/20	50/50	74/83	76/100
Windows XP (2)	20/20	50/50	83/83	85/100

WormChase on the Windows 2000 machine is the worst performer as usual, but its UPS values are fine. FSEM produces a drastic increase in the frame rate; it produces 60 FPS when 80 is requested compared to the UFS version of WormChase, which only manages 18 FPS.

The Windows 98 and XP boxes produce good to excellent frame rates at 80 FPS, but can't go any faster. FSEM improves the frame rates by around 20 percent compared to UFS, except in the case of the first XP machine.

One reason for flattening out the frame rate values may be that BufferStrategy's show( ) methodused in my screenUpdate( ) to render to the screenis tied to the frequency of the vertical synchronization (often abbreviated to vsync) of the monitor. In FSEM, show( ) blocks until the next vsync signal.

Frame rates for Windows-based FSEM applications can be collected using the FRAPS utility (http://www.fraps.com). Figure 4-15 shows WormChase with a FRAPS-generated FPS value in the top righthand corner.

Figure 4-15. FSEM WormChase with FRAPS output