roguish

GPU Test App AIR for Mobile

Optimizing Flash for mobile devices can be tricky business. With AIR for mobile we can take advantage of GPU acceleration on devices, but there’s a lot of confusion about how and when to use GPU acceleration. This test app shows the performance differences with different GPU settings and can be used for benchmark testing across devices.

The app has been published and uploaded to iOS, Android and BlackBerry PlayBook app stores. See this post for more info: http://www.roguish.com/blog/?p=326

* NOTE: This is an ongoing series of tests and optimizations. Please scroll to the bottom to see the latest results and conclusions.

Test Configuration:
Platform: Google Nexus One
Flash: CS5 (11.0.2.489)
Publish settings: Render Mode: GPU
Test params: 130 balls, all tests evaluated with UI overlay hidden
Terminology: CAB: cacheAsBitmap, CABM: cacheAsBitmapMatrix

* Note: results are outdated. See updates below.
Mode: Bitmap Display Objects

  • CAB: no, CABM: no, rotation: no, alpha fade: no. 23fps, 18ms/frame
  • CAB: yes, CABM: yes, rotation: yes, alpha fade: no. 24fps, 18ms/frame (1)
  • CAB: yes, CABM: yes, rotation: no, alpha fade: yes. 15fps, 26ms/frame (2)
  • CAB: no, CABM: no, rotation: yes, alpha fade: no. 23fps, 19ms/frame (3)
  • CAB: yes, CABM: yes, rotation: yes, alpha fade: no. 23fps, 19ms/frame (4)

Mode: Vector Display Objects

  • CAB: no, CABM: no, rotation: no, alpha fade: no. 8fps, 31ms/frame
  • CAB: yes, CABM: no, rotation: no, alpha fade: no. 23fps, 18ms/frame (5)
  • CAB: yes, CABM: yes, rotation: no, alpha fade: no. 24fps, 17ms/frame
  • CAB: yes, CABM: yes, rotation: yes, alpha fade: no. 24fps, 18ms/frame (6)
  • CAB: yes, CABM: no, rotation: yes, alpha fade: no. 1fps, 18ms/frame (7)
  • CAB: yes, CABM: yes, rotation: yes, alpha fade: yes. 14fps, 27ms/frame (8)

Mode: Blitting

  • CAB: no, CABM: no, rotation: no, alpha fade: no. 10fps, 34ms/frame

Test Result Notes:

(1) With raster, no difference when adding CAB/CABM
(2) Add alpha fade and performance dips, even with CAB/CABM
(3) Rotation doesn’t degrade performance with bitmaps
(4) Rotation is the same whether using CAB/CABM or not
(5) Major improvement with vectors with just CAB addition
(6) Adding rotation doesn’t impact performance when using CABM
(7) Rotating vector Display Objects without CABM is a lost cause
(8) Adding alpha fading *does* impact performance when using CABM — it’s not supposed to…

UPDATE March 3, 2011
AIR 2.6 for mobile was released recently and stabilized things a bit. Raster and Vector are now more similar, especially when using CAB/CABM. Alpha fading now works as expected with no performance hit for using it while CAB/CABM is enabled. Here’s a chart of the performance with AIR 2.6 on a NexusOne.

* Note: results are outdated. See updates below.

Raster
  CAB CABM alpha rotate fps code ms
1 no no no no 15 23
2 no no yes yes 15 23
3 yes yes yes yes 24 18
4 yes yes yes no 24 18
5 yes yes no no 24 17
 
Vector
  CAB CABM alpha rotate fps code ms
1 no no no no 8 23
2 no no yes yes 8 22
3 yes yes yes yes 23 18
4 yes yes yes no 24 17
5 yes yes no no 24 17
 
Blitting
  CAB CABM alpha rotate fps code ms
1 no no no no 10 17
2 yes yes no no 10 17
 
No Display
          fps code ms
1         56 16


UPDATE March 03, 2011, v.2.0.3
Added StageQuality settings. Anything but LOW setting will cause a performance hit in raster/vector modes.
A bug in the runtime (at-least this is the case on my Nexus One) prohibits the technique of setting the stage.quality to BEST before caching the vector art then setting it back to LOW after the rasters have been created and cached. I have logged the bug with Adobe.
Also, I have noticed that regardless of the stage.quality setting, when using cacheAsBitmap Vector graphics look jaggy. This suggests that setting cacheAsBitmap causes the Vectors to be drawn by the GPU and it doesn’t render with the same quality as CPU rendering.


UPDATE March 06, 2011, v.2.0.4
The latest changes resulted in some significant performance improvements and answered a few burning questions. In short, the latest version achieves 30fps for 130 balls (Raster or Vector mode).
Here’s what’s changed:

  • Created a Point object to reference on each ENTER_FRAME of blitting instead of creating a new Point each ENTER_FRAME.
  • Discovered that locking ball locations to ints was the reason balls favored the upper-left corner and removed the integer locking. This fix also saved many int calls as a result and may account for some of the performance gain in this version. I may try rounding the value up/down to an int in future versions because placing art on fixed pixel values has always been a recommended best practice in Flash, although mobile devices with high screen resolutions may negate that old advice.
  • Converted the FlashBall class to an Object rather than a Sprite, injecting a reference to a Sprite when displaying the ball as a Display Object. This caused significant frame script execution performance gains (now down to ~1-4ms/frame), and is also the likely cause of the framerate performance gains as well.
  • Display separate code execution times for physics frame script execution time and full script calculation time.
  • Applied velocities when collisions occur rather than waiting until the next frame. This was not a performance optimization, it just seems like that was missing in the original collision code.
  • Added option to inline the move and doCollide method calls.
  • Added option for opaque backgrounds for Display Objects.
  • Added stage.stageWidth = Capabilities.screenResolutionX; per blog post: http://blogs.adobe.com/actionscriptdocs/category/air-releases.
  • Show/Hide balls made a single toggle button.


You will need Flash CS5 to work with the app (no Flash Builder requirement).

Source: FlashBallsPerformanceTester204.zip

* Foundation source code for bubble physics from Alexey Gavrilov’s Bubblemark tests: http://bubblemark.com

18 ResponsesLeave one →

  1. The only app with a “SHOW BALLS” button, yeah!

    Reply
  2. TJ Beatrice

     /  March 28, 2011

    Great application.

    I am an experienced Flash Developer who is just getting started with Air for Android. I had started by attempting a VERY simple bubbles animation but have been quite dissapointed with performance on my HTC Incredible. I assumed it was basic inexperience and I needed to optimize my code better, but there really wasnt much in my code and after reading several posts/articles/etc about optimization I realized that my code was pretty sound.

    I found this post and tested your app on my phone (v.2.0.4) and performance still didnt seem like it was where it should be. Here are some numbers with 130 balls (with stage quality – LOW):

    ——-
    Raster
    ——-

    CAB CABM alpha rotate fps code-ms
    1 no no no no 16 4
    2 no no yes yes 10 4
    3 yes yes yes yes 23* 4
    4 yes yes yes no 11 4
    5 yes yes no no 13 4

    ——-
    Vector
    ——-

    CAB CABM alpha rotate fps code-ms
    1 no no no no 10 4
    2 no no yes yes 18* 4
    3 yes yes yes yes 6 4
    4 yes yes yes no 10 4
    5 yes yes no no 13 4

    ——-
    Blitting **
    ——-

    CAB CABM alpha rotate fps code-ms
    1 no no no no 19 18
    2 no no yes yes 19 19
    3 yes yes yes yes 19 19
    4 yes yes yes no 19 19
    5 yes yes no no 19 19

    * actually jumps a bit all over the place

    ** alpha and rotate, although checked, did not affect the balls… they did not fade or rotate in Blitting mode

    Reply
    • Elliot Mebane

       /  April 6, 2011

      Interesting results with the HTC Incredible. What version of AIR were you running? It’s possible that your results illustrate performance differences across different devices. It’s odd that your Raster #3 test had better performance than Raster #4 (although you report inconsistent framerates in #3). I wouldn’t expect the addition of rotation changing to increase performance.

      I hope other people will also post the results of running the app on a variety of devices.

      I have not implemented an alpha/rotation for the blitting mode. I don’t think rotation will have much of an impact. Alpha may have some impact. I’ll consider adding them for a future release.

      Reply
  3. judah

     /  April 3, 2011

    thanks! :)

    Reply
  4. jacdx

     /  May 5, 2011

    Nice tests. We are also seeing minimal difference between using CPU and GPU across several devices (Xoom, Galaxy Tab, Ipad, Ipad2). One of the biggest optimizations has come from using low stage quality. I really wish the cacheAsBitmap stuff made a bigger difference. Also, you should try Air 2.7 on labs. Definitely faster on IOS, haven’t tested Android yet.

    Reply
  5. i think it would be cool if you would submit your app on the market on any other distribution platform. this would make it easier for people to run the tests.

    also, you could collect the test results and post them here. would be interesting to see those …

    Reply
  6. Hi Elliot, you are a bit optimist on the “PlayBook best performance configuration: 255 balls at 30fps using Vector mode and cacheAsBitmap (not cacheAsBitmapMatrix)”. :)
    If only!!
    The simulator is rendering way faster than actual Playbook device: your code works at best around 20 fps with 130 moving balls, no matter modes/options.
    Full blitting is not a good idea on Playbook right now (under 2.5/2.7) – better move sprites around rather than having one bitmapdata (even in your sprite container) and redraw everything inside, plus drawrect is costly, especially at 1024*600 resolution. I do achieve around 165 objects in “half blitting” (still bitmapdata inside a sprite which is the size of the moving object, but sprites on stage and dealing with x/y coords) at full steady 60 fps (more than tripling your perf test). I’m used to full blitting as you do but alas, too much impact performance on Playbook dealing with bitmap directly on stage.
    Keep it up!

    Reply
    • Elliot Mebane

       /  June 28, 2011

      Hi Hasufel — Thanks for taking a look and commenting. My tests were on an actual PlayBook device. I have updated the app using AIR 2.7 and the new version is available on the BlackBerry AppWorld (v.2.3). Changing the stage quality has an impact now. Here’s my current results (I’m looking at a PlayBook right now :) ). I’m using cacheAsBitmap for all of them. cacheAsBitmapMatrix is not set. If you have a PlayBook and you want to confirm my results, don’t forget to hide the information panel when observing the FPS. All results below show the max number of balls I could add before the framerate dropped lower than 30.

      Draw Mode, Quality Setting, Balls
      Vector, Low, 360 (low quality isn’t really acceptable)
      Vector, Med, 250
      Vector, High/Best, 230

      Vector outperforms Raster on this device, but for a quick comparison, here’s what you get from Raster and Blitting. stage.quality doesn’t impact performance of the Raster tests, and the best performance is achieved *without* cacheAsBitmap set, so these results are without cacheAsBitmap:
      Raster, Low, 150
      Blitting, Low, 45

      BTW: The blitting approach in my app is the full stage blitting approach. My Raster approach is probably equivalent to the *half blitting* approach where a vector is drawn as BitmapData and placed inside a Sprite then that container Sprite is moved around. It’s worth testing to verify that, though.

      My Vector tests are a couple of circles and gradients, so it may be unfairly-easy for the gpu to draw them. I may adjust the vector art to include more complex vectors to be a little more *real world*.

      Reply
      • Hi Elliot. Indeed, I did push up to 365 balls using raster/in low quality after reading your comment; I totally dismissed it the first time as displaying raster in low quality didnt meet my “standards” – so sorry for that.
        I did some extensive animation/performance tests on Android/Playbook too, and was happily surprised to see your app pop up in Playbook Appworld, so I did give it a shot and looked at your code. You should try the “half blitting” way on raster/blitting, it definitely will improve your fps. I’ll test also animated bitmapdata through spritesheets – I think performance, oddly, will remain the same.

        Reply
  7. Hey guys, thanks for the nice perf util.

    Just tested both GPU and CPU apps on Xoom with AIR 2.7:

    - CPU test: based on several different quality, config settings, I’d say the rough average across all tests was in the 20 FPS range with only about 65 balls on stage. I had to reduce the # of balls drastically to even get into the 20 FPS range… Pretty awful.

    - GPU test: this was impressive. I pushed up to 465 balls and still saw nearly 60 FPS. Obviously a dramatic difference over the CPU version of the test results.

    Some observations:

    - The Settings button wouldn’t work in GPU mode (maybe for a reason?), so I couldn’t play with any settings
    - The text was badly distorted, unreadable in GPU mode until I forced the screen to update based on orientation a few times.
    - Haven’t calculated FPS yet but my Flex app using Flex 4.5, AIR 2.7 gets terrible performance (very similar to the CPU type of lag seen in the CPU test apps). So, I’m looking now at completely dropping Flex and moving to pure AS3 unfortunately. Does anyone know what UI widget set might be a good solution if I drop Flex for my UI completely. Most of my MVC code, etc doesn’t depend on Flex at all. I just need a decent UI set that is based on simple AS3 sprites so I don’t have to write the nuts/bolts of the app’s UI controls from the ground up.

    Hopefully Flex improves greatly with the AIR 3 builds… It’s just not viable for anything serious beyond basic text input type apps it seems (we have video, animation from canned SWFs delivered by designers, etc that all yield crappy performance in Flex)…

    Chris

    Reply
  8. Hi Chris,

    For games or any apps that are visually intensively you definitely want to go down the ActionScript Only path you mentioned you were considering. The best Flex like AS3 components out there I have found and use include:

    Razor Berry – http://www.razorberry.com/blog/components/
    - less components
    - plenty of styling options

    Minimal Comps – http://www.minimalcomps.com/
    - lots of components
    - super lightweight
    - minimal styling/no skinning

    Hope that helps, Mike G. – Lime Rocket

    Reply
  9. nick

     /  October 25, 2011

    Hi, great benchmark tool! Do you have any more results/comparisons to share? iPad1, iPad2 …

    Reply
  10. “GPU Test App AIR for Mobile | roguish” ended up being a splendid posting, can’t wait
    to look over even more of ur posts. Time to squander some time on the net lmao.
    Thanks for your time ,Silke

    Reply
  11. Michael Scholz

     /  April 10, 2013

    Can you send me a build using a 3.7 or so version of AIR? The 2.7 version is too old to be relevant.

    Reply
  1. Cool Stuff with the Flash Platform – Molehill 3D Special Edition – 3/4/11 | Finding Out About
  2. Mobile Performance Tester — Now Live in App Stores | roguish
  3. Adobe AIR vs. HTML5 for Mobile Apps | Avinash Kaza

Leave a Reply