The Engineering Behind Flash Player 10.1 Tweet

Today represents an exciting milestone for the Flash Platform as we officially launch the public beta release of Flash Player 10.1 on Android-based devices. The Beta release is now available and as soon as Google releases their update to the Android OS, code-named "FroYo", users will be able to get the new Flash Player installed over the air from the Android Market. The public beta release is your opportunity to try out our dramatically updated browser-based runtime with a deep focus on performance, battery life, memory management, user experience on mobile devices, and other mobile-specific features. With the pervasive use of Flash Player content on the web today, users will be able to experience the full extent of the web that Flash enables including video, games, animations, Rich Internet Apps, data presentations and visualizations, commerce, music, and so much more. It's great to see that our extensive technical collaboration on this with our partners is now becoming a reality. To give you some perspective on the magnitude of this release, let's look at some of the performance improvements and optimizations we made for Flash Player 10.1.

Power and Battery Optimizations

Mobile platforms are carefully tuned to deliver a broad range of capabilities while minimizing battery consumption. Flash Player 10.1 was developed to seamlessly integrate into this environment. To deliver on this objective, we added a new capability called instance management to intelligently load and play back Flash content only after it comes within view on the web page. This capability also allows us to work in conjunction with the browser to ensure the web page is loaded as quickly as possible.

A related capability we added is called pause and resume. Flash Player will automatically pause the content that is running when the browser is hidden from view or the current tab is placed in the background. Not only does this conserve power, but it allows the content to instantly hibernate should an important trigger event occur, such as an incoming phone call. Once the user returns to their browser session, the HTML page and the Flash content are returned to their exact point prior to the trigger event.

Another example of a power-saving capability we have developed is called timer throttling. Flash Player makes use of timers to control the speed of content execution. When the screen display dims, Flash Player utilizes this signal to automatically throttle back the speed at which the content is executing while the device is in this power-saving mode. In the case where the content is playing back audio or video, Flash Player will not throttle it back to ensure a seamless media playback experience.

These new capabilities in Flash Player 10.1 offer some insight into the priority we placed on conserving power and staying true to the goal of maximizing battery life on mobile devices.

Maximizing Performance

The architecture of mobile platforms is very different from desktops, laptops, and netbooks. To take advantage of these highly integrated hardware environments, we took a very comprehensive look at how Flash Player uses the CPU, GPU, memory, and storage. The Flash Player team, with engineering cooperation across our Open Screen Project partners, meticulously optimized the machine instructions used in our virtual machine, rendering engine, and media codecs to run efficiently on mobile hardware.

In the case of the ActionScript 3 virtual machine, we updated our just-in-time compiler to emit native ARM instructions to maximize the execution speed of applications on these platforms. With a significant portion of the Flash content on the web having been written in ActionScript 1 and 2, we also invested in increasing the execution speed of this content.

In addition, Flash Player will take advantage of the purpose-built hardware on the mobile device including the media decoders to optimize the playback of content such as H.264 video and AAC audio streams. This specialized hardware can often perform a specific task more efficiently than the CPU. Not only does this result in higher fidelity playback, but it can yield a nice power savings.

The investments we made in execution speed will be apparent in a broad range of content, from Flash applications to games to video, and everything in between. We took an extensive look at the performance characteristics of Flash Player in many different scenarios and drove considerable improvements in the execution speed. As a result of this work, most Flash content on the web today will execute more efficiently with no changes required to receive these benefits.

Conserving Memory

Mobile platforms have considerably less memory than most personal computers and Flash Player 10.1 includes numerous optimizations to minimize the amount of memory required for content and applications. We have added automatic compression of media in memory to matchthe typically smaller screen size and color depth of a mobile device. We have also enhanced the memory garbage collection system to work more effectively, particularly in low memory situations. We made changes to more aggressively release temporary buffers and media caches for images and audio data. These changes have translated into some dramatic improvements. In some cases, you'll see content that now automatically consumes 50% less memory with Flash Player 10.1 when compared to our previous release.

Another important enhancement we made relates to streaming media. In some cases, content such as video is buffered in memory to allow the user to quickly seek forward or backward. In the case of mobile platforms, we added a new circular buffering capability that allows Flash Player to constrain the size of the media buffer and recycle this memory as the content plays. This new capability strikes a balance between allowing the user to navigate within sequential media, like video, and the amount of memory that content can consume.

The Flash Player team also did a lot of development around low memory support, as this is a common situation on mobile devices. To protect against content that requires too much memory for the device, we added a new system that detects when memory is running low so defensive actions can be taken to prevent a crash. In this case, the Flash Player now has a set of heuristics to shut down content when resources are running low or depleted and gracefully exit the content. This work will also benefit personal computer users that have many tabs open in their browser such that their PC is operating in a low memory situation.

Usability and Interactivity

The last topic I'll cover is usability. In this case, it's critically important that Flash content on a smart phone both behave consistently with the HTML page and enable rich interactivity with media in Flash. For example, as a web page can be zoomed using a multi-touch gesture by the user, the Flash content needs to enable that to happen seamlessly. If the user rotates the phone to switch into landscape mode, the Flash content responds as you'd expect. If the user double-taps to zoom in or out, the Flash content responds in a consistent, intuitive way. We have enhanced this interaction to enable a new smart-zoom capability, bringing zoomed Flash content to the best width/height dimension for the device to enable easy viewing of video and other content in Flash. We have also added the ability for content to specify that it should automatically go into full screen mode on first interaction in order to enable easy full screen interactivity with games. This new mode is indicated to the user with a message about how to exit full screen mode on their particular device.

The Flash Player team added a number of new capabilities that are essential to smart phones. Text fields in Flash will automatically take advantage of virtual or soft keyboards for text entry when a physical input device is not available. In this case, the text field is intelligently repositioned on the screen to facilitate text input.

In addition, Flash Player exposes some of the unique capabilities of mobile platforms, including access to the accelerometer. This will allow the Flash development community to create content that leverages some of the unique capabilities. We've got more work to do here, but we're off to a good start and look forward to exposing new capabilities in the future, such as geolocation support.

These examples demonstrate the importance of ensuring that Flash content on a mobile device offers a predictable and intuitive experience to the end-user. To help ensure that we got the experience right, we conducted multiple rounds of usability testing to try out new ideas and improve the experience based on the feedback we received. Our goal was to ensure that existing content simply works as expected and ultimately allow smart phone users to experience the full glory of the web.

Looking Ahead

As we look forward, it's important to note that this is just the start of our work to bring the full Flash Player to mobile devices. We will continue to improve on this effort as we support a broad range of smart phones, tablets, televisions, and other devices. Of course, a number of these optimizations and improvements will also directly benefit Flash Player on desktop and netbook platforms. At the same time, we've seen content publishers like Sony Pictures, Nickelodeon, Miniclip, Warner Brothers, Kongregate, South Park Studios and others already optimizing their sites to deliver the best possible experience within the context of smaller screens. Developers, who are looking for tips about optimizing Flash content for mobile devices should check out this document.

Smartphones in market today that will be able to run Flash Player 10.1 include the Nexus One, DROID by Motorola, HTC Evo, HTC Incredible, HTC Desire, Samsung Galaxy S, Motorola Milestone and others, but will depend on FroYo availability for a given device. Upgrade mechanisms and timing will vary by device and device manufacturer. To make it as easy as possible to install Flash Player, we are enabling over-the-air downloads which include content-initiated downloads, system software updates, and on-device app catalogs. Over time, a number of new phones are planned to ship with Flash Player 10.1 preinstalled. We're thrilled to take the first step today and can't wait to see how millions of Flash developers will innovate for mobile devices.

I hope you're as excited about the future and the opportunities ahead as we are.

Paul Betlem
Sr. Director, Flash Player Engineering

PS: If you are a developer and want to start developing apps for Android today, check out this post from the AIR team and read all about the AIR SDK developer release we made available for Android today.

Labels: