Design and Implementation of WMA File Playback Function in Android System

In order to enhance the function of the Android multimedia system, WMA audio playback function is added to the Android smartphone, so that the Android platform supports the WMA format and plays the WMA format file. Based on the Stagefright framework of the Android multimedia system, the encoded data in the WMA audio file is correctly decoded into the original data and output by creating the file parsing unit and the decoding unit of the WMA. By playing WMA audio files repeatedly on the Android platform tester, the playback sound is clear and the sound quality is good.

WMA can be used in encoded files in multiple formats. Microsoft has significantly improved its engine in WMA9. In fact, it can be almost 1 / 3 smaller than MP3 in the same file quality, so it is suitable for network streaming media and mobile devices. Many player software have also developed plug-ins that support the WMA format, but Android phones do not yet support this format, so adding a WMA audio decoding format to Android phones has a certain meaning.

1 Android platform and its multimedia framework structure

1. 1 Android system

Android is an open source mobile operating system launched by Google and OHA (Open HandsetAlliance). Android is based on the Linux platform and consists of the operating system, middleware, user interface and application software. The Android platform consists of four levels from the bottom up:

Linux kernel layer, runtime library and other library layers, application framework layer, application layer.

(1) Linux Kernel.Android is a stand-alone operating system based on Linux 2. 6 kernel. This layer is mainly used to provide the underlying services of the system, including security mechanism, memory management, process management, network stack and driver.

(2) Libraries and Android Runtime. This layer is mainly related to process operation, and contains a set of C/C++ function libraries, including Libc, Media, Framework, WebKit, SGL, OpenGLES, FreeType, SQLite, etc. The core library provides most of the functionality of the Java programming core library, which is presented to developers through the Android application framework, and each Android program has a separate Dalvik virtual machine to provide it with a runtime environment.

(3) Application Framework. This layer is designed for application development on the Android platform. The developer invokes the API provided by the Android framework by using the core application, which is designed to facilitate reuse of components, which is composed of a series of services and systems.

(4) Applications.Android comes with some core application packages, such as Email client, browser, calendar, Google Maps, SMS short message program, etc.

1. 2 media player structure and the core of multimedia implementation

　　 The Android multimedia system spans all four levels of the Android system: the Java application layer, the Java framework layer, the native code layer, and the Linux driver layer. The multimedia native code layer is the focus of the multimedia system. The Libmedia library provides a native framework for the multimedia part, and Libstagefright provides the implementation of multimedia core functionality.

Figure 1 Module structure of Android media player

The upper application sets the media URI as input to the media player, then through the application framework, JNI, and the local framework, up to the StagefrightPlayer. There is no data flow in the process, just the URI path is passed. After parsing by the parsing unit in Stagefright-Player, the audio stream is read and converted into raw data by the processing of the decoder. The audio raw data will be sent to the audio output.

Stagefright is at the heart of the Android multimedia native implementation.

Stagefright includes a lot of content. From the perspective of playback, StagefrightPlayer inputs files or network media streams, and the output is audio and video output devices. The basic functions include media stream control, file parsing, audio and video file decoding. Therefore, to achieve Android multimedia playback of WMA audio format media files or streaming media, you need to extend the file parsing and audio decoding aspects of Stagefright, add WMA format file parsing unit and WMA audio file decoding unit.

2 Multimedia system to increase the design of WMA audio format

From the perspective of the specific implementation of the multimedia system, WMA audio format playback mainly through three stages: WMA format file parsing, WMA encoded stream decoding, and PCM output playback. The structure of the WMA audio player is shown in Figure 2.

Figure 2 Structure of the WMA audio player

Based on the Android multimedia system audio playback process, there are four main tasks in the WMA audio format development process:

(1) Identification of WMA files;

　　 (2) Analysis of WMA files;

　　 (3) Reading of encoded data;

　　 (4) Decoding and output of encoded data.

2. 1 WMA format audio playback function flow design

The data source is set by calling the setDataSource function of AwesomePlayer. AwesomePlayer recognizes the format of the file by calling the Create function of MediaExtractor. After MediaPlayer determines that the file is in WMA format, it will create a WMAExtractor. At the same time that WMAExtractor is created, WMAExtractor will parse the file header. , get the relevant information in the file. Then call WMAExtractor's getTrack function to create a WMASource; AwesomePlayer creates a WMADecoder through OMXCodec; AwesomePlayer then creates an AudioPlayer, passes WMADecoder as a data source to AudioPlayer, and calls AudioPlayer's start function; AudioPlayer gets the relevant parameters in WMADecoder: Type, sampling rate, number of channels, and start AudioSink according to the data, and transmit AudioSinkCallback as a callback function to AudioSink.AudioPlayer first call WMADecoder to solve the first frame data, and pass the data to AudioSink to play, when the playback is completed. After AudioSink will call the callback function AudioSinkCallback and then retrieve the decoded data, AudioSinkCallback will call the FillBuffer function to get the decoded original data. After the decoded data is retrieved, AudioPlayer will call WMADecoder to decode the next frame of data to AudioSink. Repeat until the number of files is played, the playback process is shown in Figure 3. When the scroll bar is pulled, the upper layer will send SeekTime, which is transmitted to WMADecoder via AudioPlayer and then to WMAExtractor. WMAExtractor determines the starting position of the original data to be played according to the SeekTime sent from the upper layer, and then reads a packet from the position. Passed to WMADecoder for decoding.

Figure 3 audio playback flow chart

In the whole WMA format decoding and playback process, the main design has two modules: WMAExtractor and WMADecoder.WMAExtractor mainly performs WMA format file parsing and data reading.

WMADecoder mainly performs decoding functions; WMA format audio playback function is implemented.

(1) Identification of WMA files.

Before deciding to play the file format, AwesomePlayer will register the supported format in advance through the RegisterDefault-Sniffers function in the DataSource. When judging the playback file format, the file will be matched with the supported formats one by one. The most matching format is the format of the file, so the following code should be added to the RegisterDefaultSniffers function in the Datasource:

Void DataSource: : RegisterDefaultSniffers( ) {

......

RegisterSniffer( SniffWMA) ;

}

The WMA file starts with a 16 Byte identifier, indicating WMA: 30 26 B2 75 8E 66 CF 11 A6 D9 00 AA 62 CE6C. If the first 16 characters of the audio file match the 16 Byte, then the file can be judged as WMA file. The SniffWMA function in WMAExtractor determines whether the file is a WMA file by reading the first 16 bytes of the file. In the SniffWMA function, if it is judged that the first 16 Bytes and the 16 identifier bytes of the WMA are equal, the MEDIA_MIMETYPE _AUDIO_WMA is given to the mime-Type pointer, indicating that the audio file type is WMA format.

MEDIA_MIMETYPE_AUDIO_WMA is defined in the MediaDefs.h file and assigned in the MediaDefs. cpp file:

Extern const char * MEDIA_MIMETYPE_AUDIO_WMA; / / in the MediaDefs. h header file

Const char * MEDIA_MIMETYPE_AUDIO_WMA ="audio /wma"; / /MediaDefs. cpp header file

(2) Analysis of WMA files.

WMAExtractor takes 16 Bytes from the 31st Byte of the WMA file, and then compares it with file_header, stream_header, data_header, comment_header, and extended_content_header. If it is equal to file_header, it will get the file size, creation time, and number of packets from the next Byte. ,... packet size. Then read 16 Bytes from the next Byte and compare them. If it is equal to extended_content_header, you can get non-audio information such as name, artist, copyright, and comment in order from the next Byte. Then read 16 Bytes for comparison until it is equal to data_header. After data_header is the audio file decoding data, the end position of data_header is the offset of the first packet in the file. WMAExtractor will create a MetaData and store the sample_rate, Byte_rate, channels, and duration obtained in the header of the file into MetaData. In the getMeta-Data function of WMAExtractor, put the previously obtained non-audio information into MetaData, and finally return the MetaData. In the getTrack function of WMAExtractor, create a WMASource and pass WMA data and MetaData to WMASource.

(3) Reading of encoded data.

Obtaining undecoded data is read by WMASource's read function. WMA data is in packets and is the same size as the packets in the file. There are multiple frames of data in each data packet. The starting position of each data packet is subtracted from the starting position of the first data packet and divided by the size of the packet equal to an integer. This integer is the data packet before the data packet. number. The first Byte of each packet is generally equal to 0x82. The second Byte is followed by information about the packet.

The undecoded data in the packet can be obtained based on the relevant data of the packet.

When WMASource's read reads undecoded data, it first determines whether the options from WMADecoder are empty. If it is not empty, and can get a play time seek-TimeUs from options, it will pass seekTimeUs, total play time and total. The number of data packets is calculated as the starting position of the data packet to be played, and then the data of one data packet is obtained from the starting position, and the size, starting position, time and other data of the valid data are obtained from the data packet, and finally The valid data and time are placed in the Buffer from WMADecoder.

When the Read of WMASource is called, if the transmitted Options is empty or the time seek-TimeUs cannot be obtained from Options, a packet will be read from the WMA file, according to the size and starting position of the valid data. Valid data, and get the time in the packet, then put the valid data and time in the buffer from WMADecoder. The starting position of the first packet is the offset of the first packet obtained when parsing the header file, so the first time WMASource's read is called, the first position is read from the next position of the offset. Packets. There is a pointer in WMASource that specifically records the read location. After reading 1 packet each time, the pointer will point to the next position at the end of the packet. When the next read of WMASource reads undecoded data, if it is not music fixed point playback, it will be from the position pointed by the pointer. Start reading the packet.

(4) Decoding and output of encoded data.

AwesomePlayer creates WMADecoder via the Create function in OMXCodec, so register WMADecoder information in OMXCodec:

FACTORY_CREATE( WMADecoder)

...

Static sp < MediaSource > InstantiateSoftwareCodec(const char * name,const sp < MediaSource >&source) {

...

Static const FactoryInfo kFactoryInfo[]= {

...

FACTORY_REF( WMADecoder) } } ;

...

Static const CodecInfo kDecoderInfo[]= {

...

{ MEDIA_MIMETYPE_AUDIO_WMA, "WMADecoder"}

} ;

When creating WMADecoder, pass the previously created WMASource to WMADecoder. In the WMADecoder constructor, WMADecoder obtains Metadata from WMASource and gets sampleRate, numChannels, duration, etc. from Metadata. In the start function of WMADecoder, the avcodec_open function is called to allocate the space required for decoding, and to create and initialize the relevant parameters required for decoding. The Stop function of WMADecoder is called in the WMADecoder destructor. All related space is released in the Stop function.

WMA audio decoding is mainly done in WMADecoder's read function: First, it will first determine whether it is music fixed point playback, if not, WMADecoder will call WMAExtractor's read function to read an undecoded data packet; then, the data is Decoding, storing the decoded audio data in the Data( ) of the MediaBuffer, and then setting the mRangeOffset and mRangeLength of the MediaBuffer. When reading the data packet, the timestamp in the data packet is obtained from the packet, and the timestamp is stored in the data packet. In the kKeyTime of MediaBuffer's Meta_data( ); finally, WMAdecoder passes the MediaBuffer back to AudioPlayer. If it is music fixed-point playback, first, WMADecoder will get the playback time from ReadOption passed from AudioPlayer ( option - > getSeekTo( &seekTimeUs, &mode ) ), when WMASource's read function is called to read undecoded audio data, the time ( seek-TimeUs) is passed to WMASource.WMASource's read function. After the time is obtained, the time is played by calculation. The starting position of the audio packet, then read The data packet is transmitted to the WMA Decoder for decoding, and finally the decoded audio data is transmitted to the AudioPlayer.

3 Experimental results

The WMA audio playback designed by the multimedia system based on the Android platform adds the WMA audio format to the core Stagefright framework of the Android multimedia framework. Android's support for WMA audio formats enables Android phones to play files in WMA audio format. After the actual test, the playback effect has reached the expected requirements, the sound is clear, and the sound quality is good. Figure 4 is a screenshot of the Android source code compilation result after adding the WMA audio player module. Figure 5 is a screenshot of the playback interface when playing a WMA format file. Figure 6 is a screenshot of the normal operation after pulling the scroll bar.

Figure 4 Android source code compilation results

Figure 5 WMA audio file playback

Figure 6 WMA audio file playback

4 Conclusion

Based on the Stagefright framework in the Android Multimedia Module, WMA audio format support is enabled on smartphones, enabling Android smartphones to play media files or streaming media in WMA audio formats. The design realizes the enhancement of the multimedia system function in the Android operating system on the existing basis. At present, the Android platform mobile phone still does not support RMVB, WAV and other video formats, so the functions of the Android multimedia system need to continue to be enhanced and expanded.