This release I finally got time to sit down and focus on the first bug I was assigned this semester, which was 686370 ( Implement video playback statistics proposed in WHATWG ). The playback statistics were already started earlier and a few portions of it have already been implemented ( mozParsedFrames, mozDecodedFrames, mozPresentedFrames, mozPaintedFrames, and mozPaintDelay ). What I was to do for this ticket was to finish off the remaining statistics that needed to be implemented ( scroll down to the bottom ). Mathew Schranz also expressed interest in helping with this so I’m sure you’ll catch him blogging about his experiences as well.
To get things started I was fortunate enough to have some class time allocated to go over how we would begin working on this bug. Dave went over what would need to be done in order to get this implemented and how we would go from the decoder level all the way up to the DOM. We began by taking a look at the already implemented portions of the spec and began tracing a route that they took from an idl file all the way down into the decoder. This was invaluable to me as I got input from the class on what they would do, thoughts from others who have worked with idl files before, as well as an understand of the scope of what I would be working on. After we traced a path through the code it was time to begin coding.
I went home that night and basically didn’t know which statistics I wanted to do first, so I just chose the one at the top of the list, which was bytesReceived. bytesReceived is described as “The raw bytes received from the network for decode. Together with the downloadTime, this can be used to calculate the effective bandwidth. Includes container data.” which basically means that we need a way to measure all of the data that we are downloading for a given media element. I took a similar approach to what was done before in the video idl file and applied it to the media element idl file. I began forging my own path down to the decoder and began seeing some results. Once I got into the decoder I found a portion of the code that looked like what I needed to monitor and got access to the data I needed. I battled with making this correct for the next few nights and it actually turned out that I wasn’t even in the right code, as I’m confident now the data that I need is hidden within the media cache somewhere. I took the weekend off to work on my car and think about this and it dawned on me that what I actually found and created was bytesDecoded. I ended up renaming things accordingly and testing it again and it seemed like exactly what I needed. 1 down, 5 to go :).
Next I decided to work on dropped frames, as I scrambled my brain a bit trying to comprehend some of the media cache code. droppedFrames is defined as “The number of frames of video that have been dropped due to performance reasons. This does not include (for example) frames dropped due to seeking.”. This means that we need a way to measure any frames that are not displayed, which will provide developers with a way to measure the speed of the decoding pipeline ( at least a portion of it, more accurate results would be achieved is used in conjunction with other media statistics ). Since this particular statistic was referencing frames I knew it would be directly related video only, and we could ignore the audio portion of the media element. This enabled me to do some caro culting and copy what I could from earlier implementation of the video statistics. Once I reached the decoder I did some searching and read through the code a bit and ended up finding the following snippet in the nsBuiltinStateMachine.cpp and added in a small piece of code that would keep track of all of the frames that were skipped and report them to a setter that I created in nsMediaDecoder for mozDroppedFrames. The setter would simply increment a value that I had stored in the nsMediaDecoder and I also had a getter that would be able to reach in and get the value of droppedFrames. It looked like a pretty solid first attempt and I rolled with it and all seemed to be working well. The problem that I am having is finding out if I did it right, because I seem to be getting the same results for both high-def videos and low-def videos ( 1080 vs 480 ). This doesn’t seem like it should be the case and I should probably look into this further.
The next steps for the statistics are to get the remaining few implemented and get this up for review as soon as possible. I’m sure that I’ve done a few things the wrong way, or it isn’t optimized enough, so I’m looking forward to getting some feedback. Until then, expect a few more blog posts about this and expect these stats to be landing in one of the upcoming Firefox Nightlies :).