Sunday, January 30, 2011

Edit recorded web pages

Web Page Replay now supports editing recorded web pages. Credit for the idea goes to Sergey Chernyshev. To use it: record a web page then use httparchive's edit command.

$ ./ edit --path=/ ~/archive.wpr

Since all other aspects of the page are controlled, this allows you to measure the effect of tweaking just a single aspect of the page without having to worry about conflating factors like rotating ads, server response time or variable network conditions.

I'm finding this invaluable for answering quick questions like how much faster a page loads if this script were deferred or if the order of this script and stylesheet were switched.

Friday, January 28, 2011

The WebKit PreloadScanner

WebKit's HTMLDocumentParser frequently must block parsing while waiting for a script or stylesheet to download. During this time, the HTMLPreloadScanner looks ahead in the source for subresource downloads which can be started speculatively. I've always assumed that discovering subresources sooner is a key factor in loading web pages efficiently, but until now, never had a good way to quantify it.

How effective is it?

Today I used Web Page Replay to test a build of Chromium with preload scanning disabled vs a stock build. The results were definitive. A sample of 43 URLs from Alexa's top 75 websites loaded on average in 1,086ms without the scanner and 879ms with it. That is a ~20% savings!

That number conceals some subtleties. The preload scanner has zero effect on highly optimized sites such as and In stark contrast, the preload scanner causes, a subresource heavy site, to load fully twice as fast.

Why does this matter?

There is a lot of room for improvement in the preload scanner. These results tell me that it is worth spending time giving it some serious love. Some ideas:

  • It doesn't detect iframes, @import stylesheet, fonts, HTML5 audio/video, and probably lots of other types of subresources.
  • When blocked in the <head>, it doesn't scan the <body>.
  • It doesn't work on xhtml pages (wikipedia is perhaps the most prominent example).
  • The tokens it generates are not reused by the parser, so in many cases parsing is done twice.
  • The scanner runs in the UI thread. So as data arrives from the network, it may not be scanned immediately if the UI thread is blocked by JavaScript execution.
  • External stylesheets are not scanned until they are entirely downloaded. They could be scanned as data arrives like is done for the root document.

Test setup

The test was performed with a simulated connection of 5Mbps download, 2Mbps upload, and a 40ms RTT time on OSX 10.6. The full data set is available.