Mixed Signals
Any sufficiently advanced information is indistinguishable from noise
Tuesday 2 September 2008, 8:54 PM
Google Chrome - first benchmarks. Summary: wow.
We'll be looking at Chrome very closely over the next few days: modern browsers are complex beasts, and there are a lot of things to test. But JavaScript performance benchmarks we can run right here, right now - so, we have. We took SunSpider 0.9 and ran it twice. Earlier today, in expectation of getting Chrome, we ran the same benchmarks on the same machine for a representative set of other current browsers.
We compared Google Chrome beta 1 running on Vista SP1, on a Lenovo T61 with 1GB of RAM, a 2GHz Core 2 Duo processor and GMA x3000 graphics, comparing it against IE 7, IE 8 beta 2, Opera 9.52, Safari 3.1.2, Firefox 3.0.1 and Firefox 3.1b1. We couldn't run it on Firefox 3.1 with TraceMonkey, because the nightly build failed to complete the benchmarks.
And it is fast, easily topping the chart. In particular, it's more than ten times faster than IE 7, and three times faster than IE8 b2. FF 3.1 comes closest.
Here are the figures.
There is no doubt that Chrome more than lives up to its performance claims: it is the fastest browser we've seen on standard platforms. That may not be too surprising - one of the people behind the V8 JavaScript compiler is Lars Bak, who was the lead developer on the high-performance Sun Java VM that became HotSpot.
We'll flesh out these tests over the coming days, and go into more detail about what they show and what they might mean for Google - and its competitors. Meanwhile, Chrome looks set to more than justify the initial excitement raised in the day between it being announced and becoming available.
[UPDATE]
The latest TraceMonkey vs V8 benchmarks show the Monkey ahead on points. Be sure to read the comments - for their tone, as much as for their technical content.
[UPDATE 2]
We've now got much more comprehensive benchmarks, courtesy of ZDNet Germany. Chrome is maintaining its reputation as an interesting, speedy browser that gives the competiton a serious challenge
PreviousComments on this post
ekk it's a beast.
Good to see some figures though. I look forward to seeing some more of the results in the next few days.
Was there any concrete date for the Mac version Rupert ?
There was no date given for the Mac and Linux versions. They said they were "working hard" on them, and I believe that - Google does not want to be dependent on Windows.
One of the Mac-using Google people said he'd been running VMware Fusion a lot, so he could run Chrome.
You forgot to test it against the recent WebKit Nightly, which has a new JavaScript VM, and is faster than Chrome's V8, according to my tests.
The Mac and Linux versions aren't here yet. It's apparently possible to build them yourselves if you like, but there's only so much Charles and I (OK, Charles!) could do in the day we had to prepare for the Chrome launch. It's not quite as simple as you might think to load a lot of browsers into one machine and harvest all the benchmark data. I'll see if Charles can find the time to blog his experiences doing that, but he's got a nervous twitch in one eye now and a murderous gleam in the other.
The same is true of the Webkit nightly. We did spend some time trying to get the FF 3.1 nightly with TraceMonkey going, but it borked out on SunSpider 0.9 - a problem that didn't seem to be reflected in the regression bug reports, so we weren't quite sure what was going on there.
Very interested to hear other people's experiences with benchmarking browsers. It's the glory days of MS-DOS PCs all over again...
This comment has been deleted at the users request
I downloaded and installed Chrome yesterday evening, shortly after Petersaid that he had it. Everything went very smoothly, but...
I have AVG Internet Security installed, and I don't give blanket internet permission to anything new, I let the firewall ask me for a while so I can get a feel for what is going on. Ever since installed Chrome, I have periodically gotten a request from the "Google Installer" to connect to some IP address, even when I don't have Chrome running! Why in the world is it running something like that in the background all the time, independent of Chrome, what is it trying to do, and how do I stop it?
I don't want to rain on anyone's parade, and I agree that Chrome looks like an interesting new browser, at least in the few minutes that I tried it last night. But I'm not the least bit impressed if they are putting something on my laptop that is poking around, passing on information or whatever, especially when I don't even have Chrome running. I recall that there was a question about whether I was willing to participate in a feedback program, but I assumed that had to do with feedback about Chrome use, and I would NOT expect that it would be running, gathering information and passing it along when I am not using Chrome.
jw 3/9/2008
P.S. As I was writing the above, I realized that I was really not at all comfortable with what Chrome (or Google itself) was doing on my laptop, so I uninstalled it. As is typical when removing something from Google, it popped up a browser window asking for feedback about why I was doing such a thing. I told them, and pressed send. It replied that they would investigate, and they would contact me if they needed more information. Perhaps it was a poor choice of words, or perhaps I am just being paranoid... but there was nothing on that page which told them who I am or how to contact me, so how are they going to manage that feat? Have they already picked up more details about me than I ever wanted them to have? I'm even less pleased now than I was before...
In any case, I will leave it to braver souls than I to continue testing and reporting on Chrome.
jw 3/9/2008
I read Mr. Watson's post and I also had noticed HD activity when the desktop was up. I checked the scheduled tasks and found two google updaters. One was scheduled to run at logon, and the other when the machine was idle for 10 minutes.
Google has said that Chrome regularly (I think every few minutes) downloads the latest list of known hostile websites from the Googleplex.
The best thing about Chrome, and something I find very reassuring, is that because it's entirely open source it is impossible for Google to do something with it that won't be found out in short order. Either it puts it in the source code it releases to the world, in which case anyone who suspects something's up will find out quickly, or it issues a different executable than that which can be built from the public source - equally obvious to those looking.
It's quite in order to be paranoid about what Google gets up to with its servers. Where Chrome is concerned, it feels as if Google is going for maximum openness to attack Microsoft where Microsoft is most vulnerable - its business model. But I do feel that means that Chrome is inherently more trustworthy.
Oh, and latest news from those wot know - seems that TraceMonkey's latest nightly has fixed the iloop problem which broke Sunspider. It now runs - and is faster than V8 overall (although V8 still beats it on some individual tests).
Charles has been doing more testing the while, checking how Chrome compares to IE 7 on the smaller platform. He reports that "Numbers for Eee PC 900: SunSpider IE7 = 103775ms, Chrome = 5093ms; Memory footprint IE7 = 61132KB, Chrome = 25304."
Thanks for the explanation, Rupert. Regardless of the purpose, I will have to consider for a while whether I am satisfied with something running in the background and periodically going out to the internet, regardless of whether Chrome is actually running or not. It certainly seems like overkill to me, as long as Chrome is not my primary browser, and I am running it infrequently at best. Of course, it's all moot for the moment as far as I am concerned.
@ator - Thanks for the followup, too.
jw
I have been running it since last night and it is indeed fast. But its answer to the need to get information from many websites seems to be the thumbnails page which makes it easier to click through to them. I prefer just aggregating all the content from RSS feeds as Opera allows me. No equivalent in Chrome alas.
Ian Hendry
WeCanDo.BIZ
http://www.wecando.biz
Everyone has to decide what is best for them, and since this is still in beta I will wait and see what the final version is like. If it runs this fast on windows I imagine it will fly on MAC & Linux. I have it installed on 4 machines and have it set as the default browser, and plan on running it as hard as I can to see what develops. As for it going online I trust Google a LOT more than I trust that bunch in Redmond. I have found Vista doing something in the background a lot, I hope it isn't sending anything home.
The worst browser ever released and the worst of google betas.......Google betas have a reputation of being of very high quality but this beta is way out of line.......it cant even load the google home page properly(half logo missing) and even gmail was not loading..........and not to mention that most of the sites were not loading or were loading a bit and then stopped loading......i dont think i have any prob with my connection because all the pages which i tried in chrome opened smoothly in IE and firefox.........ANd not to mention the google's eula mentioing that it had permissio to update remove he features which it thinks is appropriate for the user........
Good luck guys ....
Chrome is NOT outright beating out Firefox... there's an example here where Firefox 3.0 is waaaaaaaaaaaaaayy faster than Chrome and using much less memory to boot:
http://www.wrensoft.com/forum/showthread.php?t=2789
And that's Firefox 3.0 - the current public version... not the latest beta with the new TraceMonkey engine which is supposed to be even faster.
Don't get too hung up on the benchmarks. Any optimising compiler will have its weak spots compared to other optimising compilers, and while the overall figure from any one benchmark may be interesting it doesn't show much more than how good the code is at running that benchmark.
What's important is that this new breed of browsers is capable of running web apps many times faster than we're used to, and that's game-changing. There's the old story about a scientist with a mainframe which took a day to run a particular task. Then he bought a new computer that did it in an hour. A salesman turned up with another model that did the task in thirty minutes, and said "Look! That problem you did in a day now takes half an hour. We're even twice as fast as your last computer".
"Nah," said the scientist. "My last computer saved me 23 hours. Yours only saves me a few minutes."
Now, Microsoft has an interesting question to answer. Does it improve its own browser performance, thus making Google apps seem more competitive, or does it sit on its hands, making the other browsers more competitive?
That IS an interesting dilemma for Microsoft. I suppose if one really suspected the worse in people, you could imagine that nothing would stop them from "optimizing" their browser to work slightly under par for Google apps ("oh it's traffic from google.com, let's take a few more seconds to think about that"), but better for everything else. Then again, you would think that's not something they could afford to risk with anti-trust suits and all that.
Agreed that most benchmarks only show how well optimized the application is at running that benchmark. But the measurements I linked to above are from a real-life search engine application. In fact, it's interesting that in ALL the other comparisons I've seen where the performance is measured in a real-life app, Firefox 3.0 and Safari has beaten out Chrome, and in some, Chrome was even behind Opera.
This comment has been deleted at the users request
This comment has been deleted at the users request
I've been reading about Google Chrome for a while, especially when I was investing in Google's stock. It definitely looks cool, though I'm going to hold off a tad as others test its functionality. Since I use most of my computers for work I don't want to explore on them.
I can't wait to see how Chrome progresses.
- Aaron Schoenberger
My wife's really taken with it since I popped it on to her rig. No complaints, it's working like a dream for her. For now I have gone back to FF3 with the tweak network settings add on (this is very fast). I've also developed a liking for apples safari browser. Apart from one hang up, (well they all do from time to time) all is well. I find safari tends to put a warm fuzzy edge onto everything it draws, a very pleasing effect.
For the record. It seems as though chrome is permanently conected to home.
If you go to srware.net, there is a version of Chrome called SRware Iron which has some of the insecure/annoying elements removed to create a more private surfing experience. These are explained in German (as it is a German site).
I have read somewhere that Chrome is really quite invasive in respect of our online activities and that the records may not be entirely anonymised.
Codeweavers have created a Chrome installation for Linux. On my computer there is a small problem in rendering some text, otherwise it works fine.
Like James, I like my privacy, such as it is, so I don't actually use Chrome other than as an occasional plaything.
I still really like it, but as with most early adopters, I guess, I've got a bunch of plug-ins in FF that I'd rather not live without. I use Chrome from time to time, but it's not ready to be the main browser.
I'm not too worried about the privacy aspects - as with FF, there's no danger that something really nasty is going on in secret, because you just can't do that with open source . You just can't do that anyway, if you keep an eye on your network traffic, but with open source you have a very high confidence that either you can track down the origin of suspicious activity or someone with much better analytical skills will do the job.
It's interesting that Google's obsession with speed has carried over to the browser in Android. We've been playing with that here for a couple of days on the G1 , and it's blazing. Twice as fast as the iPod (and that's on 3G), which makes a real difference. So far, everyone who's got their paws on the G1 would prefer it to the iPod for serious browsing, although that could be a function of the fact that everyone who's done that here is a grade-A geek with mixed feelings about Apple!
