Our Intel SysMark Article Ruffles
Date: August 16, 2001
We produced an
this week citing optimizations that favored Intel processors
in SysMark 2000's Photoshop tests. After outlining the
intimate relationship between these two organizations we repeated our warning
that SysMark and its BAPCo brethren must be taken cautiously and with suspicion.
Apparently OverClockers.com felt offended by this
assertion and posted something of a
I was alerted to this article while perusing the message
board at Ace's Hardware. Instead of recreating the argument, here is
my post slightly edited to correct typos and formatting. A few additional
comments were also added in delimited fashion. The original posting can be
ARGH! Against my better judgment and your sane
advice, I'm replying simply because this issue means so much to me.
Ed, who I don't know, claims that I have an agenda. What agenda I do
have is to root out deception. And it's pretty clear that there is a level of
obfuscation at least going on here. Ed managed to avoid mentioning the
history of BAPCo and the other details in my article.
I don't care if a benchmark has SSE or whatever, but it needs to be made clear
up front or the benchmark is absolutely useless to an evaluator -- at least a
conscientious evaluator that desires to know upon what foundation his data is
Any benchmark that just spins and spits out some number is worthless or even
dangerous unless the reviewer can have some clue as to what's going on under
As far as the P4 is concerned, it *did* perform poorly in SysMark 2000, but
that it fares better in SysMark2001 is no coincidence.
I don't know why he's [Ed's] trying to bury the connection between BAPCo and
Intel. What good does this do?
I don't know Ed, and I haven't talked to him, but he is wrong when he says I
wrote about COSBI four months ago -- that was more than a year ago -- May,
2000 at Tom's Hardware.
Now I'd like to know how I'm supposed to benefit from COSBI. There is no money
involved in it at all, but a whole lot of work. I have spoken with a lot
of people to get their support, Calibrator, Latency2, Sphinx, ScienceMark (or
at least help from Tim). The only reason I push it is because no one
else is doing anything like this and it needs to get done.
I am a fanatic? I am trying to advance my hidden "cause"? I'd
really like to know what that is supposed to be. It seems Ed's whole
premise is based upon the assumption that I am profiting from COSBI and this
is crazy. Ask Tim Wilkens or Julian [Ruhe] or anyone else who knows me.
Honestly, I don't know what Ed is angry about. Perhaps it is because his site
had something up about the relative performance of the Palomino and the Athlon
using SysMark a while back. But if I remember correctly, c't beat his
site to this fact as well.
The point of my article was to make a connection that was not made before and
that I think this point is too important to overlook. The rumors that had
persisted for well over a year is that the specific Photoshop filters applied
in SysMark were cherry picked for their SSE optimizations. This appears to be
true and given BAPCo's connection to Intel it is easy to see why this was
Here is a note that I just sent Ed:
I read your post, but I’m afraid you are off base.
A benchmark is worthless unless the evaluator has some clue as to what is
being tested. The SSE optimized Photoshop filters would be fine if it
had been stated clearly that this is what was being stressed to the exclusion
of 3dNow! Is it the ALU or the FPU or memory latency or bandwidth or perhaps
SSE is better or worse than 3dNow! – maybe you are smarter than I and can see
these details, but I cannot.
When I review hardware I don’t like to crank up some product that spits out a
“dumb” number, a number whose significance is completely unclear.
Personally, I feel an obligation to my readership, people who potentially base
buying decisions on my recommendations. I desire to know as much about
the benchmark results that I am citing as possible. I wouldn’t and don’t
trust such black box benchmarks from anyone, much less an organization with
clear ties to Intel [or any other company that tries to trick people into
using tools -- that secretly come from them -- to evaluate their and their
You mention the P4 -- it did do poorly in SysMark2000 where the PIII
was strong [and at a time when the Pentium III was Intel's bread and butter],
but it is no coincidence that SysMark 2001 makes the P4 look much better.
Just in case you need something to jar your memory, here are results from a
And I don’t know why you are trying to bury the BapCo connection with Intel –
what good is it to anyone to do this?
I would also like to know how COSBI is supposed to benefit me – it’s a lot of
work and there is no money involved. For years, I have been soliciting
help from everyone that I thought would be interested – the project is way too
big for me. I am just one man trying to act in good conscience.
I am not even making any money from my site, although I certainly wouldn’t
mind. I’m maintaining my site because I am writing about things I care
about and am interested in.
Forgive me, but I think you are too cynical. I have no hidden agendas and am
trying to act in as honest a way as I know how.
Donation Page is up.