The iPad Pro vs. the Surface Pro 3: We check Apple's lofty performance claims

 The iPad Pro will offer “desktop-class performance” and have a CPU faster than 80 percent of portable PCs shipped.
If you just fell off your chair, it’s understandable. Those claims made by Apple about its newest tablet are truly bold. They’re also pretty hard to believe for anyone who believes in the inarguable performance supremacy of the PC and x86 over tablets and ARM.
Let’s take a closer look at exactly what Apple’s VP of marketing Phil Schiller said about the iPad Pro during the company’s big unveil last Wednesday. During his talk, Schiller said the new 64-bit A9X SoC doubled memory bandwidth, doubled storage read and storage, and offered 1.9X the performance of the iPad Air 2. 

What does this even mean?

”This is desktop-class performance,” Schiller boasted. ”It is faster than 80 percent of the portable PCs that shipped in the last 12 months, that’s at CPU tasks. At graphics tasks, it’s faster than 90 percent of them.”
Schiller did seem to qualify the statements that the GPU performance really sang using Apple’s new Metal but didn’t specifically say Metal support was required to be “better than 90 percent” of portable PCs.
He continued to say: “When you run tasks and applications that we all love to use, they get incredibly fast. For example, running iMovie on an iPad Pro delivers desktop-class performance. With iMovie you can now edit three streams of 4K video simultaneously.”
Schiller also demonstrated AutoCAD 360, showing wireframe mesh with 320,000 objects and saying: “That’s something you can’t do on the PC.”

Shades of the snail

 

Years ago, Apple proudly proclaimed the PowerPC G3 chip far faster than Intel’s Pentium II. The truth, besides a few cherry-picked tests, was quite the opposite though.

When I heard Schiller’s claims, it reminded me of one of Apple’s previous claims: that the PowerPC G3-based Mac made a monkey out of the PC. Citing results from the now-defunct magazine Byte, Apple said the new Macintosh was twice as fast as a 350MHz Pentium II. The truth, of course, was quite different. Even some Macintosh partisan magazines of the era agreed that at best, it was maybe 19 percent faster in Photoshop and other real-world applications and got as good as it gave against a Pentium II chip. That didn’t stop Apple from picking fights with Intel: One of its more memorable ads showed a garden snail crawling along with a Pentium II on its back. 

What exactly is “80 percent of portable PCs?”

The problem with Apple’s claims is the lack of detail. What did Schiller mean by “Portable PC?” Does that figure include Atom-based tablets? Does it include Chromebooks? Does it count Macbooks? I reached out to Apple for clarification. A few days on, I still haven’t heard back, and I suspect I won’t.
I decided to approach it from another angle and figure out what 80 percent of the portable PC market looks like today. Neither Intel nor AMD disclose that kind of granularity to the public, so I spoke with analyst Dean McCarron of Mercury Research. McCarron closely follows the chip market using public and private data civilians can’t access.
He said if you exclude Atom and count only Core i3, Pentium and Celeron, plus some of the Core i5 CPUs, and then lump in AMD APUs, you get pretty close to 75 percent sold, which is pretty close to Apple’s 80 percent number. 
“I can’t speak for Apple, but if they’re saying faster than 80 percent of notebook PCs in the past 12 months, it would have to compete with Haswell dual-core pretty far up the Haswell product stack—not all the way, but at least past most i3s and some i5,” McCarron said. “Again, they’re in control of their own metrics for measurements, and it’s pretty much the industry norm to pick ones that show your product most favorably.”
Because Intel moves the most laptop chips, I’m pretty certain Apple feels the iPad Pro stands up even to Intel’s Haswell and Broadwell CPUs. The A8X is already generally faster than the bulk of the Atom-based chips .

iPad vs. Surface: Fight!

To see how Apple’s fastest tablet stacks up against the Surface Pro 3, I dug up performance numbers I’ve run internally to compare to some publicly available data. In this story I wrote when BAPCo’s TabletMark v3 first came out, we can see how the iPad Air 2 and the A8X rate.
Tabletmarket V3 simulates photo editing, web browsing and other typical tablet functions. It’s cross-platform and obviously runs on iOS, Android and Windows. Tabletmark V3 doesn’t leverage Apple’s newer Metal API, which is like the company’s take on DirectX 12 and OpenCL rolled into one and could show a nice bump in performance.
To be fair to the PC, the Surface Pro 3 is hardly the fastest thing out there. A typical two-year old laptop will be faster. Still, the dual-core Core i5-4300U Haswell CPU in the Surface Pro 3 soundly destroys all of the ARM-powered devices.  

PCWORLD

The Core i5-4300U easily thumps all of its ARM comptetitors, but if the iPad Pro is “1.9X” faster, will it be just as fast?

As another point of reference, I used the results of MacWorld’s review of the iPad Air 2 to compare to my own test of a Surface Pro 3 with the Core i5-4300U chip. In these multi-core 32-bit results, the dual-core Haswell again wallops the iPad Air 2, but it’s surprisingly closer than I expected.
Geekbench 3 is a cross-platform test and measures integer, floating point and memory bandwidth performance. The developer says the workloads are "real-world" and are similar to what an actual program would stress when decompressing a JPEG file, or running encryption or the like. This isn't quite the same,as say, running Maxon's CineBench R15 which uses its own 3D rendering engine as part of its benchmark.

The iPad Air 2 loses to the Surface 3 Pro, but it’s closer than I expected

While the iPad Air 2 does quite well for an ARM chip,I'd hazard a guess that if you could run CineBench R15 on the iPad Air 2, it wouldn't fare as well. Still, people will want to see these Geekbench numbers because they're readily available on all platforms.
Here’s one last comparison, using 3DMark’s Ice Storm Unlimited gaming test. I made the chart using data from a Surface Pro 3 I tested and public data for the iPad Air 2.

The Core i5-4300U trounces the iPad Air 2 but the iPad Pro is 1.9x faster right?

Parsing the data

So does this mean that the iPad Pro’s performance boast of 1.9X over the iPad Air 2 means instant equal footing with a dual-core Haswell or Broadwell chip? 
It’s hard to believe, but Schiller could be right from a certain point of view.
I base this on statements the company has made regarding what it thinks the iPad Pro can do. For example, editing three simultaneous streams of 4K video on an iPad Pro is apparently possible now. Schiller cited this as proof of the iPad Pro’s “desktop-like” performance. That task would indeed be tough on 80 percent of the portable PCs out there, and a good chore for a desktop too.
At the same time, Apple’s demo used iMovie, its in-house video editor, which I’d guess is highly optimized for its hardware. And let’s face it, iMovie is not Adobe Premiere Pro by any stretch of the imagination.
That actually brings up the big disconnect with Apple’s boasts of the iPad Pro’s performance. One task that Schiller said you can’t do is run AutoCAD 360 on a PC with 320,000 objects in a mesh and smoothly pan around. Probably. But after looking at the multiple 1-star reviews of Autodesk’s free mobile AutoCAD 360 app in the Windows Store (Yes, it’s a Metro app), I’d rather just skip it and install AutoCAD 2016.

The free Metro-based AutoCAD360 doesn’t get highmarks on the PC.

iPad Pro can’t do that

And that’s the big difference here. You can install AutoCAD 2016 on a large majority of the “portable PCs” being sold today. And you can also install Photoshop, Premiere Pro and Office.
I’m pretty certain you can’t install the full versions of Photoshop, Premiere, AutoCAD, and Office on the iPad Pro and go to town. That’s because it’s not a PC. It’s not even a Mac. It’s a tablet that runs limited-functionality software really, really well.
Even if the iPad Pro is the fastest tablet when released, and an incredible testament to Apple’s mastery of disciplines in hardware and software, it won’t really be faster than “80 percent of Portable PCs.” That’s because it can’t do things those 80 percent of PCs can do.