System Performance Revisited

Now that we’ve covered battery life we can revisit another topic where our testing has changed dramatically for 2016, which is our system performance benchmarks. As previously mentioned this year a major goal of ours was to focus on benchmarks with metrics that better indicate user experience rather than being subject to additional layers of indirection in addition to updating our previously used benchmarks. Probably one of the hardest problems to tackle from a testing perspective is capturing what it means to have a smooth and fast phone, and with the right benchmarks you can actually start to test for these things in a meaningful way instead of just relying on a reviewer’s word. In addition to new benchmarks, we’ve attempted to update existing types of benchmarks with tests that are more realistic and more useful rather than simple microbenchmarks that can be easily optimized against without any meaningful user experience improvements. As the Galaxy S7 edge is identical in performance to the Galaxy S7, scores for the Galaxy S7 edge are excluded for clarity.

JetStream 1.1

Kraken 1.1 (Chrome/Safari/IE)

WebXPRT 2015 (Chrome/Safari/IE)

In browser/JavaScript performance the Galaxy S7 in its Snapdragon 820 variants performs pretty much as you'd expect with fairly respectable performance about on par with the iPhone 6 at least part of the time, which frankly still isn't enough but a lot of this is more due to Google's lack of optimization in Chrome than anything else. The Exynos 8890 version comes a lot closer but it still isn't great. Subjectively browsing performance on the Galaxy S7 with the Snapdragon 820 is still painful with Chrome, and I have to install either a variant of Snapdragon Browser or Samsung's stock browser in order to get remotely acceptable performance. Even then, performance isn't great when compared to Apple's A9-equipped devices. The lack of single thread performance relative to other devices on the market in conjunction with poor software optimization on the part of Google is really what continues to hold OEMs back here rather than anything that Samsung Mobile is capable of resolving.

PCMark - Work Performance Overall

PCMark - Web Browsing

PCMark - Video Playback

PCMark - Writing

PCMark - Photo Editing

PCMark shows that the Galaxy S7 is generally well-optimized, with good performance in native Android APIs, although devices like the OnePlus 3 pull ahead in general, likely due to differences in DVFS, lower display resolution, more RAM, and similar changes as the hardware is otherwise quite similar. In general though unless you get something with a Kirin 95x in it you aren't going to get performance much better than what you find in the Galaxy S7, although the software optimization in cases like the writing test could be better for the Snapdragon 820 version of the phone.

DiscoMark - Android startActivity() Cold Runtimes

DiscoMark - Android startActivity() Hot Runtimes

As hinted by the PCMark results, the Galaxy S7 with the Snapdragon 820 is really nothing to write home about when it comes to actual software optimizations, while the Exynos 8890 version is significantly faster in comparison. The fastest devices by far here are still the Kirin 950-equipped phones, but even from cold start launches the HTC 10 is comparable, and pulls ahead slightly when the applications are pre-loaded into memory. The OnePlus 3 and Xiaomi Mi5 are closer to what the S820 GS7 should be achieving, which is really more a testament to just how strangely slow the Galaxy S7 with Snapdragon 820 is.

Overall though, the Galaxy S7 in both iterations are acceptably fast for general purpose tasks. However, with that said the Snapdragon 820 variant is noticeably slower, and the software stack seems to be less optimized for whatever reason even after multiple post-launch OTAs and all the latest app updates. Given that these devices have locked bootloaders it's difficult to really go deep and try to figure out exactly what's causing these issues, but it's likely that Samsung Mobile has the engineering staff to do this and resolve these issues as a 600 USD phone really shouldn't be performing worse than a 400 USD phone. On the bright side, the Exynos 8890 variants perform quite well here, with performance comparable to top devices and often beating out Snapdragon 820 devices, although usually not by a huge margin.

Introduction and Battery Life Revisited System Performance Cont'd
Comments Locked

266 Comments

View All Comments

  • Buk Lau - Tuesday, July 5, 2016 - link

    Then do you think there'll be a second opinion review on these two? Because now there's a clear divide among AT's reviewers on how to evaluate the quality of still images. Either you guys can conform to a more uniform standard or it should be noted that the evaluation of these are still quite heavily subjective. I just don't think there's should be that many inconsistencies if you guys are only publishing one cumulative review for each device
  • fanofanand - Wednesday, July 6, 2016 - link

    Excellent, honest answer. People need to keep in mind that a LOT of stuff when it comes to tech (GUI etc.) comes down to subjective preference. Josh clearly likes Apple products and it shows in his reviews. Some won't like it, others will. It's near impossible to have a 100% objective review. I abhor the time to publish as of late, but for the most part their reviews are still second to none. For the most part.
  • ikjadoon - Tuesday, July 5, 2016 - link

    "Given that these devices have locked bootloaders it's difficult to really go deep and try to figure out exactly what's causing these issues, but it's likely that Samsung Mobile has the engineering staff to do this and resolve these issues as a 600 USD phone really shouldn't be performing worse than a 400 USD phone."

    And this is the crux of the problem Android market in 2016! A 50% increase in price...means maybe a 10% decrease in performance?

    We talk about the Android ecosystem giving choices (and I've benefited from that choice, having owned 4 Android devices from 3 different manufacturers)....price and performance should be related.

    At a certain point (certainly when you get closer to $1,000), these devices should be making meaningful upgrades over cheaper brethren.

    Otherwise...what are we paying for? Maybe Samsung's relatively good update support? Water-proofing?
  • theduckofdeath - Tuesday, July 5, 2016 - link

    Just in time for the Galaxy Note 7 release!

    By the way. Has this site ever delved into the extreme energy usage accompanied by excellent performance numbers when benchmarking apps are running on the iPhones? Or is that kind of cheating only an issue when it's done by a random Android OEM?
  • MonkeyPaw - Tuesday, July 5, 2016 - link

    As an LG G4 owner, I find it odd that its wifi browsing battery score is worse than the LTE. My experience has been the opposite for the many months I've operated the device.
  • theduckofdeath - Tuesday, July 5, 2016 - link

    I guess the reviewer screwed this up like most other things when reviewing non-Apple products. Samsung phones use LTE and Wifi simultaneously by default when you enable Wifi.
    I mean, the review just reeks of nias with a natural negative stance to anything not Apple. Hopefully this will be the last review posted by Joshua. If not for his persistent bias towards anything not-Apple but for the disgracefully delayed post of this review. The phone is almost six months old and it's the biggest selling phone of the year....
  • theduckofdeath - Tuesday, July 5, 2016 - link

    *bias
  • Matt Humrick - Wednesday, July 6, 2016 - link

    The purpose of these specific battery tests is to measure the difference in battery life when using WiFi vs. LTE, which is why we do not run it with them on simultaneously.
  • Matt Humrick - Wednesday, July 6, 2016 - link

    I've updated the G4's LTE battery life. The first time I ran the G4 was before we established our target value for signal strength, and the value I used was too strong. I need to rerun the Nexus 6P too.
  • Bonedatt - Tuesday, July 5, 2016 - link

    "I’m left wondering what Samsung would be capable of if they cared about getting things right even if no one would notice the extra frame drop or 50ms of roam latency".... This is how I feel about Samsung too. I've never owned an iPhone but I just have this feeling that Samsung continues to hold back. Either that or they don't do a good job in showcasing their efforts

Log in

Don't have an account? Sign up now