Benchmarking Android 5.0 Lollipop performance on the Galaxy S5, HTC One (M8), LG G3, and Nexus 5

Most major manufacturers recently rolled out the Android 5.0 Lollipop update to their respective flagships and we were more then eager to test how their performance has changed now that Lollipop is here. The speed comparison will be done on our usual battery of synthetic benchmarks.

So, does Lollipop improve performance or make it worse? Read on to find out.

Our expectations were high as Android 5.0 made the jump to a new runtime, ART, which replaces the Dalvik. Dalvik uses Just-in-time compilation – after the app is already running, so it has to be done quickly, leaving little time for optimization.

ART meanwhile does ahead-of-time compilation – that “Optimizing apps” screen you get at first boot. You may have noticed that it takes longer than before, the reason being the phone is compiling the apps and spending time to optimize them properly.

This requires a word of warning too – as ART is a new runtime, some apps suffer from some issues, but with multiple benchmarks a issue with a single test shouldn’t be a problem overall.

Ready, set, go! The first lap is around the Geekbench 3 ring and the Galaxy S5 keeps in top position with a small increase in score. The HTC One (M8) and Nexus 5 got much bigger boosts, while the LG G3 has actually gotten worse.

GeekBench 3

Higher is better

• Samsung Galaxy S5 (Lollipop)
  3120
• Samsung Galaxy S5 (KitKat)
  3011
• HTC One (M8) (Lollipop)
  2923
• Nexus 5 (Lollipop)
  2743
• LG G3 (KitKat)
  2563
• Nexus 5 (KitKat)
  2453
• LG G3 (Lollipop)
  2370
• HTC One (M8) (KitKat)
  2367

We look to AnTuTu 5 for overall performance, where all phones got a small boost. The Nexus 5 in particular got a massive boost.

AnTuTu 5

Higher is better

• HTC One (M8) (Lollipop)
  45530
• Samsung Galaxy S5 (Lollipop)
  45348
• HTC One (M8) (KitKat)
  44020
• Samsung Galaxy S5 (KitKat)
  43164
• LG G3 (Lollipop)
  42038
• LG G3 (KitKat)
  39905
• Nexus 5 (Lollipop)
  37774
• Nexus 5 (KitKat)
  29500

Basemark OS II is another full-system benchmark and it shows improvements across the board ranging from small (One (M8)) to big (LG G3).

Basemark OS II

Higher is better

• Nexus 5 (KitKat)
  1351
• LG G3 (Lollipop)
  1327
• HTC One (M8) (Lollipop)
  1202
• Samsung Galaxy S5 (Lollipop)
  1147
• HTC One (M8) (KitKat)
  1126
• LG G3 (KitKat)
  1126
• Samsung Galaxy S5 (KitKat)
  1082
• Nexus 5 (Lollipop)
  948

Single and multi-core performance is a bit of a mixed bag, but scores largely remained unchanged. Besides CPU performance, the benchmark also tests the internal storage, the graphics and web performance. It’s not clear which of these contributed to the performance bump as Lollipop did little for web performance.

Basemark OS II (single-core)

Higher is better

• Samsung Galaxy S5 (Lollipop)
  2510
• HTC One (M8) (KitKat)
  2428
• HTC One (M8) (Lollipop)
  2426
• Samsung Galaxy S5 (KitKat)
  2415
• LG G3 (Lollipop)
  2267
• LG G3 (KitKat)
  2213

Basemark OS II (multi-core)

Higher is better

• Samsung Galaxy S5 (KitKat)
  10063
• LG G3 (Lollipop)
  9975
• HTC One (M8) (KitKat)
  9860
• HTC One (M8) (Lollipop)
  9827
• Samsung Galaxy S5 (Lollipop)
  9646
• LG G3 (KitKat)
  9611

Going to GFX benchmark for a look at 3D performance we see that very little has changed. The new Android version brings the Android Extension Pack, which promises PC-level graphics. These versions of the benchmark don’t support the new APIs and most games don’t either (though Unreal did demo the tech on the Tegra K1).

GFX 2.7 T-Rex (onscreen)

Higher is better

• HTC One (M8) (KitKat)
  30.1
• HTC One (M8) (Lollipop)
  30
• Samsung Galaxy S5 (KitKat)
  28.1
• Samsung Galaxy S5 (Lollipop)
  28
• Nexus 5 (KitKat)
  24
• Nexus 5 (Lollipop)
  24
• LG G3 (KitKat)
  20.5
• LG G3 (Lollipop)
  20

GFX 3.0 Manhattan (onscreen)

Higher is better

• HTC One (M8) (Lollipop)
  13
• Samsung Galaxy S5 (Lollipop)
  12
• HTC One (M8) (KitKat)
  11.9
• Samsung Galaxy S5 (KitKat)
  11.7
• Nexus 5 (Lollipop)
  10
• LG G3 (Lollipop)
  7.7
• LG G3 (KitKat)
  7.2

Basemark X didn’t see much of a change in performance either, aside from the LG G3, which slipped back a bit.

Basemark X

Higher is better

• HTC One (M8) (KitKat)
  12396
• HTC One (M8) (Lollipop)
  12257
• Samsung Galaxy S5 (Lollipop)
  11798
• Samsung Galaxy S5 (KitKat)
  11744
• Nexus 5 (KitKat)
  11638
• LG G3 (KitKat)
  11552
• LG G3 (Lollipop)
  10580
• Nexus 5 (Lollipop)
  9786

For web browsing it seems that Lollipop brought some regressions in Kraken 1.1. The Nexus 5 slowed down significantly, the LG G3 also took a hit. The HTC One (M8) is the big winner here with a noticeably jump forward in times.

Kraken 1.1

Lower is better

• Samsung Galaxy S5 (Lollipop)
  5968
• Samsung Galaxy S5 (KitKat)
  6043
• LG G3 (KitKat)
  6987
• HTC One (M8) (Lollipop)
  7023
• Nexus 5 (KitKat)
  7148
• LG G3 (Lollipop)
  7632
• Nexus 5 (Lollipop)
  9524
• HTC One (M8) (KitKat)
  10296

BrowserMark 2.1 on the other hand saw some gains, big ones for the Galaxy S5. The LG G3 stood in place, posting a virtually identical result.

BrowserMark 2.1

Higher is better

• Samsung Galaxy S5 (Lollipop)
  2066
• LG G3 (KitKat)
  1474
• HTC One (M8) (Lollipop)
  1460
• LG G3 (Lollipop)
  1453
• Nexus 5 (Lollipop)
  1440
• Samsung Galaxy S5 (KitKat)
  1398
• Nexus 5 (KitKat)
  1286
• HTC One (M8) (KitKat)
  1069

Android 5.0 Lollipop carried some promise of speed and while there are some improvements, this is certainly not like when Android 2.2 Froyo changed the Just-in-time compilation (JIT) engine to Dalvik back in the day.

Of course, benchmarks poorly capture some of the differences between the two runtimes. JIT typically compiles only the most intensive code – like a benchmark – because this is done in real-time while the app is running, compiling the whole thing will slow down the app. Also the compiled code is cached but not kept permanently so apps may have to be re-JITed.

Lollipop also promised better battery life, we’re in the process of testing that too (see you again next week).