JavaのマイクロベンチマークツールJMHを試してみた

apply plugin: "java"

group 'microbenchmark-sample'
version '1.0-SNAPSHOT'

sourceCompatibility = 1.8

// gradleプロジェクトでjmhを使えるようにする設定
buildscript {
    repositories {
        maven {
            url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
        classpath "me.champeau.gradle:jmh-gradle-plugin:0.4.7"
    }
}
apply plugin: "me.champeau.gradle.jmh"

// ライブラリのダウンロード先指定
repositories {
    mavenCentral()
}

// jmhの依存関係解決
dependencies {
    compile 'org.openjdk.jmh:jmh-core:1.21'
}

// jmhの設定
jmh {
    jmhVersion = '1.21'
    warmupIterations = 5
    iterations = 20
    fork = 2
    benchmarkMode = ['thrpt']
    failOnError = true
}

jmhJar {
    destinationDir = projectDir
}

~/w/bcmk-sample> mkdir -p src/main/java/bcmk
~/w/bcmk-sample> touch src/main/java/bcmk/App.java

package bcmk;

import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
import java.util.Random;

public class App {

    // 乱数生成用
    private final static Random random = new Random();

    /**
     * if文のやつ
     * @return
     */
    public Optional<String> getUserName_legacy() {
        List<String> userNameList = getUserNameList();

        if(userNameList.isEmpty()) {
            return Optional.empty();
        } else {
            return Optional.of(userNameList.get(0).toUpperCase());
        }
    }

    /**
     * 三項演算子
     * @return
     */
    public Optional<String> getUserName_old() {
        List<String> userNameList = getUserNameList();

        return userNameList.isEmpty()
                ? Optional.empty()
                : Optional.of(userNameList.get(0).toUpperCase());
    }

    /**
     * Stream使う
     * @return
     */
    public Optional<String> getUserName_stream() {
        List<String> userNameList = getUserNameList();

        return userNameList.stream().findFirst().map(s -> s.toUpperCase());
    }

    /**
     * 空リストか要素入りのリストが返される..DBからの取得の代用(計測の本質は DBじゃないため
     * @return
     */
    private List<String> getUserNameList() {
        List<String> userNameList = new ArrayList<>();
        userNameList.add("taro");
        userNameList.add("moge");

        return random.nextBoolean()
                ? new ArrayList<String>()
                : userNameList;
    }
}

~/w/bcmk-sample> mkdir -p src/jmh/java/bcmk
~/w/bcmk-sample> touch src/jmh/java/bcmk/SampleBenchmark.java

package bcmk;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.State;

import java.util.concurrent.TimeUnit;


@State(Scope.Thread)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
public class SampleBenchmark {

    @Benchmark
    public void if文の古いコード() {
        final bcmk.App m = new bcmk.App();
        System.out.print("Result: " + m.getUserName_legacy().orElse("該当ユーザーはいませんでした"));
    }

    @Benchmark
    public void 三項演算子() {
        final bcmk.App m = new bcmk.App();
        System.out.print("Result: " + m.getUserName_old().orElse("該当ユーザーはいませんでした"));
    }

    @Benchmark
    public void Stream使ったやつ() {
        final bcmk.App m = new bcmk.App();
        System.out.print("Result: " + m.getUserName_stream().orElse("該当ユーザーはいませんでした"));
    }
}

~/w/bcmk-sample> ./gradlew clean jmh

Result "bcmk.SampleBenchmark.三項演算子":
  134.958 ±(99.9%) 10.147 ops/ms [Average]
  (min, avg, max) = (94.187, 134.958, 167.342), stdev = 18.036
  CI (99.9%): [124.811, 145.104] (assumes normal distribution)


# Run complete. Total time: 00:25:04

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                     Mode  Cnt    Score    Error   Units
SampleBenchmark.Stream使ったやつ  thrpt   40  154.498 ±  9.895  ops/ms
SampleBenchmark.if文の古いコード    thrpt   40  146.304 ±  7.848  ops/ms
SampleBenchmark.三項演算子        thrpt   40  134.958 ± 10.147  ops/ms

BUILD SUCCESSFUL in 25m 12s
6 actionable tasks: 1 executed, 5 up-to-date
18:03:17: Task execution finished 'jmh'.

// jmhの設定
- jmh {
-    jmhVersion = '1.21'
-    warmupIterations = 5
-    iterations = 20
-    fork = 2
-    benchmarkMode = ['thrpt']
-    failOnError = true
- }

// モードを消します(コードで設定します)
// 処理時間かかるので, iterationsとかも落とします
+ jmh {
+    jmhVersion = '1.21'
+    warmupIterations = 2
+    iterations = 5
+    fork = 2
+    failOnError = true
+ }

    @Benchmark
+    @BenchmarkMode(Mode.AverageTime)
    public void Stream使ったやつ() {
        final bcmk.App m = new bcmk.App();
        System.out.print("Result: " + m.getUserName_stream().orElse("該当ユーザーはいませんでした"));
    }

Result "bcmk.SampleBenchmark.三項演算子":
  0.008 ±(99.9%) 0.002 ms/op [Average]
  (min, avg, max) = (0.007, 0.008, 0.011), stdev = 0.001
  CI (99.9%): [0.006, 0.010] (assumes normal distribution)


# Run complete. Total time: 00:07:04

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                    Mode  Cnt  Score    Error  Units
SampleBenchmark.Stream使ったやつ  avgt   10  0.007 ±  0.001  ms/op
SampleBenchmark.if文の古いコード    avgt   10  0.006 ±  0.001  ms/op
SampleBenchmark.三項演算子        avgt   10  0.008 ±  0.002  ms/op

BUILD SUCCESSFUL in 7m 6s
6 actionable tasks: 1 executed, 5 up-to-date
18:15:01: Task execution finished 'jmh'.