benchmarkdotnet - powerful .net library for benchmarking
Post on 14-Apr-2017
97 Views
Preview:
TRANSCRIPT
BENCHMARKDOTNET - POWERFUL .NET LIBRARY
FOR BENCHMARKINGLarry Nung
AGENDAIntroductionGetting startedJobsColumnsDiagnosersExportersParamsSetupBaselineReferenceQ & A 2
INTRODUCTION3
INTRODUCTION A powerful .NET library for benchmarking
Standard benchmarking routine: generating an isolated project per each benchmark method; auto-selection of iteration amount; warmup; overhead evaluation; statistics calculation; and so on.
Supported runtimes: Full .NET Framework, .NET Core (RTM), Mono
Supported languages: C#, F#, and Visual Basic Supported OS: Windows, Linux, MacOS Easy way to compare different environments
(x86 vs x64, LegacyJit vs RyuJit, and so on; see: Jobs) Reports: markdown, csv, html, plain text, png plots. Advanced features: Baseline, Params Powerful diagnostics based on ETW events
(see BenchmarkDotNet.Diagnostics.Windows)4
GETTING STARTED5
INSTALLATION Install-Package BenchmarkDotNet
6
INSTALLATION
7
INSTALLATION
8
INSTALLATION
9
INSTALLATION
10
WRITE CODE TO BENCHMARKusing BenchmarkDotNet.Attributes; … public class ProgramBenchmarker { protected Program m_Program { get;
set; } = new Program(); [Benchmark] public void Test() { m_Program.Test(); } } …
11
RUN THE BENCHMARKusing BenchmarkDotNet.Running; … public class Program { static void Main(string[] args) { var summary =
BenchmarkRunner.Run<ProgramBenchmarker>();
} public void Test() {… } } …}
12
VIEW RESULTS
13
ANALYZE RESULTS
14
JOBS15
JOBS Describes how to run your benchmark
Avaliable Jobs DryJob ClrJob CoreJob MonoJob LegacyJitX86Job LegacyJitX64 RyuJitX64Job SimpleJob LongRunJob MediumRunJob ShortRunJob VeryLongRunJob
16
JOBSusing BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes.Jobs; ... [ShortRunJob] public class ProgramBenchmarker { ... }
17
JOBS
18
COLUMNS19
COLUMNS A column in the summary table
Available Columns NamespaceColumn MedianColumn MinColumn MaxColumn RankColumn e.t.c
20
COLUMNSusing BenchmarkDotNet.Attributes.Columns;…[NamespaceColumn] [MedianColumn] [MinColumn] [MaxColumn] [RankColumn][RankColumn(NumeralSystem.Roman)] [OrderProvider(SummaryOrderPolicy.FastestToSlowest)] public class ProgramBenchmarker { …} 21
COLUMNS
22
COLUMNSusing BenchmarkDotNet.Attributes; using BenchmarkDotNet.Columns; using BenchmarkDotNet.Configs; …[Config(typeof(Config))] public class ProgramBenchmarker { private class Config : ManualConfig { public Config() { Add(new TagColumn("HashCode", item =>
item.GetHashCode().ToString())); } } …}
23
COLUMNS
24
COLUMNSusing BenchmarkDotNet.Columns; using BenchmarkDotNet.Reports; using BenchmarkDotNet.Running;…public class HashCodeColumn : IColumn { public string ColumnName { get; } = "HashCode"; public HashCodeColumn() { } public bool IsDefault(Summary summary, Benchmark benchmark)
=> false; public string GetValue(Summary summary, Benchmark benchmark)
=> benchmark.Target.Method.Name.GetHashCode().ToString(); public bool IsAvailable(Summary summary) => true; public bool AlwaysShow => true; public ColumnCategory Category => ColumnCategory.Custom; public string Id { get; } = "1"; public int PriorityInCategory { get; } = 0; public override string ToString() => ColumnName; }
25
COLUMNS…[Config(typeof(Config))] public class ProgramBenchmarker { private class Config : ManualConfig { public Config() { Add(new HashCodeColumn())); } } …}
26
DIAGNOSERS27
DIAGNOSERS Can attach to your benchmark and get some
useful info
Available Diagnosers MemoryDiagnoser
28
DIAGNOSERSusing BenchmarkDotNet.Attributes; using BenchmarkDotNet.Configs; using BenchmarkDotNet.Diagnosers; ... [Config(typeof(Config))] public class ProgramBenchmarker { private class Config : ManualConfig { public Config() { Add(MemoryDiagnoser.Default); } } ... }
29
DIAGNOSERSusing BenchmarkDotNet.Attributes; using BenchmarkDotNet.Configs; using BenchmarkDotNet.Diagnosers; ... [MemoryDiagnoser] public class ProgramBenchmarker { ... }
30
DIAGNOSERS
31
EXPORTERS32
EXPORTERS Allows you to export results of your benchmark in
different formats.By default, files with results will be located in .\BenchmarkDotNet.Artifacts\results directory.
Available Exporters HtmlExporter CsvExporter MarkdownExporter AsciiDocExporter CsvMeasurementsExporter PlainExporter JsonExporter 33
CONFIGusing
BenchmarkDotNet.Attributes.Exporters;…[AsciiDocExporter] [CsvMeasurementsExporter] [PlainExporter] [JsonExporter] public class ProgramBenchmarker { …}
34
CONFIGusing BenchmarkDotNet.Configs; using BenchmarkDotNet.Exporters; using BenchmarkDotNet.Exporters.Csv; using BenchmarkDotNet.Exporters.Json; …[Config(typeof(Config))] public class ProgramBenchmarker { private class Config : ManualConfig { public Config() { Add(AsciiDocExporter.Default); Add(CsvMeasurementsExporter.Default); Add(PlainExporter.Default); Add(JsonExporter.Default); } } …}
35
HTMLEXPORTER
36
MARKDOWNEXPORTER
37
CSVEXPORTER
38
ASCIIDOCEXPORTER
39
CSVMEASUREMENTSEXPORTER
40
PLAINEXPORTER
41
JSONEXPORTER
42
PARAMS43
PARAMSusing BenchmarkDotNet.Attributes;... public class ProgramBenchmarker { [Params(100, 200)] public int Parameter { get; set; } protected Program m_Program { get; set; }
= new Program(); [Benchmark] public void Test() { m_Program.Test(); } } 44
PARAMS
45
SETUP46
SETUPusing BenchmarkDotNet.Attributes; ... public class ProgramBenchmarker { protected Program m_Program { get; set; } [Setup] public void Setup() { m_Program = new Program(); } [Benchmark] public void Test() { m_Program.Test(); } }
47
BASELINE48
BASELINEusing System.Threading; using BenchmarkDotNet.Attributes; ... public class ProgramBenchmarker { protected Program m_Program { get; set; } = new Program(); [Benchmark(Baseline = true)] public void Test1() { m_Program.Test(); Thread.Sleep(10); } [Benchmark] public void Test2() { m_Program.Test(); Thread.Sleep(20); } }
49
BASELINE
50
REFERENCE51
REFERENCE NuGet Gallery | BenchmarkDotNet 0.10.3
https://www.nuget.org/packages/BenchmarkDotNet
dotnet/BenchmarkDotNet: Powerful .NET library for benchmarking https://github.com/dotnet/BenchmarkDotNet
Home - BenchmarkDotNet Documentation http://benchmarkdotnet.org/
52
Q&A53
QUESTION & ANSWER
54
top related