Functions

Sampler Structs

Each sampler is associated with a MCMC sampler struct, which is a subtype of MCMCSampler. In MCMCBenchmarks, we define subtypes of MCMCSampler for three popular NUTS MCMC samplers in Julia: CmdStan, AdvancedHMC via Turing, and DynamicHMC.

MCMCBenchmarks.MCMCSampler — Type.

Abstract MCMC sampler type

source
MCMCBenchmarks.CmdStanNUTS — Type.

MCMC sampler struct for CmdStan NUTS

  • model: model configuration
  • dir: probject directory
  • id: a unique identifier for each instance of CmdStan in parallel applications
  • name: a unique identifer given to each sampler
source
MCMCBenchmarks.AHMCNUTS — Type.

MCMC sampler struct for AdvancedHMC NUTS

  • model: model function that accepts data
  • config: sampler configution settings
  • name: a unique identifer given to each sampler
source
MCMCBenchmarks.DHMCNUTS — Type.

MCMC sampler struct for DynamicHMC NUTS

  • model: model function that accepts data
  • config: sampler configution settings
  • name: a unique identifer given to each sampler
source

Functions

Top-level Benchmark Routines

MCMCBenchmarks.benchmark — Function.

Runs the benchmarking procedure and returns the results

  • samplers: a tuple of samplers or a single sampler object
  • simulate: model simulation function with keyword Nd
  • Nreps: number of times the benchmark is repeated for each factor combination
source
benchmark!(samplers,results,csr̂,simulate,Nreps,chains;kwargs...)

benchmark!(sampler::T,results,csr̂,simulate,Nreps,chains;kwargs...) where {T<:MCMCSampler}
MCMCBenchmarks.pbenchmark — Function.

Runs the benchmarking procedure defined in benchmark in parallel and returns the results

  • samplers: a tuple of samplers or a single sampler object
  • simulate: model simulation function with keyword Nd
  • Nreps: number of times the benchmark is repeated for each factor combination
source
pbenchmark(samplers,simulate,Nreps;kwargs...)
MCMCBenchmarks.benchmark! — Function.

Primary function that performs mcmc benchmark repeatedly on a set of samplers and records the results.

  • 'sampler': tuple of sampler objects
  • results: DataFrame containing benchmark results
  • csr̂: cross sampler r̂
  • simulate: data generating function
  • Nreps: number of repetitions for a given set of simulation parameters. Default = 100
  • kwargs: optional keyword arguments that are passed to modifyConfig!, updateResults! and

runSampler, providing flexibility in benchmark simulations.

source
benchmark!(samplers,results,csr̂,simulate,Nreps,chains;kwargs...)

Overloaded Benchmark Functions

MCMCBenchmarks.modifyConfig! — Function.

Modifies MCMC sampler configuration, including number of samples, target acceptance rate and others depending on the specific sampler.

  • s: sampler object
  • Nsamples: total number of MCMC samples
  • Nadapt: number of adaption samples during warm up
  • delta: target acceptance rate.
source
modifyConfig!(s::AHMCNUTS;Nsamples,Nadapt,delta,kwargs...)
MCMCBenchmarks.updateResults! — Function.

Update the results DataFrame on each iteration

  • s: MCMC sampler object
  • performance: includes MCMC Chain, execution time, and memory measurements
  • results: DataFrame containing benchmark results
source
updateResults!(s::CmdStanNUTS,performance,results;kwargs...)
MCMCBenchmarks.runSampler — Function.

Extracts model and configuration from sampler object and performs parameter estimation

  • s: sampler object
  • data: data for benchmarking
  • Nchains: number of chains ran serially. Default = 1
source
 runSampler(s::AHMCNUTS,data;kwargs...)
 ```

### Helper Functions

@docs addPerformance!

julia addPerformance!(df,p)

@docs addKW!

julia addKW!(df;kwargs...)

@docs addChainSummary!

julia addChainSummary!(newDF,chn,df,col)

@docs addHPD!

julia addHPD!(newDF,chain)

@docs addMeans!

julia addMeans!(newDF,df)

@docs runallbenchmarks

julia runallbenchmarks()

### Plotting

@docs plotdensity

julia plotdensity(df::DataFrame,metric::Symbol,group=(:sampler,);save=false, figfmt="pdf",dir="",options...)

@docs plotsummary

julia plotsummary(df::DataFrame,xvar::Symbol,metric::Symbol,group=(:sampler,);save=false, figfmt="pdf",func=mean,dir="",options...)

@docs plotscatter

julia plotscatter(df::DataFrame,xvar::Symbol,metric::Symbol,group=(:sampler,);save=false, figfmt="pdf",func=mean,dir="",options...)

@docs plotrecovery

julia plotrecovery(df::DataFrame,parms,group=(:sampler,);save=false, figfmt="pdf",dir="",options...) ```