This document outlines how to get started with PPL Bench.
- Enter a virtual (or conda) environment
- Install PPL Bench core via pip:
- Install PPLs that you wish to benchmark. For PPL-specific instructions, see Installing PPLs. You could also run the following command to install all PPLs that are currently supported by PPL Bench (except for Jags):
Alternatively, you could also install PPL Bench from source. Please refer to Installing PPLs for instructions.
Launching PPL Bench
Let's dive right in with a benchmark run of Bayesian Logistic Regression. To run this, you'll need to install PyStan (if you haven't already):
Then, run PPL Bench with example config:
This will create a benchmark run with two trials of Stan on the Bayesian Logistic Regression model. The results of the run are saved in the
This is what the Predictive Log Likelihood (PLL) plot should look like:
A number of models is available in the
pplbench/models directory and the PPL implementations are available in the
Please feel free to submit pull requests to modify an existing PPL implementation or to add a new PPL or model.
You'd like to contribute to PPL Bench? Great! Please see here for how to help out.
Join the PPL Bench community
For more information about PPL Bench, refer to