alarsyo/spot
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

stutter: fiddle with the benchmark

Browse source 
* bench/stutter/stutter_bench.sh: Add headers in the CSV files, and also
run stutter_invariance_randomgraph.
* bench/stutter/stutter_invariance_formulas.cc: Remove space from CSV
output.
* bench/stutter/stutter_invariance_randomgraph.cc: Likewise, plus fix
the call to is_stutter_invariant(), and return an average time.
* bench/stutter/stutter.ipynb: Adjust.
* bench/stutter/README: Simplify.
* bench/stutter/Makefile.am: Distribute the script and python notebook.
This commit is contained in:
Alexandre Duret-Lutz 2014-11-23 21:47:12 +01:00
parent 0beb148b6a
commit c494a347c9
6 changed files with 65 additions and 120 deletions
Download patch file Download diff file Expand all files Collapse all files

33
bench/stutter/README
View file

@ -1,31 +1,18 @@
This benchmark measures the performance of different algorithms to check
if a büchi automaton has the stutter-invariance property. If the
benchmark is run on formulas, the translation time is not included in
the measured time.
This benchmark measures the performance of different algorithms to
check if property (expressed as a formula or as a deterministic TGBA)
is stutter-invariant. When the benchmark is run on formulas, the
translation time is not included in the measured time.
You can specify which formulas are to be used for the benchmarks by
running:
% ./stutter_invariance_formulas -F FILE > bench_formulas.csv
Where FILE is a file containing a list of formulas (see
./stutter_invariance_formulas --help for other input options).
Or use the script which will call this executable on random formulas
ranging from 1 to 4 atomic propositions:
To reproduce the benchmark is to run
% ./stutter_bench.sh
This will create the file bench_formulas.csv.
to create bench_formulas.csv and bench_randgraph.csv, and then
explore these data the provided ipython notbook
Alternatively, the algorithms can be measured on random complete
deterministic automata with:
% ipython notebook --pylab=inline stutter.ipynb
% ./stutter_invariance_randomgraph > bench_randgraph.csv
Assuming ipython is installed, the csv files can be visualized using the
provided ipython notebook. Run:
% ipython notebook --matplotlib
in this directory, and open stutter.ipynb.
The time in bench_formulas.csv is reported in microseconds, while the
time in bench_randgraph.csv is in seconds.