This directory contains an evolution of the benchmark we used in
the following papers.
* Souheib Baarir and Alexandre Duret-Lutz. Test de vacuité pour
automates de Büchi ensemblistes avec tests d'inclusion. In Actes du
6e Colloque Francophone sur la Modélisation des Systèmes Réactifs
(MSR'07), pages 19-34. Hermes-Lavoisier, October 2007.
* Souheib Baarir and Alexandre Duret-Lutz. Emptiness check of powerset
Büchi automata. In Proceedings of the 7th International Conference
on Application of Concurrency to System Design (ACSD'07), pages
41-50. IEEE Computer Society, July 2007.
* Souheib Baarir and Alexandre Duret-Lutz. Emptiness check of powerset
Büchi automata. Technical report 2006/003, Université Pierre et
Marie Curie, LIP6-CNRS, Paris, France, October 2006.
It is unlikely that you will be able to run this benchmark unless you
are close to Souheib et Alexandre, because of it requires
(1) a customized version of GreatSPN that has never been released,
(2) a tool called Snow that has never been released either
(see http://spot.lip6.fr/wiki/Snow)
Here is some documentation nonetheless.
Layout
======
template/
This directory contains the original models. Some of them
are parametrized and need to be instantiated.
WCSsym/
A parameterized model for a critical section in which
processes are assigned a different priority (yielding an
asymmetry). The parameter is the number of processes.
bagrodia/
A parameterized model of Rajive L Bagrodia's distributed
rendez-vous algorithm. The parameter is the number of
processes.
predef/
Some preinstanciated models for WCS that do not use
symmetries.
models/
This directory is built by make and will contain the actual
instances of each model.
results/
This directory will be built by make and will contain the result of
each test.
tools/
Scripts used by the benchmark.
tools/modelgen-create
is a script that creates , the makefile rules for instantiating
the models in the models/ directory.
tools/bench-create
is a script that creates modelgen.mk and bench.mk.
modelgen.mk contains the rules for instantiating
the models in the models/ directory, and bench.mk
those for putting the results in the results/ directory.
Unless you add new models you should not need to run this,
because the Makefiles are distributed with Spot.
tools/sum
(for summary) is the script that should be run to display the
results of the benchmark. This script can be run at any point
during the benchmark, and will display the available results.
tools/runbench
is an auxiliary script used in the makefile rules of bench.mk.
Its job is to actually run the ltlgspn-* binaries from iface/gspn/
with the right options. Runbench will also clone the model in a
temporary directory before running any tool : this is because
GreatSPN use many temporary files and we want to be able to run
several test in parallel.
tools/trans2prop.pl
is an auxiliary script used by tools/runbench. It is used in
conjunction with Snow (see link above) to express atomic properties
as transitions in a model.
Running
=======
1) Compile GreatSPN. You should have the following three libraries:
% ls greatspn/i686_R2.6.25.10/2bin/lib
libgspnRG.a libgspnSRG.a libgspnSSP.a
2) Compile Spot using GreatSPN. I use the following command:
% ./configure --with-gspn=GREATSPN_PATH --disable-static LDFLAGS=-lglib-2.0
You should have the following tree binaries :
iface/gspn/ltlgspn-rg
iface/gspn/ltlgspn-srg
iface/gspn/ltlgspn-ssp*
3) Edit bench/gspn-ssp/defs and fix the location of Snow.
4) Go into bench/gspn-ssp/, run "make check" or "make -j2 check" (or more)
if you have several CPUs.
5) Run tools/sum to get a summary of the results. You can run this
even before the above "make check" has finished, but of course this
will only summarize the available results.
What is tested
==============
Each model is tested using differents emptiness check strategies
(these are the ltlgspn-ssp options -e4, -e5, -e6 etc.) against 50
formulaes. Each test (1 model, 1 set of option, 1 formula) produce
one ".log" file in the results/ directory. The tools/sum script
processes all these log files and show how one model with one set of
option averages on the 50 formulae.
