alarsyo/spot
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

bench/stutter: Update

Browse source 
* bench/stutter/stutter_invariance_randomgraph.cc: Update to recent
changes.  If an algorithm took more that 30s on an average for a set of
parameters, avoid running it with more states.  Take the density and
ap count as parameter.  Output all the algorithms on the same line.
Add additional statistics about automata.
* bench/stutter/stutter_invariance_formulas.cc: Update to recent
changes.  Output all the algorithms on the same line.
Add additional statistics about automata.
* bench/stutter/stutter_bench.sh: Use a Makefile to manage concurrency.
* bench/stutter/README: Update.
This commit is contained in:
Alexandre Duret-Lutz 2015-01-22 11:31:20 +01:00
parent 5d31094029
commit bd414d4d4c
4 changed files with 153 additions and 76 deletions
Download patch file Download diff file Expand all files Collapse all files

36
bench/stutter/stutter_bench.sh
View file

@ -3,15 +3,33 @@
RANDLTL=../../src/bin/randltl
LTLFILT=../../src/bin/ltlfilt
OUTPUT=bench_formulas.csv
echo 'formula,algo,ap,states,result,time' > "$OUTPUT"
pos="pos.states,pos.trans,pos.edges,pos.scc,pos.nondet"
neg="neg.states,neg.trans,neg.edges,neg.scc,neg.nondet"
algos="time1,time2,time3,time4,time5,time6,time7,time8"
OUTPUTF=bench_formulas.csv
(
all=
for ap in 1 2 3 4; do
echo "Generating benchmarks for formulas with $ap atomic propositions..."
$RANDLTL $ap --tree-size=..30 -n 20000 | $LTLFILT --ap=$ap |
./stutter_invariance_formulas -F- >> "$OUTPUT"
out=ltl-ap$ap.csv
echo "$out:; $RANDLTL $ap --tree-size=..30 -n -1 | $LTLFILT --ap=$ap | $LTLFILT -v --nox -n 10000 | ./stutter_invariance_formulas -F- > \$@"
all="$all $out"
done
echo "Generating benchmarks for random graphs..."
OUTPUT=bench_randgraph.csv
echo 'algo,ap,states,result,time' > "$OUTPUT"
./stutter_invariance_randomgraph >> "$OUTPUT"
echo "$OUTPUTF:$all; (echo 'formula,ap,$pos,$neg,$algos,res'; cat $all) > \$@"
) > run.mk
OUTPUTG=bench_randgraph.csv
(
all=
for d in 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0; do
for ap in 1 2 3 4; do
out=graph-d$d-ap$ap.csv
echo "$out:; ./stutter_invariance_randomgraph $d $ap > \$@"
all="$all $out"
done
done
echo "$OUTPUTG:$all; (echo 'd,ap,seed,$pos,$neg,$algos,res'; cat $all) > \$@"
) >> run.mk
make "$@" -f run.mk $OUTPUTF $OUTPUTG