alarsyo/spot
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

ltlcross: report exit_status and exit_code columns in CSV and JSON

Browse source 
* src/bin/ltlcross.cc: Report exit_status and exit_code columns in CSV
and JSON files.  Also output lines for failed translations, and add
a --omit-missing option to disable that.  Move the time column right
after exit_status and exit_code.
* src/bin/man/ltlcross.x: Document each column of the output.
* bench/ltl2tgba/tools: Use the "{name}cmd" notation.
* bench/ltl2tgba/sum.py: Adjust to the new columns.
* bench/ltl2tgba/README: Update to point to the man page for a
description of the columns.
* bench/ltl2tgba/Makefile.am: Build results.pdf as said announced in
README.
* bench/spin13/html.bottom: Update code to ignore these two new
columns and lines with null values.
* src/tgbatest/ltlcross3.test: Add tests.
* doc/org/ltlcross.org: Adjust examples.
* NEWS: Mention this.
This commit is contained in:
Alexandre Duret-Lutz 2013-11-19 23:22:29 +01:00
parent 686a45484d
commit f65c621a55
10 changed files with 430 additions and 202 deletions
Download patch file Download diff file Expand all files Collapse all files

49
src/tgbatest/ltlcross3.test
View file

@ -21,5 +21,52 @@
. ./defs
set -e
run 2 ../../bin/ltlcross 'ltl2tgba -s %f >%N' 'foo bar' 2>stderr -f a
check_csv()
{
# Make sure all lines in $1 have the same number of comas
sed 's/[^,]//g' < "$1" |
( read first
while read l; do
test "x$first" = "x$l" || exit 1
done)
}
ltl2tgba=../../bin/ltl2tgba
run 2 ../../bin/ltlcross "$ltl2tgba -s %f >%N" 'foo bar' 2>stderr -f a
grep 'ltlcross.*no input.*in.*foo bar' stderr
# Make sure non-zero exit codes are reported...
run 1 ../../bin/ltlcross "$ltl2tgba -s %f >%N" 'false %f >%N' \
-f a --csv=out.csv 2>stderr
grep '"exit_status"' out.csv
grep '"exit_code"' out.csv
test `grep 'error:.*returned exit code 1' stderr | wc -l` -eq 2
test `grep '"exit code",1' out.csv | wc -l` -eq 2
check_csv out.csv
# ... unless --omit-missing is supplied.
run 1 ../../bin/ltlcross "$ltl2tgba -s %f >%N" 'false %f >%N' \
-f a --csv=out.csv --omit-missing 2>stderr
grep '"exit_status"' out.csv && exit 1
grep '"exit_code"' out.csv && exit 1
test `grep 'error:.*returned exit code 1' stderr | wc -l` -eq 2
test `grep '"exit code",1' out.csv | wc -l` -eq 0
check_csv out.csv
# Likewise for timeouts
run 0 ../../bin/ltlcross 'sleep 5; false %f >%N' \
--timeout 2 -f a --csv=out.csv 2>stderr
grep '"exit_status"' out.csv
grep '"exit_code"' out.csv
test `grep 'warning:.*timeout' stderr | wc -l` -eq 2
test `grep '"timeout",-1' out.csv | wc -l` -eq 2
check_csv out.csv
run 0 ../../bin/ltlcross 'sleep 5; false %f >%N' \
--timeout 2 --omit-missing -f a --csv=out.csv 2>stderr
grep '"exit_status"' out.csv && exit 1
grep '"exit_code"' out.csv && exit 1
test `grep 'warning:.*timeout' stderr | wc -l` -eq 2
test `wc -l < out.csv` -eq 1
check_csv out.csv