alarsyo/spot
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spot/doc/org/ltlsynt.org
Alexandre Duret-Lutz b729aa3f30 ltlsynt: fix a few issues with --csv 
Some columns were superfluous, other had inconsistent names, and some
times where not tracked.

* spot/twaalgos/synthesis.cc: Improve tracking of times and verbose
messages.
* bin/ltlsynt.cc (print_csv): Adjust CSV columns.
* tests/core/ltlsynt.test, tests/core/ltlsynt2.test,
tests/core/ltlsynt-pgame.test: Adjust expected CSV and verbose
messages.
* doc/org/ltlsynt.org: Give some example.
2024-09-05 08:41:30 +02:00

422 lines
18 KiB
Org Mode
Raw Blame History

# -*- coding: utf-8 -*-
#+TITLE: =ltlsynt=
#+DESCRIPTION: Spot command-line tool for synthesizing AIGER circuits from LTL/PSL formulas.
#+INCLUDE: setup.org
#+HTML_LINK_UP: tools.html
#+PROPERTY: header-args:sh :results verbatim :exports both
* Basic usage
This tool synthesizes reactive controllers from LTL/PSL formulas.
Consider a set $I$ of /input/ atomic propositions, a set $O$ of /output/ atomic
propositions, and a PSL formula \phi over the propositions in $I \cup O$. A
*reactive controller* realizing \phi is a function $c: (2^{I})^\star \times 2^I \mapsto
2^O$ such that, for every \omega-word $(u_i)_{i \in N} \in (2^I)^\omega$ over
the input propositions, the word $(u_i \cup c(u_0 \dots u_{i-1}, u_i))_{i \in
N}$ satisfies \phi.
If a reactive controller exists, then one with finite memory
exists. Such controllers are easily represented as automata (or more
specifically as Mealy machines). In the automaton representing the
controller, the acceptance condition is irrelevant and trivially true.
Here is a small example where $I=\{i_1,i_2\}$ and $O=\{o_1\}$. The
specification asks that $o_1$ hold at some point if and only if $i_1$
and $i_2$ hold one after the other at some point.
#+NAME: example
#+BEGIN_SRC sh :exports both
ltlsynt -f 'F(i1 & Xi2) <-> F(o1)'
#+END_SRC
#+RESULTS: example
#+begin_example
REALIZABLE
HOA: v1
States: 2
Start: 0
AP: 3 "i1" "o1" "i2"
acc-name: all
Acceptance: 0 t
properties: trans-labels explicit-labels state-acc deterministic
controllable-AP: 1
--BODY--
State: 0
[!0&!1] 0
[0&!1] 1
State: 1
[!0&!1&!2] 0
[0&!1&!2] 1
[1&2] 1
--END--
#+end_example
The output is composed of two parts:
- The first part is a single line stating =REALIZABLE= or
=UNREALIZABLE=; the presence of this line, required by the [[http://http://www.syntcomp.org/][SyntComp
competition]], can be disabled with option =--hide-status=.
- The second part, only present in the =REALIZABLE= case, is an
automaton describing the controller.
The controller contains the line =controllable-AP: 2=, which means
that this automaton should be interpreted as a Mealy machine where
=o0= is part of the output. Using the =--dot= option, makes it easier
to visualize this machine.
#+NAME: exampledot
#+BEGIN_SRC sh :exports code
ltlsynt -f 'F(i1 & Xi2) <-> F(o1)' --hide-status --dot
#+END_SRC
#+BEGIN_SRC dot :file ltlsyntex.svg :var txt=exampledot :exports results
$txt
#+END_SRC
#+RESULTS:
[[file:ltlsyntex.svg]]
The following example illustrates the case of an unrealizable specification. As
=a= is an input proposition, there is no way to guarantee that it will
eventually hold.
#+BEGIN_SRC sh :epilogue true
ltlsynt --ins=a -f 'F a'
#+END_SRC
#+RESULTS:
: UNREALIZABLE
* Input options
:PROPERTIES:
:CUSTOM_ID: input-options
:END:
=ltlsynt= require two pieces of information two solve a reactive
LTL/PSL synthesis problem: an LTL (or PSL) formula, and a partition of
its atomic propositions as input and output.
The specification formula can be passed with =-f/--formula= or
=-F/--file=. If multiple specifications formulas are passed, they
will all be solved individually.
The input/output partition can be given in several ways. If it is
not specified, =ltlsynt= assumes that input variables should start
with =i=, and output variables should start with =o=.
Options =--ins= and =--outs= should be followed by a comma-separated
list of input atomic propositions, or input regexes enclosed in
slashes. E.g., =--ins=switch,/^in/,car=. If only one of these
options is given, atomic propositions not matched by that option
are assumed to belong to the other set.
Another way to specify the input/output partition is using a =*.part=
file passed to the =--part-file= option. Such a file is used by
several other synthesis tools. The format is space-separated list of
words representing atomic-propositions. Two keywords =.inputs= and
=.outputs= indicate the set of the atomic-propositions that follow.
For instance:
#+BEGIN_EXAMPLE
.inputs request cancel
.outputs grant ack
#+END_EXAMPLE
Using =--part-file=THEABOVEFILE= is equivalent to
=--ins=request,cancel --outs=grant,ack=.
As an extension to this simple =*.part= format, words enclosed in
slashes are interpreted as regexes, like for the =--ins= and =--outs=
options.
* TLSF input
=ltlsynt= was made with the [[http://syntcomp.org/][SYNTCOMP]] competition in mind, and more
specifically the TLSF track of this competition. TLSF is a high-level
specification language created for the purpose of this competition.
Fortunately, the SYNTCOMP organizers also provide a tool called
[[https://github.com/reactive-systems/syfco][=syfco=]] which can translate a TLSF specification to an LTL formula.
The following line shows how a TLSF specification called =FILE= can
be synthesized using =syfco= and =ltlsynt=:
#+BEGIN_SRC sh :export code
ltlsynt --tlsf FILE
#+END_SRC
The above =--tlsf= option will call =syfco= (which must be on your
=$PATH=) to perform the conversion and extract output signals, as if
you had used:
#+BEGIN_SRC sh :export code
LTL=$(syfco -f ltlxba -m fully FILE)
OUT=$(syfco --print-output-signals FILE)
ltlsynt --formula="$LTL" --outs="$OUT"
#+END_SRC
* Output options
By default, the controller is output in HOA format, but it can be
output as an And-Inverter-Graph in [[http://fmv.jku.at/aiger/][AIGER format]] using the =--aiger=
flag. This is the output format required for the [[http://syntcomp.org/][SYNTCOMP]] competition.
#+NAME: exampleaig
#+BEGIN_SRC sh :exports both
ltlsynt -f 'F(i1 & Xi2) <-> F(o1)' --aiger
#+END_SRC
#+RESULTS: exampleaig
#+begin_example
REALIZABLE
aag 5 2 1 1 2
2
4
6 11
8
8 4 6
10 3 9
i0 i1
i1 i2
o0 o1
#+end_example
The above format is not very human friendly. Again, by passing both
=--aiger= and =--dot=, one can display the And-Inverter-Graph representing
the controller:
#+NAME: exampleaigdot
#+BEGIN_SRC sh :exports code
ltlsynt -f 'F(i1 & Xi2) <-> F(o1)' --hide-status --aiger --dot
#+END_SRC
#+RESULTS: exampleaigdot
#+BEGIN_SRC dot :file ltlsyntexaig.svg :var txt=exampleaigdot :exports results
$txt
#+END_SRC
#+RESULTS:
[[file:ltlsyntexaig.svg]]
In the above diagram, round nodes represent AND gates. Small black
circles represent inversions (or negations), colored triangles are
used to represent input signals (at the bottom) and output signals (at
the top), and finally rectangles represent latches. A latch is a one
bit register that delays the signal by one step. Initially, all
latches are assumed to contain =false=, and they emit their value from
the =*_out= rectangles at the bottom. Their input value, to be
emitted at the next step, is received via the =*_in= boxes at the top.
In =ltlsynt='s encoding, the set of latches is used to keep track of
the current state of the Mealy machine.
The generation of a controller can be disabled with the flag
=--realizability=. In this case, =ltlsynt='s output is limited to
=REALIZABLE= or =UNREALIZABLE=.
* Internal details
The tool reduces the synthesis problem to a parity game, and solves the parity
game using Zielonka's recursive algorithm. The process can be pictured as follows.
[[file:ltlsynt.svg]]
LTL decomposition consist in splitting the specification into multiple
smaller constraints on disjoint subsets of the output values (as
described by [[https://arxiv.org/abs/2103.08459][Finkbeiner, Geier, and Passing]]), solve those constraints
separately, and then combine them while encoding the AIGER circuit.
This is enabled by default, but can be disabled by passing option
=--decompose=no=.
The ad hoc construction on the top is just a shortcut for some type of
constraints that can be solved directly by converting the constraint
into a DBA.
Otherwise, conversion to parity game (represented by the blue zone) is
done using one of several algorithms specified by the =--algo= option.
The game is then solved, producing a strategy if the game is realizable.
If =ltlsynt= is in =--realizability= mode, the process stops here
In synthesis mode, the strategy is first simplified. How this is done
can be fine-tuned with option =--simplify=:
#+BEGIN_SRC sh :exports results
ltlsynt --help | sed -n '/--simplify=/,/^$/p' | sed '$d'
#+END_SRC
#+RESULTS:
: --simplify=no|bisim|bwoa|sat|bisim-sat|bwoa-sat
: simplification to apply to the controler (no)
: nothing, (bisim) bisimulation-based reduction,
: (bwoa) bissimulation-based reduction with output
: assignment, (sat) SAT-based minimization,
: (bisim-sat) SAT after bisim, (bwoa-sat) SAT after
: bwoa. Defaults to 'bwoa'.
Finally, the strategy is encoded into [[http://fmv.jku.at/aiger/][AIGER]]. The =--aiger= option can
take an argument to specify a type of encoding to use: by default it
is =ite= for if-then-else, because it follows the structure of BDD
used to encode the conditions in the strategy. An alternative
encoding is =isop= where condition are first put into
irredundant-sum-of-product, or =both= if both encodings should be
tried. Additionally, these optiosn can accept the suffix =+ud= (use
dual) to attempt to encode each condition and its negation and keep
the smallest one, =+dc= (don't care) to take advantage of /don't care/
values in the output, and one of =+sub0=, =+sub1=, or =+sub2= to test
various grouping of variables in the encoding. Multiple encodings can
be tried by separating them using commas. For instance
=--aiger=isop,isop+dc,isop+ud= will try three different encodings.
* Other useful options
** Printing games
You can also ask =ltlsynt= to print to obtained parity game into
[[https://github.com/tcsprojects/pgsolver][PGSolver]] format, with the flag =--print-pg=, or in the HOA format,
using =--print-game-hoa=. These flags deactivate the resolution of the
parity game. Note that if any of those flag is used with =--dot=, the game
will be printed in the Dot format instead:
#+NAME: examplegamedot
#+BEGIN_SRC sh :exports code
ltlsynt -f '(i1 & i2) <-> F(o1 & X(!o1))' --print-game-hoa --dot
#+END_SRC
#+BEGIN_SRC dot :file ltlsyntexgame.svg :var txt=examplegamedot :exports results
$txt
#+END_SRC
#+RESULTS:
[[file:ltlsyntexgame.svg]]
** Saving statistics
For benchmarking purpose, the =--csv= option can be used to record
intermediate statistics about the resolution. The =--csv= option will
also save the formula into the CSV file, which can therefore become
very large. The variant =--csv-without-formula= is usually enough.
For instance the following command tests the realizability of the 23
demonstration specifications from [[http://www.ist.tugraz.at/staff/jobstmann/lily/][Lily 1.0.2]] while saving some
statistics in =bench.csv=. (If you compare our results with theirs,
keep in mind that Lily uses Moore's semantics, while =ltlsynt= uses
Mealy's semantics.)
#+BEGIN_SRC sh :results verbatim :exports code :epilogue true
genltl --lily-patterns |
ltlsynt --algo=acd --realizability --csv-without-formula=bench.csv
#+END_SRC
#+RESULTS:
#+begin_example
UNREALIZABLE
UNREALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
UNREALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
REALIZABLE
#+end_example
After execution, =bench.csv= contains a table like the following:
#+BEGIN_SRC sh :results output raw :exports results
sed 's/"//g
s/|/\\vert{}/g
s/--/@@html:--@@/g
1a\
|-|
s/^/| /
s/$/ |/
s/,/|/g
' bench.csv
#+END_SRC
#+RESULTS:
| source | algo | tot_time | trans_time | split_time | todpa_time | solve_time | realizable | game_states | game_states_env |
|--------+------+-------------+-------------+-------------+-------------+-------------+------------+-------------+-----------------|
| -:1 | acd | 0.000472663 | 0.00019603 | 2.0339e-05 | 2.0388e-05 | 1.4617e-05 | 0 | 3 | 2 |
| -:2 | acd | 0.00028595 | 0.000188466 | 1.4417e-05 | 2.0027e-05 | 5.861e-06 | 0 | 13 | 7 |
| -:3 | acd | 0.000741622 | 0.000591889 | 4.7229e-05 | 1.9516e-05 | 1.8014e-05 | 1 | 26 | 12 |
| -:4 | acd | 0.000917794 | 0.000725371 | 7.2026e-05 | 3.0328e-05 | 2.0349e-05 | 1 | 33 | 15 |
| -:5 | acd | 0.000878991 | 0.000612978 | 0.000102604 | 3.4155e-05 | 2.7913e-05 | 1 | 47 | 20 |
| -:6 | acd | 0.00100199 | 0.000761539 | 8.0191e-05 | 2.9817e-05 | 2.9075e-05 | 1 | 55 | 24 |
| -:7 | acd | 0.000587721 | 0.000425814 | 4.6268e-05 | 1.6261e-05 | 1.4106e-05 | 1 | 26 | 11 |
| -:8 | acd | 1.4046e-05 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
| -:9 | acd | 0.000519242 | 0.000400918 | 2.2322e-05 | 2.9446e-05 | 1.3886e-05 | 1 | 16 | 6 |
| -:10 | acd | 6.0835e-05 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
| -:11 | acd | 5.5245e-05 | 1.8335e-05 | 5.249e-06 | 4.007e-06 | 4.549e-06 | 0 | 3 | 2 |
| -:12 | acd | 1.6411e-05 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
| -:13 | acd | 0.000192153 | 0.000134825 | 1.06e-05 | 8.506e-06 | 5.33e-06 | 1 | 5 | 2 |
| -:14 | acd | 0.000291931 | 0.000209857 | 1.0881e-05 | 1.4076e-05 | 6.182e-06 | 1 | 4 | 2 |
| -:15 | acd | 0.000690605 | 0.000480759 | 9.4349e-05 | 3.2541e-05 | 1.8675e-05 | 1 | 30 | 9 |
| -:16 | acd | 0.00232829 | 0.00173036 | 0.000348709 | 9.2966e-05 | 6.1276e-05 | 1 | 103 | 29 |
| -:17 | acd | 0.000554708 | 0.00038608 | 2.4887e-05 | 2.9205e-05 | 1.1902e-05 | 1 | 6 | 3 |
| -:18 | acd | 0.00114041 | 0.00088879 | 3.3784e-05 | 3.4585e-05 | 1.1602e-05 | 1 | 8 | 4 |
| -:19 | acd | 0.000761799 | 0.000517278 | 4.3132e-05 | 5.1968e-05 | 2.127e-05 | 1 | 11 | 4 |
| -:20 | acd | 0.0169891 | 0.0133503 | 0.00172203 | 0.00113707 | 0.000412299 | 1 | 1002 | 311 |
| -:21 | acd | 0.118002 | 0.115604 | 0.00165549 | 0.000149402 | 0.00024346 | 1 | 371 | 75 |
| -:22 | acd | 0.00316832 | 0.00240598 | 0.000305407 | 0.000103245 | 0.00010582 | 1 | 86 | 30 |
| -:23 | acd | 0.000824969 | 0.000632956 | 3.2161e-05 | 2.9766e-05 | 2.0299e-05 | 1 | 17 | 7 |
A source of the form =-:N= designates the Nth line of the standard
input, as =ltlsynt= was reading from that. The various =*_time*=
columns refers to different steps in the processing pipeline. Note
that various bits and minor operations are not timed, so =tot_time=
(the total time) should be larger than the sum of times used for
translation, splitting, conversion to DPA, and game solving. Some of
these intermediate processing time are listed as =0= above because
(e.g., for input 8, 10, 12) because the specifications can be found to
be realizable trivially without building any game.
** Verifying the output
The =--verify= option requests that the produced strategy or aiger
circuit are compatible with the specification. This is done by
ensuring that they do not intersect the negation of the specification.
* References
The initial reduction from LTL to parity game is described in the
following paper:
- *Reactive Synthesis from LTL Specification with Spot*, /Thibaud Michaud/,
/Maximilien Colange/. Presented in SYNT@CAV'18. ([[https://www.lrde.epita.fr/dload/papers/michaud.18.synt.pdf][pdf]] | [[https://www.lrde.epita.fr/~max/bibtexbrowser.php?key=michaud.18.synt&bib=perso.bib][bib]])
Further improvements are described in the following paper:
- *Improvements to =ltlsynt=*, /Florian Renkin/, /Philipp Schlehuber/,
/Alexandre Duret-Lutz/, and /Adrien Pommellet/. Presented at the
SYNT'21 workshop. ([[https://www.lrde.epita.fr/~adl/dl/adl/renkin.21.synt.pdf][pdf]] | [[https://www.lrde.epita.fr/~adl/dl/adl_bib.html#renkin.21.synt][bib]])
Simplification of Mealy machines is discussed in the following papers:
- *Effective reductions of Mealy machines*, /Florian Renkin/,
/Philipp Schlehuber-Caissier/, /Alexandre Duret-Lutz/, and /Adrien Pommellet/.
Presented at FORTE'22. ([[https://www.lrde.epita.fr/~adl/dl/adl/renkin.22.forte.pdf][pdf]] | [[https://www.lrde.epita.fr/~adl/dl/adl_bib.html#renkin.22.forte][bib]])
- *The Mealy-machine reduction functions of Spot*, /Florian Renkin/,
/Philipp Schlehuber-Caissier/, /Alexandre Duret-Lutz/, and /Adrien Pommellet/.
Science of Computer Programming, 230(102995), August 2023. ([[https://www.lrde.epita.fr/~adl/dl/adl_bib.html#renkin.23.fmsd][bib]] | [[https://www.lrde.epita.fr/~adl/dl/adl/renkin.23.fmsd.pdf][pdf]])
A more recent paper covering many aspects of =ltlsynt= is the following
- *Dissecting ltlsynt*, /Florian Renkin/, /Philipp
Schlehuber-Caissier/, /Alexandre Duret-Lutz/, and Adrien Pommellet.
In Formal Methods in System Design, 2023. ([[https://www.lrde.epita.fr/~adl/dl/adl_bib.html#renkin.23.scp][bib]] | [[https://www.lrde.epita.fr/~adl/dl/adl/renkin.23.scp.pdf][pdf]])
# LocalWords: utf ltlsynt AIGER html args mapsto SRC acc aiger TLSF
# LocalWords: UNREALIZABLE unrealizable SYNTCOMP realizability Proc
# LocalWords: syfco ltlxba Zielonka's Thibaud Michaud Maximilien
# LocalWords: Colange PGSolver
Reference in a new issue View git blame Copy permalink