# -*- coding: utf-8 -*- #+TITLE: =autfilt= #+DESCRIPTION: Spot command-line tool for filtering, tranforming, and converting ω-automata. #+SETUPFILE: setup.org #+HTML_LINK_UP: tools.html The =autfilt= tool can filter, transform, and convert a stream of automata. The tool operates a loop over 5 phases: - input one automaton - optionally preprocess the automaton - optionally filter the automaton (i.e., decide whether to ignore the automaton or continue with it) - optionally postprocess the automaton - output the automaton The simplest way to use the tool is simply to use it for input and output (i.e., format conversion) without any transformation and filtering. * Conversion between formats =autfilt= can read automata written in the [[http://adl.github.io/hoaf/][Hanoi Omega Automata Format]], as [[http://spinroot.com/spin/Man/never.html][Spin never claims]], using [[http://www.tcs.hut.fi/Software/lbtt/doc/html/Format-for-automata.html][LBTT's format]], or using [[http://www.ltl2dstar.de/docs/ltl2dstar.html][=ltl2dstar='s format]]. Automata in those formats (even a mix of those formats) can be concatenated in the same stream, =autfilt= will process them in batch. (The only restriction is that inside a file an automaton in LBTT's format may not follow an automaton in =ltl2dstar='s format.) By default the output uses the HOA format. This can be changed using [[file:oaut.org][the common output options]] like =--spin=, =--lbtt=, =--dot=, =--hoaf=... #+BEGIN_SRC sh :results silent :exports both cat >example.hoa < 0 : 0 [label="0"] : 0 -> 0 [label="p0\n{0}"] : 0 -> 0 [label="!p0"] : } The =--spin= options implicitly requires a degeneralization: #+BEGIN_SRC sh :results verbatim :exports both autfilt example.hoa --spin #+END_SRC #+RESULTS: #+begin_example never { accept_init: if :: ((p0)) -> goto accept_init :: ((!(p0))) -> goto T0_S2 fi; T0_S2: if :: ((p0)) -> goto accept_init :: ((!(p0))) -> goto T0_S2 fi; } #+end_example #+BEGIN_SRC sh :results verbatim :exports both autfilt example.hoa --lbtt #+END_SRC #+RESULTS: : 1 1t : 0 1 : 0 0 -1 p0 : 0 -1 ! p0 : -1 * Displaying statistics One special output format of =autfilt= is the statistic output. For instance the following command calls [[file:randaut.org][=randaut=]] to generate 10 random automata, and pipe the result into =autfilt= to display various statistics. #+BEGIN_SRC sh :results verbatim :exports both randaut -n 10 -A0..2 -Q10..20 -e0.05 2 | autfilt --stats='%s states, %e edges, %a acc-sets, %c SCCs, det=%d' #+END_SRC #+RESULTS: #+begin_example 16 states, 30 edges, 1 acc-sets, 3 SCCs, det=0 20 states, 42 edges, 2 acc-sets, 1 SCCs, det=0 15 states, 27 edges, 2 acc-sets, 1 SCCs, det=0 10 states, 17 edges, 1 acc-sets, 1 SCCs, det=1 13 states, 25 edges, 1 acc-sets, 1 SCCs, det=0 11 states, 18 edges, 0 acc-sets, 1 SCCs, det=0 19 states, 41 edges, 2 acc-sets, 1 SCCs, det=0 11 states, 18 edges, 0 acc-sets, 1 SCCs, det=0 12 states, 21 edges, 1 acc-sets, 5 SCCs, det=0 18 states, 37 edges, 1 acc-sets, 5 SCCs, det=0 #+end_example The following =%= sequences are available: #+BEGIN_SRC sh :results verbatim :exports results autfilt --help | sed -n '/^ %%/,/^$/p' | sed '$d' #+END_SRC #+RESULTS: #+begin_example %% a single % %A, %a number of acceptance sets %C, %c number of SCCs %d 1 if the output is deterministic, 0 otherwise %E, %e number of edges %F name of the input file %G, %g acceptance condition (in HOA syntax) %L location in the input file %M, %m name of the automaton %n number of nondeterministic states in output %p 1 if the output is complete, 0 otherwise %r processing time (excluding parsing) in seconds %S, %s number of states %T, %t number of transitions %w one word accepted by the output automaton #+end_example When a letter is available both as uppercase and lowercase, the uppercase version refer to the input automaton, while the lowercase refer to the output automaton. Of course this distinction makes sense only if =autfilt= was instructed to perform an operation on the input automaton. * Filtering automata =autfilt= supports multiple ways to filter automata based on different characteristics of the automaton. #+BEGIN_SRC sh :results verbatim :exports results autfilt --help | sed -n '/Filtering options.*:/,/^$/p' | sed '1d;$d' #+END_SRC #+RESULTS: #+begin_example --acc-sets=RANGE keep automata whose number of acceptance sets are in RANGE --accept-word=WORD keep automata that accept WORD --ap=RANGE match automata with a number of atomic propositions in RANGE --are-isomorphic=FILENAME keep automata that are isomorphic to the automaton in FILENAME --edges=RANGE keep automata whose number of edges are in RANGE --equivalent-to=FILENAME keep automata thare are equivalent (language-wise) to the automaton in FILENAME --included-in=FILENAME keep automata whose languages are included in that of the automaton from FILENAME --intersect=FILENAME keep automata whose languages have an non-empty intersection with the automaton from FILENAME --is-complete keep complete automata --is-deterministic keep deterministic automata --is-empty keep automata with an empty language --is-inherently-weak keep only inherently weak automata --is-terminal keep only terminal automata --is-unambiguous keep only unambiguous automata --is-weak keep only weak automata --reject-word=WORD keep automata that reject WORD --states=RANGE keep automata whose number of states are in RANGE -u, --unique do not output the same automaton twice (same in the sense that they are isomorphic) -v, --invert-match select non-matching automata #+end_example For instance =--states=2..5 --acc-sets=3= will /keep/ only automata that use 3 acceptance sets, and that have between 2 and 5 states. Except for =--unique=, all these filters can be inverted. Using =--states=2..5 --acc-sets=3 -v= will /drop/ all automata that use 3 acceptance sets and that have between 2 and 5 states, and keep the others. * Simplifying automata The standard set of automata simplification routines (these are often referred to as the "post-processing" routines, because these are the procedures performed by [[file:ltl2tgba.org][=ltl2tgba=]] after translating a formula into a TGBA) are available through the following options. This set of options controls the desired type of output automaton: #+BEGIN_SRC sh :results verbatim :exports results autfilt --help | sed -n '/Output automaton type:/,/^$/p' | sed '1d;$d' #+END_SRC #+RESULTS: : -B, --ba Büchi Automaton (with state-based acceptance) : -C, --complete output a complete automaton : -G, --generic any acceptance is allowed (default) : -M, --monitor Monitor (accepts all finite prefixes of the given : property) : -S, --state-based-acceptance, --sbacc : define the acceptance using states : --tgba Transition-based Generalized Büchi Automaton These options specify any simplification goal: #+BEGIN_SRC sh :results verbatim :exports results autfilt --help | sed -n '/Simplification goal:/,/^$/p' | sed '1d;$d' #+END_SRC #+RESULTS: : -a, --any no preference, do not bother making it small or : deterministic : -D, --deterministic prefer deterministic automata : --small prefer small automata Finally, the following switches control the amount of effort applied toward the desired goal: #+BEGIN_SRC sh :results verbatim :exports results autfilt --help | sed -n '/Simplification level:/,/^$/p' | sed '1d;$d' #+END_SRC #+RESULTS: : --high all available optimizations (slow) : --low minimal optimizations (fast) : --medium moderate optimizations By default, =--any --low= is used, which cause all simplifications to be skipped. However if any goal is given, than the simplification level defaults to =--high= (unless specified otherwise). If a simplification level is given without specifying any goal, then the goal default to =--small=. So if you want to reduce the size of the automaton, try =--small= and if you want to try to make (or keep) it deterministic use =--deterministic=. Note that the =--deterministic= flag has two possible behaviors depending on the constraints on the acceptance conditions: - When =autfilt= is configured to work with generic acceptance (the =--generic= option, which is the default), then the =--deterministic= flag will do whatever it takes to output a deterministic automaton, and this includes changing the acceptance condition if needed (see below). - If options =--tgba= or =--ba= are used, the =--deterministic= option is taken as a /preference/: =autfilt= will try to favor determinism in the output, but it may not always succeed and may output non-deterministic automata. Note that if =autfilt --deterministic --tgba= fails to output a deterministic automaton, it does not necessarily implies that a deterministic TGBA does not exist: it just implies that =autfilt= could not find it. ** Determinization Spot has basically two ways to determinize automata, and that it uses when =--deterministic= is passed. - Automata that express obligation properties (this can be decided), can be *determinized and minimized* into weak Büchi automata, as discussed by [[http://www.daxc.de/eth/paper/atva07.pdf][Dax at al. (ATVA'07)]]. - Büchi automata (preferably with transition-based acceptance) can be determinized into parity automata using a Safra-like procedure close to the one presented by [[http://www.romanredz.se/papers/FI2012.pdf][Redziejowski (Fund. Inform. 119)]], with a few additional tricks. This procedure does not necessarily produce a minimal automaton. When =--deterministic= is used, the first of these two procedures is attempted on any supplied automaton. (It's even attempted for deterministic automata, because that might reduce them.) If that first procedure failed, and the input automaton is not deterministic and =--generic= (the default for =autfilt=) is used, then the second procedure is used. In this case, automata will be first converted to transition-based Büchi automata if their condition is more complex. * Transformations The following transformations are available: #+BEGIN_SRC sh :results verbatim :exports results autfilt --help | sed -n '/Transformations:/,/^$/p' | sed '1d;$d' #+END_SRC #+RESULTS: #+begin_example --cleanup-acceptance remove unused acceptance sets from the automaton --cnf-acceptance put the acceptance condition in Conjunctive Normal Form --complement complement each automaton (currently support only deterministic automata) --complement-acceptance complement the acceptance condition (without touching the automaton) --decompose-strength=t|w|s extract the (t) terminal, (w) weak, or (s) strong part of an automaton (letters may be combined to combine more strengths in the output) --destut allow less stuttering --dnf-acceptance put the acceptance condition in Disjunctive Normal Form --exclusive-ap=AP,AP,... if any of those APs occur in the automaton, restrict all edges to ensure two of them may not be true at the same time. Use this option multiple times to declare independent groups of exclusive propositions. --instut[=1|2] allow more stuttering (two possible algorithms) --keep-states=NUM[,NUM...] only keep specified states. The first state will be the new initial state. Implies --remove-unreachable-states. --mask-acc=NUM[,NUM...] remove all transitions in specified acceptance sets --merge-transitions merge transitions with same destination and acceptance --product=FILENAME, --product-and=FILENAME build the product with the automaton in FILENAME to intersect languages --product-or=FILENAME build the product with the automaton in FILENAME to sum languages --randomize[=s|t] randomize states and transitions (specify 's' or 't' to randomize only states or transitions) --remove-ap=AP[=0|=1][,AP...] remove atomic propositions either by existential quantification, or by assigning them 0 or 1 --remove-dead-states remove states that are unreachable, or that cannot belong to an infinite path --remove-fin rewrite the automaton without using Fin acceptance --remove-unreachable-states remove states that are unreachable from the initial state --sat-minimize[=options] minimize the automaton using a SAT solver (only work for deterministic automata) --separate-sets if both Inf(x) and Fin(x) appear in the acceptance condition, replace Fin(x) by a new Fin(y) and adjust the automaton --simplify-exclusive-ap if --exclusive-ap is used, assume those AP groups are actually exclusive in the system to simplify the expression of transition labels (implies --merge-transitions) --strip-acceptance remove the acceptance condition and all acceptance sets #+end_example * Examples Here is an automaton with transition-based acceptance: #+BEGIN_SRC sh :results silent :exports both cat >aut-ex1.hoa<❶) & Fin(❸) & Inf(⓿)) | (Inf(❷)&Inf(❸)) | Inf(❶)> labelloc="t" fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label="0"] 0 -> 0 [label=<1
❸>] 0 -> 1 [label=❶❸>] 0 -> 2 [label=⓿❸>] 1 [label="1"] 1 -> 0 [label=❸>] 1 -> 1 [label=⓿❸>] 1 -> 2 [label=❷❸>] 2 [label="2"] 2 -> 0 [label=] 2 -> 1 [label=⓿>] 2 -> 2 [label=⓿>] } #+end_example #+BEGIN_SRC dot :file autfilt-ex1.png :cmdline -Tpng :var txt=autfilt-ex1 :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex1.png]] Using =-S= will "push" the acceptance membership of the transitions to the states: #+NAME: autfilt-ex2 #+BEGIN_SRC sh :results verbatim :exports code autfilt -S aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex2 #+begin_example digraph G { rankdir=LR label=<(Fin(❶) & Fin(❸) & Inf(⓿)) | (Inf(❷)&Inf(❸)) | Inf(❶)> labelloc="t" node [shape="circle"] fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label=<0
❸>] 0 -> 0 [label=<1>] 0 -> 1 [label=] 0 -> 2 [label=] 1 [label=<1
❶❸>] 1 -> 0 [label=] 1 -> 6 [label=] 1 -> 7 [label=] 2 [label=<2
⓿❸>] 2 -> 3 [label=] 2 -> 4 [label=] 2 -> 5 [label=] 3 [label=<3>] 3 -> 0 [label=<1>] 3 -> 1 [label=] 3 -> 2 [label=] 4 [label=<4
⓿>] 4 -> 0 [label=] 4 -> 6 [label=] 4 -> 7 [label=] 5 [label=<5
⓿>] 5 -> 3 [label=] 5 -> 4 [label=] 5 -> 5 [label=] 6 [label=<6
⓿❸>] 6 -> 0 [label=] 6 -> 6 [label=] 6 -> 7 [label=] 7 [label=<7
❷❸>] 7 -> 3 [label=] 7 -> 4 [label=] 7 -> 5 [label=] } #+end_example #+BEGIN_SRC dot :file autfilt-ex2.png :cmdline -Tpng :var txt=autfilt-ex2 :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex2.png]] Using =--cnf-acceptance= simply rewrites the acceptance condition in Conjunctive Normal Form: #+NAME: autfilt-ex3 #+BEGIN_SRC sh :results verbatim :exports code autfilt --cnf-acceptance aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex3 #+begin_example digraph G { rankdir=LR label=<(Inf(⓿) | Inf(❶) | Inf(❸)) & (Fin(❸) | Inf(❶) | Inf(❷))> labelloc="t" fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label="0"] 0 -> 0 [label=<1
❸>] 0 -> 1 [label=❶❸>] 0 -> 2 [label=⓿❸>] 1 [label="1"] 1 -> 0 [label=❸>] 1 -> 1 [label=⓿❸>] 1 -> 2 [label=❷❸>] 2 [label="2"] 2 -> 0 [label=] 2 -> 1 [label=⓿>] 2 -> 2 [label=⓿>] } #+end_example #+BEGIN_SRC dot :file autfilt-ex3.png :cmdline -Tpng :var txt=autfilt-ex3 :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex3.png]] Using =--remove-fin= transforms the automaton to remove all traces of Fin-acceptance: this usually requires adding non-deterministic jumps to altered copies of strongly-connected components. #+NAME: autfilt-ex4 #+BEGIN_SRC sh :results verbatim :exports code autfilt --remove-fin aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex4 #+begin_example digraph G { rankdir=LR label=⓿) | Inf(❶) | (Inf(❷)&Inf(❸))> labelloc="t" fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label="0"] 0 -> 0 [label=<1
❸>] 0 -> 1 [label=❶❸>] 0 -> 2 [label=❸>] 1 [label="1"] 1 -> 0 [label=❸>] 1 -> 1 [label=❸>] 1 -> 2 [label=❷❸>] 2 [label="2"] 2 -> 0 [label=] 2 -> 1 [label=] 2 -> 2 [label=] 2 -> 3 [label=] 3 [label="3"] 3 -> 3 [label=⓿>] } #+end_example #+BEGIN_SRC dot :file autfilt-ex4.png :cmdline -Tpng :var txt=autfilt-ex4 :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex4.png]] Use =--mask-acc=NUM= to remove some acceptances sets and all transitions they contain. The acceptance condition will be updated to reflect the fact that these sets can never be visited. #+NAME: autfilt-ex5 #+BEGIN_SRC sh :results verbatim :exports code autfilt --mask-acc=1,2 aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex5 #+begin_example digraph G { rankdir=LR label=❶) & Inf(⓿)> labelloc="t" fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label="0"] 0 -> 0 [label=<1
❶>] 0 -> 1 [label=⓿❶>] 1 [label="1"] 1 -> 0 [label=] 1 -> 2 [label=⓿>] 1 -> 1 [label=⓿>] 2 [label="2"] 2 -> 0 [label=❶>] 2 -> 2 [label=⓿❶>] } #+end_example #+BEGIN_SRC dot :file autfilt-ex5.png :cmdline -Tpng :var txt=autfilt-ex5 :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex5.png]] Atomic propositions can be removed from an automaton in three ways: - use ~--remove-ap=a~ to remove =a= by existential quantification, i.e., both =a= and its negation will be replaced by true. This does not remove any transition. - use ~--remove-ap=a=0~ to keep only transitions compatible with =!a= (i.e, transitions requiring =a= will be removed). - use ~--remove-ap=a=1~ to keep only transitions compatible with =a= (i.e, transitions requiring =!a= will be removed). Here are the results of these three options on our example: #+NAME: autfilt-ex6a #+BEGIN_SRC sh :results verbatim :exports code autfilt --remove-ap=a aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex6a #+begin_example digraph G { rankdir=LR label=<(Fin(❶) & Fin(❸) & Inf(⓿)) | (Inf(❷)&Inf(❸)) | Inf(❶)> labelloc="t" fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label="0"] 0 -> 0 [label=<1
❸>] 0 -> 1 [label=<1
❶❸>] 0 -> 2 [label=<1
⓿❸>] 1 [label="1"] 1 -> 0 [label=❸>] 1 -> 1 [label=⓿❸>] 1 -> 2 [label=❷❸>] 2 [label="2"] 2 -> 0 [label=] 2 -> 1 [label=⓿>] 2 -> 2 [label=⓿>] } #+end_example #+BEGIN_SRC dot :file autfilt-ex6a.png :cmdline -Tpng :var txt=autfilt-ex6a :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex6a.png]] #+NAME: autfilt-ex6b #+BEGIN_SRC sh :results verbatim :exports code autfilt --remove-ap=a=0 aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex6b #+begin_example digraph G { rankdir=LR label=<(Fin(❶) & Fin(❸) & Inf(⓿)) | (Inf(❷)&Inf(❸)) | Inf(❶)> labelloc="t" fontname="Lato" node [fontname="Lato"] edge [fontname="Lato"] node[style=filled, fillcolor="#ffffa0"] edge[arrowhead=vee, arrowsize=.7] I [label="", style=invis, width=0] I -> 0 0 [label="0"] 0 -> 0 [label=<1
❸>] 0 -> 1 [label=<1
⓿❸>] 1 [label="1"] 1 -> 0 [label=] 1 -> 1 [label=⓿>] } #+end_example #+BEGIN_SRC dot :file autfilt-ex6b.png :cmdline -Tpng :var txt=autfilt-ex6b :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex6b.png]] #+NAME: autfilt-ex6c #+BEGIN_SRC sh :results verbatim :exports code autfilt --remove-ap=a=1 aut-ex1.hoa --dot=.a #+END_SRC #+RESULTS: autfilt-ex6c #+BEGIN_SRC dot :file autfilt-ex6c.png :cmdline -Tpng :var txt=autfilt-ex6c :exports results $txt #+END_SRC #+RESULTS: [[file:autfilt-ex6c.png]] #+BEGIN_SRC sh :results silent :exports results rm -f example.hoa aut-ex1.hoa #+END_SRC The following example checks whether the formula ~a U b U c~ accepts the word ~b; cycle{c}~. #+BEGIN_SRC sh :results verbatim :exports both ltl2tgba 'a U b U c' | autfilt --accept-word 'b; cycle{c}' -q && echo "word accepted" #+END_SRC #+RESULTS: : word accepted Here is an example where we generate an infinite stream of random LTL formulas using [[file:randltl.org][=randltl=]], convert them all to automata using [[file:ltl2tgba.org][=ltl2tgba=]], filter out the first 10 automata that accept both the words =a&!b;cycle{!a&!b}= and =!a&!b;cycle{a&b}= yet reject any word of the form =cycle{b}=, and display the associated formula (which was stored as the name of the automaton by =ltl2tgba=). #+BEGIN_SRC sh :results verbatim :exports both randltl -n -1 a b | ltlfilt --simplify --uniq | ltl2tgba -F- | autfilt --accept-word='a&!b;cycle{!a&!b}' --accept-word='!a&!b;cycle{a&b}' \ --reject-word='cycle{b}' --stats=%M -n 10 #+END_SRC #+RESULTS: #+begin_example F!b !b F(!a & !b) (!a & (XX!a | (!a W F!b))) R !b F(Fb R !b) Fa R F!b Fa U !b !b & X(!b W Ga) Fb R F!b XF!b U (!b & (!a | G!b)) #+end_example Note that the above example could be simplified using the =--accept-word= and =--reject-word= options of =ltlfilt= directly. However this demonstrates that using =--stats=%M=, it is possible to filter formulas based on some properties of automata that have been generated by from them. The translator needs not be =ltl2tgba=: other tools can be wrapped with [[file:ltldo.org][=ltldo --name=%f=]] to ensure they work well in a pipeline and preserve the formula name in the HOA output. For example Here is a list of 5 LTL formulas that =ltl2dstar= converts to Rabin automata that have exactly 4 states: #+BEGIN_SRC sh :results verbatim :exports both randltl -n -1 a b | ltlfilt --simplify --remove-wm | ltldo ltl2dstar --name=%f | autfilt --states=4 --stats=%M -n 5 #+END_SRC #+RESULTS: : Gb | G!b : b R (a | b) : (a & !b & (b | (F!a U (!b & F!a)))) | (!a & (b | (!b & (Ga R (b | Ga))))) : (a & (a U !b)) | (!a & (!a R b)) : a | G((a & GFa) | (!a & FG!a))