alarsyo/spot
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

bin: handle thousands of output files

Browse source 
Fixes #534.

* bin/common_file.hh, bin/common_file.cc: Make it possible
to reopen a closed file.
* bin/common_output.cc, bin/common_aoutput.cc: Add a heuristic
to decide when to close files.
* tests/core/serial.test: Add a test case.
* NEWS: Mention the issue.
This commit is contained in:
Alexandre Duret-Lutz 2023-07-24 16:56:24 +02:00
parent 17a5b41d8c
commit 40e30df7e3
6 changed files with 76 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

20
bin/common_output.cc
View file

@ -429,7 +429,27 @@ output_formula_checked(spot::formula f, spot::process_timer* ptimer,
auto [it, b] = outputfiles.try_emplace(fname, nullptr);
if (b)
it->second.reset(new output_file(fname.c_str()));
else
// reopen if the file has been closed; see below
it->second->reopen_for_append(fname);
out = &it->second->ostream();
// If we have opened fewer than 10 files, we keep them all open
// to avoid wasting time on open/close calls.
//
// However we cannot keep all files open, especially in
// scenarios were we use thousands of files only once. To keep
// things simple, we only close the previous file if it is not
// the current output. This way we still save the close/open
// cost when consecutive formulas are sent to the same file.
static output_file* previous = nullptr;
static const std::string* previous_name = nullptr;
if (previous
&& outputfiles.size() > 10
&& &previous->ostream() != out)
previous->close(*previous_name);
previous = it->second.get();
previous_name = &it->first;
}
output_formula(*out, f, ptimer, filename, linenum, index, prefix, suffix);
*out << output_terminator;