Metadata-Version: 2.0
Name: BenchExec
Version: 0.5
Summary: A Framework for Reliable Benchmarking and Resource Measurement.
Home-page: https://github.com/dbeyer/benchexec/
Author: Dirk Beyer
Author-email: UNKNOWN
License: Apache 2.0 License
Keywords: benchmarking resource measurement
Platform: Linux
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: System :: Benchmark
Requires-Dist: tempita (==0.5.2)

BenchExec
=========

A Framework for Reliable Benchmarking and Resource Measurement
--------------------------------------------------------------

|Build Status|

BenchExec provides three major features:

-  execution of arbitrary commands with precise and reliable measurement
   and limitation of resource usage (e.g., CPU time and memory)
-  an easy way to define benchmarks with specific tool configurations
   and resource limits, and automatically executing them on large sets
   of input files
-  generation of interactive tables and plots for the results

Contrary to other benchmarking frameworks, it is able to reliably
measure and limit resource usage of the benchmarked tool even if it
spawns subprocesses. In order to achieve this, it uses the `cgroups
feature <https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt>`__
of the Linux kernel to correctly handle groups of processes. BenchExec
is intended for benchmarking non-interactive tools on Linux systems. It
measures CPU time, wall time, and memory usage of a tool, and allows to
specify limits for these resources. It also allows to limit the CPU
cores and (on NUMA systems) memory regions. In addition to measuring
resource usage, BenchExec can verify that the result of the tool was as
expected, and extract further statistical data from the output. Results
from multiple runs can be combined into CSV and interactive HTML tables,
of which the latter provide scatter and quantile plots.

BenchExec was originally developed for use with the software
verification framework `CPAchecker <http://cpachecker.sosy-lab.org>`__
and is now developed as an independent project at the `Software Systems
Lab <http://www.sosy-lab.org>`__ at the `University of
Passau <http://www.uni-passau.de>`__.

Links
~~~~~

-  `Documentation <https://github.com/dbeyer/benchexec/tree/master/doc/INDEX.md>`__
-  `Changelog <https://github.com/dbeyer/benchexec/tree/master/CHANGELOG.md>`__
-  `BenchExec GitHub
   Repository <https://github.com/dbeyer/benchexec>`__, use this for
   `reporting issues and asking
   questions <https://github.com/dbeyer/benchexec/issues>`__
-  `BenchExec at PyPI <https://pypi.python.org/pypi/BenchExec>`__

Users of BenchExec
~~~~~~~~~~~~~~~~~~

BenchExec was successfully used for benchmarking in all four instances
of the `International Competition on Software
Verification <http://sv-comp.sosy-lab.org>`__ with a wide variety of
benchmarked tools and hundreds of thousands benchmark runs.

The developers of the following tools use BenchExec:

-  `CPAchecker <http://cpachecker.sosy-lab.org>`__, also for regression
   testing
-  `SMACK <https://github.com/smackers/smack>`__

If you would like to be listed here, `contact
us <https://github.com/dbeyer/benchexec/issues/new>`__.

.. |Build Status| image:: https://travis-ci.org/dbeyer/benchexec.svg?branch=master
   :target: https://travis-ci.org/dbeyer/benchexec


