Phoronix Test Suite

Phoronix Test Suite

The Leading Software For Automated, Open-Source Testing & Benchmarking

Latest Release
Phoronix Test Suite 5.4.0-Lipki
9 December 2014

About

Phoronix The Phoronix Test Suite was developed out of the tools developed at Phoronix.com since its founding in 2004. The Phoronix Test Suite is designed for Linux benchmarking but also supports other operating systems. Read more about the Phoronix Test Suite.

OpenBenchmarking.org

OpenBenchmarking.org OpenBenchmarking.org was launched with Phoronix Test Suite 3.0 as a means of sharing benchmark results with the greater community, collectively collaborate over test results, store test profiles and test suites within a centralized cloud repository, and carry out other tests. Visit OpenBenchmarking.org to learn more.

Phoromatic

Phoromatic Phoromatic is an enterprise-designed component to the Phoronix Test Suite to centrally manage entire test farms/clusters of systems, carry out tests on a time or triggered basis, and provide support for other centralized test management purposes. Learn more at Phoromatic.com.

Commercial Support

Commercial Support The Phoronix Test Suite's future development is dependent upon the continued support from organizations and other stakeholders taking advantage of this long-standing, open-source benchmarking software. We are happy to offer custom engineering services, commercial support, sponsorship of new open-source software features, custom test profiles, and other commercial support options for those interested in helping advance this automated Linux benchmarking software. Contact us to learn more.

Phoronix Test Suite Features

Easy-To-Use: The Phoronix Test Suite makes the process of carrying out automated tests incredibly simple. As long as you have PHP installed, in most instances, it is just a matter of running a command like phoronix-test-suite benchmark unigine-heaven, and the Phoronix Test Suite will take care of the rest from dependency management to test installation, execution, and the result aggregation.

Extensible Architecture: The Phoronix Test Suite ships with access to more than 450 test profiles and over 100 test suites. These tests range from battery power consumption monitoring for mobile devices to multi-threaded ray-tracing benchmarks and span the CPU, graphics, system memory, disk storage, and motherboard components. If there is a test though not currently covered by the Phoronix Test Suite, new tests can be quickly added via its extensible architecture (see the documentation) with each profile just being comprised of XML files and a few simple scripts. The Phoronix Test Suite also supports Cascading Test Profiles (CTP) whereby a test is able to extend and leverage an already existing profile.

Statistical Accuracy: Regardless of the test profile, if the Phoronix Test Suite detects a test(s) standard deviation between runs is exceeding a predefined threshold, the Phoronix Test Suite can automatically call on that test to be executed additional times to ensure the reported result is accurate.

Result Recording: Beyond just recording the actual test value, the Phoronix Test Suite is also able to archive system logs (such as dmesg, Xorg.0.log, lspci, and other relevant outputs) along with the test logs and the installation logs. Results from the Phoronix Test Suite are displayed in a web-based results viewer with optional support for uploading them to OpenBenchmarking.org. The Phoronix Test Suite also has an integrated graphing library for visually displaying test results in a variety of different formats. This library can also output graphs to JPEG, PNG, SVG, and Adobe SWF formats along with the option of customizing any colors and style of the graphs, so that you can go right from test execution to putting these graphs on your web-site or in a company report.

PTS Desktop Live: PTS Desktop Live is our own Linux operating system that we have engineered to make it even easier to conduct comparisons between different sets of hardware. Simply boot the operating system from a USB device or DVD and the Phoronix Test Suite will automatically launch and prompt the user for the test(s) to run. PTS Desktop Live eliminates any software differences in conducting such comparisons by providing a single, standardized software stack.

Multi-Platform: The Phoronix Test Suite traditionally has been primarily targeted at conducting Linux-based tests, but the Phoronix Test Suite will also work on Solaris, Mac OS X, Windows, and BSD operating systems.

Side-By-Side Comparisons: The Phoronix Test Suite makes it impeccably easy to compare side-by-side performance and result comparisons with your friends or associates across the Internet (especially using the OpenBenchmarking.org online database) or within an organization when seeking to compare different software/hardware components.

System Software / Hardware: The Phoronix Test Suite is capable of reporting detailed information about installed system software and hardware via the Phoronix Test Suite GUI or the phoronix-test-suite system-info command. Information includes the major hardware components and more meticulous details like clock information for the graphics processor and installed driver versions. Phodevi powers this, which is the Phoronix Device Interface library created for the Phoronix Test Suite. This library is also equipped with system software/hardware sensor monitoring support.

Phoromatic: This web and client implementation provides a remote management system that allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location. Accounts can be created at Phoromatic.com for free and organizations can license the Phoromatic server component from our commercial services for use within a corporate intranet.

Phoromatic Tracker: The Phoromatic Tracker is an extension of Phoromatic that allows the tracking of a code-base / system for any performance regressions. Tests can be run on a scheduled basis like within Phoromatic or also on a per-commit basis or when triggered by an external call. Two reference implementations include our daily Linux kernel tracker and daily Ubuntu tracker.

Autonomous Regression Bisecting: Via a Phoronix Test Suite module, it is possible for the Phoronix Test Suite to find a performance regression within a software code-base if a regression can be illustrated within a Phoronix Test Suite test case. If using a revision control system like Git, it is possible to supply the necessary pieces of information to this module, which will then bisect the repository and carry out the test(s) each step of the way until the regression is spotted. We have even used this to find a regression in the Linux kernel!

Image Quality Comparisons: The Phoronix Test Suite is not limited to conducting tests that produce a numerical result. For example, image quality comparisons can also be carried out using the Phoronix Test Suite for studying visual differences in rendered scenes.

System Monitoring: Through another Phoronix Test Suite module it is possible to log in real-time various system sensors like the CPU temperature, the battery power consumption, disk read/write speeds, and numerous other sensors. This can be done while any test profile is running and the recorded results are then provided within the results viewer. It is as simple as running a command such as MONITOR=all phoronix-test-suite benchmark x264.

90+ Commands: There are over 90 commands that are accessible via the Phoronix Test Suite command line. The Phoronix Test Suite simply is not limited to installing and running tests, but it also provides a wealth of options for analyzing test results, merging multiple test sets, easily building new test suites, validating test profiles, and much more. Run phoronix-test-suite help to find out more or cite the PDF documentation.

Cost Savings: Integrating the Phoronix Test Suite within your existing test infrastructure can greatly reduce your company's costs by no longer needing to pay developers to develop an in-house testing stack. The Phoronix Test Suite is open-source and licensed under the GNU GPL. Those looking to deploy the Phoromatic server within an intranet can do so at low cost and we offer other custom services to enterprise clients.