Phoronix Test Suite

Easy-To-Use

The Phoronix Test Suite makes the process of carrying out automated tests incredibly simple. As long as you have PHP installed, in most instances it is just a matter of running a command like phoronix-test-suite benchmark unigine-valley, and the Phoronix Test Suite will take care of the rest from dependency management to test installation, execution, and result aggregation.

Extensible Architecture

The Phoronix Test Suite ships with access to more than 450 test profiles and over 100 test suites. These tests range from battery power consumption monitoring for mobile devices to multi-threaded ray-tracing benchmarks and span the CPU, graphics, system memory, disk storage, and motherboard components. If there is a test though not currently covered by the Phoronix Test Suite, new tests can be quickly added via its extensible architecture (see the documentation) with each profile just being comprised of XML files and a few simple scripts. The Phoronix Test Suite also supports Cascading Test Profiles (CTP) whereby a test is able to extend and leverage an already existing profile.

Statistical Accuracy

Regardless of the test profile, if the Phoronix Test Suite detects a test(s) standard deviation between runs is exceeding a predefined threshold, the Phoronix Test Suite can automatically call on that test to be executed additional times to ensure the reported result is accurate. Other statistical and analytical options are also available.

Cost Savings

Integrating the Phoronix Test Suite within your existing test infrastructure can greatly reduce your company's costs by no longer needing to pay developers to develop an in-house testing stack. The Phoronix Test Suite is open-source and licensed under the GNU GPL. Those looking to deploy the Phoromatic server within an intranet can do so at low cost and we offer other custom services to enterprise clients.

System Monitoring

Through another Phoronix Test Suite module it is possible to log in real-time various system sensors like the CPU temperature, the battery power consumption, disk read/write speeds, and numerous other sensors. This can be done while any test profile is running and the recorded results are then provided within the results viewer. It is as simple as running a command such as MONITOR=all phoronix-test-suite benchmark x264.

Multi-Platform

The Phoronix Test Suite traditionally has been primarily targeted at conducting Linux-based tests, but the Phoronix Test Suite will also work on Solaris, Mac OS X, Windows, Hurd, and BSD operating systems. The Phoronix Test Suite can also be ported to other platforms too.

Side-By-Side Comparisons

The Phoronix Test Suite makes it impeccably easy to compare side-by-side performance and result comparisons with your friends or associates across the Internet (especially using the OpenBenchmarking.org online database) or within an organization when seeking to compare different software/hardware components.

System Software / Hardware

The Phoronix Test Suite is capable of reporting detailed information about installed system software and hardware. Information includes the major hardware components and more meticulous details like clock information for the graphics processor and installed driver versions. Phodevi powers this, which is the Phoronix Device Interface library created for the Phoronix Test Suite. This library is also equipped with system software/hardware sensor monitoring support.

Phoromatic

This web and client implementation provides a remote management system that allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location. Like the Phoronix Test Suite, Phoromatic is open-source and is available for use free of charge.

Phoromatic Tracker

The Phoromatic Tracker is an extension of Phoromatic that allows the tracking of a code-base / system for any performance regressions. Tests can be run on a scheduled basis like within Phoromatic or also on a per-commit basis or when triggered by an external call.

Autonomous Regression Bisecting

Via a Phoronix Test Suite module, it is possible for the Phoronix Test Suite to find a performance regression within a software code-base if a regression can be illustrated within a Phoronix Test Suite test case. If using a revision control system like Git, it is possible to supply the necessary pieces of information to this module, which will then bisect the repository and carry out the test(s) each step of the way until the regression is spotted. We have even used this to find regressions in the Linux kernel!

Image Quality Comparisons

The Phoronix Test Suite is not limited to conducting tests that produce a numerical result. For example, image quality comparisons can also be carried out using the Phoronix Test Suite for studying visual differences in rendered scenes.

Result Recording

Beyond just recording the actual test value, the Phoronix Test Suite is also able to archive system logs along with the test logs and the installation logs. From the Phoronix Test Suite result XML output you can "slice and dice" the data however you wish.

PTS Desktop Live

PTS Desktop Live is our own Linux operating system that we have engineered to make it even easier to conduct comparisons between different sets of hardware. Simply boot the operating system from a USB device or DVD and the Phoronix Test Suite will automatically launch and prompt the user for the test(s) to run. PTS Desktop Live eliminates any software differences in conducting such comparisons by providing a single, standardized software stack.

90+ Commands

There are over 90 commands that are accessible via the Phoronix Test Suite command line. The Phoronix Test Suite simply is not limited to installing and running tests, but it also provides a wealth of options for analyzing test results, merging multiple test sets, easily building new test suites, validating test profiles, and much more. Run phoronix-test-suite help to find out more or cite the PDF documentation.

PTS Features