Analyzing the energy consumption of an application with EZTrace

Required

In order to collect energy consumption information in EZTrace, you need:

A processor that provides RAPL. This means a 'recent' Intel processor (Sandy Bridge, Haswell, etc.)
A PAPI installation with the RAPL component

Checking PAPI installation

Configuring PAPI to support RAPL

When configuring PAPI, you need to provide the --with-components=rapl option. Basically, you need to install PAPI like this: $ tar xzvf papi-5.3.2.tar.gz $ cd papi-5.3.2/src $ ./configure --with-components=rapl &&make &&make install

Checking the PAPI installation

Once PAPI is installed, make sure it is capable of reading RAPL information. For instance, you can run this rapl-read program. You may need to modify the Makefile in order to specify the installation directory of PAPI. Once compiled, running the rapl-read program should result in the following output: $ ./rapl_plot Found rapl component at cid 2 Found: rapl:::THERMAL_SPEC_CNT:PACKAGE0 Found: rapl:::MINIMUM_POWER_CNT:PACKAGE0 Found: rapl:::MAXIMUM_POWER_CNT:PACKAGE0 Found: rapl:::MAXIMUM_TIME_WINDOW_CNT:PACKAGE0 Found: rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 Found: rapl:::PP1_ENERGY_CNT:PACKAGE0 Found: rapl:::DRAM_ENERGY_CNT:PACKAGE0 Found: rapl:::PP0_ENERGY_CNT:PACKAGE0 Found: rapl:::THERMAL_SPEC:PACKAGE0 Found: rapl:::MINIMUM_POWER:PACKAGE0 Found: rapl:::MAXIMUM_POWER:PACKAGE0 Found: rapl:::MAXIMUM_TIME_WINDOW:PACKAGE0 Found: rapl:::PACKAGE_ENERGY:PACKAGE0 Found: rapl:::PP1_ENERGY:PACKAGE0 Found: rapl:::DRAM_ENERGY:PACKAGE0 Found: rapl:::PP0_ENERGY:PACKAGE0 [...] 4.7213 0.0 (* Average Power for rapl:::MAXIMUM_TIME_WINDOW:PACKAGE0 *) 4.7213 3.0 (* Average Power for rapl:::PACKAGE_ENERGY:PACKAGE0 *) 4.7213 0.2 (* Average Power for rapl:::PP1_ENERGY:PACKAGE0 *) 4.7213 0.8 (* Average Power for rapl:::DRAM_ENERGY:PACKAGE0 *) 4.7213 0.1 (* Average Power for rapl:::PP0_ENERGY:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::THERMAL_SPEC_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::MINIMUM_POWER_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::MAXIMUM_POWER_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::MAXIMUM_TIME_WINDOW_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::PP1_ENERGY_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::DRAM_ENERGY_CNT:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::PP0_ENERGY_CNT:PACKAGE0 *) 4.8218 46156882941.9 (* Average Power for rapl:::THERMAL_SPEC:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::MINIMUM_POWER:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::MAXIMUM_POWER:PACKAGE0 *) 4.8218 0.0 (* Average Power for rapl:::MAXIMUM_TIME_WINDOW:PACKAGE0 *) 4.8218 2.9 (* Average Power for rapl:::PACKAGE_ENERGY:PACKAGE0 *) 4.8218 0.2 (* Average Power for rapl:::PP1_ENERGY:PACKAGE0 *) 4.8218 0.8 (* Average Power for rapl:::DRAM_ENERGY:PACKAGE0 *)

Can't open fd for cpu0: No such file or director

However, You may have the following error: $ ./rapl_plot Found rapl component at cid 2 No rapl events found: Can't open fd for cpu0: No such file or director This usually means that the msr kernel module that permits to read the energy counters is not loaded. This should be fixed by running modprobe: $ sudo modprobe msr

Can't open fd for cpu0: Operation not permitted

Another possible error is:$ ./rapl_plot Found rapl component at cid 2 No rapl events found: Can't open fd for cpu0: Operation not permitted In that case, you may have to run the program as sudo: $ sudo ./rapl_plot [...]

Installing EZTrace

Then, configure EZTrace as described in the support page and make sure to specify where PAPI is installed.

$ ./configure
--with-papi=<PATH_TO_PAPI>
--prefix=<INSTALLATION_DIR>

If everything is OK, compile and install EZTrace:


$ make &&make install

Check the installation of EZTrace:


$ eztrace_avail

3       stdio   Module for
stdio functions (read, write, select, poll, etc.)

2       pthread Module for PThread
synchronization functions (mutex, semaphore, spinlock, etc.)

1       omp    
Module for OpenMP parallel regions

4       mpi    
Module for MPI functions

5       memory  Module for memory
functions (malloc, free, etc.)

6       papi  Module for PAPI Performance counters

Running an application with EZTrace

Let's analyze this Pthread application. First, compile it:
$ tar xzvf pthread_simple.tgz [...] $ make cc -lpthread pthread_simple.c -o pthread_simple
You can run the application with:


$ ./pthread_simple

First, let's analyze pthread events in the application. To do this, simple add eztrace -t pthread to the command line:


$ eztrace -t pthread ./pthread_simple

This will generate a trace file in /tmp/<username>_eztrace_log_rank_1.
You can generate a trace file that can be visualized with ViTE, using the eztrace_convert utility:
$ eztrace_convert /tmp/${USER}_eztrace_log_rank_1 [...] $ vite eztrace_output.trace

Getting energy consumption information

In order to integrate energy consumption information in the trace, we need to tell EZTrace to use PAPI and to collect a specific hardware counter $ export EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 $ eztrace -t "pthread papi" ./pthread_simple Then, run the application with eztrace and specify the pthread and papi moduless: $ eztrace -t "pthread papi" ./pthread_simple Depending on the privileges required to access the RAPL counters, you may have to run the command as sudo: $ sudo EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 eztrace -t "pthread papi" ./pthread_simple After converting the generated trace, you should be able to visualize the energy consumption of each thread of the application: $ eztrace_convert /tmp/${USER}_eztrace_log_rank_1 [...] $ vite eztrace_output.trace

Tuning

Changing the probe frequency

Because gathering hardware counters information can be time consuming (and thus impacts the behavior of the application), EZTrace waits at least 100 microseconds (by default) between each probe. As a result, the energy consumption information display in th previous trace does not allow to distinguish the consumption of the different phases of the application.

You can change the frequency of the probes by setting the EZTRACE_PAPI_SAMPLE_INTERVAL variable:

$ sudo EZTRACE_PAPI_SAMPLE_INTERVAL=1 EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 eztrace -t "pthread papi" ./pthread_simple [...] $ eztrace_convert /tmp/${USER}_eztrace_log_rank_1 $ vite eztrace_output.trace

Using `sigalarm`

Note that even if you ask EZTrace to probe RAPL counters every microsecond, EZTrace only records the information when another event is recorded. This means that if your application enters a long computation phase during which EZTrace does not record any event, the energy consumption information will not be recorded.

In order to collect energy consumption information during computation phase, EZTrace can use a timer that interrupts the application every x milliseconds. To enable EZTrace timer, set the EZTRACE_SIGALARM variable:

$ sudo EZTRACE_SIGALARM=100 EZTRACE_PAPI_SAMPLE_INTERVAL=1 EZTRACE_PAPI_COUNTERS=rapl:::PACKAGE_ENERGY_CNT:PACKAGE0 eztrace -t "pthread papi" ./pthread_simple Starting EZTrace... [EZTrace] Setting an alarm every 100 ms [...] $ eztrace_convert /tmp/${USER}_eztrace_log_rank_1 $ vite eztrace_output.trace

This tutorial was designed for EZTrace 1.0 or higher. If you encounter any problem, feel free to contact EZTrace developpers.