Test analysis & comparison
Companioning paper written by Frank Brüseke, Steffen Becker, Gregor Engels
Licenses and Copyright
- The scripts and models on this website are made available under the regulations of the GNU Lesser General Public License LGPL
- The data sets, the resulting diagrams and tables, and also this website are made available under the regulations
of the Creative Commons Attribution-ShareAlike 3.0 Unported License
Prerequisites
- Ensure that the R "bin"-directory is on your PATH environment variable
- Be sure you know the path to your dedicated folder "<dataPath>", where you keep all example data and scripts
- Download the All-in-one script zip-file: [Zip]
- Unpack to the Zip to <dataPath>
- Download the raw data series file to <dataPath>: [File]
- Download the PCM (aka Palladio) data series: [Zip]
- Unpack the PCM data series to <dataPath>
- Download the PCM to Implementation mapping file to <dataPath>: [File]
Steps
Hints:
Most of the steps are done at the command line, the python scripts usually support parameters and the "--help" parameter will show sensible help messages.
If you cannot start python scripts directly prefix "python" or "python.exe" to the command line (assuming the python executable is on your PATH)
- Open a command line and "cd" to <dataPath>
- Inline all the stack traces:
- Short cut to step 4:
download the compressed data series file to <dataPath>: [File] - on the command line use the command:
inlineJstack.py -i raw_measurements.txt -o inlined_measurements.txt - This will produce the file "inlined_measurements.txt" with all the compressed measurements. Instead of the stack traces being shown on multiple lines each line now stands for one measurement with a compressed stack trace.
- This will also produce the file "inlined_measurements.traces.txt"
which contains the form for the PCM to implementation mapping. It
contains all the different stack traces observed in the data file along
with their compressed name. In the third column the system architect has to
fill in the matching PCM simulation data series (as relative path to the file
containing the PCM simulation data series). In this example you do not need to do
this as we will work with a pre-made mapping file called
"mapping4.txt".
- Configure conversion factors:
- In this version of the script the conversion factors are hard coded in the file
"compareTestAndSimu.py" (fixed in the subsequent version). So replace the factors in the following lines:
...
143: data1.curr <- 0.001*0.001*data1$duration[which(data1$stacktrace==mapping$key[index])]
...
146: data2.curr <- data2$Time.Span*1000
... - Start the analysis script with the following command:
compareTestAndSimu.py -t inlined_measurements.txt -m mapping4.txt -o analysis1 - If all goes well this command produces a lot of files:
- The performance report: analysis1.RR.pdf
- The extended performance report: analysis1.R.pdf
- A series of single box plots (for the nice figures in the paper)
- The blame graph: analysis1.flame.csv2.svg
- For troubleshooting refer to the command line output and the file: analysis1.R.out.txt
- All other file are intermediate data or script files
