1 files changed, 100 insertions, 163 deletions

diff --git a/README.md b/README.md
index 57d9afa..a9975d9 100644
--- a/README.md
+++ b/README.md
@@ -70,7 +70,7 @@ OUT_DIR/[SCHED_(FILE|DIR)/]
         exec-err.txt  # Standard err '''
 ```
 
-*Defaults*: `SCHED_FILE = sched.py, PARAM_FILE = params.py, DURATION = 30, OUT_DIR = run-data/`
+*Defaults*: `SCHED_FILE = sched.py`, `PARAM_FILE = params.py`, `DURATION = 30`, `OUT_DIR = run-data/`
 
 This script reads *schedule files* (described below) and executes real-time task systems, recording all overhead, logging, and trace data which is enabled in the system. For example, if trace logging is enabled, rt-kernelshark is found in the path, but feather-trace is disabled (the devices are not present), only trace logs and rt-kernelshark logs will be recorded.
 
@@ -90,7 +90,7 @@ Schedule files have one of the following two formats:
        [real_time_task] task_arguments...
 ```
 
-b) python format
+2. python format
 ```python
        {'proc':[
            ('path/to/proc','proc_value'),
@@ -142,7 +142,7 @@ $ echo "release_master{2}
 A longer form can be used for proc entries not under `/proc/litmus`:
 
 ```bash
-$ echo "/proc/sys/something{hello}"
+$ echo "/proc/sys/something{hello}
 10 20" > test.sched
 ```
 
@@ -152,7 +152,7 @@ You can specify your own spin programs to run as well instead of rtspin by putti
 $ echo "colorspin -f color1.csv 10 20" > test.sched
 ```
 
-You can specify parameters for an experiment in a file instead of on the command line using params.py (the `-p` option lets you choose the name of this file if params.py is not for you):
+You can specify parameters for an experiment in a file instead of on the command line using params.py (the `-p` option lets you choose the name of this file if `params.py` is not for you):
 
 ```bash
 $ echo "{'scheduler':'GSN-EDF', 'duration':10}" > params.py
@@ -166,26 +166,19 @@ You can include non-relevant parameters which `run_exps.py` does not understand
 ```bash
 $ mkdir test1
 # The duration will default to 30 and need not be specified
-$ echo "{'scheduler':'C-EDF', 'test-param':1} > test1/params.py
+$ echo "{'scheduler':'C-EDF', 'test-param':1}" > test1/params.py
 $ echo "10 20" > test1/sched.py
 $ cp -r test1 test2
 $ echo "{'scheduler':'GSN-EDF', 'test-param':2}"> test2/params.py
 $ run_exps.py test*
 ```
 
-Finally, you can specify system properties in `params.py` which the environment must match for the experiment to run. These are useful if you have a large batch of experiments which must be run under different kernels or kernel configurations. The first property is a regular expression for the name of the kernel:Invalid environment for experiment 'test.sched'
-Kernel name does not match '.*linux.*'.
-Experiments run:        1
-  Successful:           0
-  Failed:               0
-  Already Done:         0
-  Invalid Environment:  1
+Finally, you can specify system properties in `params.py` which the environment must match for the experiment to run. These are useful if you have a large batch of experiments which must be run under different kernels or kernel configurations. The first property is a regular expression for the name of the kernel:
 
 ```bash
 $ uname -r
 3.0.0-litmus
-$ cp params.py old_params.py
-$ echo "{'uname': r'.*linux.*'}" >> params.py
+$ echo "{'uname': r'.*linux.*'}" > params.py
 $ run_exps.py -s GSN-EDF test.sched
 Invalid environment for experiment 'test.sched'
 Kernel name does not match '.*linux.*'.
@@ -194,51 +187,41 @@ Experiments run:        1
   Failed:               0
   Already Done:         0
   Invalid Environment:  1
-$ cp old_params.py params.py
-$ echo "{'uname': r'.*litmus.*'}" >> params.py
+$ echo "{'uname': r'.*litmus.*'}" > params.py
 # run_exps.py will now succeed
 ```
 
-The second property is kernel configuration options. These assume the configuration is stored at `/boot/config-```uname -r`` `. You can specify these like so:
+The second property is kernel configuration options. These assume the configuration is stored at `/boot/config-$(uname -r)`. You can specify these in `params.py` like so:
 
-```bash
-# Only executes on ARM systems with the release master enabled
-$ echo "{'config-options':{
-'RELEASE_MASTER' : 'y',
-'ARM' : 'y'}}" >> params.py
+```python
+{'config-options':{
+        'RELEASE_MASTER' : 'y',
+        'ARM' : 'y'}
+}
 ```
 
 
-III. GEN_EXPS
-Usage: gen_exps.py [options] [files...] [generators...] [param=val[,val]...]
-Output:   exps/EXP_DIRS which each contain sched.py and params.py
-Defaults: generators = G-EDF P-EDF C-EDF
+## gen_exps.py
+*Usage*: `gen_exps.py [options] [files...] [generators...] [param=val[,val]...]`
 
-The gen_exps.py script uses 'generators', one for each LITMUS
-scheduler supported, which each have different properties which can be
-varied to generate different types of schedules. Each of these
-properties has a default value which can be modified on the command
-line for quick and easy experiment generation.
+*Output*:   OUT_DIR/EXP_DIRS which each contain sched.py and params.py
 
-This script as written should be used to create debugging task sets,
-but not for creating task sets for experiments shown in papers. That
-is because the safety features of run_exps.py described above (uname,
-config-options) are not used here. If you are creating experiments for
-a paper, you should create your own generator which outputs values for
-'config-options' required for your plugin so that you cannot ruin your
-experiments at run time.
+*Defaults*: `generators = G-EDF P-EDF C-EDF`, `OUT_DIR = exps/`
 
-The -l option lists the supported generators which can be specified:
+This script uses *generators*, one for each LITMUS scheduler supported, which each have different properties which can be varied to generate different types of schedules. Each of these properties has a default value which can be modified on the command line for quick and easy experiment generation.
 
+This script as written should be used to create debugging task sets, but not for creating task sets for experiments shown in papers. That is because the safety features of `run_exps.py` described above (uname, config-options) are not used here. If you are creating experiments for a paper, you should create your own generator which outputs values for the `config-options` required for your plugin so that you cannot ruin your experiments at run time. Trust me, you will.
+
+The `-l` option lists the supported generators which can be specified:
+
+```bash
 $ gen_exps.py -l
 G-EDF, P-EDF, C-EDF
+```
 
-The -d option will describe the properties of a generator or
-generators and their default values. Note that some of these defaults
-will vary depending on the system the script is run. For example,
-'cpus' defaults to the number of cpus on the current system, in this
-example 24.
+The `-d` option will describe the properties of a generator or generators and their default values. Note that some of these defaults will vary depending on the system the script is run. For example, the `cpus` parameter defaults to the number of cpus on the current system, in this example 24.
 
+```bash
 $ gen_exps.py -d G-EDF,P-EDF
 Generator GSN-EDF:
         num_tasks -- Number of tasks per experiment.
@@ -254,102 +237,80 @@ Generator PSN-EDF:
                 Default: [24]
                 Allowed: <type 'int'>
         ....
+```
 
-You create experiments by specifying a generator. The following will
-create experiments 4 schedules with 24, 48, 72, and 96 tasks, because
-the default value of num_tasks is an array of these values
+You create experiments by specifying a generator. The following will create experiments 4 schedules with 24, 48, 72, and 96 tasks, because the default value of `num_tasks` is an array of these values (see above).
 
+```bash
 $ gen_exps.py P-EDF
 $ ls exps/
 sched=GSN-EDF_num-tasks=24/  sched=GSN-EDF_num-tasks=48/
 sched=GSN-EDF_num-tasks=72/  sched=GSN-EDF_num-tasks=96/
+```
 
-You can modify the default using a single value (the -f option deletes
-previous experiments in the output directory, defaulting to 'exps/',
-changeable with -o):
+You can modify the default using a single value (the `-f` option deletes previous experiments in the output directory, defaulting to `exps/`, changeable with `-o`):
 
+```bash
 $ gen_exps.py -f P-EDF num_tasks=24
 $ ls exps/
 sched=GSN-EDF_num-tasks=24/
+```
 
 Or with an array of values, specified as a comma-seperated list:
 
+```bash
 $ gen_exps.py -f num_tasks=`seq -s, 24 2 30` P-EDF
 sched=PSN-EDF_num-tasks=24/  sched=PSN-EDF_num-tasks=26/
 sched=PSN-EDF_num-tasks=28/  sched=PSN-EDF_num-tasks=30/
+```
 
-The generator will create a different directory for each possible
-configuration of the parameters. Each parameter which is varied is
-included in the name of the schedule directory. For example, to vary
-the number of CPUs but not the number of tasks:
+The generator will create a different directory for each possible configuration of the parameters. Each parameter which is varied is included in the name of the schedule directory. For example, to vary the number of CPUs but not the number of tasks:
 
+```bash
 $ gen_exps.py -f num_tasks=24 cpus=3,6 P-EDF
 $ ls exps
 sched=PSN-EDF_cpus=3/  sched=PSN-EDF_cpus=6/
+```
 
-The values of non-varying parameters are still saved in
-params.py. Continuing the example above:
+The values of non-varying parameters are still saved in `params.py`. Continuing the example above:
 
+```bash
 $ cat exps/sched\=PSN-EDF_cpus\=3/params.py
 {'periods': 'harmonic', 'release_master': False, 'duration': 30,
  'utils': 'uni-medium', 'scheduler': 'PSN-EDF', 'cpus': 3}
+```
 
-You can also have multiple schedules generated with the same
-configuration using the -n option:
+You can also have multiple schedules generated with the same configuration using the `-n` option:
 
+```bash
 $ gen_exps.py -f num_tasks=24 -n 5 P-EDF
 $ ls exps/
 sched=PSN-EDF_trial=0/  sched=PSN-EDF_trial=1/  sched=PSN-EDF_trial=2/
 sched=PSN-EDF_trial=3/  sched=PSN-EDF_trial=4/
-
+```
 
 IV. PARSE_EXPS
-Usage: parse_exps.py [options] [data_dir1] [data_dir2]...
-       where data_dirs contain feather-trace and sched-trace data,
-       e.g. ft.bin, mysched.ft, or st-*.bin.
-
-Output: print out all parsed data or
-        OUT_FILE where OUT_FILE is a python map of the data or
-        OUT_DIR/[FIELD]*/[PARAM]/[TYPE]/[TYPE]/[LINE].csv
-
-        The goal is to create csv files which record how varying PARAM
-        changes the value of FIELD. Only PARAMs which vary are
-        considered.
-
-        FIELD is a parsed value, e.g. 'RELEASE' overhead or 'miss-ratio'
-        PARAM is a parameter which we are going to vary, e.g. 'tasks'
-        A single LINE is created for every configuration of parameters
-        other than PARAM.
-
-        TYPE is the type of measurement, i.e. Max, Min, Avg, or
-        Var[iance]. The two types are used to differentiate between
-        measurements across tasks in a single taskset, and
-        measurements across all tasksets. E.g. miss-ratio/*/Max/Avg
-        is the maximum of all the average miss ratios for each task set, while
-        miss-ratio/*/Avg/Max is the average of the maximum miss ratios
-        for each task set.
-
-Defaults: OUT_DIR or OUT_FILE = parse-data, data_dir1 = '.'
-
-The parse_exps.py script reads a directory or directories, parses the
-binary files inside for feather-trace or sched-trace data, then
-summarizes and organizes the results for output. The output can be to
-the console, to a python map, or to a directory tree of csvs (the
-default, ish). The python map (using -m) can be used for
-schedulability tests. The directory tree can be used to look at how
-changing parameters affects certain measurements.
+*Usage*: `parse_exps.py [options] [data_dir1] [data_dir2]...`
+
+where data_dirs contain feather-trace and sched-trace data, e.g. `ft.bin`, `mysched.ft`, or `st-*.bin`.
+
+*Output*: print out all parsed data or `OUT_FILE` where `OUT_FILE` is a python map of the data or `OUT_DIR/[FIELD]*/[PARAM]/[TYPE]/[TYPE]/[LINE].csv`, depending on input.
+
+The goal is to create csv files which record how varying PARAM changes the value of FIELD. Only PARAMs which vary are considered.
+
+FIELD is a parsed value, e.g. 'RELEASE' overhead or 'miss-ratio'. `PARAM` is a parameter which we are going to vary, e.g. 'tasks' A single `LINE` is created for every configuration of parameters other than `PARAM`.
+
+`TYPE is the type of measurement, i.e. Max, Min, Avg, or Var[iance]. The two types are used to differentiate between measurements across tasks in a single taskset, and measurements across all tasksets. E.g. `miss-ratio/*/Max/Avg` is the maximum of all the average miss ratios for each task set, while `miss-ratio/*/Avg/Max` is the average of the maximum miss ratios for each task set.
+
+*Defaults*: `OUT_DIR, OUT_FILE = parse-data`, `data_dir1 = .`
+
+This script reads a directory or directories, parses the binary files inside for feather-trace or sched-trace data, then summarizes and organizes the results for output. The output can be to the console, to a python map, or to a directory tree of csvs (the default, ish). The python map (using `-m`) can be used for schedulability tests. The directory tree can be used to look at how changing parameters affects certain measurements.
 
 The script will use half the current computers CPUs to process data.
 
-In the following example, too little data was found to create csv
-files, so the data is output to the console despite the user not
-specifying the -v option. This use is the easiest for quick overhead
-evalutation and debugging. Note that for overhead measurements like
-these, parse_exps.py will use the 'clock-frequency' parameter saved in
-a params.py file by run_exps.py to calculate overhead measurements. If
-a param file is not present, as in this case, the current CPUs
-frequency will be used.
+In the following example, too little data was found to create csv files, so the data is output to the console despite the user not specifying the `-v` option. This use is the easiest for quick overhead evalutation and debugging. Note that for overhead measurements like these, `parse_exps.py` will use the `clock-frequency` parameter saved in a params.py file by `run_exps.py` to calculate overhead measurements. If a param file is not present, as in this case, the current CPUs frequency will be used.
 
+```bash
 $ ls run-data/
 taskset_scheduler=C-FL-split-L3_host=ludwig_n=10_idx=05_split=randsplit.ft
 $ parse_exps.py run-data/
@@ -362,10 +323,11 @@ Too little data to make csv files.
                  CXS:  Avg:     5.053  Max:    59.925  Min:     0.241
                SCHED:  Avg:     4.410  Max:    39.350  Min:     0.357
                 TICK:  Avg:     1.812  Max:    21.380  Min:     0.241
+```
 
-In the next example, because the value of num-tasks varies, csvs can
-be created:
+In the next example, because the value of num-tasks varies, csvs can be created. The varying parameters used to create csvs were found by reading the `params.py` files under each `run-data` subdirectory.
 
+```bash
 $ ls run-data/
 sched=C-EDF_num-tasks=4/   sched=GSN-EDF_num-tasks=4/
 sched=C-EDF_num-tasks=8/   sched=GSN-EDF_num-tasks=8/
@@ -374,20 +336,13 @@ sched=C-EDF_num-tasks=16/  sched=GSN-EDF_num-tasks=16/
 $ parse_exps.py run-data/*
 $ ls parse-data/
 avg-block/  avg-tard/  max-block/  max-tard/  miss-ratio/
+```
 
-The varying parameters were found by reading the params.py files under
-each run-data subdirectory.
-
-You can use the -v option to print out the values measured even when
-csvs could be created.
+You can use the `-v` option to print out the values measured even when csvs could be created.
 
-You can use the -i option to ignore variations in a certain parameter
-(or parameters if a comma-seperated list is given). In the following
-example, the user has decided the 'option' does not matter after
-viewing output. Note that the 'trial' parameter, used by gen_exps.py
-to create multiple schedules with the same configuration, is always
-ignored.
+You can use the `-i` option to ignore variations in a certain parameter (or parameters if a comma-seperated list is given). In the following example, the user has decided the parameter `option` does not matter after viewing output. Note that the `trial` parameter, used by `gen_exps.py` to create multiple schedules with the same configuration, is always ignored.
 
+```bash
 $ ls run-data/
 sched=C-EDF_num-tasks=4_option=1/ sched=C-EDF_num-tasks=4_option=2/
 sched=C-EDF_num-tasks=8_option=1/ sched=C-EDF_num-tasks=8_option=2/
@@ -407,69 +362,48 @@ $i; done
 line.csv
  4 .2
  8 .3
+```
 
-The second command will also have run faster than the first. This is
-because parse_exps.py will save the data it parses in tmp/ directories
-before it attempts to sort it into csvs. Parsing takes far longer than
-sorting, so this saves a lot of time. The -f flag can be used to
-re-parse files and overwrite this saved data.
-
-All output from the feather-trace-tool programs used to parse data is
-stored in the tmp/ directories created in the input directories.  If
-the sched_trace repo is found in the users PATH, st_show will be used
-to create a human-readable version of the sched-trace data which will
-also be stored there.
-
-
-V. PLOT_EXPS
-Usage: plot_exps.py [options] [csv_dir]...
-       where a csv dir is a directory or directory of directories (and
-       so on) containing csvs, like:
-       csv_dir/[subdirs/...]
-           line1.csv
-           line2.csv
-           line3.csv
+The second command will also have run faster than the first. This is because `parse_exps.py` will save the data it parses in `tmp/` directories before it attempts to sort it into csvs. Parsing takes far longer than sorting, so this saves a lot of time. The `-f` flag can be used to re-parse files and overwrite this saved data.
 
-Outputs: OUT_DIR/[csv_dir/]*[plot]*.pdf
-         where a single plot exists for each directory of csvs, with a
-         line for for each csv file in that directory. If only a
-         single csv_dir is specified, all plots are placed directly
-         under OUT_DIR.
+All output from the *feather-trace-tools* programs used to parse data is stored in the `tmp/` directories created in the input directories.  If the *sched_trace* repo is found in the users `PATH`, `st_show` will be used to create a human-readable version of the sched-trace data which will also be stored there.
 
-Defaults: OUT_DIR = 'plot-data/', csv_dir = '.'
+## plot_exps.py
+*Usage*: `plot_exps.py [options] [csv_dir]...`
 
-The plot_exps.py script takes directories of csvs (or directories
-formatted as specified below) and creates a pdf plot of each csv
-directory found. A line is created for each .csv file contained in a
-plot. Matplotlib is used to do the plotting.
+where a csv dir is a directory or directory of directories (and so on) containing csvs, like:
+```
+csv_dir/[subdirs/...]
+        line1.csv
+        line2.csv
+        line3.csv
+```
 
-If the csv files are formatted like:
+*Outputs*: `OUT_DIR/[csv_dir/]*[plot]*.pdf`
 
-  param=value_param2=value2.csv
+where a single plot exists for each directory of csvs, with a line for for each csv file in that directory. If only a single csv_dir is specified, all plots are placed directly under `OUT_DIR`.
 
-the variation of these parameters will be used to color the lines in
-the most readable way. For instance, if there are three parameters,
-variations in one parameter will change line color, another line
-style (dashes/dots/etc), and a third line markers
-(trianges/circles/etc).
+*Defaults*: `OUT_DIR = plot-data/`, `csv_dir = .`
 
-If a directory of directories is passed in, the script will assume the
-top level directory is the measured value and the next level is the
-variable, ie:
+This script takes directories of csvs (or directories formatted as specified below) and creates a pdf plot of each csv directory found. A line is created for each .csv file contained in a plot. [Matplotlib][matplotlib] is used to do the plotting. The script will use half the current computers CPUs to process data.
 
-  value/variable/[..../]line.csv
+If the csv filenames are formatted like: `param=value_param2=value2.csv`, the variation of these parameters will be used to color the lines in the most readable way. For instance, if there are three parameters, variations in one parameter will change line color, another line style (dashes/dots/etc), and a third line markers (trianges/circles/etc).
 
-And put a title on the plot of "Value by variable (...)". Otherwise,
-the name of the top level directory will be the title, like "Value".
+If a directory of directories is passed in, the script will assume the top level directory is the measured value and the next level is the variable, ie: `value/variable/[..../]line.csv`, and will put a title on the plot of "Value by variable (...)". Otherwise, the name of the top level directory will be the title, like "Value".
 
 A directory with some lines:
+
+```bash
 $ ls
 line1.csv line2.csv
 $ plot_exps.py
 $ ls plot-data/
 plot.pdf
+```
 
 A directory with a few subdirectories:
+
+```bash
 $ ls test/
 apples/ oranges/
 $ ls test/apples/
@@ -477,8 +411,11 @@ line1.csv line2.csv
 $ plot_exps.py test/
 $ ls plot-data/
 apples.pdf oranges.pdf
+```
 
 A directory with many subdirectories:
+
+```bash
 $ ls parse-data
 avg-block/  avg-tard/  max-block/  max-tard/  miss-ratio/
 $ ls parse-data/avg-block/tasks/Avg/Avg
@@ -490,10 +427,11 @@ avg-block_tasks_Max_Avg.pdf avg-block_tasks_Max_Max.pdf avg-block_tasks_Max_Min.
 avg-block_tasks_Min_Avg.pdf avg-block_tasks_Min_Max.pdf avg-block_tasks_Min_Min.pdf
 avg-block_tasks_Var_Avg.pdf avg-block_tasks_Var_Max.pdf avg-block_tasks_Var_Min.pdf
 .......
+```
 
-If you run the previous example directly on the subdirectories,
-subdirectories will be created in the output:
+If you run the previous example directly on the subdirectories, subdirectories will be created in the output:
 
+```bash
 $ plot_exps.py parse-data/*
 $ ls plot-data/
 avg-block/  max-tard/ avg-tard/ miss-ratio/ max-block/
@@ -502,10 +440,9 @@ tasks_Avg_Avg.pdf  tasks_Avg_Min.pdf  tasks_Max_Max.pdf
 tasks_Min_Avg.pdf  tasks_Min_Min.pdf  tasks_Var_Max.pdf
 tasks_Avg_Max.pdf  tasks_Max_Avg.pdf  tasks_Max_Min.pdf
 tasks_Min_Max.pdf  tasks_Var_Avg.pdf  tasks_Var_Min.pdf
+```
 
-However, when a single directory of directories is given, the script
-assumes the experiments are related and can make line styles match in
-different plots and more effectively parallelize the plotting.
+However, when a single directory of directories is given, the script assumes the experiments are related and can make line styles match in different plots and more effectively parallelize the plotting.
 
 [litmus]: https://github.com/LITMUS-RT/litmus-rt
 [liblitmus]: https://github.com/LITMUS-RT/liblitmus