Added README.

1 files changed, 507 insertions, 0 deletions

diff --git a/README b/README
new file mode 100644
index 0000000..e1a0815
--- /dev/null
+++ b/README
@@ -0,0 +1,507 @@
+I. INTRODUCTION
+These scripts provide a common way for creating, running, parsing, and
+plotting experiments under LITMUS^RT. They are designed with the
+following principles in mind:
+
+1. Little or no configuration: all scripts use certain parameters to
+configure behavior. However, if the user does not give these
+parameters, the scripts will examine the properties of the user's
+system to pick a suitable default. Requiring user input is a last
+resort.
+
+2. Interruptability: the scripts save their work as they evaluate
+multiple directories. When the scripts are interrupted, or if new data
+is added to those directories, the scripts can be re-run and they will
+resume where they left off. This vastly decreases turnaround time for
+testing new features.
+
+3. Maximum Safety: where possible, scripts save metadata in their output
+directories about the data contained. This metadata can be used by
+the other scripts to safely use the data later.
+
+4. Independence / legacy support: none of these scripts assume their
+input was generated by another of these scripts. Three are designed to
+recognize generic input formats inspired by past LITMUS^RT
+experimental setups. (The exception to this is gen_exps.py, which
+has only user intput and creates output only for run_exps.py)
+
+5. Save everything: all output and parameters (even from subprocesses)
+is saved for debugging / reproducability. This data is saved in tmp/
+directories while scripts are running in case scripts fail.
+
+These scripts require that the following repos are in the user's PATH:
+1. liblitmus - for real-time executable simulation and task set release
+2. feather-trace-tools - for recording and parsing overheads and
+   scheduling events
+
+Optionally, additional features will be enabled if these repos are
+present in the PATH:
+1. rt-kernelshark - to record ftrace events for kernelshark visualization
+2. sched_trace - to output a file containing scheduling events as
+strings
+
+Each of these scripts is designed to operate independently of the
+others. For example, the parse_exps.py will find any feather trace
+files resembling ft-xyz.bin or xyz.ft and print out overhead
+statistics for the records inside. However, the scripts provide the
+most features (especially safety) when their results are chained
+together, like so:
+
+gen_exps.py --> [exps/*] --> run_exps.py  --> [run-data/*] --.
+.------------------------------------------------------------'
+'--> parse_exps.py --> [parse-data/*] --> plot_exps.py --> [plot-data/*.pdf]
+
+0. Create experiments with gen_exps.py or some other script.
+1. Run experiments using run_exps.py, generating binary files in run-data/.
+2. Parse binary data in run-data using parse_exps.py, generating csv
+   files in parse-data/.
+3. Plot parse-data using plot_exps.py, generating pdfs in plot-data.
+
+Each of these scripts will be described. The run_exps.py script is
+first because gen_exps.py creates schedule files which depend on run_exps.py.
+
+
+II. RUN_EXPS
+Usage: run_exps.py [OPTIONS] [SCHED_FILE]... [SCHED_DIR]...
+       where a SCHED_DIR resembles:
+         SCHED_DIR/
+            SCHED_FILE
+            PARAM_FILE
+
+Output: OUT_DIR/[files] or OUT_DIR/SCHED_DIR/[files] or
+        OUT_DIR/SCHED_FILE/[files] depending on input
+        If all features are enabled, these files are:
+        OUT_DIR/[.*/]
+           trace.slog    # LITMUS logging
+           st-[1..m].bin # sched_trace data
+           ft.bin        # feather-trace overhead data
+           trace.dat     # ftrace data for kernelshark
+           params.py     # Schedule parameters
+           exec-out.txt  # Standard out from schedule processes
+           exec-err.txt  # Standard err '''
+
+Defaults: SCHED_FILE = sched.py, PARAM_FILE = params.py,
+          DURATION = 30, OUT_DIR = run-data/
+
+The run_exps.py script reads schedule files and executes real-time
+task systems, recording all overhead, logging, and trace data which is
+enabled in the system. For example, if trace logging is enabled,
+rt-kernelshark is found in the path, but feather-trace is disabled
+(the devices are not present), only trace-logs and kernelshark logs
+will be recorded.
+
+When run_exps.py is running a schedule file, temporary data is saved
+in a 'tmp' directory in the same directory as the schedule file. When
+execution completes, this data is moved into a directory under the
+run_exps.py output directory (default: 'run-data/', can be changed with
+the -o option). When multiple schedules are run, each schedule's data
+is saved in a unique directory under the output directory.
+
+If a schedule has been run and it's data is in the output directory,
+run_exps.py will not re-run the schedule unless the -f option is
+specified. This is useful if your system crashes midway through a set
+of experiments.
+
+Schedule files have one of the following two formats:
+
+a) simple format
+       path/to/proc{proc_value}
+       ...
+       path/to/proc{proc_value}
+       [real_time_task: default rtspin] task_arguments...
+       ...
+       [real_time_task] task_arguments...
+
+b) python format
+       {'proc':[
+           ('path/to/proc','proc_value'),
+            ...,
+           ('path/to/proc','proc_value')
+        ],
+        'spin':[
+           ('real_time_task', 'task_arguments'),
+            ...
+           ('real_time_task', 'task_arguments')
+        ]
+       }
+
+The following creates a simple 3-task system with utilization 2.0,
+which is then run under the GSN-EDF plugin:
+
+$ echo "10 20
+30 40
+60 90" > test.sched
+$ run_exps.py -s GSN-EDF test.sched
+
+The following will write a release master using
+/proc/litmus/release_master:
+
+$ echo "release_master{2}
+10 20" > test.sched && run_exps.py -s GSN-EDF test.sched
+
+A longer form can be used for proc entries not in /proc/litmus:
+
+$ echo "/proc/sys/something{hello}"
+10 20" > test.sched
+
+You can specify your own spin programs to run as well instead of
+rtspin by putting their name at the beginning of the line.
+
+$ echo "colorspin -f color1.csv 10 20" > test.sched
+
+This example also shows how you can reference files in the same
+directory as the schedule file on the command line.
+
+You can specify parameters for an experiment in a file instead of on
+the command line using params.py (the -p option lets you choose the
+name of this file if params.py is not for you):
+
+$ echo "{'scheduler':'GSN-EDF', 'duration':10}" > params.py
+$ run_exps.py test.sched
+
+You can also run multiple experiments with a single command, provided
+a directory with a schedule file exists for each. By default, the
+program will look for sched.py for the schedule file and params.py for
+the parameter file, but this behavior can be changed using the -p and
+-c options.
+
+You can include non-relevant parameters which run_exps.py does not
+understand in params.py. These parameters will be saved with the data
+output by run_exps.py. This is useful for tracking variations in
+system parameters versus experimental results.
+
+In the following example, multiple experiments are demonstrated and an
+extra parameter 'test-param' is included:
+
+$ mkdir test1
+# The duration will default to 30 and need not be specified
+$ echo "{'scheduler':'C-EDF', 'test-param':1} > test1/params.py
+$ echo "10 20" > test1/sched.py
+$ cp -r test1 test2
+$ echo "{'scheduler':'GSN-EDF', 'test-param':2}"> test2/params.py
+$ run_exps.py test*
+
+Finally, you can specify system properties in params.py which the
+environment must match for the experiment to run. These are useful if
+you have a large batch of experiments which must be run under
+different kernels. The first property is a regular expression for the
+uname of the system:
+
+$ uname -r
+3.0.0-litmus
+$ cp params.py old_params.py
+$ echo "{'uname': r'.*linux.*'}" >> params.py
+# run_exps.py will now complain of an invalid environment for this
+experiment
+$ cp old_params.py params.py
+$ echo "{'uname': r'.*litmus.*'}" >> params.py
+# run_exps.py will now succeed
+
+The second property are kernel configuration options. These assume the
+configuration is stored at /boot/config-`uname -r`. You can specify
+these like so:
+
+$ echo "{'config-options':{
+'RELEASE_MASTER' : 'y',
+'ARM' : 'y'}}" >> params.py
+# Only executes on ARM systems with the release master enabled
+
+
+III. GEN_EXPS
+Usage: gen_exps.py [options] [files...] [generators...] [param=val[,val]...]
+Output:   exps/EXP_DIRS which each contain sched.py and params.py
+Defaults: generators = G-EDF P-EDF C-EDF
+
+The gen_exps.py script uses 'generators', one for each LITMUS
+scheduler supported, which each have different properties which can be
+varied to generate different types of schedules. Each of these
+properties has a default value which can be modified on the command
+line for quick and easy experiment generation.
+
+This script as written should be used to create debugging task sets,
+but not for creating task sets for experiments shown in papers. That
+is because the safety features of run_exps.py described above (uname,