17 files changed, 759 insertions, 450 deletions

diff --git a/README.md b/README.md
index 6200277..b074aa5 100644
--- a/README.md
+++ b/README.md
@@ -45,7 +45,7 @@ gen_exps.py --> [exps/*] --> run_exps.py  --> [run-data/*] --.
 3. Parse binary data in `run-data/` using `parse_exps.py`, generating csv files in `parse-data/`.
 4. Plot `parse-data` using `plot_exps.py`, generating pdfs in `plot-data/`.
 
-Each of these scripts will be described. The `run_exps.py` script is first because `gen_exps.py` creates schedule files which depend on `run_exps.py`.
+Each of these scripts will be described. The `run_exps.py` script is first because `gen_exps.py` creates schedule files that depend on `run_exps.py`.
 
 
 ## run_exps.py
@@ -74,7 +74,7 @@ OUT_DIR/[SCHED_(FILE|DIR)/]
 
 *Defaults*: `SCHED_FILE = sched.py`, `PARAM_FILE = params.py`, `DURATION = 30`, `OUT_DIR = run-data/`
 
-This script reads *schedule files* (described below) and executes real-time task systems, recording all overhead, logging, and trace data which is enabled in the system (unless a specific set of tracers is specified in the parameter file, see below). For example, if trace logging is enabled, rt-kernelshark is found in the path, but feather-trace is disabled (the devices are not present), only trace logs and rt-kernelshark logs will be recorded.
+This script reads *schedule files* (described below) and executes real-time task systems, recording all overhead, logging, and trace data that is enabled in the system (unless a specific set of tracers is specified in the parameter file, see below). For example, if trace logging is enabled, rt-kernelshark is found in the path, but feather-trace is disabled (the devices are not present), only trace logs and rt-kernelshark logs will be recorded.
 
 When `run_exps.py` is running a schedule file, temporary data is saved in a `tmp` directory in the same directory as the schedule file. When execution completes, this data is moved into a directory under the `run_exps.py` output directory (default: `run-data/`, can be changed with the `-o` option). When multiple schedules are run, each schedule's data is saved in a unique directory under the output directory.
 
@@ -192,7 +192,7 @@ $ cat post-out.txt
 Experiment ends!
 ```
 
-Finally, you can specify system properties in `params.py` which the environment must match for the experiment to run. These are useful if you have a large batch of experiments which must be run under different kernels or kernel configurations. The first property is a regular expression for the name of the kernel:
+Finally, you can specify system properties in `params.py`, which the environment must match for the experiment to run. These are useful if you have a large batch of experiments that must be run under different kernels or kernel configurations. The first property is a regular expression for the name of the kernel:
 
 ```bash
 $ uname -r
@@ -219,7 +219,7 @@ The second property is kernel configuration options. These assume the configurat
 }
 ```
 
-The third property is required tracers. The `tracers` property lets the user specify only those tracers they want to run with an experiment, as opposed to starting every available tracer (the default). If any of these specified tracers cannot be enabled, e.g. the kernel was not compiled with feather-trace support, the experiment will not run. The following example gives an experiment which will not run unless all four tracers are enabled:
+The third property is required tracers. The `tracers` property lets the user specify only those tracers they want to run with an experiment, as opposed to starting every available tracer (the default). If any of these specified tracers cannot be enabled, e.g. the kernel was not compiled with feather-trace support, the experiment will not run. The following example gives an experiment that will not run unless all four tracers are enabled:
 ```python
 {'tracers':['kernelshark', 'log', 'sched', 'overhead']}
 ```
@@ -227,15 +227,15 @@ The third property is required tracers. The `tracers` property lets the user spe
 ## gen_exps.py
 *Usage*: `gen_exps.py [options] [files...] [generators...] [param=val[,val]...]`
 
-*Output*: `OUT_DIR/EXP_DIRS` which each contain `sched.py` and `params.py`
+*Output*: `OUT_DIR/EXP_DIRS` that each contain `sched.py` and `params.py`
 
 *Defaults*: `generators = G-EDF P-EDF C-EDF`, `OUT_DIR = exps/`
 
-This script uses *generators*, one for each LITMUS scheduler supported, which each have different properties which can be varied to generate different types of schedules. Each of these properties has a default value which can be modified on the command line for quick and easy experiment generation.
+This script uses *generators*, one for each LITMUS scheduler supported, which each have different properties that can be varied to generate different types of schedules. Each of these properties has a default value that can be modified on the command line for quick and easy experiment generation.
 
-This script as written should be used to create debugging task sets, but not for creating task sets for experiments shown in papers. That is because the safety features of `run_exps.py` described above (`uname`, `config-options`) are not used here. If you are creating experiments for a paper, you should create your own generator which outputs values for the `config-options` required for your plugin so that you cannot ruin your experiments at run time. Trust me, you will.
+This script as written should be used to create debugging task sets, but not for creating task sets for experiments shown in papers. That is because the safety features of `run_exps.py` described above (`uname`, `config-options`) are not used here. If you are creating experiments for a paper, you should create your own generator that outputs values for the `config-options` required for your plugin so that you cannot ruin your experiments at run time. Trust me, you will.
 
-The `-l` option lists the supported generators which can be specified:
+The `-l` option lists the supported generators that can be specified:
 
 ```bash
 $ gen_exps.py -l
@@ -287,7 +287,7 @@ sched=PSN-EDF_num-tasks=24/  sched=PSN-EDF_num-tasks=26/
 sched=PSN-EDF_num-tasks=28/  sched=PSN-EDF_num-tasks=30/
 ```
 
-The generator will create a different directory for each possible configuration of the parameters. Each parameter which is varied is included in the name of the schedule directory. For example, to vary the number of CPUs but not the number of tasks:
+The generator will create a different directory for each possible configuration of the parameters. Each parameter that is varied is included in the name of the schedule directory. For example, to vary the number of CPUs but not the number of tasks:
 
 ```bash
 $ gen_exps.py -f tasks=24 cpus=3,6 P-EDF
@@ -319,9 +319,9 @@ where the `data_dirx` contain feather-trace and sched-trace data, e.g. `ft.bin`,
 
 *Output*: print out all parsed data or `OUT_FILE` where `OUT_FILE` is a python map of the data or `OUT_DIR/[FIELD]*/[PARAM]/[TYPE]/[TYPE]/[LINE].csv`, depending on input.
 
-The goal is to create csv files which record how varying `PARAM` changes the value of `FIELD`. Only `PARAM`s which vary are considered.
+The goal is to create csv files that record how varying `PARAM` changes the value of `FIELD`. Only `PARAM`s that vary are considered.
 
-`FIELD` is a parsed value, e.g. 'RELEASE' overhead or 'miss-ratio'. `PARAM` is a parameter which we are going to vary, e.g. 'tasks'. A single `LINE` is created for every configuration of parameters other than `PARAM`.
+`FIELD` is a parsed value, e.g. 'RELEASE' overhead or 'miss-ratio'. `PARAM` is a parameter that we are going to vary, e.g. 'tasks'. A single `LINE` is created for every configuration of parameters other than `PARAM`.
 
 `TYPE` is the statistic of the measurement, i.e. Max, Min, Avg, or Var[iance]. The two types are used to differentiate between measurements across tasks in a single taskset, and measurements across all tasksets. E.g. `miss-ratio/*/Max/Avg` is the maximum of all the average miss ratios for each task set, while `miss-ratio/*/Avg/Max` is the average of the maximum miss ratios for each task set.
 
@@ -389,7 +389,7 @@ line.csv
 
 The second command will also have run faster than the first. This is because `parse_exps.py` will save the data it parses in `tmp/` directories before it attempts to sort it into csvs. Parsing takes far longer than sorting, so this saves a lot of time. The `-f` flag can be used to re-parse files and overwrite this saved data.
 
-All output from the *feather-trace-tools* programs used to parse data is stored in the `tmp/` directories created in the input directories.  If the *sched_trace* repo is found in the users `PATH`, `st_show` will be used to create a human-readable version of the sched-trace data which will also be stored there.
+All output from the *feather-trace-tools* programs used to parse data is stored in the `tmp/` directories created in the input directories.  If the *sched_trace* repo is found in the users `PATH`, `st_show` will be used to create a human-readable version of the sched-trace data that will also be stored there.
 
 ## plot_exps.py
 *Usage*: `plot_exps.py [OPTIONS] [CSV_DIR]...`
@@ -472,4 +472,4 @@ However, when a single directory of directories is given, the script assumes the
 [rt-kernelshark]: https://github.com/LITMUS-RT/rt-kernelshark
 [feather-trace-tools]: https://github.com/LITMUS-RT/feather-trace-tools
 [rtunc]: http://www.cs.unc.edu/~anderson/real-time/
-[matplotlib]: http://matplotlib.org/
\ No newline at end of file
+[matplotlib]: http://matplotlib.org/
diff --git a/common.py b/common.py
index a2c6224..7abf0f2 100644
--- a/common.py
+++ b/common.py
@@ -193,3 +193,20 @@ def is_device(dev):
         return False
     mode = os.stat(dev)[stat.ST_MODE]
     return not (not mode & stat.S_IFCHR)
+
+__logged = []
+
+def set_logged_list(logged):
+    global __logged
+    __logged = logged
+
+def log_once(id, msg = None, indent = True):
+    global __logged
+
+    msg = msg if msg else id
+
+    if id not in  __logged:
+        __logged += [id]
+        if indent:
+            msg = '   ' + msg.strip('\t').replace('\n', '\n\t')
+        sys.stderr.write('\n' + msg.strip('\n') + '\n')
diff --git a/config/config.py b/config/config.py
index cbac6b2..5e6f9e3 100644
--- a/config/config.py
+++ b/config/config.py
@@ -9,7 +9,7 @@ BINS = {'rtspin'    : get_executable_hint('rtspin', 'liblitmus'),
         'ftsplit'   : get_executable_hint('ft2csv', 'feather-trace-tools'),
         'ftsort'    : get_executable_hint('ftsort', 'feather-trace-tools'),
         'st_trace'  : get_executable_hint('st_trace', 'feather-trace-tools'),
-        # Option, as not everyone uses kernelshark yet
+        # Optional, as not everyone uses kernelshark yet
         'trace-cmd' : get_executable_hint('trace-cmd', 'rt-kernelshark', True),
         # Optional, as sched_trace is not a publically supported repository
         'st_show'   : get_executable_hint('st_show', 'sched_trace', True)}
@@ -39,15 +39,24 @@ DEFAULTS = {'params_file' : 'params.py',
             'sched_file'  : 'sched.py',
             'duration'    : 10,
             'prog'        : 'rtspin',
+            'out-gen'     : 'exps',
+            'out-run'     : 'run-data',
+            'out-parse'   : 'parse-data',
+            'out-plot'    : 'plot-data',
             'cycles'      : ft_freq() or 2000}
 
+
 '''Default sched_trace events (this is all of them).'''
 SCHED_EVENTS = range(501, 513)
 
 '''Overhead events.'''
-OVH_BASE_EVENTS  = ['SCHED', 'RELEASE', 'SCHED2', 'TICK', 'CXS']
+OVH_BASE_EVENTS  = ['SCHED', 'RELEASE', 'SCHED2', 'TICK', 'CXS', 'LOCK', 'UNLOCK']
 OVH_ALL_EVENTS   = ["%s_%s" % (e, t) for (e,t) in
                     itertools.product(OVH_BASE_EVENTS, ["START","END"])]
 OVH_ALL_EVENTS  += ['RELEASE_LATENCY']
 # This event doesn't have a START and END
 OVH_BASE_EVENTS += ['RELEASE_LATENCY']
+
+# If a task is missing more than this many records, its measurements
+# are not included in sched_trace summaries
+MAX_RECORD_LOSS = .2
diff --git a/gen/edf_generators.py b/gen/edf_generators.py
index 3f05b76..a722c21 100644
--- a/gen/edf_generators.py
+++ b/gen/edf_generators.py
@@ -16,10 +16,10 @@ class EdfGenerator(gen.Generator):
 
     def __make_options(self):
         '''Return generic EDF options.'''
-        return [gen.Generator._dist_option('utils', ['uni-medium'],
+        return [gen.Generator._dist_option('utils', 'uni-medium',
                                            gen.NAMED_UTILIZATIONS,
                                            'Task utilization distributions.'),
-                gen.Generator._dist_option('periods', ['harmonic'],
+                gen.Generator._dist_option('periods', 'harmonic',
                                            gen.NAMED_PERIODS,
                                            'Task period distributions.')]
 
@@ -50,10 +50,22 @@ class PartitionedGenerator(EdfGenerator):
             templates + [TP_PART_TASK], options, params)
 
     def _customize(self, taskset, exp_params):
-        start = 1 if exp_params['release_master'] else 0
-        # Random partition for now: could do a smart partitioning
+        cpus  = exp_params['cpus']
+        start = 0
+        if exp_params['release_master']:
+            cpus -= 1
+            start = 1
+
+        # Partition using worst-fit for most even distribution
+        utils = [0]*cpus
+        tasks = [0]*cpus
         for t in taskset:
-            t.cpu = random.randint(start, exp_params['cpus'] - 1)
+            t.cpu = utils.index(min(utils))
+            utils[t.cpu] += t.utilization()
+            tasks[t.cpu] += 1
+
+            # Increment by one so release master has no tasks
+            t.cpu += start
 
 class PedfGenerator(PartitionedGenerator):
     def __init__(self, params={}):
@@ -61,7 +73,7 @@ class PedfGenerator(PartitionedGenerator):
 
 class CedfGenerator(PartitionedGenerator):
     TP_CLUSTER = "plugins/C-EDF/cluster{$level}"
-    CLUSTER_OPTION = gen.GenOption('level', ['L2', 'L3', 'All'], ['L2'],
+    CLUSTER_OPTION = gen.GenOption('level', ['L2', 'L3', 'All'], 'L2',
                                    'Cache clustering level.',)
 
     def __init__(self, params={}):
diff --git a/gen/generator.py b/gen/generator.py
index 693e52f..bc86cfe 100644
--- a/gen/generator.py
+++ b/gen/generator.py
@@ -5,7 +5,7 @@ import schedcat.generator.tasks as tasks
 import shutil as sh
 
 from Cheetah.Template import Template
-from common import get_config_option,num_cpus,recordtype
+from common import get_config_option,num_cpus,recordtype,log_once
 from config.config import DEFAULTS,PARAMS
 from gen.dp import DesignPointGenerator
 from parse.col_map import ColMapBuilder
@@ -69,11 +69,10 @@ class Generator(object):
         else:
             self.cpus = num_cpus()
         try:
-            config = get_config_option("RELEASE_MASTER") and True
+            rm_config  = get_config_option("RELEASE_MASTER") and True
         except:
-            config = False
-        self.release_master = list(set([False, config]))
-
+            rm_config  = False
+        self.release_master = list(set([False, bool(rm_config)]))
 
     def __make_options(self, params):
         '''Return generic Litmus options.'''
@@ -116,11 +115,11 @@ class Generator(object):
             ts = tg.make_task_set(max_tasks = params['tasks'], max_util=max_util)
             tries += 1
         if len(ts) != params['tasks']:
-            print(("Only created task set of size %d < %d for params %s. " +
-                   "Switching to light utilization.") %
-                  (len(ts), params['tasks'], params))
-            print("Switching to light util. This usually means the " +
-                  "utilization distribution is too agressive.")
+            log_once("only", ("Only created task set of size %d < %d for " +
+                              "params %s. Switching to light utilization.") %
+                              (len(ts), params['tasks'], params))
+            log_once("light", "Switching to light util. This usually means " +
+                     "the utilization distribution is too agressive.")
             return self._create_taskset(params, periods, NAMED_UTILIZATIONS['uni-light'],
                                         max_util)
         return ts
@@ -156,7 +155,10 @@ class Generator(object):
         '''Set default parameter values and check that values are valid.'''
         for option in self.options:
             if option.name not in params:
-                params[option.name] = option.default
+                val = option.default
+                val = val if type(val) == type([]) else [val]
+
+                params[option.name] = val
             else:
                 option.hidden = True
             params[option.name] = self._check_value(option.name,
diff --git a/gen/mc_generators.py b/gen/mc_generators.py
index 704bcc3..8f5bd84 100644
--- a/gen/mc_generators.py
+++ b/gen/mc_generators.py
@@ -1,8 +1,10 @@
 import gen.rv as rv
 
 from color import get_cache_info,CacheInfo,BlockColorScheme,RandomColorScheme,EvilColorScheme
-from common import try_get_config_option
+from common import try_get_config_option, log_once
 from gen.generator import GenOption,Generator,NAMED_UTILIZATIONS,NAMED_PERIODS
+from parse.col_map import ColMap
+from parse.tuple_table import TupleTable
 
 
 NAMED_SHARES = {
@@ -188,24 +190,24 @@ class McGenerator(Generator):
 
         return self._create_taskset(params, periods, utils, max_util)
 
-    def _customize(self, task_system, params):
-        pass
+    def _get_tasks(self, params):
+        return {'lvla': self.__create_lvla_sched(params),
+                'lvlb': self.__create_lvlb_sched(params),
+                'lvlc': self.__create_lvlc_sched(params)}
 
     def _create_exp(self, params):
         # Ugly way of doing it
         self.shares = self._create_dist('shares', params['shares'],
                                         NAMED_SHARES)
 
-        tasks = {'lvla': self.__create_lvla_sched(params),
-                 'lvlb': self.__create_lvlb_sched(params),
-                 'lvlc': self.__create_lvlc_sched(params)}
+        tasks = self._get_tasks(params)
 
         conf_options = {MC_OPT : 'y'}
         if params['timer_merging']:
             conf_options[TM_OPT] = 'y'
         if params['redirect']:
             if not params['release_master']:
-                print("Forcing release master option to enable redirection.")
+                log_once("Forcing release master option to enable redirection.")
                 params['release_master'] = 'y'
             conf_options[RD_OPT] = 'y'
         if params['slack_stealing']:
@@ -241,8 +243,11 @@ TP_TYPE = """#if $type != 'unmanaged'
 /proc/sys/litmus/color/preempt_cache{0}
 #end if"""
 
+# Always add some pages
+TP_ADD = """/proc/sys/litmus/color/add_pages{1}"""
+
 # Use special spin for color tasks
-TP_COLOR_BASE = """colorspin -y $t.id -x $t.colorcsv """
+TP_COLOR_BASE = """colorspin -y $t.id -x $t.colorcsv -q $t.wss -l $t.loops """
 
 TP_COLOR_B = TP_BASE.format("b", TP_COLOR_BASE + "-p $t.cpu ")
 TP_COLOR_C = TP_BASE.format("c", TP_COLOR_BASE)
@@ -255,12 +260,16 @@ TP_CHUNK = """#if $chunk_size > 0
 COLOR_TYPES = ['scheduling', 'locking', 'unmanaged']
 
 class ColorMcGenerator(McGenerator):
+    __SINGLE_PAGE_LOOP_MS = {'ringo': .023}
+
     def __init__(self, params = {}):
         super(ColorMcGenerator, self).__init__("MC",
-            templates=[TP_TYPE, TP_CHUNK, TP_COLOR_B, TP_COLOR_C],
+            templates=[TP_ADD, TP_TYPE, TP_CHUNK, TP_COLOR_B, TP_COLOR_C],
             options=self.__make_options(),
             params=self.__extend_params(params))
 
+        self.tasksets = None
+
     def __extend_params(self, params):
         '''Add in fixed mixed-criticality parameters.'''
         params['levels'] = 2
@@ -278,6 +287,10 @@ class ColorMcGenerator(McGenerator):
 
         return params
 
+    def __get_system_name(self):
+        import socket
+        return socket.gethostname().split(".")[0]
+
     def __make_system_info(self):
         info = get_cache_info()
 
@@ -298,20 +311,32 @@ class ColorMcGenerator(McGenerator):
             info = CacheInfo(cache, line=line, page=page,
                              ways=ways, sets=sets, colors=colors)
 
-        self.system = info
+        self.cache = info
+
+        hostname = self.__get_system_name()
+        if hostname not in self.__SINGLE_PAGE_LOOP_MS:
+            first_host = self.__SINGLE_PAGE_LOOP_MS.keys()[0]
+            log_once("hostname", "No timing info for host %s" % hostname +
+                     ", needed to calculate work done per task. Please get the "
+                     "timing info and add to __SINGLE_PAGE_LOOP_MS in " +
+                     "mc_generators.py. Assuming host %s." % first_host)
+            hostname = first_host
+        self.host = hostname
 
     def __make_options(self):
         self.__make_system_info()
 
         return [GenOption('type', COLOR_TYPES, COLOR_TYPES,
                           'Cache management type.'),
-                GenOption('chunk_size', float, [0], 'Chunk size.'),
-                GenOption('ways', int, [self.system.ways], 'Ways (associativity).'),
-                GenOption('colors', int, [self.system.colors],
+                GenOption('host', self.__SINGLE_PAGE_LOOP_MS.keys(), self.host,
+                          'System experiment will run on (for calculating work).'),
+                GenOption('chunk_size_ns', float, 0, 'Chunk size. 0 = no chunking.'),
+                GenOption('ways', int, self.cache.ways, 'Ways (associativity).'),
+                GenOption('colors', int, self.cache.colors,
                           'System colors (cache size / ways).'),
-                GenOption('page_size', int, [self.system.page],
+                GenOption('page_size', int, self.cache.page,
                           'System page size.'),
-                GenOption('wss', [float, int], [.5],
+                GenOption('wss', [float, int], .5,
                           'Task working set sizes. Can be expressed as a fraction ' +
                           'of the cache.')]
 
@@ -346,14 +371,37 @@ class ColorMcGenerator(McGenerator):
             for color, replicas in task.colors.iteritems():
                 f.write("%d, %d\n" % (color, replicas))
 
+    def __get_loops(self, task, pages, system):
+        all_pages_loop = self.__SINGLE_PAGE_LOOP_MS[system] * pages
+        return int(task.cost / all_pages_loop) + 1
+
+    def _get_tasks(self, params):
+        # Share tasksets amongst experiments with different types but
+        # identical other parameters for proper comparisons
+        if self.tasksets == None:
+            fields = params.keys()
+            fields.remove("type")
+            self.tasksets = TupleTable( ColMap(fields), lambda:None )
+
+        if params not in self.tasksets:
+            ts = super(ColorMcGenerator, self)._get_tasks(params)
+            self.tasksets[params] = ts
+
+        return self.tasksets[params]
+
     def _customize(self, task_system, params):
         '''Add coloring properties to the mixed-criticality task system.'''
+        pages_needed = self.__get_wss_pages(params)
+        real_wss = params['page_size'] * pages_needed
+
         # Every task needs a unique id for coloring and wss walk order
         all_tasks = []
         for level, tasks in task_system.iteritems():
             all_tasks += tasks
         for i, task in enumerate(all_tasks):
-            task.id = i
+            task.id  = i
+            task.wss = real_wss
+            task.loops = self.__get_loops(task, pages_needed, params['host'])
 
         c = params['colors']
         w = params['ways']
@@ -365,8 +413,6 @@ class ColorMcGenerator(McGenerator):
             srt_colorer = RandomColorScheme(c, w)
             hrt_colorer = BlockColorScheme(c, w, way_first=True)
 
-        pages_needed = self.__get_wss_pages(params)
-
         hrt_colorer.color(task_system['lvlb'], pages_needed)
         srt_colorer.color(task_system['lvlc'], pages_needed)
 
diff --git a/gen_exps.py b/gen_exps.py
index 6488cdc..b847661 100755
--- a/gen_exps.py
+++ b/gen_exps.py
@@ -7,6 +7,7 @@ import re
 import shutil as sh
 import sys
 
+from config.config import DEFAULTS
 from optparse import OptionParser
 
 def parse_args():
@@ -15,7 +16,7 @@ def parse_args():
 
     parser.add_option('-o', '--out-dir', dest='out_dir',
                       help='directory for data output',
-                      default=("%s/exps"%os.getcwd()))
+                      default=("%s/%s"% (os.getcwd(), DEFAULTS['out-gen'])))
     parser.add_option('-f', '--force', action='store_true', default=False,
                       dest='force', help='overwrite existing data')
     parser.add_option('-n', '--num-trials', default=1, type='int', dest='trials',
@@ -51,9 +52,9 @@ def main():
     if opts.described != None:
         for generator in opts.described.split(','):
             if generator not in gen.get_generators():
-                print("No generator '%s'" % generator)
+                sys.stderr.write("No generator '%s'\n" % generator)
             else:
-                sys.stdout.write("Generator '%s', " % generator)
+                print("Generator '%s', " % generator)
                 gen.get_generators()[generator]().print_help()
     if opts.list_gens or opts.described:
         return 0
@@ -85,7 +86,7 @@ def main():
         if gen_name not in gen.get_generators():
             raise ValueError("Invalid generator '%s'" % gen_name)
 
-        print("Creating experiments using %s generator..." % gen_name)
+        sys.stderr.write("Creating experiments with %s generator...\n" % gen_name)
 
         params = dict(gen_params.items() + global_params.items())
         clazz  = gen.get_generators()[gen_name]
@@ -94,5 +95,7 @@ def main():
 
         generator.create_exps(opts.out_dir, opts.force, opts.trials)
 
+    sys.stderr.write("Experiments saved in %s.\n" % opts.out_dir)
+
 if __name__ == '__main__':
     main()
diff --git a/parse/point.py b/parse/point.py
index ac47c70..b1d9d53 100644
--- a/parse/point.py
+++ b/parse/point.py
@@ -133,6 +133,10 @@ class ExpPoint(object):
     def get_stats(self):
         return self.stats.keys()
 
+    def __bool__(self):
+        return bool(self.stats)
+    __nonzero__ = __bool__
+
 
 class SummaryPoint(ExpPoint):
     def __init__(self, id="", points=[], typemap = default_typemap):
diff --git a/parse/sched.py b/parse/sched.py
index 1f07751..1033989 100644
--- a/parse/sched.py
+++ b/parse/sched.py
@@ -3,67 +3,119 @@ import os
 import re
 import struct
 import subprocess
-import sys
 
 from collections import defaultdict,namedtuple
-from common import recordtype
+from common import recordtype,log_once
 from point import Measurement
+from ctypes import *
+
+LOSS_MSG = """Found task missing more than %d%% of its scheduling records.
+These won't be included in scheduling statistics!"""%(100*conf.MAX_RECORD_LOSS)
+SKIP_MSG = """Measurement '%s' has no non-zero values.
+Measurements like these are not included in scheduling statistics.
+If a measurement is missing, this is why."""
+SCALE_MSG = """Task in {} with config {} has < 1.0 scale!
+These scales are skipped in measurements."""
+
+# Data stored for each task
+TaskParams = namedtuple('TaskParams',  ['wcet', 'period', 'cpu', 'level'])
+TaskData   = recordtype('TaskData',    ['params', 'jobs', 'loads',
+                                        'blocks', 'misses', 'execs'])
+
+ScaleData = namedtuple('ScaleData', ['reg_tasks', 'base_tasks'])
 
 class TimeTracker:
     '''Store stats for durations of time demarcated by sched_trace records.'''
     def __init__(self):
-        self.begin = self.avg = self.max = self.num = self.job = 0
+        self.begin = self.avg = self.max = self.num = self.next_job = 0
+
+        # Count of times the job in start_time matched that in store_time
+        self.matches = 0
+        # And the times it didn't
+        self.disjoints = 0
 
-    def store_time(self, record):
+        # Measurements are recorded in store_ time using the previous matching
+        # record which was passed to store_time. This way, the last record for
+        # any task is always skipped
+        self.last_record = None
+
+    def store_time(self, next_record):
         '''End duration of time.'''
-        dur = record.when - self.begin
+        dur = (self.last_record.when - self.begin) if self.last_record else -1
+
+        if self.next_job == next_record.job:
+            self.last_record = next_record
 
-        if self.job == record.job and dur > 0:
-            self.max  = max(self.max, dur)
-            self.avg *= float(self.num / (self.num + 1))
-            self.num += 1
-            self.avg += dur / float(self.num)
+            if self.last_record:
+                self.matches += 1
 
-            self.begin = 0
-            self.job   = 0
+            if dur > 0:
+                self.max  = max(self.max, dur)
+                self.avg *= float(self.num / (self.num + 1))
+                self.num += 1
+                self.avg += dur / float(self.num)
 
-    def start_time(self, record):
+                self.begin = 0
+                self.next_job   = 0
+        else:
+            self.disjoints += 1
+
+    def start_time(self, record, time = None):
         '''Start duration of time.'''
-        self.begin = record.when
-        self.job   = record.job
+        if self.last_record:
+            if not time:
+                self.begin = self.last_record.when
+            else:
+                self.begin = time
 
-# Data stored for each task
-TaskParams = namedtuple('TaskParams',  ['wcet', 'period', 'cpu', 'level'])
-TaskData   = recordtype('TaskData',    ['params', 'jobs', 'loads',
-                                        'blocks', 'misses', 'execs'])
+        self.next_job = record.job
 
-# Map of event ids to corresponding class, binary format, and processing methods
-RecordInfo = namedtuple('RecordInfo', ['clazz', 'fmt', 'method'])
-record_map = [0]*10
 
-# Common to all records
-HEADER_FORMAT = '<bbhi'
-HEADER_FIELDS = ['type', 'cpu', 'pid', 'job']
-RECORD_SIZE   = 24
+class LeveledArray(object):
+    """Groups statistics by the level of the task to which they apply"""
+    def __init__(self):
+        self.name = name
+        self.vals = defaultdict(lambda: defaultdict(lambda:[]))
+
+    def add(self, name, level, value):
+        if type(value) != type([]):
+            value = [value]
+        self.vals[name][level] += value
+
+    def write_measurements(self, result):
+        for stat_name, stat_data in self.vals.iteritems():
+            for level, values in stat_data.iteritems():
+                if not values or not sum(values):
+                    log_once(SKIP_MSG, SKIP_MSG % stat_name)
+                    continue
+
+                name = "%s%s" % ("%s-" % level if level else "", stat_name)
+                result[name] = Measurement(name).from_array(arr)
 
+# Map of event ids to corresponding class and format
+record_map = {}
+
+RECORD_SIZE   = 24
 NSEC_PER_MSEC = 1000000
 
-def register_record(name, id, method, fmt, fields):
-    '''Create record description from @fmt and @fields and map to @id, using
-    @method to process parsed record.'''
-    # Format of binary data (see python struct documentation)
-    rec_fmt = HEADER_FORMAT + fmt
+def register_record(id, clazz):
+    fields = clazz.FIELDS
 
-    # Corresponding field data
-    rec_fields = HEADER_FIELDS + fields
-    if "when" not in rec_fields: # Force a "when" field for everything
-        rec_fields += ["when"]
+    fsize = lambda fields : sum([sizeof(list(f)[1]) for f in fields])
+    diff  = RECORD_SIZE - fsize(SchedRecord.FIELDS) - fsize(fields)
 
-    # Create mutable class with the given fields
-    field_class = recordtype(name, list(rec_fields))
-    clazz = type(name, (field_class, object), {})
+    # Create extra padding fields to make record the proper size
+    # Creating one big field of c_uint64 and giving it a size of 8*diff
+    # _shoud_ work, but doesn't. This is an uglier way of accomplishing
+    # the same goal
+    for d in range(diff):
+        fields += [("extra%d" % d, c_char)]
 
-    record_map[id] = RecordInfo(clazz, rec_fmt, method)
+    # Create structure with fields and methods of clazz
+    clazz2 = type("Dummy%d" % id, (LittleEndianStructure,clazz),
+                  {'_fields_': SchedRecord.FIELDS + fields,
+                   '_pack_'  : 1})
+    record_map[id] = clazz2
 
 def make_iterator(fname):
     '''Iterate over (parsed record, processing method) in a
@@ -73,7 +125,6 @@ def make_iterator(fname):
         return
 
     f = open(fname, 'rb')
-    max_type = len(record_map)
 
     while True:
         data = f.read(RECORD_SIZE)
@@ -83,156 +134,169 @@ def make_iterator(fname):
         except struct.error:
             break
 
-        rdata = record_map[type_num] if type_num <= max_type else 0
-        if not rdata:
+        if type_num not in record_map:
             continue
 
-        try:
-            values = struct.unpack_from(rdata.fmt, data)
-        except struct.error:
-            continue
+        clazz = record_map[type_num]
+        obj = clazz()
+        obj.fill(data)
 
-        obj = rdata.clazz(*values)
-        yield (obj, rdata.method)
+        if obj.job != 1:
+            yield obj
+        else:
+            # Results from the first job are nonsense
+            pass
 
 def read_data(task_dict, fnames):
     '''Read records from @fnames and store per-pid stats in @task_dict.'''
     buff = []
 
+    def get_time(record):
+        return record.when if hasattr(record, 'when') else 0
+
     def add_record(itera):
         # Ordered insertion into buff
         try:
-            next_ret = itera.next()
+            arecord = itera.next()
         except StopIteration:
             return
 
-        arecord, method = next_ret
         i = 0
-        for (i, (brecord, m, t)) in enumerate(buff):
-            if brecord.when > arecord.when:
+        for (i, (brecord, _)) in enumerate(buff):
+            if get_time(brecord) > get_time(arecord):
                 break
-        buff.insert(i, (arecord, method, itera))
+        buff.insert(i, (arecord, itera))
 
     for fname in fnames:
         itera = make_iterator(fname)
         add_record(itera)
 
     while buff:
-        (record, method, itera) = buff.pop(0)
+        record, itera = buff.pop(0)
 
         add_record(itera)
-        method(task_dict, record)
-
-def process_completion(task_dict, record):
-    task_dict[record.pid].misses.store_time(record)
-    task_dict[record.pid].loads += [record.load]
-
-def process_release(task_dict, record):
-    data = task_dict[record.pid]
-    data.jobs += 1
-    data.misses.start_time(record)
-
-def process_param(task_dict, record):
-    level = chr(97 + record.level)
-    params = TaskParams(record.wcet, record.period,
-                        record.partition, level)
-    task_dict[record.pid].params = params
-
-def process_block(task_dict, record):
-    task_dict[record.pid].blocks.start_time(record)
-
-def process_resume(task_dict, record):
-    task_dict[record.pid].blocks.store_time(record)
-
-def process_switch_to(task_dict, record):
-    task_dict[record.pid].execs.start_time(record)
-
-def process_switch_away(task_dict, record):
-    task_dict[record.pid].execs.store_time(record)
-
-register_record('ResumeRecord', 9, process_resume, 'Q8x', ['when'])
-register_record('BlockRecord', 8, process_block, 'Q8x', ['when'])
-register_record('CompletionRecord', 7, process_completion, 'QQ', ['when', 'load'])
-register_record('ReleaseRecord', 3, process_release, 'QQ', ['release', 'when'])
-register_record('SwitchToRecord', 5, process_switch_to, 'Q8x', ['when'])
-register_record('SwitchAwayRecord', 6, process_switch_away, 'Q8x', ['when'])
-register_record('ParamRecord', 2, process_param, 'IIIcccx',
-                ['wcet','period','phase','partition', 'task_class', 'level'])
-
-saved_stats = []
-def get_task_data(data_dir, work_dir = None):
+        record.process(task_dict)
+
+class SchedRecord(object):
+    # Subclasses will have their FIELDs merged into this one
+    FIELDS = [('type', c_uint8),  ('cpu', c_uint8),
+              ('pid',  c_uint16), ('job', c_uint32)]
+
+    def fill(self, data):
+        memmove(addressof(self), data, RECORD_SIZE)
+
+    def process(self, task_dict):
+        raise NotImplementedError()
+
+class ParamRecord(SchedRecord):
+    FIELDS = [('wcet', c_uint32),  ('period', c_uint32),
+              ('phase', c_uint32), ('partition', c_uint8),
+              ('class', c_uint8),  ('level', c_uint8)]
+
+    def process(self, task_dict):
+        params = TaskParams(self.wcet, self.period,
+                            self.partition, self.level)
+        task_dict[self.pid].params = params
+
+class ReleaseRecord(SchedRecord):
+    FIELDS = [('when', c_uint64), ('release', c_uint64)]
+
+    def process(self, task_dict):
+        data = task_dict[self.pid]
+        data.jobs += 1
+        if data.params:
+            data.misses.start_time(self, self.when + data.params.period)
+
+class CompletionRecord(SchedRecord):
+    FIELDS = [('when', c_uint64)]
+
+    def process(self, task_dict):
+        task_dict[self.pid].misses.store_time(self)
+
+class BlockRecord(SchedRecord):
+    FIELDS = [('when', c_uint64)]
+
+    def process(self, task_dict):
+        task_dict[self.pid].blocks.start_time(self)
+
+class ResumeRecord(SchedRecord):
+    FIELDS = [('when', c_uint64)]
+
+    def process(self, task_dict):
+        task_dict[self.pid].blocks.store_time(self)
+
+# Map records to sched_trace ids (see include/litmus/sched_trace.h
+register_record(2, ParamRecord)
+register_record(3, ReleaseRecord)
+register_record(7, CompletionRecord)
+register_record(8, BlockRecord)
+register_record(9, ResumeRecord)
+
+__all_dicts = {}
+
+def create_task_dict(data_dir, work_dir = None):
     '''Parse sched trace files'''
-    if data_dir in saved_stats:
-        return data_dir
+    if data_dir in __all_dicts:
+        return __all_dicts[data_dir]
 
     bin_files   = conf.FILES['sched_data'].format(".*")
     output_file = "%s/out-st" % work_dir
 
-    bins = ["%s/%s" % (data_dir,f) for f in os.listdir(data_dir) if re.match(bin_files, f)]
-    if not len(bins):
-        return
+    task_dict = defaultdict(lambda :
+                            TaskData(None, 1, TimeTracker(), TimeTracker()))
+
+    bin_names = [f for f in os.listdir(data_dir) if re.match(bin_files, f)]
+    if not len(bin_names):
+        return task_dict
 
     # Save an in-english version of the data for debugging
     # This is optional and will only be done if 'st_show' is in PATH
     if work_dir and conf.BINS['st_show']:
         cmd_arr = [conf.BINS['st_show']]
-        cmd_arr.extend(bins)
+        cmd_arr.extend(bin_names)
         with open(output_file, "w") as f:
-            print("calling %s" % cmd_arr)
             subprocess.call(cmd_arr, cwd=data_dir, stdout=f)
 
-    task_dict = defaultdict(lambda :TaskData(0, 0, 0, [], TimeTracker(),
-                                             TimeTracker(), TimeTracker()))
-
     # Gather per-task values
-    read_data(task_dict, bins)
+    bin_paths = ["%s/%s" % (data_dir,f) for f in bin_names]
+    read_data(task_dict, bin_paths)
 
-    saved_stats[data_dir] = task_dict
-    return task_dict
+    __all_dicts[data_dir] = task_dict
 
-class LeveledArray(object):
-    """Groups statistics by the level of the task to which they apply"""
-    def __init__(self):
-        self.name = name
-            self.vals = defaultdict(lambda: defaultdict(lambda:[]))
-
-    def add(self, name, level, value):
-        if type(value) != type([]):
-            value = [value]
-        self.vals[name][task.config.level] += value
-
-    def write_measurements(self, result):
-        for stat_name, stat_data in self.vals.iteritems():
-            for level, values in stat_data.iteritems():
-                if not values:
-                    continue
-
-                name = "%s%s" % ("%s-" % level if level else "", stat_name)
-                result[name] = Measurement(name).from_array(arr)
+    return task_dict
 
 def extract_sched_data(result, data_dir, work_dir):
-    task_dict = get_task_data(data_dir, work_dir)
-
+    task_dict = create_task_dict(data_dir, work_dir)
     stat_data = LeveledArray()
+
     for tdata in task_dict.itervalues():
         if not tdata.params:
             # Currently unknown where these invalid tasks come from...
             continue
 
-        miss_ratio = float(tdata.misses.num) / tdata.jobs
-        # Scale average down to account for jobs with 0 tardiness
-        avg_tard = tdata.misses.avg * miss_ratio
+        level = tdata.config.level
+        miss  = tdata.misses
+
+        record_loss = float(miss.disjoints)/(miss.matches + miss.disjoints)
+        stat_data("record-loss", level, record_loss)
+
+        if record_loss > conf.MAX_RECORD_LOSS:
+            log_once(LOSS_MSG)
+            continue
+
+        miss_ratio = float(miss.num) / miss.matches
+        avg_tard = miss.avg * miss_ratio
 
-        level = tdata.params.level
-        stat_data.add("miss-ratio", level, miss_ratio)
-        stat_data.add("avg-tard",   level, avg_tard / tdata.params.wcet)
-        stat_data.add("max-tard",   level, tdata.misses.max / tdata.params.wcet)
-        stat_data.add("avg-block",  level, tdata.blocks.avg / NSEC_PER_MSEC)
-        stat_data.add("max-block",  level, tdata.blocks.max / NSEC_PER_MSEC)
+        stat_data("miss-ratio", level, miss_ratio)
+
+        stat_data("max-tard",   level, miss.max / tdata.params.period)
+        stat_data("avg-tard",   level, avg_tard / tdata.params.period)
+
+        stat_data("avg-block",  level, tdata.blocks.avg / NSEC_PER_MSEC)
+        stat_data("max-block",  level, tdata.blocks.max / NSEC_PER_MSEC)
 
     stat_data.write_measurements(result)
 
-ScaleData = namedtuple('ScaleData', ['reg_tasks', 'base_tasks'])
 def extract_mc_data(result, data_dir, base_dir):
     task_dict = get_task_data(data_dir)
     base_dict = get_task_data(base_dir)
@@ -245,12 +309,14 @@ def extract_mc_data(result, data_dir, base_dir):
 
     tasks_by_config = defaultdict(lambda: ScaleData([], []))
 
-    # Add tasks in order of pid to tasks_by_config
-    # Tasks must be ordered by pid or we can't make 1 to 1 comparisons
-    # when multiple tasks have the same config in each task set
+    # Add task execution times in order of pid to tasks_by_config
     for tasks, field in ((task_dict, 'reg_tasks'), (base_dict, 'base_tasks')):
+        # Sorted for tie breaking: if 3 regular tasks have the same config
+        # (so 3 base tasks also have the same config), match first pid regular
+        # with first pid base, etc. This matches tie breaking in kernel
         for pid in sorted(tasks.keys()):
             tdata = tasks[pid]
+
         tlist  = getattr(tasks_by_config[tdata.params], field)
         tlist += [tdata.execs]
 
@@ -260,7 +326,10 @@ def extract_mc_data(result, data_dir, base_dir):
             # Can't make comparison if different numbers of tasks!
             continue
 
+        # Tuples of (regular task execution times, base task execution times)
+        # where each has the same configuration
         all_pairs = zip(scale_data.reg_tasks, scale_data.base_tasks)
+
         for reg_execs, base_execs in all_pairs:
             if not reg_execs.max  or not reg_execs.avg or\
                not base_execs.max or not base_execs.avg:
@@ -271,8 +340,7 @@ def extract_mc_data(result, data_dir, base_dir):
             avg_scale = float(base_execs.avg) / reg_execs.avg
 
             if (avg_scale < 1 or max_scale < 1) and config.level == "b":
-                sys.stderr.write("Task in {} with config {} has <1.0 scale!"
-                                 .format(data_dir, config)
+                log_once(SCALE_MSG, SCALE_MSG.format(data_dir, config))
                 continue
 
             stat_data.add('max-scale', config.level, max_scale)
diff --git a/parse/tuple_table.py b/parse/tuple_table.py
index 47fb6b6..6edc037 100644
--- a/parse/tuple_table.py
+++ b/parse/tuple_table.py
@@ -13,13 +13,17 @@ class TupleTable(object):
     def get_col_map(self):
         return self.col_map
 
+    def __bool__(self):
+        return bool(self.table)
+    __nonzero__ = __bool__
+
     def __getitem__(self, kv):
         key = self.col_map.get_key(kv)
         return self.table[key]
 
     def __setitem__(self, kv, value):
         key = self.col_map.get_key(kv)
-        self.table[key]
+        self.table[key] = value
 
     def __contains__(self, kv):
         key = self.col_map.get_key(kv)
diff --git a/parse_exps.py b/parse_exps.py
index 9475cfc..7a99d8a 100755
--- a/parse_exps.py
+++ b/parse_exps.py
@@ -1,8 +1,9 @@
 #!/usr/bin/env python
 from __future__ import print_function
 
-import config.config as conf
 import copy
+import common as com
+import multiprocessing
 import os
 import parse.ft as ft
 import parse.sched as st
@@ -13,18 +14,19 @@ import sys
 import traceback
 
 from collections import namedtuple
-from common import load_params
+from config.config import DEFAULTS,PARAMS
 from optparse import OptionParser
 from parse.point import ExpPoint
 from parse.tuple_table import TupleTable
 from parse.col_map import ColMapBuilder
-from multiprocessing import Pool, cpu_count
+
 
 def parse_args():
     parser = OptionParser("usage: %prog [options] [data_dir]...")
 
     parser.add_option('-o', '--out', dest='out',
-                      help='file or directory for data output', default='parse-data')
+                      help='file or directory for data output',
+                      default=DEFAULTS['out-parse'])
     parser.add_option('-i', '--ignore', metavar='[PARAM...]', default="",
                       help='ignore changing parameter values')
     parser.add_option('-f', '--force', action='store_true', default=False,
@@ -34,7 +36,8 @@ def parse_args():
     parser.add_option('-m', '--write-map', action='store_true', default=False,
                       dest='write_map',
                       help='Output map of values instead of csv tree')
-    parser.add_option('-p', '--processors', default=max(cpu_count() - 1, 1),
+    parser.add_option('-p', '--processors',
+                      default=max(multiprocessing.cpu_count() - 1, 1),
                       type='int', dest='processors',
                       help='number of threads for processing')
     parser.add_option('-s', '--scale-against', dest='scale_against',
@@ -43,12 +46,59 @@ def parse_args():
 
     return parser.parse_args()
 
+
 ExpData = namedtuple('ExpData', ['path', 'params', 'work_dir'])
 
+
+def parse_exp(exp_force_base):
+    # Tupled for multiprocessing
+    exp, force, base_table = exp_force_base
+
+    result_file = exp.work_dir + "/exp_point.pkl"
+    should_load = not force and os.path.exists(result_file)
+
+    result = None
+    if should_load:
+        with open(result_file, 'rb') as f:
+            try:
+                # No need to go through this work twice
+                result = pickle.load(f)
+            except:
+                pass
+
+    if not result:
+        try:
+            # Create a readable name
+            name = os.path.relpath(exp.path)
+            name = name if name != "." else os.path.split(os.getcwd())[1]
+
+            result = ExpPoint(name)
+
+            # Write overheads into result
+            cycles = exp.params[PARAMS['cycles']]
+            ft.extract_ft_data(result, exp.path, exp.work_dir, cycles)
+
+            # Write scheduling statistics into result
+            st.extract_sched_data(result, exp.path, exp.work_dir)
+
+            # Write scaling factors into result
+            if base_table and exp.params in base_table:
+                base_exp = base_table[exp.params]
+                if base_exp != exp:
+                    st.extract_mc_data(result, exp.path, base_exp.path)
+
+            with open(result_file, 'wb') as f:
+                pickle.dump(result, f)
+        except:
+            traceback.print_exc()
+
+    return (exp, result)
+
+
 def get_exp_params(data_dir, cm_builder):
-    param_file = "%s/%s" % (data_dir, conf.DEFAULTS['params_file'])
+    param_file = "%s/%s" % (data_dir, DEFAULTS['params_file'])
     if os.path.isfile(param_file):
-        params = load_params(param_file)
+        params = com.load_params(param_file)
 
         # Store parameters in cm_builder, which will track which parameters change
         # across experiments
@@ -58,8 +108,8 @@ def get_exp_params(data_dir, cm_builder):
          params = {}
 
     # Cycles must be present for feather-trace measurement parsing
-    if conf.PARAMS['cycles'] not in params:
-        params[conf.PARAMS['cycles']] = conf.DEFAULTS['cycles']
+    if PARAMS['cycles'] not in params:
+        params[PARAMS['cycles']] = DEFAULTS['cycles']
 
     return params
 
@@ -87,44 +137,6 @@ def load_exps(exp_dirs, cm_builder, force):
 
     return exps
 
-def parse_exp(exp_force_base):
-    # Tupled for multiprocessing
-    exp, force, base_table  = exp_force_base
-
-    result_file = exp.work_dir + "/exp_point.pkl"
-    should_load = not force and os.path.exists(result_file)
-
-    result = None
-    if should_load:
-        with open(result_file, 'rb') as f:
-            try:
-                # No need to go through this work twice
-                result = pickle.load(f)
-            except:
-                pass
-
-    if not result:
-        try:
-            result = ExpPoint(exp.path)
-            cycles = exp.params[conf.PARAMS['cycles']]
-
-            # Write overheads into result
-            ft.extract_ft_data(result, exp.path, exp.work_dir, cycles)
-
-            # Write scheduling statistics into result
-            st.extract_sched_data(result, exp.path, exp.work_dir)
-
-            if base_table and exp.params in base_table:
-                base_exp = base_table[exp.params]
-                if base_exp != exp:
-                    st.extract_scaling_data(result, exp.path, base_exp.path)
-
-            with open(result_file, 'wb') as f:
-                pickle.dump(result, f)
-        except:
-            traceback.print_exc()
-
-    return (exp, result)
 
 def make_base_table(cmd_scale, col_map, exps):
     if not cmd_scale:
@@ -137,8 +149,7 @@ def make_base_table(cmd_scale, col_map, exps):
     if param not in col_map:
         raise IOError("Base column '%s' not present in any parameters!" % param)
 
-    base_map = copy.deepcopy(col_map)
-    base_table = TupleTable(base_map)
+    base_table = TupleTable(copy.deepcopy(col_map))
 
     # Fill table with exps who we will scale against
     for exp in exps:
@@ -147,41 +158,49 @@ def make_base_table(cmd_scale, col_map, exps):
 
     return base_table
 
-def main():
-    opts, args = parse_args()
 
-    args = args or [os.getcwd()]
-
-    # Load exp parameters into a ColMap
-    builder = ColMapBuilder()
-    exps = load_exps(args, builder, opts.force)
+def get_dirs(args):
+    if args:
+        return args
+    elif os.path.exists(DEFAULTS['out-run']):
+        sys.stderr.write("Reading data from %s/*\n" % DEFAULTS['out-run'])
+        sched_dirs = os.listdir(DEFAULTS['out-run'])
+        return ['%s/%s' % (DEFAULTS['out-run'], d) for d in sched_dirs]
+    else:
+        sys.stderr.write("Reading data from current directory.\n")
+        return [os.getcwd()]
 
-    # Don't track changes in ignored parameters
-    if opts.ignore:
-        for param in opts.ignore.split(","):
-            builder.try_remove(param)
-    # Always average multiple trials
-    builder.try_remove(conf.PARAMS['trial'])
 
-    col_map = builder.build()
-    result_table = TupleTable(col_map)
+def fill_table(table, exps, opts):
+    sys.stderr.write("Parsing data...\n")
 
-    base_table = make_base_table(opts.scale_against, col_map, exps)
+    procs  = min(len(exps), opts.processors)
+    logged = multiprocessing.Manager().list()
 
     sys.stderr.write("Parsing data...\n")
 
-    procs = min(len(exps), opts.processors)
-    pool = Pool(processes=procs)
+    base_table = make_base_table(opts.scale_against,
+                                 table.get_col_map(), exps)
+
+    pool = multiprocessing.Pool(processes=procs,
+    # Share a list of previously logged messages amongst processes
+    # This is for the com.log_once method to use
+                initializer=com.set_logged_list, initargs=(logged,))
+
     pool_args = zip(exps, [opts.force]*len(exps), [base_table]*len(exps))
     enum = pool.imap_unordered(parse_exp, pool_args, 1)
 
     try:
         for i, (exp, result) in enumerate(enum):
+            if not result:
+                continue
+
             if opts.verbose:
                 print(result)
             else:
                 sys.stderr.write('\r {0:.2%}'.format(float(i)/len(exps)))
-                result_table[exp.params] += [result]
+                table[exp.params] += [result]
+
         pool.close()
     except:
         pool.terminate()
@@ -192,29 +211,61 @@ def main():
 
     sys.stderr.write('\n')
 
-    if opts.force and os.path.exists(opts.out):
-        sh.rmtree(opts.out)
 
-    reduced_table = result_table.reduce()
+def write_output(table, opts):
+    reduced_table = table.reduce()
 
     if opts.write_map:
         sys.stderr.write("Writing python map into %s...\n" % opts.out)
-        # Write summarized results into map
         reduced_table.write_map(opts.out)
     else:
+        if opts.force and os.path.exists(opts.out):
+            sh.rmtree(opts.out)
+
         # Write out csv directories for all variable params
         dir_map = reduced_table.to_dir_map()
 
         # No csvs to write, assume user meant to print out data
         if dir_map.is_empty():
             if not opts.verbose:
-                sys.stderr.write("Too little data to make csv files.\n")
-                for key, exp in result_table:
+                sys.stderr.write("Too little data to make csv files, " +
+                                 "printing results.\n")
+                for key, exp in table:
                     for e in exp:
                         print(e)
         else:
             sys.stderr.write("Writing csvs into %s...\n" % opts.out)
             dir_map.write(opts.out)
 
+
+def main():
+    opts, args = parse_args()
+    exp_dirs = get_dirs(args)
+
+    # Load experiment parameters into a ColMap
+    builder = ColMapBuilder()
+    exps = load_exps(exp_dirs, builder, opts.force)
+
+    # Don't track changes in ignored parameters
+    if opts.ignore:
+        for param in opts.ignore.split(","):
+            builder.try_remove(param)
+
+    # Always average multiple trials
+    builder.try_remove(PARAMS['trial'])
+    # Only need this for feather-trace parsing
+    builder.try_remove(PARAMS['cycles'])
+
+    col_map = builder.build()
+    table = TupleTable(col_map)
+
+    fill_table(table, exps, opts)
+
+    if not table:
+        sys.stderr.write("Found no data to parse!")
+        sys.exit(1)
+
+    write_output(table, opts)
+
 if __name__ == '__main__':
     main()
diff --git a/plot_exps.py b/plot_exps.py
index 76e7396..15c54d0 100755
--- a/plot_exps.py
+++ b/plot_exps.py
@@ -6,7 +6,9 @@ import os
 import shutil as sh
 import sys
 import traceback
+
 from collections import namedtuple
+from config.config import DEFAULTS
 from multiprocessing import Pool, cpu_count
 from optparse import OptionParser
 from parse.col_map import ColMap,ColMapBuilder
@@ -17,7 +19,8 @@ def parse_args():
     parser = OptionParser("usage: %prog [options] [csv_dir]...")
 
     parser.add_option('-o', '--out-dir', dest='out_dir',
-                      help='directory for plot output', default='plot-data')
+                      help='directory for plot output',
+                      default=DEFAULTS['out-plot'])
     parser.add_option('-f', '--force', action='store_true', default=False,
                       dest='force', help='overwrite existing data')
     parser.add_option('-p', '--processors', default=max(cpu_count() - 1, 1),
@@ -139,21 +142,31 @@ def plot_dir(data_dir, out_dir, max_procs, force):
 
     sys.stderr.write('\n')
 
+def get_dirs(args):
+    if args:
+        return args
+    elif os.path.exists(DEFAULTS['out-parse']):
+        return [DEFAULTS['out-parse']]
+    else:
+        return os.getcwd()
+
 def main():
     opts, args = parse_args()
-    args = args or [os.getcwd()]
+    dirs = get_dirs(args)
 
     if opts.force and os.path.exists(opts.out_dir):
         sh.rmtree(opts.out_dir)
     if not os.path.exists(opts.out_dir):
         os.mkdir(opts.out_dir)
 
-    for dir in args:
-        if len(args) > 1:
+    for dir in dirs:
+        if len(dirs) > 1:
             out_dir = "%s/%s" % (opts.out_dir, os.path.split(dir)[1])
         else:
             out_dir = opts.out_dir
         plot_dir(dir, out_dir, opts.processors, opts.force)
 
+    sys.stderr.write("Plots saved in %s.\n" % opts.out_dir)
+
 if __name__ == '__main__':
     main()
diff --git a/run/executable/executable.py b/run/executable/executable.py
index 263e305..a2426f1 100644
--- a/run/executable/executable.py
+++ b/run/executable/executable.py
@@ -6,9 +6,10 @@ from common import get_executable
 class Executable(object):
     '''Parent object that represents an executable for use in task-sets.'''
 
-    def __init__(self, exec_file, extra_args=None, stdout_file = None, stderr_file = None):
+    def __init__(self, exec_file, extra_args=None, stdout_file = None,
+                 stderr_file = None, cwd = None):
         self.exec_file = get_executable(exec_file)
-        self.cwd = None
+        self.cwd = cwd
         self.stdout_file = stdout_file
         self.stderr_file = stderr_file
         self.sp = None
@@ -58,6 +59,9 @@ class Executable(object):
     def interrupt(self):
         self.sp.send_signal(signal.SIGINT)
 
+    def poll(self):
+        return self.sp.poll()
+
     def terminate(self):
         '''Send the terminate signal to the binary.'''
         self.sp.terminate()
diff --git a/run/experiment.py b/run/experiment.py
index ff0e9f3..b0e46b6 100644
--- a/run/experiment.py
+++ b/run/experiment.py
@@ -1,9 +1,7 @@
-import common as com
 import os
 import time
 import run.litmus_util as lu
 import shutil as sh
-import traceback
 from operator import methodcaller
 
 class ExperimentException(Exception):
@@ -11,16 +9,11 @@ class ExperimentException(Exception):
     def __init__(self, name):
         self.name = name
 
-
 class ExperimentDone(ExperimentException):
     '''Raised when an experiment looks like it's been run already.'''
     def __str__(self):
         return "Experiment finished already: %d" % self.name
 
-class ExperimentFailed(ExperimentException):
-    def __str__(self):
-        return "Experiment failed during execution: %d" % self.name
-
 class SystemCorrupted(Exception):
     pass
 
@@ -31,7 +24,6 @@ class Experiment(object):
     def __init__(self, name, scheduler, working_dir, finished_dir,
                  proc_entries, executables, tracer_types):
         '''Run an experiment, optionally wrapped in tracing.'''
-
         self.name = name
         self.scheduler = scheduler
         self.working_dir  = working_dir
@@ -40,16 +32,10 @@ class Experiment(object):
         self.executables  = executables
         self.exec_out = None
         self.exec_err = None
+        self.tracer_types = tracer_types
 
-        self.task_batch = com.num_cpus()
-
-        self.__make_dirs()
-        self.__assign_executable_cwds()
-        self.__setup_tracers(tracer_types)
-
-
-    def __setup_tracers(self, tracer_types):
-        tracers = [ t(self.working_dir) for t in tracer_types ]
+    def __setup_tracers(self):
+        tracers = [ t(self.working_dir) for t in self.tracer_types ]
 
         self.regular_tracers  = [t for t in tracers if not t.is_exact()]
         self.exact_tracers = [t for t in tracers if t.is_exact()]
@@ -84,13 +70,11 @@ class Experiment(object):
         map(assign_cwd, self.executables)
 
     def __kill_all(self):
-        # Give time for whatever failed to finish failing
-        time.sleep(2)
-
-        released = lu.release_tasks()
-        self.log("Re-released %d tasks" % released)
+        if lu.waiting_tasks():
+            released = lu.release_tasks()
+            self.log("Re-released %d tasks" % released)
 
-        time.sleep(5)
+            time.sleep(1)
 
         self.log("Killing all tasks")
         for e in self.executables:
@@ -99,7 +83,62 @@ class Experiment(object):
             except:
                 pass
 
-        time.sleep(2)
+        time.sleep(1)
+
+    def __strip_path(self, path):
+        '''Shorten path to something more readable.'''
+        file_dir = os.path.split(self.working_dir)[0]
+        if path.index(file_dir) == 0:
+            path = path[len(file_dir):]
+
+        return path.strip("/")
+
+    def __check_tasks_status(self):
+        '''Raises an exception if any tasks have failed.'''
+        msgs = []
+
+        for e in self.executables:
+            status = e.poll()
+            if status != None and status:
+                err_msg = "Task %s failed with status: %s" % (e.wait(), status)
+                msgs += [err_msg]
+
+        if msgs:
+            # Show at most 3 messages so that every task failing doesn't blow
+            # up the terminal
+            if len(msgs) > 3:
+                num_errs = len(msgs) - 3
+                msgs = msgs[0:4] + ["...%d more task errors..." % num_errs]
+
+            out_name = self.__strip_path(self.exec_out.name)
+            err_name = self.__strip_path(self.exec_err.name)
+            help = "Check dmesg, %s, and %s" % (out_name, err_name)
+
+            raise Exception("\n".join(msgs + [help]))
+
+    def __wait_for_ready(self):
+        self.log("Sleeping until tasks are ready for release...")
+
+        wait_start = time.time()
+        num_ready  = lu.waiting_tasks()
+
+        while num_ready < len(self.executables):
+            # Quit if too much time passes without a task becoming ready
+            if time.time() - wait_start > 180.0:
+                s = "waiting: %d, submitted: %d" %\
+                  (lu.waiting_tasks(), len(self.executables))
+                raise Exception("Too much time spent waiting for tasks! %s" % s)
+
+            time.sleep(1)
+
+            # Quit if any tasks fail
+            self.__check_tasks_status()
+
+            # Reset the waiting time whenever more tasks become ready
+            now_ready = lu.waiting_tasks()
+            if now_ready != num_ready:
+                wait_start = time.time()
+                num_ready  = lu.now_ready
 
     def __run_tasks(self):
         self.log("Starting %d tasks" % len(self.executables))
@@ -108,16 +147,9 @@ class Experiment(object):
             try:
                 e.execute()
             except:
-                raise Exception("Executable failed: %s" % e)
+                raise Exception("Executable failed to start: %s" % e)
 
-        self.log("Sleeping until tasks are ready for release...")
-        start = time.time()
-        while lu.waiting_tasks() < len(self.executables):
-            if time.time() - start > 30.0:
-                s = "waiting: %d, submitted: %d" %\
-                  (lu.waiting_tasks(), len(self.executables))
-                raise Exception("Too much time spent waiting for tasks! %s" % s)
-            time.sleep(1)
+        self.__wait_for_ready()
 
         # Exact tracers (like overheads) must be started right after release or
         # measurements will be full of irrelevant records
@@ -148,60 +180,40 @@ class Experiment(object):
     def __save_results(self):
         os.rename(self.working_dir, self.finished_dir)
 
-    def log(self, msg):
-        print("[Exp %s]: %s" % (self.name, msg))
-
-    def run_exp(self):
-        self.__check_system()
-
-        succ = False
-        try:
-            self.__setup()
+    def __to_linux(self):
+        msgs = []
 
+        sched = lu.scheduler()
+        if sched != "Linux":
             try:
-                self.__run_tasks()
-                self.log("Saving results in %s" % self.finished_dir)
-                succ = True
+                lu.switch_scheduler("Linux")
             except:
-                traceback.print_exc()
-                self.__kill_all()
-                raise ExperimentFailed(self.name)
-            finally:
-                self.__teardown()
-        finally:
-            self.log("Switching to Linux scheduler")
-
-            # Give the tasks 10 seconds to finish before bailing
-            start = time.time()
-            while lu.all_tasks() > 0:
-                if time.time() - start < 10.0:
-                    raise SystemCorrupted("%d tasks still running!" %
-                                          lu.all_tasks())
-
-            lu.switch_scheduler("Linux")
-
-        if succ:
-            self.__save_results()
-            self.log("Experiment done!")
+                msgs += ["Scheduler is %s, cannot switch to Linux!" % sched]
 
-    def __check_system(self):
         running = lu.all_tasks()
         if running:
-            raise SystemCorrupted("%d tasks already running!" % running)
+            msgs += ["%d real-time tasks still running!" % running]
 
-        sched = lu.scheduler()
-        if sched != "Linux":
-            raise SystemCorrupted("Scheduler is %s, should be Linux" % sched)
+        if msgs:
+            raise SystemCorrupted("\n".join(msgs))
 
     def __setup(self):
+        self.__make_dirs()
+        self.__assign_executable_cwds()
+        self.__setup_tracers()
+
         self.log("Writing %d proc entries" % len(self.proc_entries))
         map(methodcaller('write_proc'), self.proc_entries)
 
+        self.log("Starting %d regular tracers" % len(self.regular_tracers))
+        map(methodcaller('start_tracing'), self.regular_tracers)
+
+        time.sleep(1)
+
         self.log("Switching to %s" % self.scheduler)
         lu.switch_scheduler(self.scheduler)
 
-        self.log("Starting %d regular tracers" % len(self.regular_tracers))
-        map(methodcaller('start_tracing'), self.regular_tracers)
+        time.sleep(1)
 
         self.exec_out = open('%s/exec-out.txt' % self.working_dir, 'w')
         self.exec_err = open('%s/exec-err.txt' % self.working_dir, 'w')
@@ -217,3 +229,32 @@ class Experiment(object):
         self.log("Stopping regular tracers")
         map(methodcaller('stop_tracing'), self.regular_tracers)
 
+    def log(self, msg):
+        print("[Exp %s]: %s" % (self.name, msg))
+
+    def run_exp(self):
+        self.__to_linux()
+
+        succ = False
+        try:
+            self.__setup()
+
+            try:
+                self.__run_tasks()
+                self.log("Saving results in %s" % self.finished_dir)
+                succ = True
+            except Exception as e:
+                # Give time for whatever failed to finish failing
+                time.sleep(2)
+                self.__kill_all()
+
+                raise e
+            finally:
+                self.__teardown()
+        finally:
+            self.log("Switching back to Linux scheduler")
+            self.__to_linux()
+
+        if succ:
+            self.__save_results()
+            self.log("Experiment done!")
diff --git a/run/litmus_util.py b/run/litmus_util.py
index 70da262..81f690b 100644
--- a/run/litmus_util.py
+++ b/run/litmus_util.py
@@ -20,7 +20,7 @@ def switch_scheduler(switch_to_in):
     with open('/proc/litmus/active_plugin', 'w') as active_plugin:
         subprocess.Popen(["echo", switch_to], stdout=active_plugin)
 
-    # it takes a bit to do the switch, sleep an arbitrary amount of time
+    # It takes a bit to do the switch, sleep an arbitrary amount of time
     time.sleep(2)
 
     cur_plugin = scheduler()
@@ -29,24 +29,21 @@ def switch_scheduler(switch_to_in):
                         (switch_to, cur_plugin))
 
 def waiting_tasks():
-    reg = re.compile(r'^ready.*?(?P<READY>\d+)$', re.M)
+    reg = re.compile(r'^ready.*?(?P<WAITING>\d+)$', re.M)
     with open('/proc/litmus/stats', 'r') as f:
         data = f.read()
 
     # Ignore if no tasks are waiting for release
-    match = re.search(reg, data)
-    ready = match.group("READY")
+    waiting = re.search(reg, data).group("WAITING")
 
-    return 0 if not ready else int(ready)
+    return 0 if not waiting else int(waiting)
 
 def all_tasks():
     reg = re.compile(r'^real-time.*?(?P<TASKS>\d+)$', re.M)
     with open('/proc/litmus/stats', 'r') as f:
         data = f.read()
 
-    # Ignore if no tasks are waiting for release
-    match = re.search(reg, data)
-    ready = match.group("TASKS")
+    ready = re.search(reg, data).group("TASKS")
 
     return 0 if not ready else int(ready)
 
diff --git a/run/tracer.py b/run/tracer.py
index 6e1d05c..5e92a74 100644
--- a/run/tracer.py
+++ b/run/tracer.py
@@ -38,8 +38,8 @@ class LinuxTracer(Tracer):
         stdout = open('%s/trace-cmd-stdout.txt' % self.output_dir, 'w')
         stderr = open('%s/trace-cmd-stderr.txt' % self.output_dir, 'w')
 
-        execute = Executable(conf.BINS['trace-cmd'], extra_args, stdout, stderr)
-        execute.cwd = output_dir
+        execute = Executable(conf.BINS['trace-cmd'], extra_args,
+                             stdout, stderr, output_dir)
         self.bins.append(execute)
 
     @staticmethod
diff --git a/run_exps.py b/run_exps.py
index 6531415..a15018d 100755
--- a/run_exps.py
+++ b/run_exps.py
@@ -2,22 +2,23 @@
 from __future__ import print_function
 
 import common as com
-import config.config as conf
 import os
 import re
 import shutil
 import sys
 import run.tracer as trace
 
+from config.config import PARAMS,DEFAULTS
 from collections import namedtuple
 from optparse import OptionParser
 from run.executable.executable import Executable
-from run.experiment import Experiment,ExperimentDone,ExperimentFailed,SystemCorrupted
+from run.experiment import Experiment,ExperimentDone,SystemCorrupted
 from run.proc_entry import ProcEntry
 
 '''Customizable experiment parameters'''
 ExpParams = namedtuple('ExpParams', ['scheduler', 'duration', 'tracers',
-                                     'kernel', 'config_options'])
+                                     'kernel', 'config_options', 'file_params',
+                                     'pre_script', 'post_script'])
 '''Comparison of requested versus actual kernel compile parameter value'''
 ConfigResult = namedtuple('ConfigResult', ['param', 'wanted', 'actual'])
 
@@ -56,12 +57,13 @@ def parse_args():
     parser.add_option('-d', '--duration', dest='duration', type='int',
                       help='duration (seconds) of tasks')
     parser.add_option('-o', '--out-dir', dest='out_dir',
-                      help='directory for data output', default=("%s/run-data"%os.getcwd()))
+                      help='directory for data output',
+                      default=DEFAULTS['out-run'])
     parser.add_option('-p', '--params', dest='param_file',
                       help='file with experiment parameters')
     parser.add_option('-c', '--schedule-file', dest='sched_file',
                       help='name of schedule files within directories',
-                      default=conf.DEFAULTS['sched_file'])
+                      default=DEFAULTS['sched_file'])
     parser.add_option('-f', '--force', action='store_true', default=False,
                       dest='force', help='overwrite existing data')
     parser.add_option('-j', '--jabber', metavar='username@domain',
@@ -96,7 +98,7 @@ def convert_data(data):
             proc = (loc, match.group("CONTENT"))
             procs.append(proc)
         else:
-            prog = match.group("PROG") or conf.DEFAULTS['prog']
+            prog = match.group("PROG") or DEFAULTS['prog']
             spin = (prog, match.group("ARGS"))
             tasks.append(spin)
 
@@ -112,8 +114,8 @@ def fix_paths(schedule, exp_dir, sched_file):
                 args = args.replace(arg, abspath)
                 break
             elif re.match(r'.*\w+\.[a-zA-Z]\w*', arg):
-                print("WARNING: non-existent file '%s' may be referenced:\n\t%s"
-                      % (arg, sched_file))
+                sys.stderr.write("WARNING: non-existent file '%s' " % arg +
+                                 "may be referenced:\n\t%s" % sched_file)
 
         schedule['task'][idx] = (task, args)
 
@@ -181,25 +183,23 @@ def verify_environment(exp_params):
             raise InvalidConfig(results)
 
 
-def run_parameter(exp_dir, out_dir, params, param_name):
-    '''Run an executable (arguments optional) specified as a configurable
-    @param_name in @params.'''
-    if conf.PARAMS[param_name] not in params:
+def run_script(script_params, exp, exp_dir, out_dir):
+    '''Run an executable (arguments optional)'''
+    if not script_params:
         return
 
-    script_params = params[conf.PARAMS[param_name]]
-
     # Split into arguments and program name
     if type(script_params) != type([]):
         script_params = [script_params]
-    script_name = script_params.pop(0)
 
+    exp.log("Running %s" % script_params.join(" "))
+
+    script_name = script_params.pop(0)
     script = com.get_executable(script_name, cwd=exp_dir)
 
-    out  = open('%s/%s-out.txt' % (out_dir, param_name), 'w')
-    prog = Executable(script, script_params,
+    out  = open('%s/%s-out.txt' % (out_dir, script_name), 'w')
+    prog = Executable(script, script_params, cwd=out_dir,
                       stderr_file=out, stdout_file=out)
-    prog.cwd = out_dir
 
     prog.execute()
     prog.wait()
@@ -207,28 +207,41 @@ def run_parameter(exp_dir, out_dir, params, param_name):
     out.close()
 
 
-def get_exp_params(cmd_scheduler, cmd_duration, file_params):
+def make_exp_params(cmd_scheduler, cmd_duration, sched_dir, param_file):
     '''Return ExpParam with configured values of all hardcoded params.'''
     kernel = copts = ""
 
-    scheduler = cmd_scheduler or file_params[conf.PARAMS['sched']]
-    duration  = cmd_duration  or file_params[conf.PARAMS['dur']] or\
-                conf.DEFAULTS['duration']
+    # Load parameter file
+    param_file = param_file or "%s/%s" % (sched_dir, DEFAULTS['params_file'])
+    if os.path.isfile(param_file):
+        fparams = com.load_params(param_file)
+    else:
+        fparams = {}
+
+    scheduler = cmd_scheduler or fparams[PARAMS['sched']]
+    duration  = cmd_duration  or fparams[PARAMS['dur']] or\
+                DEFAULTS['duration']
 
     # Experiments can specify required kernel name
-    if conf.PARAMS['kernel'] in file_params:
-        kernel = file_params[conf.PARAMS['kernel']]
+    if PARAMS['kernel'] in fparams:
+        kernel = fparams[PARAMS['kernel']]
 
     # Or required config options
-    if conf.PARAMS['copts'] in file_params:
-        copts = file_params[conf.PARAMS['copts']]
+    if PARAMS['copts'] in fparams:
+        copts = fparams[PARAMS['copts']]
 
     # Or required tracers
     requested = []
-    if conf.PARAMS['trace'] in file_params:
-        requested = file_params[conf.PARAMS['trace']]
+    if PARAMS['trace'] in fparams:
+        requested = fparams[PARAMS['trace']]
     tracers = trace.get_tracer_types(requested)
 
+    # Or scripts to run before and after experiments
+    def get_script(name):
+        return fparams[name] if name in fparams else None
+    pre_script  = get_script('pre')
+    post_script = get_script('post')
+
     # But only these two are mandatory
     if not scheduler:
         raise IOError("No scheduler found in param file!")
@@ -236,61 +249,70 @@ def get_exp_params(cmd_scheduler, cmd_duration, file_params):
         raise IOError("No duration found in param file!")
 
     return ExpParams(scheduler=scheduler, kernel=kernel, duration=duration,
-                     config_options=copts, tracers=tracers)
+                     config_options=copts, tracers=tracers, file_params=fparams,
+                     pre_script=pre_script, post_script=post_script)
 
-
-def load_experiment(sched_file, cmd_scheduler, cmd_duration,
-                    param_file, out_dir, ignore, jabber):
+def run_experiment(name, sched_file, exp_params, out_dir,
+                   start_message, ignore, jabber):
     '''Load and parse data from files and run result.'''
     if not os.path.isfile(sched_file):
         raise IOError("Cannot find schedule file: %s" % sched_file)
 
     dir_name, fname = os.path.split(sched_file)
-    exp_name = os.path.split(dir_name)[1] + "/" + fname
     work_dir = "%s/tmp" % dir_name
 
-    # Load parameter file
-    param_file = param_file or \
-      "%s/%s" % (dir_name, conf.DEFAULTS['params_file'])
-    if os.path.isfile(param_file):
-        file_params = com.load_params(param_file)
-    else:
-        file_params = {}
-
-    # Create input needed by Experiment
-    exp_params = get_exp_params(cmd_scheduler, cmd_duration, file_params)
-    procs, execs = load_schedule(exp_name, sched_file, exp_params.duration)
+    procs, execs = load_schedule(name, sched_file, exp_params.duration)
 
-    exp = Experiment(exp_name, exp_params.scheduler, work_dir, out_dir,
+    exp = Experiment(name, exp_params.scheduler, work_dir, out_dir,
                      procs, execs, exp_params.tracers)
 
+    exp.log(start_message)
+
     if not ignore:
         verify_environment(exp_params)
 
-    run_parameter(dir_name, work_dir, file_params, 'pre')
+    run_script(exp_params.pre_script, exp, dir_name, work_dir)
 
     exp.run_exp()
 
-    run_parameter(dir_name, out_dir, file_params, 'post')
+    run_script(exp_params.post_script, exp, dir_name, out_dir)
 
     if jabber:
-        jabber.send("Completed '%s'" % exp_name)
+        jabber.send("Completed '%s'" % name)
 
     # Save parameters used to run experiment in out_dir
-    out_params = dict(file_params.items() +
-                      [(conf.PARAMS['sched'],  exp_params.scheduler),
-                       (conf.PARAMS['tasks'],  len(execs)),
-                       (conf.PARAMS['dur'],    exp_params.duration)])
+    out_params = dict(exp_params.file_params.items() +
+                      [(PARAMS['sched'],  exp_params.scheduler),
+                       (PARAMS['tasks'],  len(execs)),
+                       (PARAMS['dur'],    exp_params.duration)])
 
     # Feather-trace clock frequency saved for accurate overhead parsing
     ft_freq = com.ft_freq()
     if ft_freq:
-        out_params[conf.PARAMS['cycles']] = ft_freq
+        out_params[PARAMS['cycles']] = ft_freq
 
-    with open("%s/%s" % (out_dir, conf.DEFAULTS['params_file']), 'w') as f:
+    with open("%s/%s" % (out_dir, DEFAULTS['params_file']), 'w') as f:
         f.write(str(out_params))
 
 
+def get_exps(opts, args):
+    '''Return list of experiment files or directories'''
+    if args:
+        return args
+
+    # Default to sched_file > generated dirs
+    if os.path.exists(opts.sched_file):
+        sys.stderr.write("Reading schedule from %s.\n" % opts.sched_file)
+        return [opts.sched_file]
+    elif os.path.exists(DEFAULTS['out-gen']):
+        sys.stderr.write("Reading schedules from %s/*.\n" % DEFAULTS['out-gen'])
+        sched_dirs = os.listdir(DEFAULTS['out-gen'])
+        return ['%s/%s' % (DEFAULTS['out-gen'], d) for d in sched_dirs]
+    else:
+        sys.stderr.write("Run with -h to view options.\n");
+        sys.exit(1)
+
+
 def setup_jabber(target):
     try:
         from run.jabber import Jabber
@@ -301,6 +323,7 @@ def setup_jabber(target):
                          "Disabling instant messages.\n")
         return None
 
+
 def setup_email(target):
     try:
         from run.emailer import Emailer
@@ -314,71 +337,82 @@ def setup_email(target):
     sys.stderr.write(message + " Disabling email message.\n")
     return None
 
+
+def make_paths(exp, out_base_dir, opts):
+    '''Translate experiment name to (schedule file, output directory) paths'''
+    path = "%s/%s" % (os.getcwd(), exp)
+    out_dir = "%s/%s" % (out_base_dir, os.path.split(exp.strip('/'))[1])
+
+    if not os.path.exists(path):
+        raise IOError("Invalid experiment: %s" % path)
+
+    if opts.force and os.path.exists(out_dir):
+        shutil.rmtree(out_dir)
+
+    if os.path.isdir(path):
+        sched_file = "%s/%s" % (path, opts.sched_file)
+    else:
+        sched_file = path
+
+    return sched_file, out_dir
+
+
 def main():
     opts, args = parse_args()
 
-    scheduler  = opts.scheduler
-    duration   = opts.duration
-    param_file = opts.param_file
-    out_base   = os.path.abspath(opts.out_dir)
+    exps = get_exps(opts, args)
 
-    args = args or [opts.sched_file]
+    jabber = setup_jabber(opts.jabber) if opts.jabber else None
+    email  = setup_email(opts.email)   if opts.email  else None
 
-    created = False
+    out_base = os.path.abspath(opts.out_dir)
+    created  = False
     if not os.path.exists(out_base):
         created = True
         os.mkdir(out_base)
 
-    ran  = 0
-    done = 0
-    succ = 0
-    failed = 0
-    invalid = 0
-
-    jabber = setup_jabber(opts.jabber) if opts.jabber else None
-    email  = setup_email(opts.email) if opts.email else None
-
-    for exp in args:
-        path = "%s/%s" % (os.getcwd(), exp)
-        out_dir = "%s/%s" % (out_base, os.path.split(exp.strip('/'))[1])
+    ran = done = succ = failed = invalid = 0
 
-        if not os.path.exists(path):
-            raise IOError("Invalid experiment: %s" % path)
+    for i, exp in enumerate(exps):
+        sched_file, out_dir = make_paths(exp, out_base, opts)
+        sched_dir = os.path.split(sched_file)[0]
 
-        if opts.force and os.path.exists(out_dir):
-            shutil.rmtree(out_dir)
+        try:
+            start_message = "Loading experiment %d of %d." % (i+1, len(exps))
+            exp_params = make_exp_params(opts.scheduler, opts.duration,
+                                         sched_dir, opts.param_file)
 
-        if os.path.isdir(exp):
-            path = "%s/%s" % (path, opts.sched_file)
+            run_experiment(exp, sched_file, exp_params, out_dir,
+                           start_message, opts.ignore, jabber)
 
-        try:
-            load_experiment(path, scheduler, duration, param_file,
-                            out_dir, opts.ignore, jabber)
             succ += 1
         except ExperimentDone:
+            sys.stderr.write("Experiment '%s' already completed " % exp +
+                             "at '%s'\n" % out_base)
             done += 1
-            print("Experiment '%s' already completed at '%s'" % (exp, out_base))
         except (InvalidKernel, InvalidConfig) as e:
+            sys.stderr.write("Invalid environment for experiment '%s'\n" % exp)
+            sys.stderr.write("%s\n" % e)
             invalid += 1
-            print("Invalid environment for experiment '%s'" % exp)
-            print(e)
         except KeyboardInterrupt:
-            print("Keyboard interrupt, quitting")
+            sys.stderr.write("Keyboard interrupt, quitting\n")
             break
         except SystemCorrupted as e:
-            print("System is corrupted! Fix state before continuing.")
-            print(e)
+            sys.stderr.write("System is corrupted! Fix state before continuing.\n")
+            sys.stderr.write("%s\n" % e)
             break
-        except ExperimentFailed:
-            print("Failed experiment %s" % exp)
+        except Exception as e:
+            sys.stderr.write("Failed experiment %s\n" % exp)
+            sys.stderr.write("%s\n" % e)
             failed += 1
 
         ran += 1
 
+    # Clean out directory if it failed immediately
     if not os.listdir(out_base) and created and not succ:
         os.rmdir(out_base)
 
-    message = "Experiments ran:\t%d of %d" % (ran, len(args)) +\
+    message = "Experiments ran:\t%d of %d" % (ran, len(exps)) +\
       "\n  Successful:\t\t%d" % succ +\
       "\n  Failed:\t\t%d" % failed +\
       "\n  Already Done:\t\t%d" % done +\
@@ -386,6 +420,10 @@ def main():
 
     print(message)
 
+    if succ:
+        sys.stderr.write("Successful experiment data saved in %s.\n" %
+                         opts.out_dir)
+
     if email:
         email.send(message)
         email.close()