<feed xmlns='http://www.w3.org/2005/Atom'>
<title>experiment-scripts.git/parse_exps.py, branch wip-ecrts14-pgm</title>
<subtitle>Helper scripts to make LITMUS^RT experiments easy!</subtitle>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/'/>
<entry>
<title>Added --collapse option to parse_exps.py for creating simpler graphs.</title>
<updated>2013-05-02T19:57:36+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-05-02T19:49:11+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=38e4029048bb693525369ea7eb0e639f73844ed2'/>
<id>38e4029048bb693525369ea7eb0e639f73844ed2</id>
<content type='text'>
This creates csvs which have merged the lines which only differ by variables
whose values are numbers (e.g. 'wss', 'number of tasks', or 'duration',
but not 'scheduler').

For example, consider a csv which is tracking the change in working set sizes.
There are 3 different experiment configurations per WSS, one with 12 tasks,
one for 14, and another for 16. Without --collapse, a seperate csv
(and line to plot in plot_exps.py) will be created for each of the 12, 14, and
16 configurations. With --collapse, a single csv (and a single line plotted
by plot_exps.py) will be created which averages the values of 12, 14, and 16
tasks for each WSS.

This is very useful for noticing trends or presenting overhead bar charts
or other examples in papers.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This creates csvs which have merged the lines which only differ by variables
whose values are numbers (e.g. 'wss', 'number of tasks', or 'duration',
but not 'scheduler').

For example, consider a csv which is tracking the change in working set sizes.
There are 3 different experiment configurations per WSS, one with 12 tasks,
one for 14, and another for 16. Without --collapse, a seperate csv
(and line to plot in plot_exps.py) will be created for each of the 12, 14, and
16 configurations. With --collapse, a single csv (and a single line plotted
by plot_exps.py) will be created which averages the values of 12, 14, and 16
tasks for each WSS.

This is very useful for noticing trends or presenting overhead bar charts
or other examples in papers.
</pre>
</div>
</content>
</entry>
<entry>
<title>Cleaned up run_exps.py parameters.</title>
<updated>2013-04-30T15:44:37+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-04-30T15:44:37+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=94cc65997d237ddeab24d396f06bb93bc0644a9d'/>
<id>94cc65997d237ddeab24d396f06bb93bc0644a9d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Improved accuracy of sched_trace measurement parsing.</title>
<updated>2013-04-23T18:01:35+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-04-23T18:01:35+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=7545402506aa76261e18d85af585ff0ac1cf05c1'/>
<id>7545402506aa76261e18d85af585ff0ac1cf05c1</id>
<content type='text'>
* Measurements from tasks missing &gt; 20% of their scheduling records are
ignored. This is configurable in config/config.py.
* Measurements which only have zero values are ignored.
* If either of these 2 situations are encountered print out a message the
first time using the common.log_once() method. See parse_exps.py for how
this is used with multiple threads.
* Measurements from a task's last job are ignored.
* Miss ratio is calculated only as a fraction of the number of jobs whose
matching release and completion records were found, not just release.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
* Measurements from tasks missing &gt; 20% of their scheduling records are
ignored. This is configurable in config/config.py.
* Measurements which only have zero values are ignored.
* If either of these 2 situations are encountered print out a message the
first time using the common.log_once() method. See parse_exps.py for how
this is used with multiple threads.
* Measurements from a task's last job are ignored.
* Miss ratio is calculated only as a fraction of the number of jobs whose
matching release and completion records were found, not just release.
</pre>
</div>
</content>
</entry>
<entry>
<title>Improved robustness of run_exps.py execution.</title>
<updated>2013-04-22T19:32:12+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-04-22T19:32:12+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=25ccdb0cbc6b959b1f96c89b8bce91963cb67b4c'/>
<id>25ccdb0cbc6b959b1f96c89b8bce91963cb67b4c</id>
<content type='text'>
Thanks to bcw and gelliott for debugging and ideas.

* Print out experiment number and total experiments when starting experiments.
* Only sleep and re-release tasks if tasks are waiting to release.
* Fail experiment with verbose messages if any tasks fail before becoming ready
to release.
* When waiting for tasks to become ready for release, reset the waiting time
whenever a new task (or task(s)) become ready.
* Start regular tracers BEFORE the plugin switch to log data from the switch.
* Check the number of running tasks AFTER trying to switch the linux scheduler.
This gives plugin deactivate code the opportunity to kill these tasks.
* If an invalid executable is specified in the schedule file, fail before
attempting to run the experiment and print out the problem.
* Propogate exceptions up from experiment failures instead of creating
ExperimentFailed exceptions.

This commit also made clock-frequency automatically ignored by parse_exps.py.
The value of this would change by +- a Mhz between experiments, ruining graphs.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Thanks to bcw and gelliott for debugging and ideas.

* Print out experiment number and total experiments when starting experiments.
* Only sleep and re-release tasks if tasks are waiting to release.
* Fail experiment with verbose messages if any tasks fail before becoming ready
to release.
* When waiting for tasks to become ready for release, reset the waiting time
whenever a new task (or task(s)) become ready.
* Start regular tracers BEFORE the plugin switch to log data from the switch.
* Check the number of running tasks AFTER trying to switch the linux scheduler.
This gives plugin deactivate code the opportunity to kill these tasks.
* If an invalid executable is specified in the schedule file, fail before
attempting to run the experiment and print out the problem.
* Propogate exceptions up from experiment failures instead of creating
ExperimentFailed exceptions.

This commit also made clock-frequency automatically ignored by parse_exps.py.
The value of this would change by +- a Mhz between experiments, ruining graphs.
</pre>
</div>
</content>
</entry>
<entry>
<title>Scripts read directories created by other scripts if no arguments.</title>
<updated>2013-04-21T17:28:38+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-04-21T17:28:38+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=fbd1df6f63eb551b99f71330d2370c570ff323f5'/>
<id>fbd1df6f63eb551b99f71330d2370c570ff323f5</id>
<content type='text'>
With no arguments, all scripts first try to load the current directory. If the
current directory has no data, the scripts search for the output of the previous
scripts in the toolchain, e.g. parse_exps.py loads run-data/*, created by
run_exps.py.

This commit also switched messages to stderr where they belong, and adds in
missing lock and unlock overheads.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
With no arguments, all scripts first try to load the current directory. If the
current directory has no data, the scripts search for the output of the previous
scripts in the toolchain, e.g. parse_exps.py loads run-data/*, created by
run_exps.py.

This commit also switched messages to stderr where they belong, and adds in
missing lock and unlock overheads.
</pre>
</div>
</content>
</entry>
<entry>
<title>Improved error handling in parse_ and plot_exps.py.</title>
<updated>2013-04-12T19:12:22+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-04-12T19:12:22+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=7eb34b5312974f601d1117eeaf6393b9648be31c'/>
<id>7eb34b5312974f601d1117eeaf6393b9648be31c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Minor bug fixes from end-to-end testing.</title>
<updated>2013-04-02T14:38:36+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-04-02T14:36:36+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=0d8c9ba50f48f3c2afb571a878117eb2aa50a969'/>
<id>0d8c9ba50f48f3c2afb571a878117eb2aa50a969</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Seperate directory output in plot_exps.py.</title>
<updated>2013-03-18T21:27:42+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-03-18T21:27:42+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=5f31b2e274399cd18ac96e887d9672ce221469d0'/>
<id>5f31b2e274399cd18ac96e887d9672ce221469d0</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Remove unused 'clean' option.</title>
<updated>2013-03-18T17:20:26+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-03-18T17:20:26+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=0b49f91cd3f0e35acc6e3eeebf11f8d69aac834f'/>
<id>0b49f91cd3f0e35acc6e3eeebf11f8d69aac834f</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Use smarter defaults which can handle data from Bjorns scripts.</title>
<updated>2013-03-18T17:15:42+00:00</updated>
<author>
<name>Jonathan Herman</name>
<email>hermanjl@cs.unc.edu</email>
</author>
<published>2013-03-18T17:15:42+00:00</published>
<link rel='alternate' type='text/html' href='http://rtsrv.cs.unc.edu/cgit/cgit.cgi/experiment-scripts.git/commit/?id=16dfc8df20c6befeed423217a2e0f5ae59b5a04d'/>
<id>16dfc8df20c6befeed423217a2e0f5ae59b5a04d</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
