1 files changed, 76 insertions, 35 deletions
diff --git a/doc/index.txt b/doc/index.txt
index 7ac39b1..78551c9 100644
--- a/doc/index.txt
+++ b/doc/index.txt
@@ -16,32 +16,26 @@ It is the complete documentation for Unit-Trace.
 ## Architecture ##
 Before trying to use Unit-Trace, it will help to understand the architecture of Unit-Trace.
-The `unit_trace` Python module is installed in the Python `site_packages` directory, so that it can be
+Unit-Trace performs various options on **trace files**. Oftentimes, when scheduler tracing takes
-imported from and Python script on the system.
+place, multiple trace files are generated for each experiment (e.g. one per CPU). We call a
-Thus, work with Unit-Trace is done by writing a short frontend script that makes use of the submodules
+related group of trace files a **trace**.
-you need for the task at hand.
+The user interacts with the tool using `unit-trace`, a Python script which is to be installed
-The `unit_trace` module consists of a number of submodules used to work with traces.
+on the local executable path.
-(Here, a `trace` is a record of scheduling events, produced by a scheduler, and may be contained in one or more `trace files` (for example,
+`unit-trace` invokes operations provided by the `unit_trace` Python module, which is installed
-one per CPU).
+in the local `site_packages` directory (the default install location for Python modules).
-In a typical Unit-Trace script, a `parser` submodule produces a **stream** of **records**, which are then piped to
+The `unit_trace` module provides submodule(s) for parsing trace files into a **stream** of Python objects (which we call **records**).
-subsequent submodules, and finally piped to one or more modules that produce output.
+This stream is then passed from one `unit_trace` submodule to the next (like a pipe), undergoing
+various transformations in the process, and ultimately arriving at one or more output submodules.
-For example, a user might write a script to invoke a parser on a set of trace files; pipe the resulting stream of event
+Intermediate modules generally add records; for example, the `gedf_test` module adds records to indicate
-records to the global EDF testing submodule, which adds error errors to the stream; pipe the stream to a submodule that computes statistics (for example,
+priority inversions, which can be treated appropriately by output submodules.
-the 10 lengthiest priority inversions) and produces relevant records; and finally, pipe the stream to a submodule that outputs all scheduling events, errors, and the
-statistical information that was computed previously.
+This architecture provides a clean and easy-to-use interface for both users and contributors.
-Seeing errors, the user may then wish to generate a visualization for a given time interval.
+The stream is implemented using Python iterators.
-(All the functionality described in this scenario is available in the current version of Unit-Trace.)
+This allows records are evaluated lazily (i.e. on an as-needed basis), making it possible to pull into memory only
+the records that are needed, and only as long as they are needed.
-This architecture provides a clean and flexible interface for working with scheduler traces. Because Python iterators are "lazy," producing items
+This is important for dealing with very large traces.
-(in this case, various records) only when necessary, this architecture avoids requiring that all trace information be read into memory at one time.
-We provide a several frontend scripts for common tasks, but encourage users to customize these to better fix their specific needs. 
-We also provide useful submodules, but expect that users will need new submodules that do not yet exist 
-(for example, not PFAIR testing submodules exists). 
-We hope users will contribute any useful code that they produce back to the project.
 ## Obtaining Unit-Trace ##
 Members of UNC's Real-Time Group can obtain Unit-Trace using:  
@@ -50,19 +44,66 @@ Members of UNC's Real-Time Group can obtain Unit-Trace using:
 ## Installing Unit-Trace ##
 Unit-Trace is based on Python 2.6, so make sure that is available on your system.
-Unit-Trace is installed by copying the unit_trace folder (a Python module) to the system's Python 2.6 `site-packages` directory, usually located at
+Unit-Trace can be installed manually by copying the `unit-trace` script and `unit_trace` Python module, as described previously.
-`/usr/lib/python2.6/site-packages/`.
+Alternatively, you can use `sudo install.py` to (re)install Unit-Trace.
-Frontend scripts (i.e., scripts that `import unit_trace`) can then be used anywhere on the system.
-The Unit-Trace code includes `install.py`, which automates (re)installation when called with `sudo`.
 ## Using Unit-Trace ##
-Example frontend scripts are included in the `scripts` folder, and can be used as-is for many tasks.
+Type `unit-trace` (without options) to view usage information.
- gedf_test.py reads trace files passed as command-line arguments and prints out all scheduling events, priority inversions, and some statistics
+In summary, trace files must be specified.
- visualize.py reads traces files passed as command-line arguments and draws the corresponding schedule.
+Flags are used to enable any desired submodules.
+Some flags have accompanying parameter(s) that are passed to the submodule.
+The order in which submodules process records is pre-determined by the `unit-trace` script,
+so the user can specify flags on the command line in any order.
+## Example/Use Case ##
+Suppose that Alice wants to perform G-EDF testing on the LITMUS<sup>RT</sup> traces included in the sample_traces/ folder.
+The LITMUS<sup>RT</sup> tracing mechanism outputs superfluous events at the beginning of the trace that will not appear "correct" for
+G-EDF testing (for example, multiple job releases that never complete, indicating the initialization of a task).
+Alice uses the following command to print out (-o) the first 50 records (-m 50), looking for the end of the bogus records:  
+<codeblock>unit-trace -m 50 -o *.bin</codeblock>.
+Seeing that she hasn't yet reached useful records, she uses the following command to print out (-o) 50 records (-m 50), skipping the
+first 50 (-s 50).  
+<codeblock>unit-trace -m 50 -s 50 -o *.bin</codeblock>.  
+She is able to see that meaningful releases begin at time `37917282934190`.
+She now commences G-EDF testing (-g), starting at the time of interest (-e <earliest time>).
+Because of the lengthy output to be expected, she redirects to standard out.
+She also uses the -c option to clean up additional records that are known to be erroneous, and
+will break the G-EDF tester.  
+<codeblock>unit-trace -c -e 37917282934190 -g -o *.bin > output</codeblock>.
+OK, everything worked. Alice can now grep through the output file and see priority inversions.
+She sees a particularly long priority inversion, and decides to generate a visualization (-v) of part of the schedule.  
+<codeblock>unit-trace -c -e 37918340000000 -l 37919000000000 -v *.bin</codeblock>.  
+(NOTE: Currently, this still shows the entire schedule, which likely won't be feasible for larger traces and is a bug.)
 ## Submodules ##
-TODO: All submodules will be documented thoroughly here.
+### Input Submodules ###
+<table border=1>
+<tr><td>Name</td><td>Flag</td><td>Options</td><td>Description</td></tr>
+<tr><td>trace_parser</td><td>(on by default)</td><td>(None)</td><td>Parses LITMUS<sup>RT</sup> traces</td></tr>
+</table>
+### Intermediate Submodules ###
+<table border=1>
+<tr><td>Name</td><td>Flag</td><td>Options</td><td>Description</td></tr>
+<tr><td>earliest</td><td>-e</td><td>time</td><td>Filters out records before the given time</td></tr>
+<tr><td>latest</td><td>-l</td><td>time</td><td>Filters out records after the given time</td></tr>
+<tr><td>skipper</td><td>-s</td><td>number n</td><td>Skips the first n records</td></tr>
+<tr><td>maxer</td><td>-m</td><td>number n</td><td>Allows at most n records to be parsed</td></tr>
+<tr><td>sanitizer</td><td>-c</td><td>(None)</td><td>Cleans up LITMUS<sup>RT</sup> traces for G-EDF testing.</td></tr>
+<tr><td>progress</td><td>-p</td><td>(None)</td><td>Outputs progress info (e.g number of records parsed so far, total time to process trace) to std error.</td></tr>
+<tr><td>stats</td><td>-i</td><td>(None)</td><td>Outputs statistics about G-EDF inversions. To be deprecated (incorporated into G-EDF tester).</td></tr>
+<tr><td>gedf_test</td><td>-g</td><td>(None)</td><td>Performs G-EDF testing.</td></tr>
+</table>
+### Output Submodules ###
+<table border=1>
+<tr><td>Name</td><td>Flag</td><td>Options</td><td>Description</td></tr>
+<tr><td>stdout_printer</td><td>-o</td><td>(None)</td><td>Prints records to standard out</td></tr>
+<tr><td>visualizer</td><td>-v</td><td>(None)</td><td>Visualizes records</td></tr>
+</table>
 ## Development ##
 TODO: Information on how to contribute will be documented thoroughly here.

diff --git a/doc/index.txt b/doc/index.txt index 7ac39b1..78551c9 100644 --- a/doc/index.txt +++ b/doc/index.txt
@@ -16,32 +16,26 @@ It is the complete documentation for Unit-Trace.
16	## Architecture ##	16	## Architecture ##
17	Before trying to use Unit-Trace, it will help to understand the architecture of Unit-Trace.	17	Before trying to use Unit-Trace, it will help to understand the architecture of Unit-Trace.
18		18
19	The `unit_trace` Python module is installed in the Python `site_packages` directory, so that it can be	19	Unit-Trace performs various options on trace files. Oftentimes, when scheduler tracing takes
20	imported from and Python script on the system.	20	place, multiple trace files are generated for each experiment (e.g. one per CPU). We call a
21	Thus, work with Unit-Trace is done by writing a short frontend script that makes use of the submodules	21	related group of trace files a trace.
22	you need for the task at hand.	22
23		23	The user interacts with the tool using `unit-trace`, a Python script which is to be installed
24	The `unit_trace` module consists of a number of submodules used to work with traces.	24	on the local executable path.
25	(Here, a `trace` is a record of scheduling events, produced by a scheduler, and may be contained in one or more `trace files` (for example,	25	`unit-trace` invokes operations provided by the `unit_trace` Python module, which is installed
26	one per CPU).	26	in the local `site_packages` directory (the default install location for Python modules).
27		27
28	In a typical Unit-Trace script, a `parser` submodule produces a stream of records, which are then piped to	28	The `unit_trace` module provides submodule(s) for parsing trace files into a stream of Python objects (which we call records).
29	subsequent submodules, and finally piped to one or more modules that produce output.	29	This stream is then passed from one `unit_trace` submodule to the next (like a pipe), undergoing
30		30	various transformations in the process, and ultimately arriving at one or more output submodules.
31	For example, a user might write a script to invoke a parser on a set of trace files; pipe the resulting stream of event	31	Intermediate modules generally add records; for example, the `gedf_test` module adds records to indicate
32	records to the global EDF testing submodule, which adds error errors to the stream; pipe the stream to a submodule that computes statistics (for example,	32	priority inversions, which can be treated appropriately by output submodules.
33	the 10 lengthiest priority inversions) and produces relevant records; and finally, pipe the stream to a submodule that outputs all scheduling events, errors, and the	33
34	statistical information that was computed previously.	34	This architecture provides a clean and easy-to-use interface for both users and contributors.
35	Seeing errors, the user may then wish to generate a visualization for a given time interval.	35	The stream is implemented using Python iterators.
36	(All the functionality described in this scenario is available in the current version of Unit-Trace.)	36	This allows records are evaluated lazily (i.e. on an as-needed basis), making it possible to pull into memory only
37		37	the records that are needed, and only as long as they are needed.
38	This architecture provides a clean and flexible interface for working with scheduler traces. Because Python iterators are "lazy," producing items	38	This is important for dealing with very large traces.
39	(in this case, various records) only when necessary, this architecture avoids requiring that all trace information be read into memory at one time.
40
41	We provide a several frontend scripts for common tasks, but encourage users to customize these to better fix their specific needs.
42	We also provide useful submodules, but expect that users will need new submodules that do not yet exist
43	(for example, not PFAIR testing submodules exists).
44	We hope users will contribute any useful code that they produce back to the project.
45		39
46	## Obtaining Unit-Trace ##	40	## Obtaining Unit-Trace ##
47	Members of UNC's Real-Time Group can obtain Unit-Trace using:	41	Members of UNC's Real-Time Group can obtain Unit-Trace using:
@@ -50,19 +44,66 @@ Members of UNC's Real-Time Group can obtain Unit-Trace using:
50	## Installing Unit-Trace ##	44	## Installing Unit-Trace ##
51	Unit-Trace is based on Python 2.6, so make sure that is available on your system.	45	Unit-Trace is based on Python 2.6, so make sure that is available on your system.
52		46
53	Unit-Trace is installed by copying the unit_trace folder (a Python module) to the system's Python 2.6 `site-packages` directory, usually located at	47	Unit-Trace can be installed manually by copying the `unit-trace` script and `unit_trace` Python module, as described previously.
54	`/usr/lib/python2.6/site-packages/`.	48	Alternatively, you can use `sudo install.py` to (re)install Unit-Trace.
55	Frontend scripts (i.e., scripts that `import unit_trace`) can then be used anywhere on the system.
56	The Unit-Trace code includes `install.py`, which automates (re)installation when called with `sudo`.
57		49
58	## Using Unit-Trace ##	50	## Using Unit-Trace ##
59	Example frontend scripts are included in the `scripts` folder, and can be used as-is for many tasks.	51	Type `unit-trace` (without options) to view usage information.
60		52
61	- gedf_test.py reads trace files passed as command-line arguments and prints out all scheduling events, priority inversions, and some statistics	53	In summary, trace files must be specified.
62	- visualize.py reads traces files passed as command-line arguments and draws the corresponding schedule.	54	Flags are used to enable any desired submodules.
		55	Some flags have accompanying parameter(s) that are passed to the submodule.
		56	The order in which submodules process records is pre-determined by the `unit-trace` script,
		57	so the user can specify flags on the command line in any order.
		58
		59	## Example/Use Case ##
		60	Suppose that Alice wants to perform G-EDF testing on the LITMUS<sup>RT</sup> traces included in the sample_traces/ folder.
		61
		62	The LITMUS<sup>RT</sup> tracing mechanism outputs superfluous events at the beginning of the trace that will not appear "correct" for
		63	G-EDF testing (for example, multiple job releases that never complete, indicating the initialization of a task).
		64	Alice uses the following command to print out (-o) the first 50 records (-m 50), looking for the end of the bogus records:
		65	<codeblock>unit-trace -m 50 -o *.bin</codeblock>.
		66
		67	Seeing that she hasn't yet reached useful records, she uses the following command to print out (-o) 50 records (-m 50), skipping the
		68	first 50 (-s 50).
		69	<codeblock>unit-trace -m 50 -s 50 -o *.bin</codeblock>.
		70	She is able to see that meaningful releases begin at time `37917282934190`.
		71
		72	She now commences G-EDF testing (-g), starting at the time of interest (-e <earliest time>).
		73	Because of the lengthy output to be expected, she redirects to standard out.
		74	She also uses the -c option to clean up additional records that are known to be erroneous, and
		75	will break the G-EDF tester.
		76	<codeblock>unit-trace -c -e 37917282934190 -g -o *.bin > output</codeblock>.
		77
		78	OK, everything worked. Alice can now grep through the output file and see priority inversions.
		79	She sees a particularly long priority inversion, and decides to generate a visualization (-v) of part of the schedule.
		80	<codeblock>unit-trace -c -e 37918340000000 -l 37919000000000 -v *.bin</codeblock>.
		81	(NOTE: Currently, this still shows the entire schedule, which likely won't be feasible for larger traces and is a bug.)
63		82
64	## Submodules ##	83	## Submodules ##
65	TODO: All submodules will be documented thoroughly here.	84	### Input Submodules ###
		85	<table border=1>
		86	<tr><td>Name</td><td>Flag</td><td>Options</td><td>Description</td></tr>
		87	<tr><td>trace_parser</td><td>(on by default)</td><td>(None)</td><td>Parses LITMUS<sup>RT</sup> traces</td></tr>
		88	</table>
		89	### Intermediate Submodules ###
		90	<table border=1>
		91	<tr><td>Name</td><td>Flag</td><td>Options</td><td>Description</td></tr>
		92	<tr><td>earliest</td><td>-e</td><td>time</td><td>Filters out records before the given time</td></tr>
		93	<tr><td>latest</td><td>-l</td><td>time</td><td>Filters out records after the given time</td></tr>
		94	<tr><td>skipper</td><td>-s</td><td>number n</td><td>Skips the first n records</td></tr>
		95	<tr><td>maxer</td><td>-m</td><td>number n</td><td>Allows at most n records to be parsed</td></tr>
		96	<tr><td>sanitizer</td><td>-c</td><td>(None)</td><td>Cleans up LITMUS<sup>RT</sup> traces for G-EDF testing.</td></tr>
		97	<tr><td>progress</td><td>-p</td><td>(None)</td><td>Outputs progress info (e.g number of records parsed so far, total time to process trace) to std error.</td></tr>
		98	<tr><td>stats</td><td>-i</td><td>(None)</td><td>Outputs statistics about G-EDF inversions. To be deprecated (incorporated into G-EDF tester).</td></tr>
		99	<tr><td>gedf_test</td><td>-g</td><td>(None)</td><td>Performs G-EDF testing.</td></tr>
		100	</table>
		101	### Output Submodules ###
		102	<table border=1>
		103	<tr><td>Name</td><td>Flag</td><td>Options</td><td>Description</td></tr>
		104	<tr><td>stdout_printer</td><td>-o</td><td>(None)</td><td>Prints records to standard out</td></tr>
		105	<tr><td>visualizer</td><td>-v</td><td>(None)</td><td>Visualizes records</td></tr>
		106	</table>
66		107
67	## Development ##	108	## Development ##
68	TODO: Information on how to contribute will be documented thoroughly here.	109	TODO: Information on how to contribute will be documented thoroughly here.