aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'README.md')
-rw-r--r--README.md80
1 files changed, 80 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..705f2b6
--- /dev/null
+++ b/README.md
@@ -0,0 +1,80 @@
1# libsmctrl: Quick & Easy Hardware Compute Partitioning on NVIDIA GPUs
2
3This library was developed as part of the following paper:
4
5_J. Bakita and J. H. Anderson, "Hardware Compute Partitioning on NVIDIA GPUs", Proceedings of the 29th IEEE Real-Time and Embedded Technology and Applications Symposium, pp. 54-66, May 2023._
6
7Please cite this paper in any work which leverages our library. Here's the BibTeX entry:
8```
9@inproceedings{bakita2023hardware,
10 title={Hardware Compute Partitioning on {NVIDIA} {GPUs}},
11 author={Bakita, Joshua and Anderson, James H},
12 booktitle={Proceedings of the 29th IEEE Real-Time and Embedded Technology and Applications Symposium},
13 year={2023},
14 month={May},
15 pages={54--66},
16 _series={RTAS}
17}
18```
19
20Please see [the paper](https://www.cs.unc.edu/~jbakita/rtas23.pdf) and libsmctrl.h for details and examples of how to use this library.
21We strongly encourage consulting those resources first; the below comments serve merely as an appendum.
22
23## Run Tests
24To test partitioning:
25```
26make tests
27./libsmctrl_test_global_mask
28./libsmctrl_test_stream_mask
29./libsmctrl_test_next_mask
30```
31
32And if `nvdebug` has been installed:
33```
34./libsmctrl_test_gpu_info
35```
36
37## Supported GPUs
38
39#### Known Working
40
41- NVIDIA GPUs from compute capability 3.5 through 8.9, including embedded "Jetson" GPUs
42- CUDA 8.1 through 12.2
43- `x86_64` and Jetson `aarch64` platforms
44
45#### Known Issues
46
47- `next_mask` will not override `stream_mask` on CUDA 12.0+
48- `global_mask` and `next_mask` cannot disable TPCs with IDs above 128
49 - Only relevant on GPUs with over 128 TPCs, such as the RTX 6000 Ada
50- Untested on H100 (compute capability 9.0)
51- Untested on non-Jetson `aarch64` platforms
52
53## Important Limitations
54
551. Only supports partitioning _within_ a single GPU context.
56 At time of writing, it is challenging to impossible to share a GPU context across multiple CPU address spaces.
57 The implication is that your applications must first be combined together into a single CPU process.
582. No aspect of this system prevents implicit synchronization on the GPU.
59 See prior work, particularly that of Amert et al. (perhaps the CUPiD^RT paper), for ways to avoid this.
60
61## Porting to New Architectures
62
63Build the tests with `make tests`. And then run the following:
64```
65for (( i=0; $?!=0; i+=8 )); do MASK_OFF=$i ./libsmctrl_test_stream_mask; done
66```
67
68How this works:
69
701. If `MASK_OFF` is set, `libsmctrl` applies this as a byte offset to a base address for the location
71 of the SM mask fields in CUDA's stream data structure.
72 - That base address is the one for CUDA 12.2 at time of writing
732. The stream masking test is run.
743. If the test succeeded (returned zero) the loop aborts, otherwise it increments the offset to attempt and repeats.
75
76Once this loop aborts, take the found offset and add it into the switch statement for the appropriate CUDA version and CPU architecture.
77
78## TODO
79
80- Add a test to check that more-granularly-set compute masks override more-corsely-set ones.