diff options
Diffstat (limited to 'Documentation/networking/s2io.txt')
-rw-r--r-- | Documentation/networking/s2io.txt | 199 |
1 files changed, 152 insertions, 47 deletions
diff --git a/Documentation/networking/s2io.txt b/Documentation/networking/s2io.txt index 6726b524ec45..bd528ffbeb4b 100644 --- a/Documentation/networking/s2io.txt +++ b/Documentation/networking/s2io.txt | |||
@@ -1,48 +1,153 @@ | |||
1 | S2IO Technologies XFrame 10 Gig adapter. | 1 | Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver. |
2 | ------------------------------------------- | 2 | |
3 | 3 | Contents | |
4 | I. Module loadable parameters. | 4 | ======= |
5 | When loaded as a module, the driver provides a host of Module loadable | 5 | - 1. Introduction |
6 | parameters, so the device can be tuned as per the users needs. | 6 | - 2. Identifying the adapter/interface |
7 | A list of the Module params is given below. | 7 | - 3. Features supported |
8 | (i) ring_num: This can be used to program the number of | 8 | - 4. Command line parameters |
9 | receive rings used in the driver. | 9 | - 5. Performance suggestions |
10 | (ii) ring_len: This defines the number of descriptors each ring | 10 | - 6. Available Downloads |
11 | can have. There can be a maximum of 8 rings. | 11 | |
12 | (iii) frame_len: This is an array of size 8. Using this we can | 12 | |
13 | set the maximum size of the received frame that can | 13 | 1. Introduction: |
14 | be steered into the corrsponding receive ring. | 14 | This Linux driver supports Neterion's Xframe I PCI-X 1.0 and |
15 | (iv) fifo_num: This defines the number of Tx FIFOs thats used in | 15 | Xframe II PCI-X 2.0 adapters. It supports several features |
16 | the driver. | 16 | such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. |
17 | (v) fifo_len: Each element defines the number of | 17 | See below for complete list of features. |
18 | Tx descriptors that can be associated with each | 18 | All features are supported for both IPv4 and IPv6. |
19 | corresponding FIFO. There are a maximum of 8 FIFOs. | 19 | |
20 | (vi) tx_prio: This is a bool, if module is loaded with a non-zero | 20 | 2. Identifying the adapter/interface: |
21 | value for tx_prio multi FIFO scheme is activated. | 21 | a. Insert the adapter(s) in your system. |
22 | (vii) rx_prio: This is a bool, if module is loaded with a non-zero | 22 | b. Build and load driver |
23 | value for tx_prio multi RING scheme is activated. | 23 | # insmod s2io.ko |
24 | (viii) latency_timer: The value given against this param will be | 24 | c. View log messages |
25 | loaded into the latency timer register in PCI Config | 25 | # dmesg | tail -40 |
26 | space, else the register is left with its reset value. | 26 | You will see messages similar to: |
27 | 27 | eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA | |
28 | II. Performance tuning. | 28 | eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA |
29 | By changing a few sysctl parameters. | 29 | eth4: Device is on 64 bit 133MHz PCIX(M1) bus |
30 | Copy the following lines into a file and run the following command, | 30 | |
31 | "sysctl -p <file_name>" | 31 | The above messages identify the adapter type(Xframe I/II), adapter revision, |
32 | ### IPV4 specific settings | 32 | driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). |
33 | net.ipv4.tcp_timestamps = 0 # turns TCP timestamp support off, default 1, reduces CPU use | 33 | In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed |
34 | net.ipv4.tcp_sack = 0 # turn SACK support off, default on | 34 | as well. |
35 | # on systems with a VERY fast bus -> memory interface this is the big gainer | 35 | |
36 | net.ipv4.tcp_rmem = 10000000 10000000 10000000 # sets min/default/max TCP read buffer, default 4096 87380 174760 | 36 | To associate an interface with a physical adapter use "ethtool -p <ethX>". |
37 | net.ipv4.tcp_wmem = 10000000 10000000 10000000 # sets min/pressure/max TCP write buffer, default 4096 16384 131072 | 37 | The corresponding adapter's LED will blink multiple times. |
38 | net.ipv4.tcp_mem = 10000000 10000000 10000000 # sets min/pressure/max TCP buffer space, default 31744 32256 32768 | 38 | |
39 | 39 | 3. Features supported: | |
40 | ### CORE settings (mostly for socket and UDP effect) | 40 | a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes, |
41 | net.core.rmem_max = 524287 # maximum receive socket buffer size, default 131071 | 41 | modifiable using ifconfig command. |
42 | net.core.wmem_max = 524287 # maximum send socket buffer size, default 131071 | 42 | |
43 | net.core.rmem_default = 524287 # default receive socket buffer size, default 65535 | 43 | b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit |
44 | net.core.wmem_default = 524287 # default send socket buffer size, default 65535 | 44 | and receive, TSO. |
45 | net.core.optmem_max = 524287 # maximum amount of option memory buffers, default 10240 | 45 | |
46 | net.core.netdev_max_backlog = 300000 # number of unprocessed input packets before kernel starts dropping them, default 300 | 46 | c. Multi-buffer receive mode. Scattering of packet across multiple |
47 | ---End of performance tuning file--- | 47 | buffers. Currently driver supports 2-buffer mode which yields |
48 | significant performance improvement on certain platforms(SGI Altix, | ||
49 | IBM xSeries). | ||
50 | |||
51 | d. MSI/MSI-X. Can be enabled on platforms which support this feature | ||
52 | (IA64, Xeon) resulting in noticeable performance improvement(upto 7% | ||
53 | on certain platforms). | ||
54 | |||
55 | e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt | ||
56 | moderation. | ||
57 | |||
58 | f. Statistics. Comprehensive MAC-level and software statistics displayed | ||
59 | using "ethtool -S" option. | ||
60 | |||
61 | g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, | ||
62 | with multiple steering options. | ||
63 | |||
64 | 4. Command line parameters | ||
65 | a. tx_fifo_num | ||
66 | Number of transmit queues | ||
67 | Valid range: 1-8 | ||
68 | Default: 1 | ||
69 | |||
70 | b. rx_ring_num | ||
71 | Number of receive rings | ||
72 | Valid range: 1-8 | ||
73 | Default: 1 | ||
74 | |||
75 | c. tx_fifo_len | ||
76 | Size of each transmit queue | ||
77 | Valid range: Total length of all queues should not exceed 8192 | ||
78 | Default: 4096 | ||
79 | |||
80 | d. rx_ring_sz | ||
81 | Size of each receive ring(in 4K blocks) | ||
82 | Valid range: Limited by memory on system | ||
83 | Default: 30 | ||
84 | |||
85 | e. intr_type | ||
86 | Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X) | ||
87 | Valid range: 1-3 | ||
88 | Default: 1 | ||
89 | |||
90 | 5. Performance suggestions | ||
91 | General: | ||
92 | a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) | ||
93 | b. Set TCP windows size to optimal value. | ||
94 | For instance, for MTU=1500 a value of 210K has been observed to result in | ||
95 | good performance. | ||
96 | # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000" | ||
97 | # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000" | ||
98 | For MTU=9000, TCP window size of 10 MB is recommended. | ||
99 | # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000" | ||
100 | # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000" | ||
101 | |||
102 | Transmit performance: | ||
103 | a. By default, the driver respects BIOS settings for PCI bus parameters. | ||
104 | However, you may want to experiment with PCI bus parameters | ||
105 | max-split-transactions(MOST) and MMRBC (use setpci command). | ||
106 | A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. | ||
107 | It could be different for your hardware. | ||
108 | Set MMRBC to 4K**. | ||
109 | |||
110 | For example you can set | ||
111 | For opteron | ||
112 | #setpci -d 17d5:* 62=1d | ||
113 | For Itanium | ||
114 | #setpci -d 17d5:* 62=3d | ||
115 | |||
116 | For detailed description of the PCI registers, please see Xframe User Guide. | ||
117 | |||
118 | b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this | ||
119 | parameter. | ||
120 | c. Turn on TSO(using "ethtool -K") | ||
121 | # ethtool -K <ethX> tso on | ||
122 | |||
123 | Receive performance: | ||
124 | a. By default, the driver respects BIOS settings for PCI bus parameters. | ||
125 | However, you may want to set PCI latency timer to 248. | ||
126 | #setpci -d 17d5:* LATENCY_TIMER=f8 | ||
127 | For detailed description of the PCI registers, please see Xframe User Guide. | ||
128 | b. Use 2-buffer mode. This results in large performance boost on | ||
129 | on certain platforms(eg. SGI Altix, IBM xSeries). | ||
130 | c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to | ||
131 | set/verify this option. | ||
132 | d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network | ||
133 | device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to | ||
134 | bring down CPU utilization. | ||
135 | |||
136 | ** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are | ||
137 | recommended as safe parameters. | ||
138 | For more information, please review the AMD8131 errata at | ||
139 | http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf | ||
140 | |||
141 | 6. Available Downloads | ||
142 | Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up | ||
143 | to date, also the latest "s2io" code (including support for 2.4 kernels) is | ||
144 | available via "Support" link on the Neterion site: http://www.neterion.com. | ||
145 | |||
146 | For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com, | ||
147 | user: linuxdocs password: HALdocs | ||
148 | |||
149 | 7. Support | ||
150 | For further support please contact either your 10GbE Xframe NIC vendor (IBM, | ||
151 | HP, SGI etc.) or click on the "Support" link on the Neterion site: | ||
152 | http://www.neterion.com. | ||
48 | 153 | ||