diff options
| -rw-r--r-- | Documentation/networking/s2io.txt | 199 |
1 files changed, 152 insertions, 47 deletions
diff --git a/Documentation/networking/s2io.txt b/Documentation/networking/s2io.txt index 6726b524ec45..bd528ffbeb4b 100644 --- a/Documentation/networking/s2io.txt +++ b/Documentation/networking/s2io.txt | |||
| @@ -1,48 +1,153 @@ | |||
| 1 | S2IO Technologies XFrame 10 Gig adapter. | 1 | Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver. |
| 2 | ------------------------------------------- | 2 | |
| 3 | 3 | Contents | |
| 4 | I. Module loadable parameters. | 4 | ======= |
| 5 | When loaded as a module, the driver provides a host of Module loadable | 5 | - 1. Introduction |
| 6 | parameters, so the device can be tuned as per the users needs. | 6 | - 2. Identifying the adapter/interface |
| 7 | A list of the Module params is given below. | 7 | - 3. Features supported |
| 8 | (i) ring_num: This can be used to program the number of | 8 | - 4. Command line parameters |
| 9 | receive rings used in the driver. | 9 | - 5. Performance suggestions |
| 10 | (ii) ring_len: This defines the number of descriptors each ring | 10 | - 6. Available Downloads |
| 11 | can have. There can be a maximum of 8 rings. | 11 | |
| 12 | (iii) frame_len: This is an array of size 8. Using this we can | 12 | |
| 13 | set the maximum size of the received frame that can | 13 | 1. Introduction: |
| 14 | be steered into the corrsponding receive ring. | 14 | This Linux driver supports Neterion's Xframe I PCI-X 1.0 and |
| 15 | (iv) fifo_num: This defines the number of Tx FIFOs thats used in | 15 | Xframe II PCI-X 2.0 adapters. It supports several features |
| 16 | the driver. | 16 | such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on. |
| 17 | (v) fifo_len: Each element defines the number of | 17 | See below for complete list of features. |
| 18 | Tx descriptors that can be associated with each | 18 | All features are supported for both IPv4 and IPv6. |
| 19 | corresponding FIFO. There are a maximum of 8 FIFOs. | 19 | |
| 20 | (vi) tx_prio: This is a bool, if module is loaded with a non-zero | 20 | 2. Identifying the adapter/interface: |
| 21 | value for tx_prio multi FIFO scheme is activated. | 21 | a. Insert the adapter(s) in your system. |
| 22 | (vii) rx_prio: This is a bool, if module is loaded with a non-zero | 22 | b. Build and load driver |
| 23 | value for tx_prio multi RING scheme is activated. | 23 | # insmod s2io.ko |
| 24 | (viii) latency_timer: The value given against this param will be | 24 | c. View log messages |
| 25 | loaded into the latency timer register in PCI Config | 25 | # dmesg | tail -40 |
| 26 | space, else the register is left with its reset value. | 26 | You will see messages similar to: |
| 27 | 27 | eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA | |
| 28 | II. Performance tuning. | 28 | eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA |
| 29 | By changing a few sysctl parameters. | 29 | eth4: Device is on 64 bit 133MHz PCIX(M1) bus |
| 30 | Copy the following lines into a file and run the following command, | 30 | |
| 31 | "sysctl -p <file_name>" | 31 | The above messages identify the adapter type(Xframe I/II), adapter revision, |
| 32 | ### IPV4 specific settings | 32 | driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X). |
| 33 | net.ipv4.tcp_timestamps = 0 # turns TCP timestamp support off, default 1, reduces CPU use | 33 | In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed |
| 34 | net.ipv4.tcp_sack = 0 # turn SACK support off, default on | 34 | as well. |
| 35 | # on systems with a VERY fast bus -> memory interface this is the big gainer | 35 | |
| 36 | net.ipv4.tcp_rmem = 10000000 10000000 10000000 # sets min/default/max TCP read buffer, default 4096 87380 174760 | 36 | To associate an interface with a physical adapter use "ethtool -p <ethX>". |
| 37 | net.ipv4.tcp_wmem = 10000000 10000000 10000000 # sets min/pressure/max TCP write buffer, default 4096 16384 131072 | 37 | The corresponding adapter's LED will blink multiple times. |
| 38 | net.ipv4.tcp_mem = 10000000 10000000 10000000 # sets min/pressure/max TCP buffer space, default 31744 32256 32768 | 38 | |
| 39 | 39 | 3. Features supported: | |
| 40 | ### CORE settings (mostly for socket and UDP effect) | 40 | a. Jumbo frames. Xframe I/II supports MTU upto 9600 bytes, |
| 41 | net.core.rmem_max = 524287 # maximum receive socket buffer size, default 131071 | 41 | modifiable using ifconfig command. |
| 42 | net.core.wmem_max = 524287 # maximum send socket buffer size, default 131071 | 42 | |
| 43 | net.core.rmem_default = 524287 # default receive socket buffer size, default 65535 | 43 | b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit |
| 44 | net.core.wmem_default = 524287 # default send socket buffer size, default 65535 | 44 | and receive, TSO. |
| 45 | net.core.optmem_max = 524287 # maximum amount of option memory buffers, default 10240 | 45 | |
| 46 | net.core.netdev_max_backlog = 300000 # number of unprocessed input packets before kernel starts dropping them, default 300 | 46 | c. Multi-buffer receive mode. Scattering of packet across multiple |
| 47 | ---End of performance tuning file--- | 47 | buffers. Currently driver supports 2-buffer mode which yields |
| 48 | significant performance improvement on certain platforms(SGI Altix, | ||
| 49 | IBM xSeries). | ||
| 50 | |||
| 51 | d. MSI/MSI-X. Can be enabled on platforms which support this feature | ||
| 52 | (IA64, Xeon) resulting in noticeable performance improvement(upto 7% | ||
| 53 | on certain platforms). | ||
| 54 | |||
| 55 | e. NAPI. Compile-time option(CONFIG_S2IO_NAPI) for better Rx interrupt | ||
| 56 | moderation. | ||
| 57 | |||
| 58 | f. Statistics. Comprehensive MAC-level and software statistics displayed | ||
| 59 | using "ethtool -S" option. | ||
| 60 | |||
| 61 | g. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings, | ||
| 62 | with multiple steering options. | ||
| 63 | |||
| 64 | 4. Command line parameters | ||
| 65 | a. tx_fifo_num | ||
| 66 | Number of transmit queues | ||
| 67 | Valid range: 1-8 | ||
| 68 | Default: 1 | ||
| 69 | |||
| 70 | b. rx_ring_num | ||
| 71 | Number of receive rings | ||
| 72 | Valid range: 1-8 | ||
| 73 | Default: 1 | ||
| 74 | |||
| 75 | c. tx_fifo_len | ||
| 76 | Size of each transmit queue | ||
| 77 | Valid range: Total length of all queues should not exceed 8192 | ||
| 78 | Default: 4096 | ||
| 79 | |||
| 80 | d. rx_ring_sz | ||
| 81 | Size of each receive ring(in 4K blocks) | ||
| 82 | Valid range: Limited by memory on system | ||
| 83 | Default: 30 | ||
| 84 | |||
| 85 | e. intr_type | ||
| 86 | Specifies interrupt type. Possible values 1(INTA), 2(MSI), 3(MSI-X) | ||
| 87 | Valid range: 1-3 | ||
| 88 | Default: 1 | ||
| 89 | |||
| 90 | 5. Performance suggestions | ||
| 91 | General: | ||
| 92 | a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration) | ||
| 93 | b. Set TCP windows size to optimal value. | ||
| 94 | For instance, for MTU=1500 a value of 210K has been observed to result in | ||
| 95 | good performance. | ||
| 96 | # sysctl -w net.ipv4.tcp_rmem="210000 210000 210000" | ||
| 97 | # sysctl -w net.ipv4.tcp_wmem="210000 210000 210000" | ||
| 98 | For MTU=9000, TCP window size of 10 MB is recommended. | ||
| 99 | # sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000" | ||
| 100 | # sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000" | ||
| 101 | |||
| 102 | Transmit performance: | ||
| 103 | a. By default, the driver respects BIOS settings for PCI bus parameters. | ||
| 104 | However, you may want to experiment with PCI bus parameters | ||
| 105 | max-split-transactions(MOST) and MMRBC (use setpci command). | ||
| 106 | A MOST value of 2 has been found optimal for Opterons and 3 for Itanium. | ||
| 107 | It could be different for your hardware. | ||
| 108 | Set MMRBC to 4K**. | ||
| 109 | |||
| 110 | For example you can set | ||
| 111 | For opteron | ||
| 112 | #setpci -d 17d5:* 62=1d | ||
| 113 | For Itanium | ||
| 114 | #setpci -d 17d5:* 62=3d | ||
| 115 | |||
| 116 | For detailed description of the PCI registers, please see Xframe User Guide. | ||
| 117 | |||
| 118 | b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this | ||
| 119 | parameter. | ||
| 120 | c. Turn on TSO(using "ethtool -K") | ||
| 121 | # ethtool -K <ethX> tso on | ||
| 122 | |||
| 123 | Receive performance: | ||
| 124 | a. By default, the driver respects BIOS settings for PCI bus parameters. | ||
| 125 | However, you may want to set PCI latency timer to 248. | ||
| 126 | #setpci -d 17d5:* LATENCY_TIMER=f8 | ||
| 127 | For detailed description of the PCI registers, please see Xframe User Guide. | ||
| 128 | b. Use 2-buffer mode. This results in large performance boost on | ||
| 129 | on certain platforms(eg. SGI Altix, IBM xSeries). | ||
| 130 | c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to | ||
| 131 | set/verify this option. | ||
| 132 | d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network | ||
| 133 | device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to | ||
| 134 | bring down CPU utilization. | ||
| 135 | |||
| 136 | ** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are | ||
| 137 | recommended as safe parameters. | ||
| 138 | For more information, please review the AMD8131 errata at | ||
| 139 | http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26310.pdf | ||
| 140 | |||
| 141 | 6. Available Downloads | ||
| 142 | Neterion "s2io" driver in Red Hat and Suse 2.6-based distributions is kept up | ||
| 143 | to date, also the latest "s2io" code (including support for 2.4 kernels) is | ||
| 144 | available via "Support" link on the Neterion site: http://www.neterion.com. | ||
| 145 | |||
| 146 | For Xframe User Guide (Programming manual), visit ftp site ns1.s2io.com, | ||
| 147 | user: linuxdocs password: HALdocs | ||
| 148 | |||
| 149 | 7. Support | ||
| 150 | For further support please contact either your 10GbE Xframe NIC vendor (IBM, | ||
| 151 | HP, SGI etc.) or click on the "Support" link on the Neterion site: | ||
| 152 | http://www.neterion.com. | ||
| 48 | 153 | ||
