aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/net/ethernet/mellanox
diff options
context:
space:
mode:
authorLinus Torvalds <torvalds@linux-foundation.org>2013-07-09 21:24:39 -0400
committerLinus Torvalds <torvalds@linux-foundation.org>2013-07-09 21:24:39 -0400
commit496322bc91e35007ed754184dcd447a02b6dd685 (patch)
treef5298d0a74c0a6e65c0e98050b594b8d020904c1 /drivers/net/ethernet/mellanox
parent2e17c5a97e231f3cb426f4b7895eab5be5c5442e (diff)
parent56e0ef527b184b3de2d7f88c6190812b2b2ac6bf (diff)
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
Pull networking updates from David Miller: "This is a re-do of the net-next pull request for the current merge window. The only difference from the one I made the other day is that this has Eliezer's interface renames and the timeout handling changes made based upon your feedback, as well as a few bug fixes that have trickeled in. Highlights: 1) Low latency device polling, eliminating the cost of interrupt handling and context switches. Allows direct polling of a network device from socket operations, such as recvmsg() and poll(). Currently ixgbe, mlx4, and bnx2x support this feature. Full high level description, performance numbers, and design in commit 0a4db187a999 ("Merge branch 'll_poll'") From Eliezer Tamir. 2) With the routing cache removed, ip_check_mc_rcu() gets exercised more than ever before in the case where we have lots of multicast addresses. Use a hash table instead of a simple linked list, from Eric Dumazet. 3) Add driver for Atheros CQA98xx 802.11ac wireless devices, from Bartosz Markowski, Janusz Dziedzic, Kalle Valo, Marek Kwaczynski, Marek Puzyniak, Michal Kazior, and Sujith Manoharan. 4) Support reporting the TUN device persist flag to userspace, from Pavel Emelyanov. 5) Allow controlling network device VF link state using netlink, from Rony Efraim. 6) Support GRE tunneling in openvswitch, from Pravin B Shelar. 7) Adjust SOCK_MIN_RCVBUF and SOCK_MIN_SNDBUF for modern times, from Daniel Borkmann and Eric Dumazet. 8) Allow controlling of TCP quickack behavior on a per-route basis, from Cong Wang. 9) Several bug fixes and improvements to vxlan from Stephen Hemminger, Pravin B Shelar, and Mike Rapoport. In particular, support receiving on multiple UDP ports. 10) Major cleanups, particular in the area of debugging and cookie lifetime handline, to the SCTP protocol code. From Daniel Borkmann. 11) Allow packets to cross network namespaces when traversing tunnel devices. From Nicolas Dichtel. 12) Allow monitoring netlink traffic via AF_PACKET sockets, in a manner akin to how we monitor real network traffic via ptype_all. From Daniel Borkmann. 13) Several bug fixes and improvements for the new alx device driver, from Johannes Berg. 14) Fix scalability issues in the netem packet scheduler's time queue, by using an rbtree. From Eric Dumazet. 15) Several bug fixes in TCP loss recovery handling, from Yuchung Cheng. 16) Add support for GSO segmentation of MPLS packets, from Simon Horman. 17) Make network notifiers have a real data type for the opaque pointer that's passed into them. Use this to properly handle network device flag changes in arp_netdev_event(). From Jiri Pirko and Timo Teräs. 18) Convert several drivers over to module_pci_driver(), from Peter Huewe. 19) tcp_fixup_rcvbuf() can loop 500 times over loopback, just use a O(1) calculation instead. From Eric Dumazet. 20) Support setting of explicit tunnel peer addresses in ipv6, just like ipv4. From Nicolas Dichtel. 21) Protect x86 BPF JIT against spraying attacks, from Eric Dumazet. 22) Prevent a single high rate flow from overruning an individual cpu during RX packet processing via selective flow shedding. From Willem de Bruijn. 23) Don't use spinlocks in TCP md5 signing fast paths, from Eric Dumazet. 24) Don't just drop GSO packets which are above the TBF scheduler's burst limit, chop them up so they are in-bounds instead. Also from Eric Dumazet. 25) VLAN offloads are missed when configured on top of a bridge, fix from Vlad Yasevich. 26) Support IPV6 in ping sockets. From Lorenzo Colitti. 27) Receive flow steering targets should be updated at poll() time too, from David Majnemer. 28) Fix several corner case regressions in PMTU/redirect handling due to the routing cache removal, from Timo Teräs. 29) We have to be mindful of ipv4 mapped ipv6 sockets in upd_v6_push_pending_frames(). From Hannes Frederic Sowa. 30) Fix L2TP sequence number handling bugs, from James Chapman." * git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next: (1214 commits) drivers/net: caif: fix wrong rtnl_is_locked() usage drivers/net: enic: release rtnl_lock on error-path vhost-net: fix use-after-free in vhost_net_flush net: mv643xx_eth: do not use port number as platform device id net: sctp: confirm route during forward progress virtio_net: fix race in RX VQ processing virtio: support unlocked queue poll net/cadence/macb: fix bug/typo in extracting gem_irq_read_clear bit Documentation: Fix references to defunct linux-net@vger.kernel.org net/fs: change busy poll time accounting net: rename low latency sockets functions to busy poll bridge: fix some kernel warning in multicast timer sfc: Fix memory leak when discarding scattered packets sit: fix tunnel update via netlink dt:net:stmmac: Add dt specific phy reset callback support. dt:net:stmmac: Add support to dwmac version 3.610 and 3.710 dt:net:stmmac: Allocate platform data only if its NULL. net:stmmac: fix memleak in the open method ipv6: rt6_check_neigh should successfully verify neigh if no NUD information are available net: ipv6: fix wrong ping_v6_sendmsg return value ...
Diffstat (limited to 'drivers/net/ethernet/mellanox')
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/cmd.c196
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_cq.c3
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c3
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_ethtool.c20
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_main.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_netdev.c88
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_rx.c184
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_tx.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/eq.c9
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/fw.c13
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/main.c11
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/mlx4.h22
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/mlx4_en.h145
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/resource_tracker.c163
14 files changed, 735 insertions, 126 deletions
diff --git a/drivers/net/ethernet/mellanox/mlx4/cmd.c b/drivers/net/ethernet/mellanox/mlx4/cmd.c
index 0e572a527154..299d0184f983 100644
--- a/drivers/net/ethernet/mellanox/mlx4/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx4/cmd.c
@@ -39,6 +39,7 @@
39#include <linux/errno.h> 39#include <linux/errno.h>
40 40
41#include <linux/mlx4/cmd.h> 41#include <linux/mlx4/cmd.h>
42#include <linux/mlx4/device.h>
42#include <linux/semaphore.h> 43#include <linux/semaphore.h>
43#include <rdma/ib_smi.h> 44#include <rdma/ib_smi.h>
44 45
@@ -111,6 +112,14 @@ enum {
111 GO_BIT_TIMEOUT_MSECS = 10000 112 GO_BIT_TIMEOUT_MSECS = 10000
112}; 113};
113 114
115enum mlx4_vlan_transition {
116 MLX4_VLAN_TRANSITION_VST_VST = 0,
117 MLX4_VLAN_TRANSITION_VST_VGT = 1,
118 MLX4_VLAN_TRANSITION_VGT_VST = 2,
119 MLX4_VLAN_TRANSITION_VGT_VGT = 3,
120};
121
122
114struct mlx4_cmd_context { 123struct mlx4_cmd_context {
115 struct completion done; 124 struct completion done;
116 int result; 125 int result;
@@ -256,6 +265,8 @@ static int mlx4_comm_cmd_wait(struct mlx4_dev *dev, u8 op,
256 265
257 if (!wait_for_completion_timeout(&context->done, 266 if (!wait_for_completion_timeout(&context->done,
258 msecs_to_jiffies(timeout))) { 267 msecs_to_jiffies(timeout))) {
268 mlx4_warn(dev, "communication channel command 0x%x timed out\n",
269 op);
259 err = -EBUSY; 270 err = -EBUSY;
260 goto out; 271 goto out;
261 } 272 }
@@ -485,6 +496,8 @@ static int mlx4_cmd_poll(struct mlx4_dev *dev, u64 in_param, u64 *out_param,
485 } 496 }
486 497
487 if (cmd_pending(dev)) { 498 if (cmd_pending(dev)) {
499 mlx4_warn(dev, "command 0x%x timed out (go bit not cleared)\n",
500 op);
488 err = -ETIMEDOUT; 501 err = -ETIMEDOUT;
489 goto out; 502 goto out;
490 } 503 }
@@ -548,6 +561,8 @@ static int mlx4_cmd_wait(struct mlx4_dev *dev, u64 in_param, u64 *out_param,
548 561
549 if (!wait_for_completion_timeout(&context->done, 562 if (!wait_for_completion_timeout(&context->done,
550 msecs_to_jiffies(timeout))) { 563 msecs_to_jiffies(timeout))) {
564 mlx4_warn(dev, "command 0x%x timed out (go bit not cleared)\n",
565 op);
551 err = -EBUSY; 566 err = -EBUSY;
552 goto out; 567 goto out;
553 } 568 }
@@ -785,6 +800,15 @@ static int mlx4_MAD_IFC_wrapper(struct mlx4_dev *dev, int slave,
785 vhcr->op, MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE); 800 vhcr->op, MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE);
786} 801}
787 802
803int MLX4_CMD_UPDATE_QP_wrapper(struct mlx4_dev *dev, int slave,
804 struct mlx4_vhcr *vhcr,
805 struct mlx4_cmd_mailbox *inbox,
806 struct mlx4_cmd_mailbox *outbox,
807 struct mlx4_cmd_info *cmd)
808{
809 return -EPERM;
810}
811
788int mlx4_DMA_wrapper(struct mlx4_dev *dev, int slave, 812int mlx4_DMA_wrapper(struct mlx4_dev *dev, int slave,
789 struct mlx4_vhcr *vhcr, 813 struct mlx4_vhcr *vhcr,
790 struct mlx4_cmd_mailbox *inbox, 814 struct mlx4_cmd_mailbox *inbox,
@@ -1219,6 +1243,15 @@ static struct mlx4_cmd_info cmd_info[] = {
1219 .wrapper = mlx4_GEN_QP_wrapper 1243 .wrapper = mlx4_GEN_QP_wrapper
1220 }, 1244 },
1221 { 1245 {
1246 .opcode = MLX4_CMD_UPDATE_QP,
1247 .has_inbox = false,
1248 .has_outbox = false,
1249 .out_is_imm = false,
1250 .encode_slave_id = false,
1251 .verify = NULL,
1252 .wrapper = MLX4_CMD_UPDATE_QP_wrapper
1253 },
1254 {
1222 .opcode = MLX4_CMD_CONF_SPECIAL_QP, 1255 .opcode = MLX4_CMD_CONF_SPECIAL_QP,
1223 .has_inbox = false, 1256 .has_inbox = false,
1224 .has_outbox = false, 1257 .has_outbox = false,
@@ -1488,6 +1521,102 @@ out:
1488 return ret; 1521 return ret;
1489} 1522}
1490 1523
1524static int calculate_transition(u16 oper_vlan, u16 admin_vlan)
1525{
1526 return (2 * (oper_vlan == MLX4_VGT) + (admin_vlan == MLX4_VGT));
1527}
1528
1529int mlx4_master_immediate_activate_vlan_qos(struct mlx4_priv *priv,
1530 int slave, int port)
1531{
1532 struct mlx4_vport_oper_state *vp_oper;
1533 struct mlx4_vport_state *vp_admin;
1534 struct mlx4_vf_immed_vlan_work *work;
1535 struct mlx4_dev *dev = &(priv->dev);
1536 int err;
1537 int admin_vlan_ix = NO_INDX;
1538 enum mlx4_vlan_transition vlan_trans;
1539
1540 vp_oper = &priv->mfunc.master.vf_oper[slave].vport[port];
1541 vp_admin = &priv->mfunc.master.vf_admin[slave].vport[port];
1542
1543 if (vp_oper->state.default_vlan == vp_admin->default_vlan &&
1544 vp_oper->state.default_qos == vp_admin->default_qos &&
1545 vp_oper->state.link_state == vp_admin->link_state)
1546 return 0;
1547
1548 vlan_trans = calculate_transition(vp_oper->state.default_vlan,
1549 vp_admin->default_vlan);
1550
1551 if (!(priv->mfunc.master.slave_state[slave].active &&
1552 dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_UPDATE_QP &&
1553 vlan_trans == MLX4_VLAN_TRANSITION_VST_VST)) {
1554 /* even if the UPDATE_QP command isn't supported, we still want
1555 * to set this VF link according to the admin directive
1556 */
1557 vp_oper->state.link_state = vp_admin->link_state;
1558 return -1;
1559 }
1560
1561 mlx4_dbg(dev, "updating immediately admin params slave %d port %d\n",
1562 slave, port);
1563 mlx4_dbg(dev, "vlan %d QoS %d link down %d\n", vp_admin->default_vlan,
1564 vp_admin->default_qos, vp_admin->link_state);
1565
1566 work = kzalloc(sizeof(*work), GFP_KERNEL);
1567 if (!work)
1568 return -ENOMEM;
1569
1570 if (vp_oper->state.default_vlan != vp_admin->default_vlan) {
1571 err = __mlx4_register_vlan(&priv->dev, port,
1572 vp_admin->default_vlan,
1573 &admin_vlan_ix);
1574 if (err) {
1575 kfree(work);
1576 mlx4_warn((&priv->dev),
1577 "No vlan resources slave %d, port %d\n",
1578 slave, port);
1579 return err;
1580 }
1581 work->flags |= MLX4_VF_IMMED_VLAN_FLAG_VLAN;
1582 mlx4_dbg((&(priv->dev)),
1583 "alloc vlan %d idx %d slave %d port %d\n",
1584 (int)(vp_admin->default_vlan),
1585 admin_vlan_ix, slave, port);
1586 }
1587
1588 /* save original vlan ix and vlan id */
1589 work->orig_vlan_id = vp_oper->state.default_vlan;
1590 work->orig_vlan_ix = vp_oper->vlan_idx;
1591
1592 /* handle new qos */
1593 if (vp_oper->state.default_qos != vp_admin->default_qos)
1594 work->flags |= MLX4_VF_IMMED_VLAN_FLAG_QOS;
1595
1596 if (work->flags & MLX4_VF_IMMED_VLAN_FLAG_VLAN)
1597 vp_oper->vlan_idx = admin_vlan_ix;
1598
1599 vp_oper->state.default_vlan = vp_admin->default_vlan;
1600 vp_oper->state.default_qos = vp_admin->default_qos;
1601 vp_oper->state.link_state = vp_admin->link_state;
1602
1603 if (vp_admin->link_state == IFLA_VF_LINK_STATE_DISABLE)
1604 work->flags |= MLX4_VF_IMMED_VLAN_FLAG_LINK_DISABLE;
1605
1606 /* iterate over QPs owned by this slave, using UPDATE_QP */
1607 work->port = port;
1608 work->slave = slave;
1609 work->qos = vp_oper->state.default_qos;
1610 work->vlan_id = vp_oper->state.default_vlan;
1611 work->vlan_ix = vp_oper->vlan_idx;
1612 work->priv = priv;
1613 INIT_WORK(&work->work, mlx4_vf_immed_vlan_work_handler);
1614 queue_work(priv->mfunc.master.comm_wq, &work->work);
1615
1616 return 0;
1617}
1618
1619
1491static int mlx4_master_activate_admin_state(struct mlx4_priv *priv, int slave) 1620static int mlx4_master_activate_admin_state(struct mlx4_priv *priv, int slave)
1492{ 1621{
1493 int port, err; 1622 int port, err;
@@ -2102,10 +2231,12 @@ int mlx4_set_vf_mac(struct mlx4_dev *dev, int port, int vf, u64 mac)
2102} 2231}
2103EXPORT_SYMBOL_GPL(mlx4_set_vf_mac); 2232EXPORT_SYMBOL_GPL(mlx4_set_vf_mac);
2104 2233
2234
2105int mlx4_set_vf_vlan(struct mlx4_dev *dev, int port, int vf, u16 vlan, u8 qos) 2235int mlx4_set_vf_vlan(struct mlx4_dev *dev, int port, int vf, u16 vlan, u8 qos)
2106{ 2236{
2107 struct mlx4_priv *priv = mlx4_priv(dev); 2237 struct mlx4_priv *priv = mlx4_priv(dev);
2108 struct mlx4_vport_state *s_info; 2238 struct mlx4_vport_oper_state *vf_oper;
2239 struct mlx4_vport_state *vf_admin;
2109 int slave; 2240 int slave;
2110 2241
2111 if ((!mlx4_is_master(dev)) || 2242 if ((!mlx4_is_master(dev)) ||
@@ -2119,12 +2250,19 @@ int mlx4_set_vf_vlan(struct mlx4_dev *dev, int port, int vf, u16 vlan, u8 qos)
2119 if (slave < 0) 2250 if (slave < 0)
2120 return -EINVAL; 2251 return -EINVAL;
2121 2252
2122 s_info = &priv->mfunc.master.vf_admin[slave].vport[port]; 2253 vf_admin = &priv->mfunc.master.vf_admin[slave].vport[port];
2254 vf_oper = &priv->mfunc.master.vf_oper[slave].vport[port];
2255
2123 if ((0 == vlan) && (0 == qos)) 2256 if ((0 == vlan) && (0 == qos))
2124 s_info->default_vlan = MLX4_VGT; 2257 vf_admin->default_vlan = MLX4_VGT;
2125 else 2258 else
2126 s_info->default_vlan = vlan; 2259 vf_admin->default_vlan = vlan;
2127 s_info->default_qos = qos; 2260 vf_admin->default_qos = qos;
2261
2262 if (mlx4_master_immediate_activate_vlan_qos(priv, slave, port))
2263 mlx4_info(dev,
2264 "updating vf %d port %d config will take effect on next VF restart\n",
2265 vf, port);
2128 return 0; 2266 return 0;
2129} 2267}
2130EXPORT_SYMBOL_GPL(mlx4_set_vf_vlan); 2268EXPORT_SYMBOL_GPL(mlx4_set_vf_vlan);
@@ -2178,7 +2316,55 @@ int mlx4_get_vf_config(struct mlx4_dev *dev, int port, int vf, struct ifla_vf_in
2178 ivf->qos = s_info->default_qos; 2316 ivf->qos = s_info->default_qos;
2179 ivf->tx_rate = s_info->tx_rate; 2317 ivf->tx_rate = s_info->tx_rate;
2180 ivf->spoofchk = s_info->spoofchk; 2318 ivf->spoofchk = s_info->spoofchk;
2319 ivf->linkstate = s_info->link_state;
2181 2320
2182 return 0; 2321 return 0;
2183} 2322}
2184EXPORT_SYMBOL_GPL(mlx4_get_vf_config); 2323EXPORT_SYMBOL_GPL(mlx4_get_vf_config);
2324
2325int mlx4_set_vf_link_state(struct mlx4_dev *dev, int port, int vf, int link_state)
2326{
2327 struct mlx4_priv *priv = mlx4_priv(dev);
2328 struct mlx4_vport_state *s_info;
2329 int slave;
2330 u8 link_stat_event;
2331
2332 slave = mlx4_get_slave_indx(dev, vf);
2333 if (slave < 0)
2334 return -EINVAL;
2335
2336 switch (link_state) {
2337 case IFLA_VF_LINK_STATE_AUTO:
2338 /* get current link state */
2339 if (!priv->sense.do_sense_port[port])
2340 link_stat_event = MLX4_PORT_CHANGE_SUBTYPE_ACTIVE;
2341 else
2342 link_stat_event = MLX4_PORT_CHANGE_SUBTYPE_DOWN;
2343 break;
2344
2345 case IFLA_VF_LINK_STATE_ENABLE:
2346 link_stat_event = MLX4_PORT_CHANGE_SUBTYPE_ACTIVE;
2347 break;
2348
2349 case IFLA_VF_LINK_STATE_DISABLE:
2350 link_stat_event = MLX4_PORT_CHANGE_SUBTYPE_DOWN;
2351 break;
2352
2353 default:
2354 mlx4_warn(dev, "unknown value for link_state %02x on slave %d port %d\n",
2355 link_state, slave, port);
2356 return -EINVAL;
2357 };
2358 s_info = &priv->mfunc.master.vf_admin[slave].vport[port];
2359 s_info->link_state = link_state;
2360
2361 /* send event */
2362 mlx4_gen_port_state_change_eqe(dev, slave, port, link_stat_event);
2363
2364 if (mlx4_master_immediate_activate_vlan_qos(priv, slave, port))
2365 mlx4_dbg(dev,
2366 "updating vf %d port %d no link state HW enforcment\n",
2367 vf, port);
2368 return 0;
2369}
2370EXPORT_SYMBOL_GPL(mlx4_set_vf_link_state);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_cq.c b/drivers/net/ethernet/mellanox/mlx4/en_cq.c
index 1e6c594d6d04..3e2d5047cdb3 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_cq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_cq.c
@@ -139,6 +139,7 @@ int mlx4_en_activate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq,
139 139
140 if (!cq->is_tx) { 140 if (!cq->is_tx) {
141 netif_napi_add(cq->dev, &cq->napi, mlx4_en_poll_rx_cq, 64); 141 netif_napi_add(cq->dev, &cq->napi, mlx4_en_poll_rx_cq, 64);
142 napi_hash_add(&cq->napi);
142 napi_enable(&cq->napi); 143 napi_enable(&cq->napi);
143 } 144 }
144 145
@@ -162,6 +163,8 @@ void mlx4_en_deactivate_cq(struct mlx4_en_priv *priv, struct mlx4_en_cq *cq)
162{ 163{
163 if (!cq->is_tx) { 164 if (!cq->is_tx) {
164 napi_disable(&cq->napi); 165 napi_disable(&cq->napi);
166 napi_hash_del(&cq->napi);
167 synchronize_rcu();
165 netif_napi_del(&cq->napi); 168 netif_napi_del(&cq->napi);
166 } 169 }
167 170
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
index 0f91222ea3d7..9d4a1ea030d8 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
@@ -207,9 +207,6 @@ static int mlx4_en_dcbnl_ieee_getmaxrate(struct net_device *dev,
207 struct mlx4_en_priv *priv = netdev_priv(dev); 207 struct mlx4_en_priv *priv = netdev_priv(dev);
208 int i; 208 int i;
209 209
210 if (!priv->maxrate)
211 return -EINVAL;
212
213 for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) 210 for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++)
214 maxrate->tc_maxrate[i] = 211 maxrate->tc_maxrate[i] =
215 priv->maxrate[i] * MLX4_RATELIMIT_UNITS_IN_KB; 212 priv->maxrate[i] * MLX4_RATELIMIT_UNITS_IN_KB;
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
index c9e6b62dd000..727874f575ce 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_ethtool.c
@@ -222,7 +222,12 @@ static int mlx4_en_get_sset_count(struct net_device *dev, int sset)
222 switch (sset) { 222 switch (sset) {
223 case ETH_SS_STATS: 223 case ETH_SS_STATS:
224 return (priv->stats_bitmap ? bit_count : NUM_ALL_STATS) + 224 return (priv->stats_bitmap ? bit_count : NUM_ALL_STATS) +
225 (priv->tx_ring_num + priv->rx_ring_num) * 2; 225 (priv->tx_ring_num * 2) +
226#ifdef CONFIG_NET_LL_RX_POLL
227 (priv->rx_ring_num * 5);
228#else
229 (priv->rx_ring_num * 2);
230#endif
226 case ETH_SS_TEST: 231 case ETH_SS_TEST:
227 return MLX4_EN_NUM_SELF_TEST - !(priv->mdev->dev->caps.flags 232 return MLX4_EN_NUM_SELF_TEST - !(priv->mdev->dev->caps.flags
228 & MLX4_DEV_CAP_FLAG_UC_LOOPBACK) * 2; 233 & MLX4_DEV_CAP_FLAG_UC_LOOPBACK) * 2;
@@ -271,6 +276,11 @@ static void mlx4_en_get_ethtool_stats(struct net_device *dev,
271 for (i = 0; i < priv->rx_ring_num; i++) { 276 for (i = 0; i < priv->rx_ring_num; i++) {
272 data[index++] = priv->rx_ring[i].packets; 277 data[index++] = priv->rx_ring[i].packets;
273 data[index++] = priv->rx_ring[i].bytes; 278 data[index++] = priv->rx_ring[i].bytes;
279#ifdef CONFIG_NET_LL_RX_POLL
280 data[index++] = priv->rx_ring[i].yields;
281 data[index++] = priv->rx_ring[i].misses;
282 data[index++] = priv->rx_ring[i].cleaned;
283#endif
274 } 284 }
275 spin_unlock_bh(&priv->stats_lock); 285 spin_unlock_bh(&priv->stats_lock);
276 286
@@ -334,6 +344,14 @@ static void mlx4_en_get_strings(struct net_device *dev,
334 "rx%d_packets", i); 344 "rx%d_packets", i);
335 sprintf(data + (index++) * ETH_GSTRING_LEN, 345 sprintf(data + (index++) * ETH_GSTRING_LEN,
336 "rx%d_bytes", i); 346 "rx%d_bytes", i);
347#ifdef CONFIG_NET_LL_RX_POLL
348 sprintf(data + (index++) * ETH_GSTRING_LEN,
349 "rx%d_napi_yield", i);
350 sprintf(data + (index++) * ETH_GSTRING_LEN,
351 "rx%d_misses", i);
352 sprintf(data + (index++) * ETH_GSTRING_LEN,
353 "rx%d_cleaned", i);
354#endif
337 } 355 }
338 break; 356 break;
339 } 357 }
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_main.c b/drivers/net/ethernet/mellanox/mlx4/en_main.c
index a5c9df07a7d0..a071cda2dd04 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_main.c
@@ -310,7 +310,7 @@ static void *mlx4_en_add(struct mlx4_dev *dev)
310err_mr: 310err_mr:
311 (void) mlx4_mr_free(dev, &mdev->mr); 311 (void) mlx4_mr_free(dev, &mdev->mr);
312err_map: 312err_map:
313 if (!mdev->uar_map) 313 if (mdev->uar_map)
314 iounmap(mdev->uar_map); 314 iounmap(mdev->uar_map);
315err_uar: 315err_uar:
316 mlx4_uar_free(dev, &mdev->priv_uar); 316 mlx4_uar_free(dev, &mdev->priv_uar);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index 89c47ea84b50..caf204770569 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -38,6 +38,7 @@
38#include <linux/slab.h> 38#include <linux/slab.h>
39#include <linux/hash.h> 39#include <linux/hash.h>
40#include <net/ip.h> 40#include <net/ip.h>
41#include <net/ll_poll.h>
41 42
42#include <linux/mlx4/driver.h> 43#include <linux/mlx4/driver.h>
43#include <linux/mlx4/device.h> 44#include <linux/mlx4/device.h>
@@ -67,6 +68,34 @@ int mlx4_en_setup_tc(struct net_device *dev, u8 up)
67 return 0; 68 return 0;
68} 69}
69 70
71#ifdef CONFIG_NET_LL_RX_POLL
72/* must be called with local_bh_disable()d */
73static int mlx4_en_low_latency_recv(struct napi_struct *napi)
74{
75 struct mlx4_en_cq *cq = container_of(napi, struct mlx4_en_cq, napi);
76 struct net_device *dev = cq->dev;
77 struct mlx4_en_priv *priv = netdev_priv(dev);
78 struct mlx4_en_rx_ring *rx_ring = &priv->rx_ring[cq->ring];
79 int done;
80
81 if (!priv->port_up)
82 return LL_FLUSH_FAILED;
83
84 if (!mlx4_en_cq_lock_poll(cq))
85 return LL_FLUSH_BUSY;
86
87 done = mlx4_en_process_rx_cq(dev, cq, 4);
88 if (likely(done))
89 rx_ring->cleaned += done;
90 else
91 rx_ring->misses++;
92
93 mlx4_en_cq_unlock_poll(cq);
94
95 return done;
96}
97#endif /* CONFIG_NET_LL_RX_POLL */
98
70#ifdef CONFIG_RFS_ACCEL 99#ifdef CONFIG_RFS_ACCEL
71 100
72struct mlx4_en_filter { 101struct mlx4_en_filter {
@@ -376,7 +405,7 @@ static int mlx4_en_vlan_rx_add_vid(struct net_device *dev,
376 en_err(priv, "Failed configuring VLAN filter\n"); 405 en_err(priv, "Failed configuring VLAN filter\n");
377 } 406 }
378 if (mlx4_register_vlan(mdev->dev, priv->port, vid, &idx)) 407 if (mlx4_register_vlan(mdev->dev, priv->port, vid, &idx))
379 en_err(priv, "failed adding vlan %d\n", vid); 408 en_dbg(HW, priv, "failed adding vlan %d\n", vid);
380 mutex_unlock(&mdev->state_lock); 409 mutex_unlock(&mdev->state_lock);
381 410
382 return 0; 411 return 0;
@@ -399,7 +428,7 @@ static int mlx4_en_vlan_rx_kill_vid(struct net_device *dev,
399 if (!mlx4_find_cached_vlan(mdev->dev, priv->port, vid, &idx)) 428 if (!mlx4_find_cached_vlan(mdev->dev, priv->port, vid, &idx))
400 mlx4_unregister_vlan(mdev->dev, priv->port, idx); 429 mlx4_unregister_vlan(mdev->dev, priv->port, idx);
401 else 430 else
402 en_err(priv, "could not find vid %d in cache\n", vid); 431 en_dbg(HW, priv, "could not find vid %d in cache\n", vid);
403 432
404 if (mdev->device_up && priv->port_up) { 433 if (mdev->device_up && priv->port_up) {
405 err = mlx4_SET_VLAN_FLTR(mdev->dev, priv); 434 err = mlx4_SET_VLAN_FLTR(mdev->dev, priv);
@@ -1207,10 +1236,19 @@ static void mlx4_en_tx_timeout(struct net_device *dev)
1207{ 1236{
1208 struct mlx4_en_priv *priv = netdev_priv(dev); 1237 struct mlx4_en_priv *priv = netdev_priv(dev);
1209 struct mlx4_en_dev *mdev = priv->mdev; 1238 struct mlx4_en_dev *mdev = priv->mdev;
1239 int i;
1210 1240
1211 if (netif_msg_timer(priv)) 1241 if (netif_msg_timer(priv))
1212 en_warn(priv, "Tx timeout called on port:%d\n", priv->port); 1242 en_warn(priv, "Tx timeout called on port:%d\n", priv->port);
1213 1243
1244 for (i = 0; i < priv->tx_ring_num; i++) {
1245 if (!netif_tx_queue_stopped(netdev_get_tx_queue(dev, i)))
1246 continue;
1247 en_warn(priv, "TX timeout on queue: %d, QP: 0x%x, CQ: 0x%x, Cons: 0x%x, Prod: 0x%x\n",
1248 i, priv->tx_ring[i].qpn, priv->tx_ring[i].cqn,
1249 priv->tx_ring[i].cons, priv->tx_ring[i].prod);
1250 }
1251
1214 priv->port_stats.tx_timeout++; 1252 priv->port_stats.tx_timeout++;
1215 en_dbg(DRV, priv, "Scheduling watchdog\n"); 1253 en_dbg(DRV, priv, "Scheduling watchdog\n");
1216 queue_work(mdev->workqueue, &priv->watchdog_task); 1254 queue_work(mdev->workqueue, &priv->watchdog_task);
@@ -1346,12 +1384,13 @@ static void mlx4_en_do_get_stats(struct work_struct *work)
1346 1384
1347 mutex_lock(&mdev->state_lock); 1385 mutex_lock(&mdev->state_lock);
1348 if (mdev->device_up) { 1386 if (mdev->device_up) {
1349 err = mlx4_en_DUMP_ETH_STATS(mdev, priv->port, 0); 1387 if (priv->port_up) {
1350 if (err) 1388 err = mlx4_en_DUMP_ETH_STATS(mdev, priv->port, 0);
1351 en_dbg(HW, priv, "Could not update stats\n"); 1389 if (err)
1390 en_dbg(HW, priv, "Could not update stats\n");
1352 1391
1353 if (priv->port_up)
1354 mlx4_en_auto_moderation(priv); 1392 mlx4_en_auto_moderation(priv);
1393 }
1355 1394
1356 queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY); 1395 queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
1357 } 1396 }
@@ -1445,6 +1484,8 @@ int mlx4_en_start_port(struct net_device *dev)
1445 for (i = 0; i < priv->rx_ring_num; i++) { 1484 for (i = 0; i < priv->rx_ring_num; i++) {
1446 cq = &priv->rx_cq[i]; 1485 cq = &priv->rx_cq[i];
1447 1486
1487 mlx4_en_cq_init_lock(cq);
1488
1448 err = mlx4_en_activate_cq(priv, cq, i); 1489 err = mlx4_en_activate_cq(priv, cq, i);
1449 if (err) { 1490 if (err) {
1450 en_err(priv, "Failed activating Rx CQ\n"); 1491 en_err(priv, "Failed activating Rx CQ\n");
@@ -1603,6 +1644,9 @@ void mlx4_en_stop_port(struct net_device *dev, int detach)
1603 return; 1644 return;
1604 } 1645 }
1605 1646
1647 /* close port*/
1648 mlx4_CLOSE_PORT(mdev->dev, priv->port);
1649
1606 /* Synchronize with tx routine */ 1650 /* Synchronize with tx routine */
1607 netif_tx_lock_bh(dev); 1651 netif_tx_lock_bh(dev);
1608 if (detach) 1652 if (detach)
@@ -1694,14 +1738,20 @@ void mlx4_en_stop_port(struct net_device *dev, int detach)
1694 1738
1695 /* Free RX Rings */ 1739 /* Free RX Rings */
1696 for (i = 0; i < priv->rx_ring_num; i++) { 1740 for (i = 0; i < priv->rx_ring_num; i++) {
1697 mlx4_en_deactivate_rx_ring(priv, &priv->rx_ring[i]); 1741 struct mlx4_en_cq *cq = &priv->rx_cq[i];
1698 while (test_bit(NAPI_STATE_SCHED, &priv->rx_cq[i].napi.state)) 1742
1743 local_bh_disable();
1744 while (!mlx4_en_cq_lock_napi(cq)) {
1745 pr_info("CQ %d locked\n", i);
1746 mdelay(1);
1747 }
1748 local_bh_enable();
1749
1750 while (test_bit(NAPI_STATE_SCHED, &cq->napi.state))
1699 msleep(1); 1751 msleep(1);
1700 mlx4_en_deactivate_cq(priv, &priv->rx_cq[i]); 1752 mlx4_en_deactivate_rx_ring(priv, &priv->rx_ring[i]);
1753 mlx4_en_deactivate_cq(priv, cq);
1701 } 1754 }
1702
1703 /* close port*/
1704 mlx4_CLOSE_PORT(mdev->dev, priv->port);
1705} 1755}
1706 1756
1707static void mlx4_en_restart(struct work_struct *work) 1757static void mlx4_en_restart(struct work_struct *work)
@@ -2061,6 +2111,13 @@ static int mlx4_en_get_vf_config(struct net_device *dev, int vf, struct ifla_vf_
2061 return mlx4_get_vf_config(mdev->dev, en_priv->port, vf, ivf); 2111 return mlx4_get_vf_config(mdev->dev, en_priv->port, vf, ivf);
2062} 2112}
2063 2113
2114static int mlx4_en_set_vf_link_state(struct net_device *dev, int vf, int link_state)
2115{
2116 struct mlx4_en_priv *en_priv = netdev_priv(dev);
2117 struct mlx4_en_dev *mdev = en_priv->mdev;
2118
2119 return mlx4_set_vf_link_state(mdev->dev, en_priv->port, vf, link_state);
2120}
2064static const struct net_device_ops mlx4_netdev_ops = { 2121static const struct net_device_ops mlx4_netdev_ops = {
2065 .ndo_open = mlx4_en_open, 2122 .ndo_open = mlx4_en_open,
2066 .ndo_stop = mlx4_en_close, 2123 .ndo_stop = mlx4_en_close,
@@ -2083,6 +2140,9 @@ static const struct net_device_ops mlx4_netdev_ops = {
2083#ifdef CONFIG_RFS_ACCEL 2140#ifdef CONFIG_RFS_ACCEL
2084 .ndo_rx_flow_steer = mlx4_en_filter_rfs, 2141 .ndo_rx_flow_steer = mlx4_en_filter_rfs,
2085#endif 2142#endif
2143#ifdef CONFIG_NET_LL_RX_POLL
2144 .ndo_ll_poll = mlx4_en_low_latency_recv,
2145#endif
2086}; 2146};
2087 2147
2088static const struct net_device_ops mlx4_netdev_ops_master = { 2148static const struct net_device_ops mlx4_netdev_ops_master = {
@@ -2101,6 +2161,7 @@ static const struct net_device_ops mlx4_netdev_ops_master = {
2101 .ndo_set_vf_mac = mlx4_en_set_vf_mac, 2161 .ndo_set_vf_mac = mlx4_en_set_vf_mac,
2102 .ndo_set_vf_vlan = mlx4_en_set_vf_vlan, 2162 .ndo_set_vf_vlan = mlx4_en_set_vf_vlan,
2103 .ndo_set_vf_spoofchk = mlx4_en_set_vf_spoofchk, 2163 .ndo_set_vf_spoofchk = mlx4_en_set_vf_spoofchk,
2164 .ndo_set_vf_link_state = mlx4_en_set_vf_link_state,
2104 .ndo_get_vf_config = mlx4_en_get_vf_config, 2165 .ndo_get_vf_config = mlx4_en_get_vf_config,
2105#ifdef CONFIG_NET_POLL_CONTROLLER 2166#ifdef CONFIG_NET_POLL_CONTROLLER
2106 .ndo_poll_controller = mlx4_en_netpoll, 2167 .ndo_poll_controller = mlx4_en_netpoll,
@@ -2271,6 +2332,8 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
2271 mdev->pndev[port] = dev; 2332 mdev->pndev[port] = dev;
2272 2333
2273 netif_carrier_off(dev); 2334 netif_carrier_off(dev);
2335 mlx4_en_set_default_moderation(priv);
2336
2274 err = register_netdev(dev); 2337 err = register_netdev(dev);
2275 if (err) { 2338 if (err) {
2276 en_err(priv, "Netdev registration failed for port %d\n", port); 2339 en_err(priv, "Netdev registration failed for port %d\n", port);
@@ -2302,7 +2365,6 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
2302 en_err(priv, "Failed Initializing port\n"); 2365 en_err(priv, "Failed Initializing port\n");
2303 goto out; 2366 goto out;
2304 } 2367 }
2305 mlx4_en_set_default_moderation(priv);
2306 queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY); 2368 queue_delayed_work(mdev->workqueue, &priv->stats_task, STATS_DELAY);
2307 2369
2308 if (mdev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_TS) 2370 if (mdev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_TS)
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 02aee1ebd203..76997b93fdfe 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -31,6 +31,7 @@
31 * 31 *
32 */ 32 */
33 33
34#include <net/ll_poll.h>
34#include <linux/mlx4/cq.h> 35#include <linux/mlx4/cq.h>
35#include <linux/slab.h> 36#include <linux/slab.h>
36#include <linux/mlx4/qp.h> 37#include <linux/mlx4/qp.h>
@@ -42,40 +43,64 @@
42 43
43#include "mlx4_en.h" 44#include "mlx4_en.h"
44 45
46static int mlx4_alloc_pages(struct mlx4_en_priv *priv,
47 struct mlx4_en_rx_alloc *page_alloc,
48 const struct mlx4_en_frag_info *frag_info,
49 gfp_t _gfp)
50{
51 int order;
52 struct page *page;
53 dma_addr_t dma;
54
55 for (order = MLX4_EN_ALLOC_PREFER_ORDER; ;) {
56 gfp_t gfp = _gfp;
57
58 if (order)
59 gfp |= __GFP_COMP | __GFP_NOWARN;
60 page = alloc_pages(gfp, order);
61 if (likely(page))
62 break;
63 if (--order < 0 ||
64 ((PAGE_SIZE << order) < frag_info->frag_size))
65 return -ENOMEM;
66 }
67 dma = dma_map_page(priv->ddev, page, 0, PAGE_SIZE << order,
68 PCI_DMA_FROMDEVICE);
69 if (dma_mapping_error(priv->ddev, dma)) {
70 put_page(page);
71 return -ENOMEM;
72 }
73 page_alloc->size = PAGE_SIZE << order;
74 page_alloc->page = page;
75 page_alloc->dma = dma;
76 page_alloc->offset = frag_info->frag_align;
77 /* Not doing get_page() for each frag is a big win
78 * on asymetric workloads.
79 */
80 atomic_set(&page->_count, page_alloc->size / frag_info->frag_stride);
81 return 0;
82}
83
45static int mlx4_en_alloc_frags(struct mlx4_en_priv *priv, 84static int mlx4_en_alloc_frags(struct mlx4_en_priv *priv,
46 struct mlx4_en_rx_desc *rx_desc, 85 struct mlx4_en_rx_desc *rx_desc,
47 struct mlx4_en_rx_alloc *frags, 86 struct mlx4_en_rx_alloc *frags,
48 struct mlx4_en_rx_alloc *ring_alloc) 87 struct mlx4_en_rx_alloc *ring_alloc,
88 gfp_t gfp)
49{ 89{
50 struct mlx4_en_rx_alloc page_alloc[MLX4_EN_MAX_RX_FRAGS]; 90 struct mlx4_en_rx_alloc page_alloc[MLX4_EN_MAX_RX_FRAGS];
51 struct mlx4_en_frag_info *frag_info; 91 const struct mlx4_en_frag_info *frag_info;
52 struct page *page; 92 struct page *page;
53 dma_addr_t dma; 93 dma_addr_t dma;
54 int i; 94 int i;
55 95
56 for (i = 0; i < priv->num_frags; i++) { 96 for (i = 0; i < priv->num_frags; i++) {
57 frag_info = &priv->frag_info[i]; 97 frag_info = &priv->frag_info[i];
58 if (ring_alloc[i].offset == frag_info->last_offset) { 98 page_alloc[i] = ring_alloc[i];
59 page = alloc_pages(GFP_ATOMIC | __GFP_COMP, 99 page_alloc[i].offset += frag_info->frag_stride;
60 MLX4_EN_ALLOC_ORDER); 100 if (page_alloc[i].offset + frag_info->frag_stride <= ring_alloc[i].size)
61 if (!page) 101 continue;
62 goto out; 102 if (mlx4_alloc_pages(priv, &page_alloc[i], frag_info, gfp))
63 dma = dma_map_page(priv->ddev, page, 0, 103 goto out;
64 MLX4_EN_ALLOC_SIZE, PCI_DMA_FROMDEVICE);
65 if (dma_mapping_error(priv->ddev, dma)) {
66 put_page(page);
67 goto out;
68 }
69 page_alloc[i].page = page;
70 page_alloc[i].dma = dma;
71 page_alloc[i].offset = frag_info->frag_align;
72 } else {
73 page_alloc[i].page = ring_alloc[i].page;
74 get_page(ring_alloc[i].page);
75 page_alloc[i].dma = ring_alloc[i].dma;
76 page_alloc[i].offset = ring_alloc[i].offset +
77 frag_info->frag_stride;
78 }
79 } 104 }
80 105
81 for (i = 0; i < priv->num_frags; i++) { 106 for (i = 0; i < priv->num_frags; i++) {
@@ -87,14 +112,16 @@ static int mlx4_en_alloc_frags(struct mlx4_en_priv *priv,
87 112
88 return 0; 113 return 0;
89 114
90
91out: 115out:
92 while (i--) { 116 while (i--) {
93 frag_info = &priv->frag_info[i]; 117 frag_info = &priv->frag_info[i];
94 if (ring_alloc[i].offset == frag_info->last_offset) 118 if (page_alloc[i].page != ring_alloc[i].page) {
95 dma_unmap_page(priv->ddev, page_alloc[i].dma, 119 dma_unmap_page(priv->ddev, page_alloc[i].dma,
96 MLX4_EN_ALLOC_SIZE, PCI_DMA_FROMDEVICE); 120 page_alloc[i].size, PCI_DMA_FROMDEVICE);
97 put_page(page_alloc[i].page); 121 page = page_alloc[i].page;
122 atomic_set(&page->_count, 1);
123 put_page(page);
124 }
98 } 125 }
99 return -ENOMEM; 126 return -ENOMEM;
100} 127}
@@ -103,12 +130,12 @@ static void mlx4_en_free_frag(struct mlx4_en_priv *priv,
103 struct mlx4_en_rx_alloc *frags, 130 struct mlx4_en_rx_alloc *frags,
104 int i) 131 int i)
105{ 132{
106 struct mlx4_en_frag_info *frag_info = &priv->frag_info[i]; 133 const struct mlx4_en_frag_info *frag_info = &priv->frag_info[i];
107 134
108 if (frags[i].offset == frag_info->last_offset) { 135 if (frags[i].offset + frag_info->frag_stride > frags[i].size)
109 dma_unmap_page(priv->ddev, frags[i].dma, MLX4_EN_ALLOC_SIZE, 136 dma_unmap_page(priv->ddev, frags[i].dma, frags[i].size,
110 PCI_DMA_FROMDEVICE); 137 PCI_DMA_FROMDEVICE);
111 } 138
112 if (frags[i].page) 139 if (frags[i].page)
113 put_page(frags[i].page); 140 put_page(frags[i].page);
114} 141}
@@ -116,35 +143,28 @@ static void mlx4_en_free_frag(struct mlx4_en_priv *priv,
116static int mlx4_en_init_allocator(struct mlx4_en_priv *priv, 143static int mlx4_en_init_allocator(struct mlx4_en_priv *priv,
117 struct mlx4_en_rx_ring *ring) 144 struct mlx4_en_rx_ring *ring)
118{ 145{
119 struct mlx4_en_rx_alloc *page_alloc;
120 int i; 146 int i;
147 struct mlx4_en_rx_alloc *page_alloc;
121 148
122 for (i = 0; i < priv->num_frags; i++) { 149 for (i = 0; i < priv->num_frags; i++) {
123 page_alloc = &ring->page_alloc[i]; 150 const struct mlx4_en_frag_info *frag_info = &priv->frag_info[i];
124 page_alloc->page = alloc_pages(GFP_ATOMIC | __GFP_COMP,
125 MLX4_EN_ALLOC_ORDER);
126 if (!page_alloc->page)
127 goto out;
128 151
129 page_alloc->dma = dma_map_page(priv->ddev, page_alloc->page, 0, 152 if (mlx4_alloc_pages(priv, &ring->page_alloc[i],
130 MLX4_EN_ALLOC_SIZE, PCI_DMA_FROMDEVICE); 153 frag_info, GFP_KERNEL))
131 if (dma_mapping_error(priv->ddev, page_alloc->dma)) {
132 put_page(page_alloc->page);
133 page_alloc->page = NULL;
134 goto out; 154 goto out;
135 }
136 page_alloc->offset = priv->frag_info[i].frag_align;
137 en_dbg(DRV, priv, "Initialized allocator:%d with page:%p\n",
138 i, page_alloc->page);
139 } 155 }
140 return 0; 156 return 0;
141 157
142out: 158out:
143 while (i--) { 159 while (i--) {
160 struct page *page;
161
144 page_alloc = &ring->page_alloc[i]; 162 page_alloc = &ring->page_alloc[i];
145 dma_unmap_page(priv->ddev, page_alloc->dma, 163 dma_unmap_page(priv->ddev, page_alloc->dma,
146 MLX4_EN_ALLOC_SIZE, PCI_DMA_FROMDEVICE); 164 page_alloc->size, PCI_DMA_FROMDEVICE);
147 put_page(page_alloc->page); 165 page = page_alloc->page;
166 atomic_set(&page->_count, 1);
167 put_page(page);
148 page_alloc->page = NULL; 168 page_alloc->page = NULL;
149 } 169 }
150 return -ENOMEM; 170 return -ENOMEM;
@@ -157,13 +177,18 @@ static void mlx4_en_destroy_allocator(struct mlx4_en_priv *priv,
157 int i; 177 int i;
158 178
159 for (i = 0; i < priv->num_frags; i++) { 179 for (i = 0; i < priv->num_frags; i++) {
180 const struct mlx4_en_frag_info *frag_info = &priv->frag_info[i];
181
160 page_alloc = &ring->page_alloc[i]; 182 page_alloc = &ring->page_alloc[i];
161 en_dbg(DRV, priv, "Freeing allocator:%d count:%d\n", 183 en_dbg(DRV, priv, "Freeing allocator:%d count:%d\n",
162 i, page_count(page_alloc->page)); 184 i, page_count(page_alloc->page));
163 185
164 dma_unmap_page(priv->ddev, page_alloc->dma, 186 dma_unmap_page(priv->ddev, page_alloc->dma,
165 MLX4_EN_ALLOC_SIZE, PCI_DMA_FROMDEVICE); 187 page_alloc->size, PCI_DMA_FROMDEVICE);
166 put_page(page_alloc->page); 188 while (page_alloc->offset + frag_info->frag_stride < page_alloc->size) {
189 put_page(page_alloc->page);
190 page_alloc->offset += frag_info->frag_stride;
191 }
167 page_alloc->page = NULL; 192 page_alloc->page = NULL;
168 } 193 }
169} 194}
@@ -194,13 +219,14 @@ static void mlx4_en_init_rx_desc(struct mlx4_en_priv *priv,
194} 219}
195 220
196static int mlx4_en_prepare_rx_desc(struct mlx4_en_priv *priv, 221static int mlx4_en_prepare_rx_desc(struct mlx4_en_priv *priv,
197 struct mlx4_en_rx_ring *ring, int index) 222 struct mlx4_en_rx_ring *ring, int index,
223 gfp_t gfp)
198{ 224{
199 struct mlx4_en_rx_desc *rx_desc = ring->buf + (index * ring->stride); 225 struct mlx4_en_rx_desc *rx_desc = ring->buf + (index * ring->stride);
200 struct mlx4_en_rx_alloc *frags = ring->rx_info + 226 struct mlx4_en_rx_alloc *frags = ring->rx_info +
201 (index << priv->log_rx_info); 227 (index << priv->log_rx_info);
202 228
203 return mlx4_en_alloc_frags(priv, rx_desc, frags, ring->page_alloc); 229 return mlx4_en_alloc_frags(priv, rx_desc, frags, ring->page_alloc, gfp);
204} 230}
205 231
206static inline void mlx4_en_update_rx_prod_db(struct mlx4_en_rx_ring *ring) 232static inline void mlx4_en_update_rx_prod_db(struct mlx4_en_rx_ring *ring)
@@ -234,7 +260,8 @@ static int mlx4_en_fill_rx_buffers(struct mlx4_en_priv *priv)
234 ring = &priv->rx_ring[ring_ind]; 260 ring = &priv->rx_ring[ring_ind];
235 261
236 if (mlx4_en_prepare_rx_desc(priv, ring, 262 if (mlx4_en_prepare_rx_desc(priv, ring,
237 ring->actual_size)) { 263 ring->actual_size,
264 GFP_KERNEL)) {
238 if (ring->actual_size < MLX4_EN_MIN_RX_SIZE) { 265 if (ring->actual_size < MLX4_EN_MIN_RX_SIZE) {
239 en_err(priv, "Failed to allocate " 266 en_err(priv, "Failed to allocate "
240 "enough rx buffers\n"); 267 "enough rx buffers\n");
@@ -449,11 +476,11 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv *priv,
449 DMA_FROM_DEVICE); 476 DMA_FROM_DEVICE);
450 477
451 /* Save page reference in skb */ 478 /* Save page reference in skb */
452 get_page(frags[nr].page);
453 __skb_frag_set_page(&skb_frags_rx[nr], frags[nr].page); 479 __skb_frag_set_page(&skb_frags_rx[nr], frags[nr].page);
454 skb_frag_size_set(&skb_frags_rx[nr], frag_info->frag_size); 480 skb_frag_size_set(&skb_frags_rx[nr], frag_info->frag_size);
455 skb_frags_rx[nr].page_offset = frags[nr].offset; 481 skb_frags_rx[nr].page_offset = frags[nr].offset;
456 skb->truesize += frag_info->frag_stride; 482 skb->truesize += frag_info->frag_stride;
483 frags[nr].page = NULL;
457 } 484 }
458 /* Adjust size of last fragment to match actual length */ 485 /* Adjust size of last fragment to match actual length */
459 if (nr > 0) 486 if (nr > 0)
@@ -546,7 +573,7 @@ static void mlx4_en_refill_rx_buffers(struct mlx4_en_priv *priv,
546 int index = ring->prod & ring->size_mask; 573 int index = ring->prod & ring->size_mask;
547 574
548 while ((u32) (ring->prod - ring->cons) < ring->actual_size) { 575 while ((u32) (ring->prod - ring->cons) < ring->actual_size) {
549 if (mlx4_en_prepare_rx_desc(priv, ring, index)) 576 if (mlx4_en_prepare_rx_desc(priv, ring, index, GFP_ATOMIC))
550 break; 577 break;
551 ring->prod++; 578 ring->prod++;
552 index = ring->prod & ring->size_mask; 579 index = ring->prod & ring->size_mask;
@@ -656,8 +683,11 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
656 * - DIX Ethernet (type interpretation) 683 * - DIX Ethernet (type interpretation)
657 * - TCP/IP (v4) 684 * - TCP/IP (v4)
658 * - without IP options 685 * - without IP options
659 * - not an IP fragment */ 686 * - not an IP fragment
660 if (dev->features & NETIF_F_GRO) { 687 * - no LLS polling in progress
688 */
689 if (!mlx4_en_cq_ll_polling(cq) &&
690 (dev->features & NETIF_F_GRO)) {
661 struct sk_buff *gro_skb = napi_get_frags(&cq->napi); 691 struct sk_buff *gro_skb = napi_get_frags(&cq->napi);
662 if (!gro_skb) 692 if (!gro_skb)
663 goto next; 693 goto next;
@@ -737,6 +767,8 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
737 timestamp); 767 timestamp);
738 } 768 }
739 769
770 skb_mark_ll(skb, &cq->napi);
771
740 /* Push it up the stack */ 772 /* Push it up the stack */
741 netif_receive_skb(skb); 773 netif_receive_skb(skb);
742 774
@@ -781,8 +813,13 @@ int mlx4_en_poll_rx_cq(struct napi_struct *napi, int budget)
781 struct mlx4_en_priv *priv = netdev_priv(dev); 813 struct mlx4_en_priv *priv = netdev_priv(dev);
782 int done; 814 int done;
783 815
816 if (!mlx4_en_cq_lock_napi(cq))
817 return budget;
818
784 done = mlx4_en_process_rx_cq(dev, cq, budget); 819 done = mlx4_en_process_rx_cq(dev, cq, budget);
785 820
821 mlx4_en_cq_unlock_napi(cq);
822
786 /* If we used up all the quota - we're probably not done yet... */ 823 /* If we used up all the quota - we're probably not done yet... */
787 if (done == budget) 824 if (done == budget)
788 INC_PERF_COUNTER(priv->pstats.napi_quota); 825 INC_PERF_COUNTER(priv->pstats.napi_quota);
@@ -794,21 +831,7 @@ int mlx4_en_poll_rx_cq(struct napi_struct *napi, int budget)
794 return done; 831 return done;
795} 832}
796 833
797 834static const int frag_sizes[] = {
798/* Calculate the last offset position that accommodates a full fragment
799 * (assuming fagment size = stride-align) */
800static int mlx4_en_last_alloc_offset(struct mlx4_en_priv *priv, u16 stride, u16 align)
801{
802 u16 res = MLX4_EN_ALLOC_SIZE % stride;
803 u16 offset = MLX4_EN_ALLOC_SIZE - stride - res + align;
804
805 en_dbg(DRV, priv, "Calculated last offset for stride:%d align:%d "
806 "res:%d offset:%d\n", stride, align, res, offset);
807 return offset;
808}
809
810
811static int frag_sizes[] = {
812 FRAG_SZ0, 835 FRAG_SZ0,
813 FRAG_SZ1, 836 FRAG_SZ1,
814 FRAG_SZ2, 837 FRAG_SZ2,
@@ -836,9 +859,6 @@ void mlx4_en_calc_rx_buf(struct net_device *dev)
836 priv->frag_info[i].frag_stride = 859 priv->frag_info[i].frag_stride =
837 ALIGN(frag_sizes[i], SMP_CACHE_BYTES); 860 ALIGN(frag_sizes[i], SMP_CACHE_BYTES);
838 } 861 }
839 priv->frag_info[i].last_offset = mlx4_en_last_alloc_offset(
840 priv, priv->frag_info[i].frag_stride,
841 priv->frag_info[i].frag_align);
842 buf_size += priv->frag_info[i].frag_size; 862 buf_size += priv->frag_info[i].frag_size;
843 i++; 863 i++;
844 } 864 }
@@ -850,13 +870,13 @@ void mlx4_en_calc_rx_buf(struct net_device *dev)
850 en_dbg(DRV, priv, "Rx buffer scatter-list (effective-mtu:%d " 870 en_dbg(DRV, priv, "Rx buffer scatter-list (effective-mtu:%d "
851 "num_frags:%d):\n", eff_mtu, priv->num_frags); 871 "num_frags:%d):\n", eff_mtu, priv->num_frags);
852 for (i = 0; i < priv->num_frags; i++) { 872 for (i = 0; i < priv->num_frags; i++) {
853 en_dbg(DRV, priv, " frag:%d - size:%d prefix:%d align:%d " 873 en_err(priv,
854 "stride:%d last_offset:%d\n", i, 874 " frag:%d - size:%d prefix:%d align:%d stride:%d\n",
855 priv->frag_info[i].frag_size, 875 i,
856 priv->frag_info[i].frag_prefix_size, 876 priv->frag_info[i].frag_size,
857 priv->frag_info[i].frag_align, 877 priv->frag_info[i].frag_prefix_size,
858 priv->frag_info[i].frag_stride, 878 priv->frag_info[i].frag_align,
859 priv->frag_info[i].last_offset); 879 priv->frag_info[i].frag_stride);
860 } 880 }
861} 881}
862 882
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
index 4e6877a032a8..7c492382da09 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
@@ -544,7 +544,7 @@ u16 mlx4_en_select_queue(struct net_device *dev, struct sk_buff *skb)
544 if (vlan_tx_tag_present(skb)) 544 if (vlan_tx_tag_present(skb))
545 up = vlan_tx_tag_get(skb) >> VLAN_PRIO_SHIFT; 545 up = vlan_tx_tag_get(skb) >> VLAN_PRIO_SHIFT;
546 546
547 return __skb_tx_hash(dev, skb, rings_p_up) + up * rings_p_up; 547 return __netdev_pick_tx(dev, skb) % rings_p_up + up * rings_p_up;
548} 548}
549 549
550static void mlx4_bf_copy(void __iomem *dst, unsigned long *src, unsigned bytecnt) 550static void mlx4_bf_copy(void __iomem *dst, unsigned long *src, unsigned bytecnt)
diff --git a/drivers/net/ethernet/mellanox/mlx4/eq.c b/drivers/net/ethernet/mellanox/mlx4/eq.c
index 6000342f9725..7e042869ef0c 100644
--- a/drivers/net/ethernet/mellanox/mlx4/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/eq.c
@@ -448,6 +448,7 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq)
448 int i; 448 int i;
449 enum slave_port_gen_event gen_event; 449 enum slave_port_gen_event gen_event;
450 unsigned long flags; 450 unsigned long flags;
451 struct mlx4_vport_state *s_info;
451 452
452 while ((eqe = next_eqe_sw(eq, dev->caps.eqe_factor))) { 453 while ((eqe = next_eqe_sw(eq, dev->caps.eqe_factor))) {
453 /* 454 /*
@@ -556,7 +557,9 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq)
556 mlx4_dbg(dev, "%s: Sending MLX4_PORT_CHANGE_SUBTYPE_DOWN" 557 mlx4_dbg(dev, "%s: Sending MLX4_PORT_CHANGE_SUBTYPE_DOWN"
557 " to slave: %d, port:%d\n", 558 " to slave: %d, port:%d\n",
558 __func__, i, port); 559 __func__, i, port);
559 mlx4_slave_event(dev, i, eqe); 560 s_info = &priv->mfunc.master.vf_oper[slave].vport[port].state;
561 if (IFLA_VF_LINK_STATE_AUTO == s_info->link_state)
562 mlx4_slave_event(dev, i, eqe);
560 } else { /* IB port */ 563 } else { /* IB port */
561 set_and_calc_slave_port_state(dev, i, port, 564 set_and_calc_slave_port_state(dev, i, port,
562 MLX4_PORT_STATE_DEV_EVENT_PORT_DOWN, 565 MLX4_PORT_STATE_DEV_EVENT_PORT_DOWN,
@@ -580,7 +583,9 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq)
580 for (i = 0; i < dev->num_slaves; i++) { 583 for (i = 0; i < dev->num_slaves; i++) {
581 if (i == mlx4_master_func_num(dev)) 584 if (i == mlx4_master_func_num(dev))
582 continue; 585 continue;
583 mlx4_slave_event(dev, i, eqe); 586 s_info = &priv->mfunc.master.vf_oper[slave].vport[port].state;
587 if (IFLA_VF_LINK_STATE_AUTO == s_info->link_state)
588 mlx4_slave_event(dev, i, eqe);
584 } 589 }
585 else /* IB port */ 590 else /* IB port */
586 /* port-up event will be sent to a slave when the 591 /* port-up event will be sent to a slave when the
diff --git a/drivers/net/ethernet/mellanox/mlx4/fw.c b/drivers/net/ethernet/mellanox/mlx4/fw.c
index 2c97901c6a6d..8873d6802c80 100644
--- a/drivers/net/ethernet/mellanox/mlx4/fw.c
+++ b/drivers/net/ethernet/mellanox/mlx4/fw.c
@@ -133,7 +133,8 @@ static void dump_dev_cap_flags2(struct mlx4_dev *dev, u64 flags)
133 [4] = "Automatic MAC reassignment support", 133 [4] = "Automatic MAC reassignment support",
134 [5] = "Time stamping support", 134 [5] = "Time stamping support",
135 [6] = "VST (control vlan insertion/stripping) support", 135 [6] = "VST (control vlan insertion/stripping) support",
136 [7] = "FSM (MAC anti-spoofing) support" 136 [7] = "FSM (MAC anti-spoofing) support",
137 [8] = "Dynamic QP updates support"
137 }; 138 };
138 int i; 139 int i;
139 140
@@ -659,6 +660,8 @@ int mlx4_QUERY_DEV_CAP(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap)
659 QUERY_DEV_CAP_MAX_COUNTERS_OFFSET); 660 QUERY_DEV_CAP_MAX_COUNTERS_OFFSET);
660 661
661 MLX4_GET(field32, outbox, QUERY_DEV_CAP_EXT_2_FLAGS_OFFSET); 662 MLX4_GET(field32, outbox, QUERY_DEV_CAP_EXT_2_FLAGS_OFFSET);
663 if (field32 & (1 << 16))
664 dev_cap->flags2 |= MLX4_DEV_CAP_FLAG2_UPDATE_QP;
662 if (field32 & (1 << 26)) 665 if (field32 & (1 << 26))
663 dev_cap->flags2 |= MLX4_DEV_CAP_FLAG2_VLAN_CONTROL; 666 dev_cap->flags2 |= MLX4_DEV_CAP_FLAG2_VLAN_CONTROL;
664 if (field32 & (1 << 20)) 667 if (field32 & (1 << 20))
@@ -830,8 +833,10 @@ int mlx4_QUERY_PORT_wrapper(struct mlx4_dev *dev, int slave,
830 u8 port_type; 833 u8 port_type;
831 u16 short_field; 834 u16 short_field;
832 int err; 835 int err;
836 int admin_link_state;
833 837
834#define MLX4_VF_PORT_NO_LINK_SENSE_MASK 0xE0 838#define MLX4_VF_PORT_NO_LINK_SENSE_MASK 0xE0
839#define MLX4_PORT_LINK_UP_MASK 0x80
835#define QUERY_PORT_CUR_MAX_PKEY_OFFSET 0x0c 840#define QUERY_PORT_CUR_MAX_PKEY_OFFSET 0x0c
836#define QUERY_PORT_CUR_MAX_GID_OFFSET 0x0e 841#define QUERY_PORT_CUR_MAX_GID_OFFSET 0x0e
837 842
@@ -861,6 +866,12 @@ int mlx4_QUERY_PORT_wrapper(struct mlx4_dev *dev, int slave,
861 /* set port type to currently operating port type */ 866 /* set port type to currently operating port type */
862 port_type |= (dev->caps.port_type[vhcr->in_modifier] & 0x3); 867 port_type |= (dev->caps.port_type[vhcr->in_modifier] & 0x3);
863 868
869 admin_link_state = priv->mfunc.master.vf_oper[slave].vport[vhcr->in_modifier].state.link_state;
870 if (IFLA_VF_LINK_STATE_ENABLE == admin_link_state)
871 port_type |= MLX4_PORT_LINK_UP_MASK;
872 else if (IFLA_VF_LINK_STATE_DISABLE == admin_link_state)
873 port_type &= ~MLX4_PORT_LINK_UP_MASK;
874
864 MLX4_PUT(outbox->buf, port_type, 875 MLX4_PUT(outbox->buf, port_type,
865 QUERY_PORT_SUPPORTED_TYPE_OFFSET); 876 QUERY_PORT_SUPPORTED_TYPE_OFFSET);
866 877
diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index 264ddeb846a3..e85af922dcdc 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -842,11 +842,11 @@ static ssize_t set_port_ib_mtu(struct device *dev,
842 return -EINVAL; 842 return -EINVAL;
843 } 843 }
844 844
845 err = sscanf(buf, "%d", &mtu); 845 err = kstrtoint(buf, 0, &mtu);
846 if (err > 0) 846 if (!err)
847 ibta_mtu = int_to_ibta_mtu(mtu); 847 ibta_mtu = int_to_ibta_mtu(mtu);
848 848
849 if (err <= 0 || ibta_mtu < 0) { 849 if (err || ibta_mtu < 0) {
850 mlx4_err(mdev, "%s is invalid IBTA mtu\n", buf); 850 mlx4_err(mdev, "%s is invalid IBTA mtu\n", buf);
851 return -EINVAL; 851 return -EINVAL;
852 } 852 }
@@ -2080,6 +2080,11 @@ static int __mlx4_init_one(struct pci_dev *pdev, int pci_dev_data)
2080 num_vfs, MLX4_MAX_NUM_VF); 2080 num_vfs, MLX4_MAX_NUM_VF);
2081 return -EINVAL; 2081 return -EINVAL;
2082 } 2082 }
2083
2084 if (num_vfs < 0) {
2085 pr_err("num_vfs module parameter cannot be negative\n");
2086 return -EINVAL;
2087 }
2083 /* 2088 /*
2084 * Check for BARs. 2089 * Check for BARs.
2085 */ 2090 */
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4.h b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
index df15bb6631cc..17d9277e33ef 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
@@ -482,6 +482,7 @@ struct mlx4_vport_state {
482 u8 default_qos; 482 u8 default_qos;
483 u32 tx_rate; 483 u32 tx_rate;
484 bool spoofchk; 484 bool spoofchk;
485 u32 link_state;
485}; 486};
486 487
487struct mlx4_vf_admin_state { 488struct mlx4_vf_admin_state {
@@ -570,6 +571,25 @@ struct mlx4_cmd {
570 u8 comm_toggle; 571 u8 comm_toggle;
571}; 572};
572 573
574enum {
575 MLX4_VF_IMMED_VLAN_FLAG_VLAN = 1 << 0,
576 MLX4_VF_IMMED_VLAN_FLAG_QOS = 1 << 1,
577 MLX4_VF_IMMED_VLAN_FLAG_LINK_DISABLE = 1 << 2,
578};
579struct mlx4_vf_immed_vlan_work {
580 struct work_struct work;
581 struct mlx4_priv *priv;
582 int flags;
583 int slave;
584 int vlan_ix;
585 int orig_vlan_ix;
586 u8 port;
587 u8 qos;
588 u16 vlan_id;
589 u16 orig_vlan_id;
590};
591
592
573struct mlx4_uar_table { 593struct mlx4_uar_table {
574 struct mlx4_bitmap bitmap; 594 struct mlx4_bitmap bitmap;
575}; 595};
@@ -1217,4 +1237,6 @@ static inline spinlock_t *mlx4_tlock(struct mlx4_dev *dev)
1217 1237
1218#define NOT_MASKED_PD_BITS 17 1238#define NOT_MASKED_PD_BITS 17
1219 1239
1240void mlx4_vf_immed_vlan_work_handler(struct work_struct *_work);
1241
1220#endif /* MLX4_H */ 1242#endif /* MLX4_H */
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index b1d7657b2bf5..35fb60e2320c 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
@@ -96,13 +96,14 @@
96 96
97/* Use the maximum between 16384 and a single page */ 97/* Use the maximum between 16384 and a single page */
98#define MLX4_EN_ALLOC_SIZE PAGE_ALIGN(16384) 98#define MLX4_EN_ALLOC_SIZE PAGE_ALIGN(16384)
99#define MLX4_EN_ALLOC_ORDER get_order(MLX4_EN_ALLOC_SIZE)
100 99
101/* Receive fragment sizes; we use at most 4 fragments (for 9600 byte MTU 100#define MLX4_EN_ALLOC_PREFER_ORDER PAGE_ALLOC_COSTLY_ORDER
101
102/* Receive fragment sizes; we use at most 3 fragments (for 9600 byte MTU
102 * and 4K allocations) */ 103 * and 4K allocations) */
103enum { 104enum {
104 FRAG_SZ0 = 512 - NET_IP_ALIGN, 105 FRAG_SZ0 = 1536 - NET_IP_ALIGN,
105 FRAG_SZ1 = 1024, 106 FRAG_SZ1 = 4096,
106 FRAG_SZ2 = 4096, 107 FRAG_SZ2 = 4096,
107 FRAG_SZ3 = MLX4_EN_ALLOC_SIZE 108 FRAG_SZ3 = MLX4_EN_ALLOC_SIZE
108}; 109};
@@ -234,9 +235,10 @@ struct mlx4_en_tx_desc {
234#define MLX4_EN_CX3_HIGH_ID 0x1005 235#define MLX4_EN_CX3_HIGH_ID 0x1005
235 236
236struct mlx4_en_rx_alloc { 237struct mlx4_en_rx_alloc {
237 struct page *page; 238 struct page *page;
238 dma_addr_t dma; 239 dma_addr_t dma;
239 u16 offset; 240 u32 offset;
241 u32 size;
240}; 242};
241 243
242struct mlx4_en_tx_ring { 244struct mlx4_en_tx_ring {
@@ -290,6 +292,11 @@ struct mlx4_en_rx_ring {
290 void *rx_info; 292 void *rx_info;
291 unsigned long bytes; 293 unsigned long bytes;
292 unsigned long packets; 294 unsigned long packets;
295#ifdef CONFIG_NET_LL_RX_POLL
296 unsigned long yields;
297 unsigned long misses;
298 unsigned long cleaned;
299#endif
293 unsigned long csum_ok; 300 unsigned long csum_ok;
294 unsigned long csum_none; 301 unsigned long csum_none;
295 int hwtstamp_rx_filter; 302 int hwtstamp_rx_filter;
@@ -310,6 +317,19 @@ struct mlx4_en_cq {
310 u16 moder_cnt; 317 u16 moder_cnt;
311 struct mlx4_cqe *buf; 318 struct mlx4_cqe *buf;
312#define MLX4_EN_OPCODE_ERROR 0x1e 319#define MLX4_EN_OPCODE_ERROR 0x1e
320
321#ifdef CONFIG_NET_LL_RX_POLL
322 unsigned int state;
323#define MLX4_EN_CQ_STATE_IDLE 0
324#define MLX4_EN_CQ_STATE_NAPI 1 /* NAPI owns this CQ */
325#define MLX4_EN_CQ_STATE_POLL 2 /* poll owns this CQ */
326#define MLX4_CQ_LOCKED (MLX4_EN_CQ_STATE_NAPI | MLX4_EN_CQ_STATE_POLL)
327#define MLX4_EN_CQ_STATE_NAPI_YIELD 4 /* NAPI yielded this CQ */
328#define MLX4_EN_CQ_STATE_POLL_YIELD 8 /* poll yielded this CQ */
329#define CQ_YIELD (MLX4_EN_CQ_STATE_NAPI_YIELD | MLX4_EN_CQ_STATE_POLL_YIELD)
330#define CQ_USER_PEND (MLX4_EN_CQ_STATE_POLL | MLX4_EN_CQ_STATE_POLL_YIELD)
331 spinlock_t poll_lock; /* protects from LLS/napi conflicts */
332#endif /* CONFIG_NET_LL_RX_POLL */
313}; 333};
314 334
315struct mlx4_en_port_profile { 335struct mlx4_en_port_profile {
@@ -421,8 +441,6 @@ struct mlx4_en_frag_info {
421 u16 frag_prefix_size; 441 u16 frag_prefix_size;
422 u16 frag_stride; 442 u16 frag_stride;
423 u16 frag_align; 443 u16 frag_align;
424 u16 last_offset;
425
426}; 444};
427 445
428#ifdef CONFIG_MLX4_EN_DCB 446#ifdef CONFIG_MLX4_EN_DCB
@@ -562,6 +580,115 @@ struct mlx4_mac_entry {
562 struct rcu_head rcu; 580 struct rcu_head rcu;
563}; 581};
564 582
583#ifdef CONFIG_NET_LL_RX_POLL
584static inline void mlx4_en_cq_init_lock(struct mlx4_en_cq *cq)
585{
586 spin_lock_init(&cq->poll_lock);
587 cq->state = MLX4_EN_CQ_STATE_IDLE;
588}
589
590/* called from the device poll rutine to get ownership of a cq */
591static inline bool mlx4_en_cq_lock_napi(struct mlx4_en_cq *cq)
592{
593 int rc = true;
594 spin_lock(&cq->poll_lock);
595 if (cq->state & MLX4_CQ_LOCKED) {
596 WARN_ON(cq->state & MLX4_EN_CQ_STATE_NAPI);
597 cq->state |= MLX4_EN_CQ_STATE_NAPI_YIELD;
598 rc = false;
599 } else
600 /* we don't care if someone yielded */
601 cq->state = MLX4_EN_CQ_STATE_NAPI;
602 spin_unlock(&cq->poll_lock);
603 return rc;
604}
605
606/* returns true is someone tried to get the cq while napi had it */
607static inline bool mlx4_en_cq_unlock_napi(struct mlx4_en_cq *cq)
608{
609 int rc = false;
610 spin_lock(&cq->poll_lock);
611 WARN_ON(cq->state & (MLX4_EN_CQ_STATE_POLL |
612 MLX4_EN_CQ_STATE_NAPI_YIELD));
613
614 if (cq->state & MLX4_EN_CQ_STATE_POLL_YIELD)
615 rc = true;
616 cq->state = MLX4_EN_CQ_STATE_IDLE;
617 spin_unlock(&cq->poll_lock);
618 return rc;
619}
620
621/* called from mlx4_en_low_latency_poll() */
622static inline bool mlx4_en_cq_lock_poll(struct mlx4_en_cq *cq)
623{
624 int rc = true;
625 spin_lock_bh(&cq->poll_lock);
626 if ((cq->state & MLX4_CQ_LOCKED)) {
627 struct net_device *dev = cq->dev;
628 struct mlx4_en_priv *priv = netdev_priv(dev);
629 struct mlx4_en_rx_ring *rx_ring = &priv->rx_ring[cq->ring];
630
631 cq->state |= MLX4_EN_CQ_STATE_POLL_YIELD;
632 rc = false;
633 rx_ring->yields++;
634 } else
635 /* preserve yield marks */
636 cq->state |= MLX4_EN_CQ_STATE_POLL;
637 spin_unlock_bh(&cq->poll_lock);
638 return rc;
639}
640
641/* returns true if someone tried to get the cq while it was locked */
642static inline bool mlx4_en_cq_unlock_poll(struct mlx4_en_cq *cq)
643{
644 int rc = false;
645 spin_lock_bh(&cq->poll_lock);
646 WARN_ON(cq->state & (MLX4_EN_CQ_STATE_NAPI));
647
648 if (cq->state & MLX4_EN_CQ_STATE_POLL_YIELD)
649 rc = true;
650 cq->state = MLX4_EN_CQ_STATE_IDLE;
651 spin_unlock_bh(&cq->poll_lock);
652 return rc;
653}
654
655/* true if a socket is polling, even if it did not get the lock */
656static inline bool mlx4_en_cq_ll_polling(struct mlx4_en_cq *cq)
657{
658 WARN_ON(!(cq->state & MLX4_CQ_LOCKED));
659 return cq->state & CQ_USER_PEND;
660}
661#else
662static inline void mlx4_en_cq_init_lock(struct mlx4_en_cq *cq)
663{
664}
665
666static inline bool mlx4_en_cq_lock_napi(struct mlx4_en_cq *cq)
667{
668 return true;
669}
670
671static inline bool mlx4_en_cq_unlock_napi(struct mlx4_en_cq *cq)
672{
673 return false;
674}
675
676static inline bool mlx4_en_cq_lock_poll(struct mlx4_en_cq *cq)
677{
678 return false;
679}
680
681static inline bool mlx4_en_cq_unlock_poll(struct mlx4_en_cq *cq)
682{
683 return false;
684}
685
686static inline bool mlx4_en_cq_ll_polling(struct mlx4_en_cq *cq)
687{
688 return false;
689}
690#endif /* CONFIG_NET_LL_RX_POLL */
691
565#define MLX4_EN_WOL_DO_MODIFY (1ULL << 63) 692#define MLX4_EN_WOL_DO_MODIFY (1ULL << 63)
566 693
567void mlx4_en_update_loopback_state(struct net_device *dev, 694void mlx4_en_update_loopback_state(struct net_device *dev,
diff --git a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
index 1157f028a90f..f984a89c27df 100644
--- a/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
+++ b/drivers/net/ethernet/mellanox/mlx4/resource_tracker.c
@@ -101,6 +101,8 @@ struct res_qp {
101 spinlock_t mcg_spl; 101 spinlock_t mcg_spl;
102 int local_qpn; 102 int local_qpn;
103 atomic_t ref_count; 103 atomic_t ref_count;
104 u32 qpc_flags;
105 u8 sched_queue;
104}; 106};
105 107
106enum res_mtt_states { 108enum res_mtt_states {
@@ -355,7 +357,7 @@ static void update_gid(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *inbox,
355 357
356static int update_vport_qp_param(struct mlx4_dev *dev, 358static int update_vport_qp_param(struct mlx4_dev *dev,
357 struct mlx4_cmd_mailbox *inbox, 359 struct mlx4_cmd_mailbox *inbox,
358 u8 slave) 360 u8 slave, u32 qpn)
359{ 361{
360 struct mlx4_qp_context *qpc = inbox->buf + 8; 362 struct mlx4_qp_context *qpc = inbox->buf + 8;
361 struct mlx4_vport_oper_state *vp_oper; 363 struct mlx4_vport_oper_state *vp_oper;
@@ -369,12 +371,30 @@ static int update_vport_qp_param(struct mlx4_dev *dev,
369 371
370 if (MLX4_VGT != vp_oper->state.default_vlan) { 372 if (MLX4_VGT != vp_oper->state.default_vlan) {
371 qp_type = (be32_to_cpu(qpc->flags) >> 16) & 0xff; 373 qp_type = (be32_to_cpu(qpc->flags) >> 16) & 0xff;
372 if (MLX4_QP_ST_RC == qp_type) 374 if (MLX4_QP_ST_RC == qp_type ||
375 (MLX4_QP_ST_UD == qp_type &&
376 !mlx4_is_qp_reserved(dev, qpn)))
373 return -EINVAL; 377 return -EINVAL;
374 378
379 /* the reserved QPs (special, proxy, tunnel)
380 * do not operate over vlans
381 */
382 if (mlx4_is_qp_reserved(dev, qpn))
383 return 0;
384
375 /* force strip vlan by clear vsd */ 385 /* force strip vlan by clear vsd */
376 qpc->param3 &= ~cpu_to_be32(MLX4_STRIP_VLAN); 386 qpc->param3 &= ~cpu_to_be32(MLX4_STRIP_VLAN);
377 if (0 != vp_oper->state.default_vlan) { 387
388 if (vp_oper->state.link_state == IFLA_VF_LINK_STATE_DISABLE &&
389 dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_UPDATE_QP) {
390 qpc->pri_path.vlan_control =
391 MLX4_VLAN_CTRL_ETH_TX_BLOCK_TAGGED |
392 MLX4_VLAN_CTRL_ETH_TX_BLOCK_PRIO_TAGGED |
393 MLX4_VLAN_CTRL_ETH_TX_BLOCK_UNTAGGED |
394 MLX4_VLAN_CTRL_ETH_RX_BLOCK_PRIO_TAGGED |
395 MLX4_VLAN_CTRL_ETH_RX_BLOCK_UNTAGGED |
396 MLX4_VLAN_CTRL_ETH_RX_BLOCK_TAGGED;
397 } else if (0 != vp_oper->state.default_vlan) {
378 qpc->pri_path.vlan_control = 398 qpc->pri_path.vlan_control =
379 MLX4_VLAN_CTRL_ETH_TX_BLOCK_TAGGED | 399 MLX4_VLAN_CTRL_ETH_TX_BLOCK_TAGGED |
380 MLX4_VLAN_CTRL_ETH_RX_BLOCK_PRIO_TAGGED | 400 MLX4_VLAN_CTRL_ETH_RX_BLOCK_PRIO_TAGGED |
@@ -2114,6 +2134,8 @@ int mlx4_RST2INIT_QP_wrapper(struct mlx4_dev *dev, int slave,
2114 if (err) 2134 if (err)
2115 return err; 2135 return err;
2116 qp->local_qpn = local_qpn; 2136 qp->local_qpn = local_qpn;
2137 qp->sched_queue = 0;
2138 qp->qpc_flags = be32_to_cpu(qpc->flags);
2117 2139
2118 err = get_res(dev, slave, mtt_base, RES_MTT, &mtt); 2140 err = get_res(dev, slave, mtt_base, RES_MTT, &mtt);
2119 if (err) 2141 if (err)
@@ -2836,6 +2858,9 @@ int mlx4_INIT2RTR_QP_wrapper(struct mlx4_dev *dev, int slave,
2836{ 2858{
2837 int err; 2859 int err;
2838 struct mlx4_qp_context *qpc = inbox->buf + 8; 2860 struct mlx4_qp_context *qpc = inbox->buf + 8;
2861 int qpn = vhcr->in_modifier & 0x7fffff;
2862 struct res_qp *qp;
2863 u8 orig_sched_queue;
2839 2864
2840 err = verify_qp_parameters(dev, inbox, QP_TRANS_INIT2RTR, slave); 2865 err = verify_qp_parameters(dev, inbox, QP_TRANS_INIT2RTR, slave);
2841 if (err) 2866 if (err)
@@ -2844,11 +2869,30 @@ int mlx4_INIT2RTR_QP_wrapper(struct mlx4_dev *dev, int slave,
2844 update_pkey_index(dev, slave, inbox); 2869 update_pkey_index(dev, slave, inbox);
2845 update_gid(dev, inbox, (u8)slave); 2870 update_gid(dev, inbox, (u8)slave);
2846 adjust_proxy_tun_qkey(dev, vhcr, qpc); 2871 adjust_proxy_tun_qkey(dev, vhcr, qpc);
2847 err = update_vport_qp_param(dev, inbox, slave); 2872 orig_sched_queue = qpc->pri_path.sched_queue;
2873 err = update_vport_qp_param(dev, inbox, slave, qpn);
2848 if (err) 2874 if (err)
2849 return err; 2875 return err;
2850 2876
2851 return mlx4_GEN_QP_wrapper(dev, slave, vhcr, inbox, outbox, cmd); 2877 err = get_res(dev, slave, qpn, RES_QP, &qp);
2878 if (err)
2879 return err;
2880 if (qp->com.from_state != RES_QP_HW) {
2881 err = -EBUSY;
2882 goto out;
2883 }
2884
2885 err = mlx4_DMA_wrapper(dev, slave, vhcr, inbox, outbox, cmd);
2886out:
2887 /* if no error, save sched queue value passed in by VF. This is
2888 * essentially the QOS value provided by the VF. This will be useful
2889 * if we allow dynamic changes from VST back to VGT
2890 */
2891 if (!err)
2892 qp->sched_queue = orig_sched_queue;
2893
2894 put_res(dev, slave, qpn, RES_QP);
2895 return err;
2852} 2896}
2853 2897
2854int mlx4_RTR2RTS_QP_wrapper(struct mlx4_dev *dev, int slave, 2898int mlx4_RTR2RTS_QP_wrapper(struct mlx4_dev *dev, int slave,
@@ -3932,3 +3976,112 @@ void mlx4_delete_all_resources_for_slave(struct mlx4_dev *dev, int slave)
3932 rem_slave_xrcdns(dev, slave); 3976 rem_slave_xrcdns(dev, slave);
3933 mutex_unlock(&priv->mfunc.master.res_tracker.slave_list[slave].mutex); 3977 mutex_unlock(&priv->mfunc.master.res_tracker.slave_list[slave].mutex);
3934} 3978}
3979
3980void mlx4_vf_immed_vlan_work_handler(struct work_struct *_work)
3981{
3982 struct mlx4_vf_immed_vlan_work *work =
3983 container_of(_work, struct mlx4_vf_immed_vlan_work, work);
3984 struct mlx4_cmd_mailbox *mailbox;
3985 struct mlx4_update_qp_context *upd_context;
3986 struct mlx4_dev *dev = &work->priv->dev;
3987 struct mlx4_resource_tracker *tracker =
3988 &work->priv->mfunc.master.res_tracker;
3989 struct list_head *qp_list =
3990 &tracker->slave_list[work->slave].res_list[RES_QP];
3991 struct res_qp *qp;
3992 struct res_qp *tmp;
3993 u64 qp_mask = ((1ULL << MLX4_UPD_QP_PATH_MASK_ETH_TX_BLOCK_UNTAGGED) |
3994 (1ULL << MLX4_UPD_QP_PATH_MASK_ETH_TX_BLOCK_1P) |
3995 (1ULL << MLX4_UPD_QP_PATH_MASK_ETH_TX_BLOCK_TAGGED) |
3996 (1ULL << MLX4_UPD_QP_PATH_MASK_ETH_RX_BLOCK_UNTAGGED) |
3997 (1ULL << MLX4_UPD_QP_PATH_MASK_ETH_RX_BLOCK_1P) |
3998 (1ULL << MLX4_UPD_QP_PATH_MASK_ETH_RX_BLOCK_TAGGED) |
3999 (1ULL << MLX4_UPD_QP_PATH_MASK_VLAN_INDEX) |
4000 (1ULL << MLX4_UPD_QP_PATH_MASK_SCHED_QUEUE));
4001
4002 int err;
4003 int port, errors = 0;
4004 u8 vlan_control;
4005
4006 if (mlx4_is_slave(dev)) {
4007 mlx4_warn(dev, "Trying to update-qp in slave %d\n",
4008 work->slave);
4009 goto out;
4010 }
4011
4012 mailbox = mlx4_alloc_cmd_mailbox(dev);
4013 if (IS_ERR(mailbox))
4014 goto out;
4015 if (work->flags & MLX4_VF_IMMED_VLAN_FLAG_LINK_DISABLE) /* block all */
4016 vlan_control = MLX4_VLAN_CTRL_ETH_TX_BLOCK_TAGGED |
4017 MLX4_VLAN_CTRL_ETH_TX_BLOCK_PRIO_TAGGED |
4018 MLX4_VLAN_CTRL_ETH_TX_BLOCK_UNTAGGED |
4019 MLX4_VLAN_CTRL_ETH_RX_BLOCK_PRIO_TAGGED |
4020 MLX4_VLAN_CTRL_ETH_RX_BLOCK_UNTAGGED |
4021 MLX4_VLAN_CTRL_ETH_RX_BLOCK_TAGGED;
4022 else if (!work->vlan_id)
4023 vlan_control = MLX4_VLAN_CTRL_ETH_TX_BLOCK_TAGGED |
4024 MLX4_VLAN_CTRL_ETH_RX_BLOCK_TAGGED;
4025 else
4026 vlan_control = MLX4_VLAN_CTRL_ETH_TX_BLOCK_TAGGED |
4027 MLX4_VLAN_CTRL_ETH_RX_BLOCK_PRIO_TAGGED |
4028 MLX4_VLAN_CTRL_ETH_RX_BLOCK_UNTAGGED;
4029
4030 upd_context = mailbox->buf;
4031 upd_context->primary_addr_path_mask = cpu_to_be64(qp_mask);
4032 upd_context->qp_context.pri_path.vlan_control = vlan_control;
4033 upd_context->qp_context.pri_path.vlan_index = work->vlan_ix;
4034
4035 spin_lock_irq(mlx4_tlock(dev));
4036 list_for_each_entry_safe(qp, tmp, qp_list, com.list) {
4037 spin_unlock_irq(mlx4_tlock(dev));
4038 if (qp->com.owner == work->slave) {
4039 if (qp->com.from_state != RES_QP_HW ||
4040 !qp->sched_queue || /* no INIT2RTR trans yet */
4041 mlx4_is_qp_reserved(dev, qp->local_qpn) ||
4042 qp->qpc_flags & (1 << MLX4_RSS_QPC_FLAG_OFFSET)) {
4043 spin_lock_irq(mlx4_tlock(dev));
4044 continue;
4045 }
4046 port = (qp->sched_queue >> 6 & 1) + 1;
4047 if (port != work->port) {
4048 spin_lock_irq(mlx4_tlock(dev));
4049 continue;
4050 }
4051 upd_context->qp_context.pri_path.sched_queue =
4052 qp->sched_queue & 0xC7;
4053 upd_context->qp_context.pri_path.sched_queue |=
4054 ((work->qos & 0x7) << 3);
4055
4056 err = mlx4_cmd(dev, mailbox->dma,
4057 qp->local_qpn & 0xffffff,
4058 0, MLX4_CMD_UPDATE_QP,
4059 MLX4_CMD_TIME_CLASS_C, MLX4_CMD_NATIVE);
4060 if (err) {
4061 mlx4_info(dev, "UPDATE_QP failed for slave %d, "
4062 "port %d, qpn %d (%d)\n",
4063 work->slave, port, qp->local_qpn,
4064 err);
4065 errors++;
4066 }
4067 }
4068 spin_lock_irq(mlx4_tlock(dev));
4069 }
4070 spin_unlock_irq(mlx4_tlock(dev));
4071 mlx4_free_cmd_mailbox(dev, mailbox);
4072
4073 if (errors)
4074 mlx4_err(dev, "%d UPDATE_QP failures for slave %d, port %d\n",
4075 errors, work->slave, work->port);
4076
4077 /* unregister previous vlan_id if needed and we had no errors
4078 * while updating the QPs
4079 */
4080 if (work->flags & MLX4_VF_IMMED_VLAN_FLAG_VLAN && !errors &&
4081 NO_INDX != work->orig_vlan_ix)
4082 __mlx4_unregister_vlan(&work->priv->dev, work->port,
4083 work->orig_vlan_ix);
4084out:
4085 kfree(work);
4086 return;
4087}
generated by cgit v1.2.2 (git 2.25.0) at 2024-10-13 07:27:57 -0400