aboutsummaryrefslogtreecommitdiffstats
path: root/drivers/char/pty.c
diff options
context:
space:
mode:
authorJason Baron <jbaron@redhat.com>2005-09-09 16:01:57 -0400
committerLinus Torvalds <torvalds@g5.osdl.org>2005-09-09 16:57:31 -0400
commitff55fe2075e3901db4dbdc00e0b44a71bef97afd (patch)
treec6dfc8ba5d04fda4c5cb15cdebd5425ab674f8df /drivers/char/pty.c
parent69ac59647e66c1b53fb98fe8b6d0f2099cffad60 (diff)
[PATCH] pty_chars_in_buffer oops fix
The idea of this patch is to lock both sides of a ptmx/pty pair during line discipline changing. This is needed to ensure that say a poll on one side of the pty doesn't occur while the line discipline is actively being changed. This resulted in an oops reported on lkml, see: http://marc.theaimsgroup.com/?l=linux-kernel&m=111342171410005&w=2 A 'hacky' approach was previously implmemented which served to eliminate the poll vs. line discipline changing race. However, this patch takes a more general approach to the issue. The patch only adds locking on a less often used path, the line-discipline changing path, as opposed to locking the ptmx/pty pair on read/write/poll paths. The patch below, takes both ldisc locks in either order b/c the locks are both taken under the same spinlock(). I thought about locking the ptmx/pty separately, such as master always first but that introduces a 3 way deadlock. For example, process 1 does a blocking read on the slave side. Then, process 2 does an ldisc change on the slave side, which acquires the master ldisc lock but not the slave's. Finally, process 3 does a write which blocks on the process 2's ldisc reference. This patch does introduce some changes in semantics. For example, a line discipline change on side 'a' of a ptmx/pty pair, will now wait for a read/write to complete on the other side, or side 'b'. The current behavior is to simply wait for any read/writes on only side 'a', not both sides 'a' and 'b'. I think this behavior makes sense, but I wanted to point it out. I've tested the patch with a bunch of read/write/poll while changing the line discipline out from underneath. This patch obviates the need for the above "hide the problem" patch. Signed-off-by: Jason Baron <jbaron@redhat.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Diffstat (limited to 'drivers/char/pty.c')
-rw-r--r--drivers/char/pty.c5
1 files changed, 2 insertions, 3 deletions
diff --git a/drivers/char/pty.c b/drivers/char/pty.c
index da32889d22c0..49f3997fd251 100644
--- a/drivers/char/pty.c
+++ b/drivers/char/pty.c
@@ -149,15 +149,14 @@ static int pty_write_room(struct tty_struct *tty)
149static int pty_chars_in_buffer(struct tty_struct *tty) 149static int pty_chars_in_buffer(struct tty_struct *tty)
150{ 150{
151 struct tty_struct *to = tty->link; 151 struct tty_struct *to = tty->link;
152 ssize_t (*chars_in_buffer)(struct tty_struct *);
153 int count; 152 int count;
154 153
155 /* We should get the line discipline lock for "tty->link" */ 154 /* We should get the line discipline lock for "tty->link" */
156 if (!to || !(chars_in_buffer = to->ldisc.chars_in_buffer)) 155 if (!to || !to->ldisc.chars_in_buffer)
157 return 0; 156 return 0;
158 157
159 /* The ldisc must report 0 if no characters available to be read */ 158 /* The ldisc must report 0 if no characters available to be read */
160 count = chars_in_buffer(to); 159 count = to->ldisc.chars_in_buffer(to);
161 160
162 if (tty->driver->subtype == PTY_TYPE_SLAVE) return count; 161 if (tty->driver->subtype == PTY_TYPE_SLAVE) return count;
163 162