[PATCH] fix RLIM_NOFILE handling

* dup2() should return -EBADF on exceeded sysctl_nr_open
* dup() should *not* return -EINVAL even if you have rlimit set to 0;
  it should get -EMFILE instead.

Check for orig_start exceeding rlimit taken to sys_fcntl().
Failing expand_files() in dup{2,3}() now gets -EMFILE remapped to -EBADF.
Consequently, remaining checks for rlimit are taken to expand_files().

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>

1 files changed, 9 insertions, 0 deletions

diff --git a/fs/file.c b/fs/file.c
index 7b3887e054d0..d8773b19fe47 100644
--- a/fs/file.c
+++ b/fs/file.c
@@ -250,9 +250,18 @@ int expand_files(struct files_struct *files, int nr)
         struct fdtable *fdt;
 
         fdt = files_fdtable(files);
+
+        /*
+         * N.B. For clone tasks sharing a files structure, this test
+         * will limit the total number of files that can be opened.
+         */
+        if (nr >= current->signal->rlim[RLIMIT_NOFILE].rlim_cur)
+                return -EMFILE;
+
         /* Do we need to expand? */
         if (nr < fdt->max_fds)
                 return 0;
+
         /* Can we expand? */
         if (nr >= sysctl_nr_open)
                 return -EMFILE;