1 files changed, 284 insertions, 0 deletions

diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
new file mode 100644
index 000000000000..1b9bcd1fe98b
--- /dev/null
+++ b/Documentation/vm/hugetlbpage.txt
@@ -0,0 +1,284 @@
+
+The intent of this file is to give a brief summary of hugetlbpage support in
+the Linux kernel.  This support is built on top of multiple page size support
+that is provided by most modern architectures.  For example, i386
+architecture supports 4K and 4M (2M in PAE mode) page sizes, ia64
+architecture supports multiple page sizes 4K, 8K, 64K, 256K, 1M, 4M, 16M,
+256M and ppc64 supports 4K and 16M.  A TLB is a cache of virtual-to-physical
+translations.  Typically this is a very scarce resource on processor.
+Operating systems try to make best use of limited number of TLB resources.
+This optimization is more critical now as bigger and bigger physical memories
+(several GBs) are more readily available.
+
+Users can use the huge page support in Linux kernel by either using the mmap
+system call or standard SYSv shared memory system calls (shmget, shmat).
+
+First the Linux kernel needs to be built with CONFIG_HUGETLB_PAGE (present
+under Processor types and feature)  and CONFIG_HUGETLBFS (present under file
+system option on config menu) config options.
+
+The kernel built with hugepage support should show the number of configured
+hugepages in the system by running the "cat /proc/meminfo" command.  
+
+/proc/meminfo also provides information about the total number of hugetlb
+pages configured in the kernel.  It also displays information about the
+number of free hugetlb pages at any time.  It also displays information about
+the configured hugepage size - this is needed for generating the proper
+alignment and size of the arguments to the above system calls.
+
+The output of "cat /proc/meminfo" will have output like:
+
+.....
+HugePages_Total: xxx
+HugePages_Free:  yyy
+Hugepagesize:    zzz KB
+
+/proc/filesystems should also show a filesystem of type "hugetlbfs" configured
+in the kernel.
+
+/proc/sys/vm/nr_hugepages indicates the current number of configured hugetlb
+pages in the kernel.  Super user can dynamically request more (or free some
+pre-configured) hugepages. 
+The allocation( or deallocation) of hugetlb pages is posible only if there are
+enough physically contiguous free pages in system (freeing of hugepages is
+possible only if there are enough hugetlb pages free that can be transfered 
+back to regular memory pool).
+
+Pages that are used as hugetlb pages are reserved inside the kernel and can
+not be used for other purposes. 
+
+Once the kernel with Hugetlb page support is built and running, a user can
+use either the mmap system call or shared memory system calls to start using
+the huge pages.  It is required that the system administrator preallocate
+enough memory for huge page purposes.  
+
+Use the following command to dynamically allocate/deallocate hugepages:
+
+        echo 20 > /proc/sys/vm/nr_hugepages
+
+This command will try to configure 20 hugepages in the system.  The success
+or failure of allocation depends on the amount of physically contiguous
+memory that is preset in system at this time.  System administrators may want
+to put this command in one of the local rc init file.  This will enable the
+kernel to request huge pages early in the boot process (when the possibility
+of getting physical contiguous pages is still very high).
+
+If the user applications are going to request hugepages using mmap system
+call, then it is required that system administrator mount a file system of
+type hugetlbfs:
+
+        mount none /mnt/huge -t hugetlbfs <uid=value> <gid=value> <mode=value>
+                 <size=value> <nr_inodes=value>
+
+This command mounts a (pseudo) filesystem of type hugetlbfs on the directory
+/mnt/huge.  Any files created on /mnt/huge uses hugepages.  The uid and gid
+options sets the owner and group of the root of the file system.  By default
+the uid and gid of the current process are taken.  The mode option sets the
+mode of root of file system to value & 0777.  This value is given in octal.
+By default the value 0755 is picked. The size option sets the maximum value of
+memory (huge pages) allowed for that filesystem (/mnt/huge). The size is
+rounded down to HPAGE_SIZE.  The option nr_inode sets the maximum number of
+inodes that /mnt/huge can use.  If the size or nr_inode options are not
+provided on command line then no limits are set.  For size and nr_inodes
+options, you can use [G|g]/[M|m]/[K|k] to represent giga/mega/kilo. For 
+example, size=2K has the same meaning as size=2048. An example is given at 
+the end of this document. 
+
+read and write system calls are not supported on files that reside on hugetlb
+file systems.
+
+A regular chown, chgrp and chmod commands (with right permissions) could be
+used to change the file attributes on hugetlbfs.
+
+Also, it is important to note that no such mount command is required if the
+applications are going to use only shmat/shmget system calls.  Users who
+wish to use hugetlb page via shared memory segment should be a member of
+a supplementary group and system admin needs to configure that gid into
+/proc/sys/vm/hugetlb_shm_group.  It is possible for same or different
+applications to use any combination of mmaps and shm* calls.  Though the
+mount of filesystem will be required for using mmaps.
+
+*******************************************************************
+
+/*
+ * Example of using hugepage memory in a user application using Sys V shared
+ * memory system calls.  In this example the app is requesting 256MB of
+ * memory that is backed by huge pages.  The application uses the flag
+ * SHM_HUGETLB in the shmget system call to inform the kernel that it is
+ * requesting hugepages.
+ *
+ * For the ia64 architecture, the Linux kernel reserves Region number 4 for
+ * hugepages.  That means the addresses starting with 0x800000... will need
+ * to be specified.  Specifying a fixed address is not required on ppc64,
+ * i386 or x86_64.
+ *
+ * Note: The default shared memory limit is quite low on many kernels,
+ * you may need to increase it via:
+ *
+ * echo 268435456 > /proc/sys/kernel/shmmax
+ *
+ * This will increase the maximum size per shared memory segment to 256MB.
+ * The other limit that you will hit eventually is shmall which is the
+ * total amount of shared memory in pages. To set it to 16GB on a system
+ * with a 4kB pagesize do:
+ *
+ * echo 4194304 > /proc/sys/kernel/shmall
+ */
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/ipc.h>
+#include <sys/shm.h>
+#include <sys/mman.h>
+
+#ifndef SHM_HUGETLB
+#define SHM_HUGETLB 04000
+#endif
+
+#define LENGTH (256UL*1024*1024)
+
+#define dprintf(x)  printf(x)
+
+/* Only ia64 requires this */
+#ifdef __ia64__
+#define ADDR (void *)(0x8000000000000000UL)
+#define SHMAT_FLAGS (SHM_RND)
+#else
+#define ADDR (void *)(0x0UL)
+#define SHMAT_FLAGS (0)
+#endif
+
+int main(void)
+{
+        int shmid;
+        unsigned long i;
+        char *shmaddr;
+
+        if ((shmid = shmget(2, LENGTH,
+                            SHM_HUGETLB | IPC_CREAT | SHM_R | SHM_W)) < 0) {
+                perror("shmget");
+                exit(1);
+        }
+        printf("shmid: 0x%x\n", shmid);
+
+        shmaddr = shmat(shmid, ADDR, SHMAT_FLAGS);
+        if (shmaddr == (char *)-1) {
+                perror("Shared memory attach failure");
+                shmctl(shmid, IPC_RMID, NULL);
+                exit(2);
+        }
+        printf("shmaddr: %p\n", shmaddr);
+
+        dprintf("Starting the writes:\n");
+        for (i = 0; i < LENGTH; i++) {
+                shmaddr[i] = (char)(i);
+                if (!(i % (1024 * 1024)))
+                        dprintf(".");
+        }
+        dprintf("\n");
+
+        dprintf("Starting the Check...");
+        for (i = 0; i < LENGTH; i++)
+                if (shmaddr[i] != (char)i)
+                        printf("\nIndex %lu mismatched\n", i);
+        dprintf("Done.\n");
+
+        if (shmdt((const void *)shmaddr) != 0) {
+                perror("Detach failure");
+                shmctl(shmid, IPC_RMID, NULL);
+                exit(3);
+        }
+
+        shmctl(shmid, IPC_RMID, NULL);
+
+        return 0;
+}
+
+*******************************************************************
+
+/*
+ * Example of using hugepage memory in a user application using the mmap
+ * system call.  Before running this application, make sure that the
+ * administrator has mounted the hugetlbfs filesystem (on some directory
+ * like /mnt) using the command mount -t hugetlbfs nodev /mnt. In this
+ * example, the app is requesting memory of size 256MB that is backed by
+ * huge pages.
+ *
+ * For ia64 architecture, Linux kernel reserves Region number 4 for hugepages.
+ * That means the addresses starting with 0x800000... will need to be
+ * specified.  Specifying a fixed address is not required on ppc64, i386
+ * or x86_64.
+ */
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+
+#define FILE_NAME "/mnt/hugepagefile"
+#define LENGTH (256UL*1024*1024)
+#define PROTECTION (PROT_READ | PROT_WRITE)
+
+/* Only ia64 requires this */
+#ifdef __ia64__
+#define ADDR (void *)(0x8000000000000000UL)
+#define FLAGS (MAP_SHARED | MAP_FIXED)
+#else
+#define ADDR (void *)(0x0UL)
+#define FLAGS (MAP_SHARED)
+#endif
+
+void check_bytes(char *addr)
+{
+        printf("First hex is %x\n", *((unsigned int *)addr));
+}
+
+void write_bytes(char *addr)
+{
+        unsigned long i;
+
+        for (i = 0; i < LENGTH; i++)
+                *(addr + i) = (char)i;
+}
+
+void read_bytes(char *addr)
+{
+        unsigned long i;
+
+        check_bytes(addr);
+        for (i = 0; i < LENGTH; i++)
+                if (*(addr + i) != (char)i) {
+                        printf("Mismatch at %lu\n", i);
+                        break;
+                }
+}
+
+int main(void)
+{
+        void *addr;
+        int fd;
+
+        fd = open(FILE_NAME, O_CREAT | O_RDWR, 0755);
+        if (fd < 0) {
+                perror("Open failed");
+                exit(1);
+        }
+
+        addr = mmap(ADDR, LENGTH, PROTECTION, FLAGS, fd, 0);
+        if (addr == MAP_FAILED) {
+                perror("mmap");
+                unlink(FILE_NAME);
+                exit(1);
+        }
+
+        printf("Returned address is %p\n", addr);
+        check_bytes(addr);
+        write_bytes(addr);
+        read_bytes(addr);
+
+        munmap(addr, LENGTH);
+        close(fd);
+        unlink(FILE_NAME);
+
+        return 0;
+}