diff options
author | Linus Torvalds <torvalds@linux-foundation.org> | 2008-07-15 13:38:46 -0400 |
---|---|---|
committer | Linus Torvalds <torvalds@linux-foundation.org> | 2008-07-15 13:38:46 -0400 |
commit | 38c46578ffd8ffbfec514c2a9876d527303322d6 (patch) | |
tree | d55ad8a52048f70bc6ef51d388c0f21ff467155e /Documentation/filesystems | |
parent | e7849f16c13476288fe4fbd420975e8456c75aa0 (diff) | |
parent | 4abaca17e758e3326c96ced88b2cd9b7b84922f6 (diff) |
Merge git://git.kernel.org/pub/scm/linux/kernel/git/steve/gfs2-2.6-nmw
* git://git.kernel.org/pub/scm/linux/kernel/git/steve/gfs2-2.6-nmw:
[GFS2] Fix GFS2's use of do_div() in its quota calculations
[GFS2] Remove unused declaration
[GFS2] Remove support for unused and pointless flag
[GFS2] Replace rgrp "recent list" with mru list
[GFS2] Allow local DF locks when holding a cached EX glock
[GFS2] Fix delayed demote race
[GFS2] don't call permission()
[GFS2] Fix module building
[GFS2] Glock documentation
[GFS2] Remove all_list from lock_dlm
[GFS2] Remove obsolete conversion deadlock avoidance code
[GFS2] Remove remote lock dropping code
[GFS2] kernel panic mounting volume
[GFS2] Revise readpage locking
[GFS2] Fix ordering of args for list_add
[GFS2] trivial sparse lock annotations
[GFS2] No lock_nolock
[GFS2] Fix ordering bug in lock_dlm
[GFS2] Clean up the glock core
Diffstat (limited to 'Documentation/filesystems')
-rw-r--r-- | Documentation/filesystems/gfs2-glocks.txt | 114 |
1 files changed, 114 insertions, 0 deletions
diff --git a/Documentation/filesystems/gfs2-glocks.txt b/Documentation/filesystems/gfs2-glocks.txt new file mode 100644 index 000000000000..4dae9a3840bf --- /dev/null +++ b/Documentation/filesystems/gfs2-glocks.txt | |||
@@ -0,0 +1,114 @@ | |||
1 | Glock internal locking rules | ||
2 | ------------------------------ | ||
3 | |||
4 | This documents the basic principles of the glock state machine | ||
5 | internals. Each glock (struct gfs2_glock in fs/gfs2/incore.h) | ||
6 | has two main (internal) locks: | ||
7 | |||
8 | 1. A spinlock (gl_spin) which protects the internal state such | ||
9 | as gl_state, gl_target and the list of holders (gl_holders) | ||
10 | 2. A non-blocking bit lock, GLF_LOCK, which is used to prevent other | ||
11 | threads from making calls to the DLM, etc. at the same time. If a | ||
12 | thread takes this lock, it must then call run_queue (usually via the | ||
13 | workqueue) when it releases it in order to ensure any pending tasks | ||
14 | are completed. | ||
15 | |||
16 | The gl_holders list contains all the queued lock requests (not | ||
17 | just the holders) associated with the glock. If there are any | ||
18 | held locks, then they will be contiguous entries at the head | ||
19 | of the list. Locks are granted in strictly the order that they | ||
20 | are queued, except for those marked LM_FLAG_PRIORITY which are | ||
21 | used only during recovery, and even then only for journal locks. | ||
22 | |||
23 | There are three lock states that users of the glock layer can request, | ||
24 | namely shared (SH), deferred (DF) and exclusive (EX). Those translate | ||
25 | to the following DLM lock modes: | ||
26 | |||
27 | Glock mode | DLM lock mode | ||
28 | ------------------------------ | ||
29 | UN | IV/NL Unlocked (no DLM lock associated with glock) or NL | ||
30 | SH | PR (Protected read) | ||
31 | DF | CW (Concurrent write) | ||
32 | EX | EX (Exclusive) | ||
33 | |||
34 | Thus DF is basically a shared mode which is incompatible with the "normal" | ||
35 | shared lock mode, SH. In GFS2 the DF mode is used exclusively for direct I/O | ||
36 | operations. The glocks are basically a lock plus some routines which deal | ||
37 | with cache management. The following rules apply for the cache: | ||
38 | |||
39 | Glock mode | Cache data | Cache Metadata | Dirty Data | Dirty Metadata | ||
40 | -------------------------------------------------------------------------- | ||
41 | UN | No | No | No | No | ||
42 | SH | Yes | Yes | No | No | ||
43 | DF | No | Yes | No | No | ||
44 | EX | Yes | Yes | Yes | Yes | ||
45 | |||
46 | These rules are implemented using the various glock operations which | ||
47 | are defined for each type of glock. Not all types of glocks use | ||
48 | all the modes. Only inode glocks use the DF mode for example. | ||
49 | |||
50 | Table of glock operations and per type constants: | ||
51 | |||
52 | Field | Purpose | ||
53 | ---------------------------------------------------------------------------- | ||
54 | go_xmote_th | Called before remote state change (e.g. to sync dirty data) | ||
55 | go_xmote_bh | Called after remote state change (e.g. to refill cache) | ||
56 | go_inval | Called if remote state change requires invalidating the cache | ||
57 | go_demote_ok | Returns boolean value of whether its ok to demote a glock | ||
58 | | (e.g. checks timeout, and that there is no cached data) | ||
59 | go_lock | Called for the first local holder of a lock | ||
60 | go_unlock | Called on the final local unlock of a lock | ||
61 | go_dump | Called to print content of object for debugfs file, or on | ||
62 | | error to dump glock to the log. | ||
63 | go_type; | The type of the glock, LM_TYPE_..... | ||
64 | go_min_hold_time | The minimum hold time | ||
65 | |||
66 | The minimum hold time for each lock is the time after a remote lock | ||
67 | grant for which we ignore remote demote requests. This is in order to | ||
68 | prevent a situation where locks are being bounced around the cluster | ||
69 | from node to node with none of the nodes making any progress. This | ||
70 | tends to show up most with shared mmaped files which are being written | ||
71 | to by multiple nodes. By delaying the demotion in response to a | ||
72 | remote callback, that gives the userspace program time to make | ||
73 | some progress before the pages are unmapped. | ||
74 | |||
75 | There is a plan to try and remove the go_lock and go_unlock callbacks | ||
76 | if possible, in order to try and speed up the fast path though the locking. | ||
77 | Also, eventually we hope to make the glock "EX" mode locally shared | ||
78 | such that any local locking will be done with the i_mutex as required | ||
79 | rather than via the glock. | ||
80 | |||
81 | Locking rules for glock operations: | ||
82 | |||
83 | Operation | GLF_LOCK bit lock held | gl_spin spinlock held | ||
84 | ----------------------------------------------------------------- | ||
85 | go_xmote_th | Yes | No | ||
86 | go_xmote_bh | Yes | No | ||
87 | go_inval | Yes | No | ||
88 | go_demote_ok | Sometimes | Yes | ||
89 | go_lock | Yes | No | ||
90 | go_unlock | Yes | No | ||
91 | go_dump | Sometimes | Yes | ||
92 | |||
93 | N.B. Operations must not drop either the bit lock or the spinlock | ||
94 | if its held on entry. go_dump and do_demote_ok must never block. | ||
95 | Note that go_dump will only be called if the glock's state | ||
96 | indicates that it is caching uptodate data. | ||
97 | |||
98 | Glock locking order within GFS2: | ||
99 | |||
100 | 1. i_mutex (if required) | ||
101 | 2. Rename glock (for rename only) | ||
102 | 3. Inode glock(s) | ||
103 | (Parents before children, inodes at "same level" with same parent in | ||
104 | lock number order) | ||
105 | 4. Rgrp glock(s) (for (de)allocation operations) | ||
106 | 5. Transaction glock (via gfs2_trans_begin) for non-read operations | ||
107 | 6. Page lock (always last, very important!) | ||
108 | |||
109 | There are two glocks per inode. One deals with access to the inode | ||
110 | itself (locking order as above), and the other, known as the iopen | ||
111 | glock is used in conjunction with the i_nlink field in the inode to | ||
112 | determine the lifetime of the inode in question. Locking of inodes | ||
113 | is on a per-inode basis. Locking of rgrps is on a per rgrp basis. | ||
114 | |||