md: make it easier to wait for bad blocks to be acknowledged.

It is only safe to choose not to write to a bad block if that bad
block is safely recorded in metadata - i.e. if it has been
'acknowledged'.

If it hasn't we need to wait for the acknowledgement.

We support that using rdev->blocked wait and
md_wait_for_blocked_rdev by introducing a new device flag
'BlockedBadBlock'.

This flag is only advisory.
It is cleared whenever we acknowledge a bad block, so that a waiter
can re-check the particular bad blocks that it is interested it.

It should be set by a caller when they find they need to wait.
This (set after test) is inherently racy, but as
md_wait_for_blocked_rdev already has a timeout, losing the race will
have minimal impact.

When we clear "Blocked" was also clear "BlockedBadBlocks" incase it
was set incorrectly (see above race).

We also modify the way we manage 'Blocked' to fit better with the new
handling of 'BlockedBadBlocks' and to make it consistent between
externally managed and internally managed metadata.   This requires
that each raidXd loop checks if the metadata needs to be written and
triggers a write (md_check_recovery) if needed.  Otherwise a queued
write request might cause raidXd to wait for the metadata to write,
and only that thread can write it.

Before writing metadata, we set FaultRecorded for all devices that
are Faulty, then after writing the metadata we clear Blocked for any
device for which the Fault was certainly Recorded.

The 'faulty' device flag now appears in sysfs if the device is faulty
*or* it has unacknowledged bad blocks.  So user-space which does not
understand bad blocks can continue to function correctly.
User space which does, should not assume a device is faulty until it
sees the 'faulty' flag, and then sees the list of unacknowledged bad
blocks is empty.

Signed-off-by: NeilBrown <neilb@suse.de>

1 files changed, 23 insertions, 2 deletions

diff --git a/drivers/md/md.h b/drivers/md/md.h
index fa4b607854ac..1e586bb4452e 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -81,12 +81,29 @@ struct mdk_rdev_s
 #define In_sync         2               /* device is in_sync with rest of array */
 #define WriteMostly     4               /* Avoid reading if at all possible */
 #define AutoDetected    7               /* added by auto-detect */
-#define Blocked         8               /* An error occurred on an externally
-                                         * managed array, don't allow writes
+#define Blocked         8               /* An error occurred but has not yet
+                                         * been acknowledged by the metadata
+                                         * handler, so don't allow writes
                                          * until it is cleared */
 #define WriteErrorSeen  9               /* A write error has been seen on this
                                          * device
                                          */
+#define FaultRecorded   10              /* Intermediate state for clearing
+                                         * Blocked.  The Fault is/will-be
+                                         * recorded in the metadata, but that
+                                         * metadata hasn't been stored safely
+                                         * on disk yet.
+                                         */
+#define BlockedBadBlocks 11             /* A writer is blocked because they
+                                         * found an unacknowledged bad-block.
+                                         * This can safely be cleared at any
+                                         * time, and the writer will re-check.
+                                         * It may be set at any time, and at
+                                         * worst the writer will timeout and
+                                         * re-check.  So setting it as
+                                         * accurately as possible is good, but
+                                         * not absolutely critical.
+                                         */
         wait_queue_head_t blocked_wait;
 
         int desc_nr;                    /* descriptor index in the superblock */
@@ -124,6 +141,10 @@ struct mdk_rdev_s
 
         struct badblocks {
                 int     count;          /* count of bad blocks */
+                int     unacked_exist;  /* there probably are unacknowledged
+                                         * bad blocks.  This is only cleared
+                                         * when a read discovers none
+                                         */
                 int     shift;          /* shift from sectors to block size
                                          * a -ve shift means badblocks are
                                          * disabled.*/