memory-hotplug: fix pages missed by race rather than failing

If race between allocation and isolation in memory-hotplug offline
happens, some pages could be in MIGRATE_MOVABLE of free_list although the
pageblock's migratetype of the page is MIGRATE_ISOLATE.

The race could be detected by get_freepage_migratetype in
__test_page_isolated_in_pageblock.  If it is detected, now EBUSY gets
bubbled all the way up and the hotplug operations fails.

But better idea is instead of returning and failing memory-hotremove, move
the free page to the correct list at the time it is detected.  It could
enhance memory-hotremove operation success ratio although the race is
really rare.

Suggested by Mel Gorman.

[akpm@linux-foundation.org: small cleanup]
Signed-off-by: Minchan Kim <minchan@kernel.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Reviewed-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Cc: Xishi Qiu <qiuxishi@huawei.com>
Cc: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 14 insertions, 2 deletions

diff --git a/mm/page_isolation.c b/mm/page_isolation.c
index 6744235d2d0e..5f34a9053ce0 100644
--- a/mm/page_isolation.c
+++ b/mm/page_isolation.c
@@ -201,8 +201,20 @@ __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn)
                 }
                 page = pfn_to_page(pfn);
                 if (PageBuddy(page)) {
-                        if (get_freepage_migratetype(page) != MIGRATE_ISOLATE)
-                                break;
+                        /*
+                         * If race between isolatation and allocation happens,
+                         * some free pages could be in MIGRATE_MOVABLE list
+                         * although pageblock's migratation type of the page
+                         * is MIGRATE_ISOLATE. Catch it and move the page into
+                         * MIGRATE_ISOLATE list.
+                         */
+                        if (get_freepage_migratetype(page) != MIGRATE_ISOLATE) {
+                                struct page *end_page;
+
+                                end_page = page + (1 << page_order(page)) - 1;
+                                move_freepages(page_zone(page), page, end_page,
+                                                MIGRATE_ISOLATE);
+                        }
                         pfn += 1 << page_order(page);
                 }
                 else if (page_count(page) == 0 &&