58 lines
2.3 KiB
Diff
58 lines
2.3 KiB
Diff
|
From eb51c53e5ded1743830368815c550b871f950738 Mon Sep 17 00:00:00 2001
|
||
|
|
From: Sultan Alsawaf <sultan@kerneltoast.com>
|
||
|
|
Date: Wed, 20 Oct 2021 20:50:11 -0700
|
||
|
|
Subject: ZEN: INTERACTIVE: mm: Lower the non-hugetlbpage pageblock size to
|
||
|
|
reduce scheduling delays
|
||
|
|
|
||
|
|
The page allocator processes free pages in groups of pageblocks, where
|
||
|
|
the size of a pageblock is typically quite large (1024 pages without
|
||
|
|
hugetlbpage support). Pageblocks are processed atomically with the zone
|
||
|
|
lock held, which can cause severe scheduling delays on both the CPU
|
||
|
|
going through the pageblock and any other CPUs waiting to acquire the
|
||
|
|
zone lock. A frequent offender is move_freepages_block(), which is used
|
||
|
|
by rmqueue() for page allocation.
|
||
|
|
|
||
|
|
As it turns out, there's no requirement for pageblocks to be so large,
|
||
|
|
so the pageblock order can simply be reduced to ease the scheduling
|
||
|
|
delays and zone lock contention. PAGE_ALLOC_COSTLY_ORDER is used as a
|
||
|
|
reasonable setting to ensure non-costly page allocation requests can
|
||
|
|
still be serviced without always needing to free up more than one
|
||
|
|
pageblock's worth of pages at a time.
|
||
|
|
|
||
|
|
This has a noticeable effect on overall system latency when memory
|
||
|
|
pressure is elevated. The various mm functions which operate on
|
||
|
|
pageblocks no longer appear in the preemptoff tracer, where previously
|
||
|
|
they would spend up to 100 ms on a mobile arm64 CPU processing a
|
||
|
|
pageblock with preemption disabled and the zone lock held.
|
||
|
|
|
||
|
|
Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
|
||
|
|
---
|
||
|
|
include/linux/pageblock-flags.h | 4 ++++
|
||
|
|
init/Kconfig | 1 +
|
||
|
|
2 files changed, 5 insertions(+)
|
||
|
|
|
||
|
|
--- a/include/linux/pageblock-flags.h
|
||
|
|
+++ b/include/linux/pageblock-flags.h
|
||
|
|
@@ -52,7 +52,11 @@ extern unsigned int pageblock_order;
|
||
|
|
#else /* CONFIG_TRANSPARENT_HUGEPAGE */
|
||
|
|
|
||
|
|
/* If huge pages are not used, group by MAX_ORDER_NR_PAGES */
|
||
|
|
+#ifdef CONFIG_ZEN_INTERACTIVE
|
||
|
|
+#define pageblock_order PAGE_ALLOC_COSTLY_ORDER
|
||
|
|
+#else
|
||
|
|
#define pageblock_order MAX_PAGE_ORDER
|
||
|
|
+#endif
|
||
|
|
|
||
|
|
#endif /* CONFIG_HUGETLB_PAGE */
|
||
|
|
|
||
|
|
--- a/init/Kconfig
|
||
|
|
+++ b/init/Kconfig
|
||
|
|
@@ -152,6 +152,7 @@ config ZEN_INTERACTIVE
|
||
|
|
Compact unevictable............: yes -> no
|
||
|
|
Compaction proactiveness.......: 20 -> 0
|
||
|
|
Watermark boost factor.........: 1.5 -> 0
|
||
|
|
+ Pageblock order................: 10 -> 3
|
||
|
|
|
||
|
|
--- EEVDF CPU Scheduler --------------------------------
|
||
|
|
|