From eb51c53e5ded1743830368815c550b871f950738 Mon Sep 17 00:00:00 2001 From: Sultan Alsawaf Date: Wed, 20 Oct 2021 20:50:11 -0700 Subject: ZEN: INTERACTIVE: mm: Lower the non-hugetlbpage pageblock size to reduce scheduling delays The page allocator processes free pages in groups of pageblocks, where the size of a pageblock is typically quite large (1024 pages without hugetlbpage support). Pageblocks are processed atomically with the zone lock held, which can cause severe scheduling delays on both the CPU going through the pageblock and any other CPUs waiting to acquire the zone lock. A frequent offender is move_freepages_block(), which is used by rmqueue() for page allocation. As it turns out, there's no requirement for pageblocks to be so large, so the pageblock order can simply be reduced to ease the scheduling delays and zone lock contention. PAGE_ALLOC_COSTLY_ORDER is used as a reasonable setting to ensure non-costly page allocation requests can still be serviced without always needing to free up more than one pageblock's worth of pages at a time. This has a noticeable effect on overall system latency when memory pressure is elevated. The various mm functions which operate on pageblocks no longer appear in the preemptoff tracer, where previously they would spend up to 100 ms on a mobile arm64 CPU processing a pageblock with preemption disabled and the zone lock held. Signed-off-by: Sultan Alsawaf --- include/linux/pageblock-flags.h | 4 ++++ init/Kconfig | 1 + 2 files changed, 5 insertions(+) --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -52,7 +52,11 @@ extern unsigned int pageblock_order; #else /* CONFIG_TRANSPARENT_HUGEPAGE */ /* If huge pages are not used, group by MAX_ORDER_NR_PAGES */ +#ifdef CONFIG_ZEN_INTERACTIVE +#define pageblock_order PAGE_ALLOC_COSTLY_ORDER +#else #define pageblock_order MAX_PAGE_ORDER +#endif #endif /* CONFIG_HUGETLB_PAGE */ --- a/init/Kconfig +++ b/init/Kconfig @@ -152,6 +152,7 @@ config ZEN_INTERACTIVE Compact unevictable............: yes -> no Compaction proactiveness.......: 20 -> 0 Watermark boost factor.........: 1.5 -> 0 + Pageblock order................: 10 -> 3 --- EEVDF CPU Scheduler --------------------------------