mm/khugepaged: write all dirty file folios when collapsing

Overview

In the Linux kernel, the following vulnerability has been resolved:

mm/khugepaged: write all dirty file folios when collapsing

[There is no upstream commit, as this code was removed by upstream commit 044925f9b565 ("mm: fs: remove filemap_nr_thps*() functions and their users")]

As-is, khugepaged and writable-file opening exclude each other. A file cannot be open writeable and have THPs (because the filesystem is not aware of them). khugepaged will never collapse file pages for files that are opened writeable. On an open(O_RDWR/O_WRONLY), the page cache for that particular file is dropped. This is fine because nothing could've been dirtied.

However, there is an edge-case: collapse_file() might not be able to coexist with concurrent writers, but it can coexist with dirty folios (from previous writers). Therefore, the following can happen:

open(file, O_RDWR) write(file) close(file) madvise(file_mapping, MADV_COLLAPSE, some non-dirty range) open(file, O_RDWR) nr_thps > 0 truncate_inode_pages() /* THPs are cleared out, but so are the dirty folios */

When this edge-case happens, there is data loss, as the dirty folios are fully discarded.

Fix it by fully writing back the page cache (and waiting) when collapsing file THPs. Doing so provides the guarantee that no dirty folio will be observed while there are active THPs. To fully ensure this is safe, the invalidate_lock needs to be held while doing the writeout, so that do_dentry_open()'s page cache truncation excludes this write-and-wait.

As a side effect, move the nr_thps counter bumping outside the i_pages lock. This is correct since the counter itself is an atomic_t and the producer <-> consumer correctness is provided by a full memory barrier: smp_mb() in collapse_file()/memory barrier implied by full ordering in get_write_access() -> atomic_inc_unless_negative().

Affected

• Linux >= 99cb0dbd47a15d395bf3faa78dc122bc5efe3fc0 < 2dfe9f5c91d0963058f8a5e46e1c2a908382cc46
• Linux 5.4

Remediation

Refer to the linked advisories for vendor-supplied fixes and affected version ranges.