Compiling drop(Box<large array>) causes runaway compiler memory usage #109004
Description
Attempting to compile this code causes extremely large memory usage by rustc.
By experimentation, the important features of the code are: large LIMIT, the element type needs to have drop glue, and the drop can't be skipped with forget or some other mechanism. It also must be a program, not a library function.
src/main.rs
fn main() {
const LIMIT: usize = 64_000_000;
// NB. if it compiled successfully, this would just panic, it wouldn't even allocate...
let _arr: Box<[String; LIMIT]> = Vec::new().into_boxed_slice().try_into().ok().unwrap();
}Cargo.toml
[package]
name = "repro"
version = "0.1.0"
edition = "2021"
The problematic compiler invocation: cargo build.
I expected to see this happen: The compiler memory usage shouldn't run away due to large LIMIT; in fact the compiler memory usage should be almost entirely independent of LIMIT for this program.
Instead, this happened: When LIMIT is large, rustc uses a lot of memory and takes a long time. I have not successfully compiled the above program with the given LIMIT of 64M.
Meta
I repro'd this on 1.65.0, 1.68.0, and finally:
rustc 1.70.0-nightly (39f2657d1 2023-03-09)
binary: rustc
commit-hash: 39f2657d1101b50f9b71ae460b762d330cc8426b
commit-date: 2023-03-09
host: x86_64-unknown-linux-gnu
release: 1.70.0-nightly
LLVM version: 15.0.7
(I did not capture a compiler backtrace.)
Thoughts
I surmise that there's a correlation between LIMIT and compiler behavior but I haven't explored it beyond verifying a small LIMIT doesn't repro the bad behavior.
I thought perhaps this program tries to use the stack for the slice->array cast, and somehow the compiler is getting stuck on such a large stack frame. Thankfully, this is ruled out by checking that src/alloc/boxed.rs:boxed_slice_as_array_unchecked does indeed do the slice->array cast entirely using pointers. So the compiler shouldn't "see" how big the array is; it's behind a pointer.
Appending std::mem::forget(_arr); to main bypasses the issue, so it appears to be drop glue for the box which causes the problem. I don't see a particular reason why the drop glue for this type should be problematic; it should be no different than dropping the initialized part of a large Vec<String>.