The futex facility returned an unexpected error code.

[gyakovlev]

Gentoo users and developers report rustc failing periodically with this error:
The futex facility returned an unexpected error code.

example:

Compiling serde_json v1.0.64
     Running `rustc --crate-name build_script_build --edition=2018 /var/tmp/portage/dev-util/maturin-0.10.6/work/cargo_home/gentoo/serde_json-1.0.64/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="std"' --cfg 'feature="unbounded_depth"' -C metadata=d4ad0344b57abcdb -C extra-filename=-d4ad0344b57abcdb --out-dir /var/tmp/portage/dev-util/maturin-0.10.6/work/maturin-0.10.6/target/release/build/serde_json-d4ad0344b57abcdb -L dependency=/var/tmp/portage/dev-util/maturin-0.10.6/work/maturin-0.10.6/target/release/deps --cap-lints allow`
The futex facility returned an unexpected error code.
error: could not compile `serde_json`

happens to different crates, randomly, at different times.
not clearly reproducible and not clear what triggers it.

most often seen on our tinderbox ( package build testing machine) with random packages.
happens with user-build rustc but also with rust-bin package, which uses official standalone installers published on downloads page https://forge.rust-lang.org/infra/other-installation-methods.html#standalone
retrying build usually works fine.

error seems to be coming from glibc:

https://github.com/bminor/glibc/blob/e8d52b64a54ba9ed7778ca9ce1f084eb5808f8d1/sysdeps/nptl/futex-internal.h#L82

/* Calls __libc_fatal with an error message.  Convenience function for
   concrete implementations of the futex interface.  */
static __always_inline __attribute__ ((__noreturn__)) void
futex_fatal_error (void)
{
  __libc_fatal ("The futex facility returned an unexpected error code.\n");
}



....

static __always_inline int
futex_wait (unsigned int *futex_word, unsigned int expected, int private)
{
  int err = lll_futex_timed_wait (futex_word, expected, NULL, private);
  switch (err)
    {
    case 0:
    case -EAGAIN:
    case -EINTR:
      return -err;

    case -ETIMEDOUT: /* Cannot have happened as we provided no timeout.  */
    case -EFAULT: /* Must have been caused by a glibc or application bug.  */
    case -EINVAL: /* Either due to wrong alignment or due to the timeout not
		     being normalized.  Must have been caused by a glibc or
		     application bug.  */
    case -ENOSYS: /* Must have been caused by a glibc bug.  */
    /* No other errors are documented at this time.  */
    default:
      futex_fatal_error ();
    }
}

Unfortunately I was never able to reproduce myself.

Any ideas what can it be, what's happening and how to catch it properly?
maybe it's a specific configuration issue on user's systems?

some example downlstream bug reports, those contain build logs, but those looks same, just futex error for random crate:

• example build log: https://799878.bugs.gentoo.org/attachment.cgi?id=721056

some bugs:

• https://bugs.gentoo.org/799878#c8
• https://bugs.gentoo.org/809212

this one is very recent:

• using rust-1.58.1 official tarball to build gentoo's rust-1.58.1
  user reported a falure:
  build log:
  https://809212.bugs.gentoo.org/attachment.cgi?id=763250
  again, happens just randomly with a random crate during build.

[m-ou-se]

That's very surprising.

For thread parkers we call the futex() syscall directly without any of the glibc code you linked. As far as I know, the only way that code gets invoked is through pthread primitives like pthread_mutex_t, on which Rust's standard Mutex is based. Looking at all possible errors that futex() can return for the different (non-PI) operations, I don't see any that would match this 'works most of the time' behaviour. This makes me wonder if there's memory corruption involved, somehow, somewhere.

It's a shame it's hard to reproduce. It'd be interesting to see a log of strace -efutex to see the exact syscall arguments and error.

[jyn514]

Triage: the upstream issue https://bugs.gentoo.org/799878#c8 has been closed, so I'm not sure if it makes sense to keep this open on our side. It seems likely that this is a bug in glibc, not rust itself.

[gyakovlev]

I haven’t seen those again reported for a long time indeed. And never was able to catch one via strace too.I’ll close then.