rustdoc-json: Include size, alignment, and field offset info for types

[obi1kenobi]

Size, alignment, and field offset are often part of types' public API, so it would be lovely to make that information available in rustdoc JSON (and perhaps in rustdoc HTML as well, when relevant).

For example, public repr(C) types have well-defined layout which may be depended on by FFI use cases:

#[repr(C)]
pub struct Example {
    first: i64,
    pub second: Option<&'static str>,
}

Say the type of first were to change from i64 to (i64, i64). Since the first field is not public, there's no SemVer hazard from the field access. However, there are two hazards that cargo-semver-checks cannot detect without knowing size, alignment, and offset information:

• The offset of second changes from 8 to 16 (IIUC), so applications using Example via FFI need to update where they find that field's information.
• The size of the Example struct grows by 16 bytes, which likely affects how it can be allocated as well as the layout of [Example; N] arrays and slices thereof.

Similar information is useful when considering whether #[repr(transparent)] is public API. The current rule is that it's public API if its sole non-1-ZST field is pub. cargo-semver-checks would like to be able to:

• Clearly determine whether fields are 1-ZST or not.
• Thereby deduce whether #[repr(transparent)] is public API or not.
• In cases where #[repr(transparent)] used to be and has since stopped being public API, report to the user the reason why #[repr(transparent)] is no longer public API. This point in particular cannot be addressed if rustdoc performs 1-ZST analysis and strips #[repr(transparent)] in cases where it is considered non-public.

@rustbot label A-rustdoc-json

cc @aDotInTheVoid per our conversation at RustWeek

[Mark-Simulacrum]

For example, public repr(C) types have well-defined layout which may be depended on by FFI use cases

I'm not sure this is true -- at least in the standard library, generally repr(transparent) and repr(C) on types are considered internal implementation details unless explicitly documented otherwise. I'm not sure if that's documented by RFC, though the lack of that documentation implies it's not stable in some sense.

The nomicon shouldn't really be considered normative but IMO "This repr is only considered part of the public ABI of a type if either the single field is pub, or if its layout is documented in prose. Otherwise, the layout should not be relied upon by other crates." should really apply to all of the reprs -- not just transparent.

[obi1kenobi]

This is a fair point, so perhaps the better statement is "repr(C) types may have well-defined layout ..."

As I mentioned at RustWeek, I'd love to have concrete and ideally machine-readable annotations describing what is and is not public API.

In the meantime, I think we still want to have cargo-semver-checks lint on everything I described here — just as "warn-by-default" instead of "deny-by-default" in some of those gray areas.

[hanna-kruppe]

Beyond the general point, the specific example seems extra dubious to me. Since second is public but other fields aren’t, Rust consumers really should use offset_of instead of assuming a particular offset. There’s nothing about the struct’s definition that indicates that assuming a fixed offset would be warranted. Neither does it look like a type that’s likely to be accessed from C code that assumes it knows the layout of the type. If cargo-semver-checks describes the lints as outlined then my gut feeling is that warn-by-default would still be overconfident (too many false positives).

[obi1kenobi]

I feel like we're getting quite a bit off topic here. If we're discussing whether lints should be allow-, warn-, or deny-by-default, let's open an issue in the cargo-semver-checks repo specifically for that.

This information has multiple use cases as outlined in the issue. It would be useful to have even with a single one of them.

Also, the pattern I described exists in real world Rust code. Users perhaps shouldn't rely on it but in practice I'd bet they do. cargo-semver-checks approaches this from a regret minimization perspective: we want to inform maintainers about the existence of a hazard even if it doesn't require a major change, so that they don't find out about it via a GitHub issue after the fact. This means that at worst we'd still want an allow-by-default lint here that maintainers can choose to enable when they wish.

[hanna-kruppe]

You’re right that this isn’t the best place to discuss cargo-semver-checks behavior. More on-topic, how would you expect this to work for generic types? Size and alignment of field types can be computed for each field independently and omitted if it depends on type/const parameters — so you could identify the non-1ZST in repr(transparent), if there is one, as the one with unknown size (just as rustc does). But I don’t think offsets can be salvaged at all if any field has unknown size/alignment.

[obi1kenobi]

Of course, I think that's fine. It's okay to leave the data missing in those cases, and cargo-semver-checks should be able to handle it. If there is a public API size/alignment/layout obligation, it ultimately has to be on a field with known & fixed size/alignment/layout. If a type has const or type generics (that aren't solely used in e.g. `PhantomData`), then I'd expect that in general such obligations don't make sense.

[workingjubilee]

The layout for the same type may be different per target, even if the fields are public, so this may require multi-target checking in order to be effective. Even the exact same type, with the exact same size, for instance, may have different alignment on a per-target basis. This is even if everything is monomorphic.

[obi1kenobi]

Of course — I appreciate you bringing that up.

This is unfortunately already a problem regardless of this feature request, since APIs can also be different on a per-target basis. Rustdoc JSON only contains information for one given target at a time, so users already have to run cargo-semver-checks once per target they care to check anyway.