Allow SOURCE_DATE_EPOCH to override file timestamps

[bmwiedemann]

Limit the maximum date to SOURCE_DATE_EPOCH, if defined
similar to the tar --clamp-mtime option

based on a patch by Nicolas Vigier <boklm at torproject.org>

together with previously merged
#143
b8a54d6 from @boklm
2258825

This allows rpmbuild to produce bit-by-bit identical rpms on different hosts when setting rpmmacros to

%_buildhost reproducible
%source_date_epoch_from_changelog Y
%clamp_mtime_to_source_date_epoch Y

macro is needed, because not everyone might want it, as it is known to affect python packages that include .pyc files which reference the original timestamp. Those will still work but re-compile on demand.

[ffesti]

I am not too keen on the use of global variable(s) here. May be we can renew loading the environ variable for each build and store it in the spec or package variable instead.
While it may not matter much in practise I'd rather not like to add more technical debt there.

Other than that the patch looks good from a code POV.

I wonder how we want to address the Python .pyc file issue (and probably others). Not that this is a prerequisite for getting this patch in. We could try to leverage the file attribute mechanism or have a list of REs matching the FILECLASS tag. Resulting files could be given an mtime of SOURCE_DATE_EPOCH+1

[bmwiedemann]

On 2017-02-02 13:46, Florian Festi wrote: I am not too keen on the use of global variable

Do you refer to "oneshot" or "SOURCE_DATE_EPOCH"?

I wonder how we want to address the Python .pyc file issue

I was thinking that it would be better to do it like Debian and generate those in %post which also reduces the size of rpms and allows it to be noarch The alternative approach would be to change the macro generating them and first touch source .py files to set their date to $SOURCE_DATE_EPOCH so that this gets embedded in .pyc headers

[toabctl]

I was thinking that it would be better to do it like Debian and generate
those in %post
which also reduces the size of rpms and allows it to be noarch

Yes that would be good. Here is the relevant debian policy for that: https://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html#s-byte_compilation

[Conan-Kudo]

@toabctl I'm not particularly a fan of the late byte-compilation technique Debian uses, and I'd rather not propagate that down to everyone. For one thing, we're trying to reduce the number of scripts run outside of the realm of the package manager, not increase it. Having even fewer files known by the package manager is just crazy and stupid.

However, @bmwiedemann's idea of getting the .py files set to $SOURCE_DATE_EPOCH to embed that in .pyc files is interesting.

[fweimer]

@bmwiedemann wrote:

The alternative approach would be to change the macro generating them and first touch source .py files to set their date to $SOURCE_DATE_EPOCH so that this gets embedded in .pyc headers

I doubt that RPM can do this without delayed script execution because the bytecode generator is not necessarily installed when the %post runs.

[bmwiedemann]

@ffesti I restructured the code to get rid of the ugly 'static' globals.

[Conan-Kudo]

@fweimer Wouldn't that happen during the package build, rather than at runtime? It seems like that's a package build-time action.

[fweimer]

@Conan-Kudo:

Wouldn't that happen during the package build, rather than at runtime? It seems like that's a package build-time action.

I quoted the wrong part (Github doesn't seem to support automated quoting). I was actually referring to this:

I was thinking that it would be better to do it like Debian and generate those in %post which also reduces the size of rpms and allows it to be noarch

[Conan-Kudo]

@fweimer I see, that makes much more sense.

[ffesti]

Thanks for the patch!

[bmwiedemann]

on the semi-offtopic python .pyc timestamp issue see: python/cpython#296