Skip to content

Rebase to v2.24.0-rc0 #2360

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 948 commits into from
Oct 22, 2019
Merged

Rebase to v2.24.0-rc0 #2360

merged 948 commits into from
Oct 22, 2019

Conversation

dscho
Copy link
Member

@dscho dscho commented Oct 18, 2019

This is a work-in-progress:

  • Make sure that the branch structure is logical (i.e. move the phase-out-* and the readme branches to the tip of the thicket.
  • Make sure that the range-diff relative to master makes sense.
  • Update the add-i/add-i patches to the latest iteration.
  • Move the "ci(visual-studio)" fixup into its own branch (needs more investigation why the tests are so slow all of a sudden, just because we build via vcxproj instead of via MSVC=1).

My plan was to make sure that every single commit compiles cleanly, but since I already spent a dozen hours today on getting things into a good shape, I think I'll skip this opportunity, deferring to the -rc1 rebase.

@dscho dscho changed the title [Work-In-Progress] Rebase to v2.24.0-rc0 Rebase to v2.24.0-rc0 Oct 18, 2019
@dscho
Copy link
Member Author

dscho commented Oct 18, 2019

Note: during the latest Git IRC standup, the Git maintainer indicated that v2.23.1 will not happen before v2.24.0 (enough patches accumulated a bugfix release, but the -rc0..-rc3 time is as good a time as any to focus on the upcoming "big" release).

Therefore, Git for Windows' master will follow the -rc<n> releases this time round (also making maintenance substantially easier because I won't have to synchronize fixes between the master and prepare-for-v2.24.0 branches).

dscho and others added 28 commits October 18, 2019 22:59
These fixes were necessary for Sverre Rabbelier's remote-hg to work,
but for some magic reason they are not necessary for the current
remote-hg. Makes you wonder how that one gets away with it.

Signed-off-by: Johannes Schindelin <[email protected]>
Turns out that it don't work so well on Vista, see
git-for-windows#1742 for details.

According to https://devblogs.microsoft.com/oldnewthing/?p=8873, it
*should* work on Windows Vista and later.

But apparently there are issues on Windows Vista when pipes are
involved. Given that Windows Vista is past its end of life (official
support ended on April 11th, 2017), let's not spend *too* much time on
this issue and just disable the file handle inheritance restriction on
any Windows version earlier than Windows 7.

Signed-off-by: Johannes Schindelin <[email protected]>
…er/register_rename_src

diffcore-rename: speed up register_rename_src
This topic branch allows us to specify absolute paths without the drive
prefix e.g. when cloning.

Example:

	C:\Users\me> git clone https://github.com/git/git \upstream-git

This will clone into a new directory C:\upstream-git, in line with how
Windows interprets absolute paths.

Signed-off-by: Johannes Schindelin <[email protected]>
Fix fetching from UNC paths.

Signed-off-by: Johannes Schindelin <[email protected]>
When compiling Git with a runtime prefix (so that it can be installed
into any location, finding its libexec/ directory relative to the
location of the `git` executable), it is convenient to provide
"absolute" Unix-y paths e.g. for http.sslCAInfo, and have those absolute
paths be resolved relative to the runtime prefix.

This patch makes it so for Windows. It is up for discussion whether we
want this for other platforms, too, as long as building with
RUNTIME_PREFIX.

Signed-off-by: Johannes Schindelin <[email protected]>
Add a helper function to start GDB that was already attached to the current process
This topic branch avoids spawning `gzip` when asking `git archive` to
create `.tar.gz` files.

Signed-off-by: Johannes Schindelin <[email protected]>
This is hardly the first conversion of a Git command that is implemented
as a script to a built-in. So far, the most successful strategy for such
conversions has been to add a built-in helper and call that for more and
more functionality from the script, as more and more parts are
converted.

With the interactive add, we choose a different strategy. The sole
reason for this is that on Windows (where such a conversion has the most
benefits in terms of speed and robustness) we face the very specific
problem that a `system()` call in Perl seems to close `stdin` in the
parent process when the spawned process consumes even one character from
`stdin`. And that just does not work for us here, as it would stop the
main loop as soon as any interactive command was performed by the
helper. Which is almost all of the commands in `git add -i`.

It is almost as if Perl told us once again that it does not want us to
use it on Windows.

Instead, we follow the opposite route where we start with a bare-bones
version of the built-in interactive add, guarded by the new
`add.interactive.useBuiltin` config variable, and then add more and more
functionality to it, until it is feature complete.

At this point, the built-in version of `git add -i` only states that it
cannot do anything yet ;-)

Signed-off-by: Johannes Schindelin <[email protected]>
Make the diffstat interface (namely, the diffstat_t struct and
compute_diffstat) no longer be internal to diff.c and allow it to be used
by other parts of git.

This is helpful for code that may want to easily extract information
from files using the diff machinery, while flushing it differently from
how the show_* functions used by diff_flush() do it. One example is the
builtin implementation of git-add--interactive's status.

Signed-off-by: Daniel Ferreira <[email protected]>
Signed-off-by: Slavica Đukić <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
This implements the `status` command of `git add -i`. The data
structures introduced in this commit will be extended later, as needed.

At this point, we re-implement only part of the `list_and_choose()`
function of the Perl script `git-add--interactive.perl` and call it
`list()`. It does not yet color anything, or do columns, or allow user
input.

Over the course of the next commits, we will introduce a
`list_and_choose()` function that uses `list()` to display the list of
options and let the user choose one or more of the displayed items. This
will be used to implement the main loop of the built-in `git add -i`, at
which point the new `status` command can actually be used.

Signed-off-by: Daniel Ferreira <[email protected]>
Signed-off-by: Slavica Đukić <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
For simplicity, we only implemented the `status` command without colors.
This patch starts adding color, matching what the Perl script
`git-add--interactive.perl` does.

Original-Patch-By: Daniel Ferreira <[email protected]>
Signed-off-by: Slavica Đukić <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
The reason why we did not start with the main loop to begin with is that
it is the first user of `list_and_choose()`, which uses the `list()`
function that we conveniently introduced for use by the `status`
command.

Apart from the "and choose" part, there are more differences between the
way the `status` command calls the `list_and_choose()` function in the
Perl version of `git add -i` compared to the other callers of said
function. The most important ones:

- The list is not only shown, but the user is also asked to make a
  choice, possibly selecting multiple entries.

- The list of items is prefixed with a marker indicating what items have
  been selected, if multi-selection is allowed.

- Initially, for each item a unique prefix (if there exists any within
  the given parameters) is determined, and shown in the list, and
  accepted as a shortcut for the selection.

These features will be implemented later, except the part where the user
can choose a command. At this stage, though, the built-in `git add -i`
still only supports the `status` command, with the remaining commands to
follow over the course of the next commits.

In addition, we also modify `list()` to support displaying the commands
in columns, even if there is currently only one.

The Perl script `git-add--interactive.perl` mixed the purposes of the
"list" and the "and choose" part into the same function. In the C
version, we will keep them separate instead, calling the `list()`
function from the `list_and_choose()` function.

Note that we only have a prompt ending in a single ">" at this stage;
later commits will add commands that display a double ">>" to indicate
that the user is in a different loop than the main one.

Signed-off-by: Johannes Schindelin <[email protected]>
Just like in the Perl script `git-add--interactive.perl`, for each
command a unique prefix is determined (if there exists any within the
given parameters), and shown in the list, and accepted as a shortcut for
the command.

To determine the unique prefixes, as well as to look up the command in
question, we use a copy of the list and sort it.

While this might seem like overkill for a single command, it will make
much more sense when all the commands are implemented, and when we reuse
the same logic to present a list of files to edit, with convenient
unique prefixes.

At the start of the development of this patch series, a dedicated data
structure was introduced that imitated the Trie that the Perl version
implements. However, this was deemed overkill, and we now simply sort
the list before determining the length of the unique prefixes by looking
at each item's neighbor. As a bonus, we now use the same sorted list to
perform a binary search using the user-provided prefix as search key.

Original-patch-by: Slavica Đukić <[email protected]>
Helped-by: SZEDER Gábor <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
With this change, we print out the same colored help text that the
Perl-based `git add -i` prints in the main loop when question mark is
entered.

Signed-off-by: Johannes Schindelin <[email protected]>
The error messages as well as the unique prefixes are colored in `git
add -i` by default; We need to do the same in the built-in version.

Signed-off-by: Slavica Đukić <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
This imitates the code to show the help text from the Perl script
`git-add--interactive.perl` in the built-in version.

To make sure that it renders exactly like the Perl version of `git add
-i`, we also add a test case for that to `t3701-add-interactive.sh`.

Signed-off-by: Slavica Đukić <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
In the `update` command of `git add -i`, we are primarily interested in the
list of modified files that have worktree (i.e. unstaged) changes.

At the same time, we need to determine _also_ the staged changes, to be
able to produce the full added/deleted information.

The Perl script version of `git add -i` has a parameter of the
`list_modified()` function for that matter. In C, we can be a lot more
precise, using an `enum`.

The C implementation of the filter also has an easier time to avoid
unnecessary work, simply by using an adaptive order of the `diff-index`
and `diff-files` phases, and then skipping files in the second phase
when they have not been seen in the first phase.

Signed-off-by: Johannes Schindelin <[email protected]>
The `upgrade`, `revert` and `add-untracked` commands allow selecting
multiple entries. Let's extend the `list_and_choose()` function to
accommodate those use cases.

Signed-off-by: Johannes Schindelin <[email protected]>
After `status` and `help`, it is now turn to port the `update` command
to C, the second command that is shown in the main loop menu of `git add
-i`.

This `git add -i` command is the first one which lets the user choose a
subset of a list of files, and as such, this patch lays the groundwork
for the other commands of that category:

- It teaches the `print_file_item()` function to show a unique prefix
  if we found any (the code to find it had been added already in the
  previous patch where we colored the unique prefixes of the main loop
  commands, but that patch uses the `print_command_item()` function to
  display the menu items).

- This patch also adds the help text that is shown when the user input
  to select items from the shown list could not be parsed.

- As `get_modified_files()` clears the list of files, it now has to take
  care of clearing the _full_ `prefix_item_list` lest the `sorted` and
  `selected` fields go stale and inconsistent.

Signed-off-by: Johannes Schindelin <[email protected]>
This is a relatively straight-forward port from the Perl version, with
the notable exception that we imitate `git reset -- <paths>` in the C
version rather than the convoluted `git ls-tree HEAD -- <paths> | git
update-index --index-info` followed by `git update-index --force-remove
-- <paths>` for the missed ones.

While at it, we fix the pretty obvious bug where the `revert` command
offers to unstage files that do not have staged changes.

Signed-off-by: Johannes Schindelin <[email protected]>
This is yet another command, ported to C. It builds nicely on the
support functions introduced for other commands, with the notable
difference that only names are displayed for untracked files, no
file type or diff summary.

Signed-off-by: Johannes Schindelin <[email protected]>
Well, it is not a full implementation yet. In the interest of making
this easy to review (and easy to keep bugs out), we still hand off to
the Perl script to do the actual work.

The `patch` functionality actually makes up for more than half of the
1,800+ lines of `git-add--interactive.perl`. It will be ported from Perl
to C incrementally, later.

Signed-off-by: Johannes Schindelin <[email protected]>
It is not only laziness that we simply spawn `git diff -p --cached`
here: this command needs to use the pager, and the pager needs to exit
when the diff is done. Currently we do not have any way to make that
happen if we run the diff in-process. So let's just spawn.

Signed-off-by: Johannes Schindelin <[email protected]>
We do not really want to `exit()` here, of course, as this is safely
libified code.

Signed-off-by: Johannes Schindelin <[email protected]>
In this developer's workflows, it often happens that a hunk needs to be
edited in a way that adds lines, and even reduces the context

Let's add a regression test for this.

Note that just like the preceding test case, the new test case is *not*
handled gracefully by the current `git add -p`. It will be handled
correctly by the upcoming built-in `git add -p`, though.

Signed-off-by: Johannes Schindelin <[email protected]>
The TTY prerequisite is a rather heavy one: it not only requires Perl to
work, but also the IO/Pty.pm module (with native support, and it
requires pseudo terminals, too).

In particular, test cases marked with the TTY prerequisite would be
skipped in Git for Windows' SDK.

In the case of `git add -p`, we do not actually need that big a hammer,
as we do not want to test any functionality that requires a pseudo
terminal; all we want is to talk the interactive add command to use
color, even when being called from within the test suite.

And we found exactly such a trick earlier already: when we added a test
case to verify that the main loop of `git add -i` is colored
appropriately. Let's use that trick instead of the TTY prerequisite.

While at it, we avoid the pipes, as we do not want a SIGPIPE to break
the regression test cases (which will be much more likely when we do not
run everything through Perl because that is inherently slower).

Signed-off-by: Johannes Schindelin <[email protected]>
The `git add -p` command offers different prompts for regular diff hunks
vs mode change pseudo hunks vs diffs deleting files.

Let's cover this in the regresion test suite, in preparation for
re-implementing `git add -p` in C.

For the mode change prompt, we use a trick that lets this test case pass
even on systems without executable bit, i.e. where `core.filemode =
false` (such as Windows): we first add the file to the index with `git
add --chmod=+x`, and then call `git add -p` with `core.filemode` forced
to `true`. The file on disk has no executable bit set, therefore we will
see a mode change.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho and others added 23 commits October 18, 2019 23:06
fetch-pack.c: enable fscache for stats under .git/objects
…t_flush

checkout.c: enable fscache for checkout again

Signed-off-by: Johannes Schindelin <[email protected]>
…_index

Enable the filesystem cache (fscache) in refresh_index().
…-gfw

fscache: use FindFirstFileExW to avoid retrieving the short name
…ter-status-gfw

status: disable and free fscache at the end of the status command
…e-gfw

fscache: add GIT_TEST_FSCACHE support
…ter-add-gfw

At the end of the add command, disable and free the fscache
…ics-gfw

fscache: add fscache hit statistics
This brings substantial wins in performance because the FSCache is now
per-thread, being merged to the primary thread only at the end, so we do
not have to lock (except while merging).

Signed-off-by: Johannes Schindelin <[email protected]>
…safe-enable-gfw

fscache: make fscache_enable() thread safe
…DirectoryFile-gfw

 fscache: teach fscache to use NtQueryDirectoryFile
When updating the skip-worktree bits in the index to align with new
values in a sparse-checkout file, Git scans the entire working
directory with lstat() calls. In a sparse-checkout, many of these
lstat() calls are for paths that do not exist.

Enable the fscache feature during this scan.

In a local test of a repo with ~2.2 million paths, updating the index
with `git read-tree -m -u HEAD` with a sparse-checkout file containing
only `/.gitattributes` improved from 2-3 minutes to 15-20 seconds.

More work could be done to stop running lstat() calls when recursing
into directories that are known to not exist.
We already avoid traversing NTFS junction points in `git clean -dfx`.
With this topic branch, we do that when the FSCache is enabled, too.

Signed-off-by: Johannes Schindelin <[email protected]>
This was pull request git-for-windows#1645 from ZCube/master

Support windows container.

Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Handle Ctrl+C in Git Bash nicely

Signed-off-by: Johannes Schindelin <[email protected]>
This branch allows third-party tools to call `git status
--no-lock-index` to avoid lock contention with the interactive Git usage
of the actual human user.

Signed-off-by: Johannes Schindelin <[email protected]>
…ored-directory-gracefully

Phase out `--show-ignored-directory` gracefully
Add a README.md for GitHub goodness.

Signed-off-by: Johannes Schindelin <[email protected]>
@dscho
Copy link
Member Author

dscho commented Oct 22, 2019

I managed to release https://github.com/git-for-windows/git/releases/tag/v2.24.0-rc0.windows.1 last night, now it is time to merge.

@dscho dscho merged commit 51b78e6 into git-for-windows:master Oct 22, 2019
@dscho dscho deleted the prepare-for-v2.24.0 branch October 22, 2019 10:39
@dscho dscho mentioned this pull request Oct 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.