Add a handler to post a summary comment of Clippy lintcheck runs

[Alexendoo]

In Clippy we have a tool called lintcheck that we use in CI as a sort of mini crater run, this handler would post a summary of the run as a comment in the PR thread

It works by having the workflow upload a JSON artifact that the handler downloads, in Clippy we would add

[lintcheck-summary]
workflow = "Lintcheck"
artifact = "summary"

JSON is used instead of uploading a markdown file so that PRs from forks can't have triagebot post arbitrary comments

For https://github.com/rust-lang/rust-clippy/actions/runs/14763277413#summary-41449477485 the following comment would be posted:

---

Lintcheck changes for f980edd0da51eb4da9c15eb423a7829f2c5657fc

Lint	Added	Removed	Changed
clippy::collapsible_if	0	0	1
clippy::elidable_lifetime_names	30	0	1125
clippy::extra_unused_lifetimes	1	3	4
clippy::needless_lifetimes	23	0	214

This comment will be updated if you push new changes

[Urgau]

Have you thought about using a actions/github-script job at the end of your workflow? It gives you an Javascript environment with octocrab already setup-ed.

You could I think pretty easily download your artifacts and post your summary comment with it.

The only tricky part would be updating the previous comment, but using a hidden marker (like triagebot does) would I think do it.

[Alexendoo]

A PR workflow doesn't have permission to create comments when it's from a fork but it is possible with a separate on.workflow_run workflow, I had a non JSON implementation working here

That's when I thought it might not be a good idea to allow comments of arbitrary content, but I did not fancy writing a bunch of JS in YAML

[Urgau]

Interesting, the workflow approach seems much more approachable.

So if I understand correctly the motivating factor for doing this in triagebot is over security, but wouldn't adversarial PR still be able to put arbitrary data in some of the output (like name and what not)?

This reminds me that in rust-lang/rust we have had for a few weeks something similar (cc @Kobzol). We have a post-merge workflow that outputs details about the workflow; granted it doesn't download artifact and runs post merge (so much less security risks).

[Kobzol]

I know that posting comments from actions on PRs from forks on GitHub is a PITA, but I would like to avoid (ab)using triagebot for this, as this is a very repo-specific functionality and I think it belongs in Clippy, not in triagebot.

The way I do this in the team is to run a normal pull_request workflow that computes the necessary data, and then uploads it as an artifact that contains the PR number and the gathered data. Then there is a second workflow that runs from the main branch and has the workflow_run trigger (https://github.com/rust-lang/team/blob/master/.github/workflows/dry-run.yml), which downloads the results and sends the comment. The bash code for that isn't so terrible, but if you want to avoid it, you can also just create e.g. a Python script in the Clippy repo and run it in the workflow. If you upload a JSON and then read some known data out of it, then I think it should be fine security-wise.

[Alexendoo]

wouldn't adversarial PR still be able to put arbitrary data in some of the output (like name and what not)?

Yeah any strings in the JSON are checked to only contain safe chars

The bash code for that isn't so terrible

True, I was picturing swapping the whole thing over to JS but really I only need to parse the JSON + write a file with it, the rest can still be done via gh commands. That should be pretty reasonable

FWIW I found that the PR number is available in the event, and gh has gained the ability to do --edit-last --create-if-none

[Kobzol]

Oh, nice, good to know about that flag!

Watch out about the PR number though, I also tried to use it from that attribute, but it won't be filled on PRs from forks 🤦