[Backend Tester] Add FACTO operator test skeleton #11953
Conversation
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11953
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 67 PendingAs of commit 32e7033 with merge base 91c9ffa ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
GregoryComer
added
the
release notes: none
GregoryComer
requested review from
digantdesai,
mcr229 and
cccclai
as code owners
|
Is this gonna be part of unit test in pull job? How long does it take? Thoughts on making it part of trunk job? |
If so, can you explicitly say "--ignore" inside pytest.ini file? |
I'm hoping to evaluate this once I get the full test suite landed (big stack at #11894), foundational PRs are currently being reviewed. I'm intending to hopefully run the P0 delegate tests at least on periodic, if not more frequently. We can also look at trimming down the full test suite depending on execution time and CI cost. Though FYI some delegates (QNN and Vulkan) aren't fully set up for pybindings in OSS yet (it's on the roadmap). I'll get back to you on this in the next two weeks once the test suite is landed. |
| return self.op(*(args + self.fixed_args), **(kwargs | self.fixed_kwargs)) | ||
|
|
||
|
|
||
| class ConvModel(OpModel): |
There was a problem hiding this comment.
do we not have specs for Conv? i.e. why do we need this wrapper?
| if isinstance(posargs[0], torch.Tensor): | ||
| # Temporary for getting around XNN crashes | ||
| if posargs[0].dtype not in {torch.float32, torch.float16}: | ||
| print("SKIPPING NON FLOAT CASE") |
There was a problem hiding this comment.
I will be curious to see how FACTO and quantization will work together but I guess you have some ideas?
Add initial skeleton for running Facto operator tests. This is only set up for XNNPACK in this commit (other delegates are later in the stack). It also relies on a manual install of Facto. I'm currently intentionally not running this in CI, due to a combination of finding a number of crashes and due to a high volume of tests. This will be addressed further up the stack. Instructions for running the tests locally are included at the top of test_facto.py.
Add initial skeleton for running Facto operator tests. This is only set up for XNNPACK in this commit (other delegates are later in the stack). It also relies on a manual install of Facto. I'm currently intentionally not running this in CI, due to a combination of finding a number of crashes and due to a high volume of tests. This will be addressed further up the stack. Instructions for running the tests locally are included at the top of test_facto.py.