Skip to content

[test] Run Python unit tests in validation tests #778

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

modocache
Copy link
Contributor

This repository includes several Python modules, each with unit tests.
Add a step to the validation tests to ensure these tests pass.


For now, there's only one module being tested: cmpcodesize. #761 adds unit tests for several components of utils/build-script, and I plan to add those tests to this command as well.

@gribozavr
Copy link
Contributor

@modocache Thank you! Could you instead integrate tests through lit? That is, via a file with a RUN line that does the same? This would allow unified running and reporting for all our tests.

@gribozavr gribozavr self-assigned this Dec 26, 2015
@dabrahams
Copy link
Contributor

Gyb has doctests too; you might want to run those

@modocache modocache force-pushed the add-python-unit-tests-to-build branch from 6bf7cc6 to 3e7b4f6 Compare December 26, 2015 17:52
@modocache
Copy link
Contributor Author

@dabrahams I noticed this CMake file, which purports to add a --test flag to gyb. I'm still trying to understand the build system so I'm not sure, but is it possible that gyb is already being tested in some way?

@gribozavr I updated the PR to use lit, picking up on runner commands in the README.

@gribozavr
Copy link
Contributor

@modocache Oh, I see -- it seems like the --test flag first runs the tests, and then does regular processing. So yes, gyb is being tested every time it is invoked from the build system.

@gribozavr
Copy link
Contributor

@modocache Thanks! I don't think you need to go as far as creating a new directory, and a readme. You can just add a new file validation-test/Python/cmpcodesize.swift that does the same.

@dabrahams
Copy link
Contributor

@modocache If we have a separate phase for tests of python scripts, it would be better to test gyb there, rather than running its tests each time it processes a file, right?

@modocache
Copy link
Contributor Author

If we have a separate phase for tests of python scripts, it would be better to test gyb there, rather than running its tests each time it processes a file, right?

I think so as well. I'd add it to this pull request, but I'm still trying to figure out how to run the gyb tests in the first place--see https://bugs.swift.org/browse/SR-364.

@modocache modocache force-pushed the add-python-unit-tests-to-build branch from 3e7b4f6 to af1cd9a Compare December 27, 2015 00:57
@modocache
Copy link
Contributor Author

@gribozavr Updated as per your comments. 👍

@gribozavr
Copy link
Contributor

@modocache Thank you, almost perfect! Please add the new substitution to docs/Testing.rst and I'll merge, thanks!

This repository includes several Python modules, each with unit tests.
Add a step to the validation tests to ensure these tests pass.
@modocache modocache force-pushed the add-python-unit-tests-to-build branch from af1cd9a to 0ede0c4 Compare December 27, 2015 06:57
@modocache
Copy link
Contributor Author

@gribozavr Oh, good catch!! Thanks. Updated!

gribozavr added a commit that referenced this pull request Dec 27, 2015
[test] Run Python unit tests in validation tests
@gribozavr gribozavr merged commit 4b6d7b9 into swiftlang:master Dec 27, 2015
@gribozavr
Copy link
Contributor

Thanks!

@modocache modocache deleted the add-python-unit-tests-to-build branch December 27, 2015 07:26
tiagomartinho pushed a commit to tiagomartinho/swift-corelibs-xctest that referenced this pull request Oct 6, 2017
swift-corelibs-xctest uses a Python program called xctest_checker to run its functional test suite. That program itself has unit tests, written in Python.
These unit tests should be run as part of the greater XCTest test suite. This is probably possible by modifying the lit.cfg in the swift-corelibs-xctest project such that it picks up on a file with a RUN: directive that triggers the Python tests. See swiftlang/swift#778 for an example.
See llvm.org/docs/CommandGuide/lit.html for additional details on lit--it's the test runner for Swift, LLVM, and other projects, so it's nice to know about it.
tiagomartinho pushed a commit to tiagomartinho/swift-corelibs-xctest that referenced this pull request Oct 11, 2017
swift-corelibs-xctest uses a Python program called xctest_checker to run its functional test suite. That program itself has unit tests, written in Python.
These unit tests should be run as part of the greater XCTest test suite. This is probably possible by modifying the lit.cfg in the swift-corelibs-xctest project such that it picks up on a file with a RUN: directive that triggers the Python tests. See swiftlang/swift#778 for an example.
See llvm.org/docs/CommandGuide/lit.html for additional details on lit--it's the test runner for Swift, LLVM, and other projects, so it's nice to know about it.
kateinoigakukun pushed a commit to kateinoigakukun/swift that referenced this pull request Sep 7, 2020
[pull] swiftwasm-release/5.3 from release/5.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants