Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide regression tests (in nsiqunittest/nsiqcppstyle_unittest.py) for NsiqCppStyle engine internals #44

Open
mosherubin opened this issue Jul 14, 2022 · 1 comment

Comments

@mosherubin
Copy link
Collaborator

Background

NsiqCppStyle is a serious Open Source software project, intended for general use. Every time a change is made to the engine code, there is a real risk of breaking the existing implementation and/or features. NsiqCppStyle requires high-quality and reliable regression tests to ensure that changes to the existing engine code base won't affect the existing features of the project.

As expected of a well-designed product, NsiqCppStyle has a dedicated unit/regression test source file: nsiqunittest/nsiqcppstyle_unittest.py. The file defines a class unitTest with member functions, each one representing a unit/regression test. This script file currently has a small number of tests:

Test Name Description
unitTest.__testFunctionSpecifier Internal utility function used by the testIgnore function
unitTest.testIgnoreFinalFunctionSpecifier Verifies that the final token is ignored
unitTest.testIgnoreOverrideFunctionSpecifier Verifies that the override token is ignored
unitTest.testIgnoreNoexceptFunctionSpecifier Verifies that the noexcept token is ignored
unitTest.testGetPrevMatchingLT Verifies lexer.GetPrevMatchingLT() works correctly for simple variable definition/declaration statement using '<...>'
unitTest.testGetPrevMatchingLTWithInner Verifies lexer.GetPrevMatchingLT() works correctly for variable definition/declaration statement using nested '<...>' pairs, no occurrence of adjacent LT or GT
unitTest.testGetPrevMatchingLTWithInnerOnes Verifies lexer.GetPrevMatchingLT() works correctly for variable definition/declaration statement using nested '<...>' pairs, has occurrence of adjacent LT or GT
unitTest.test2 Verifies all tokens can be iterated using GetNextToken
unitTest.test3 Verifies all tokens can be iterated using GetNextTokenSkipWhiteSpaceAndComment
unitTest.test4 Same as test3 with console level set to console.Level.Verbose
unitTest.test5 Verifies the first token of a complex definition/declaration has type ID and expected value

It is clear that these tests are the proverbial "tip of the iceberg", and that many more such tests should be added to ideally provide complete coverage of internal engine code.

Request

Effort should be invested to write as many targeted unit/regression tests for NsiqCppStyle internals as possible. As more people submit code changes, having a rich set of regression tests will give us the confidence needed to make changes without breaking the existing code base.

The writing of such tests should be an ongoing task: every time any contributor (or the maintainer) touches a section of NsiqCppStyle engine code, time should be allocated for enriching the set of unit/regression tests, all related to the section/features modified.

Similarly, while implementing a new feature, should a contributor break the product during development, s/he should stop and write the necessary unit/regression tests to never allow that break to happen again.

IMHO, this should be a high-priority task as soon as the current pull requests have settled and are merged.

@mosherubin
Copy link
Collaborator Author

Considering the numerous different functional areas in the engine (e.g., tokenization, iterating the token list, verifying lexer member functions and fields), we might want to have a separate unit test source file for each area, rather than lumping them all together into a single source file. Tests should be inserted in the appropriate unit test source file. All unit test source files should then be imported into nsiqunittest/nsiqcppstyle_unittest.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant