This involves taking the current built participant infrastructure, reviewing the test plans and unit tests, and performing the tests.
- Review the above test plan. The list of test cases is incomplete. Don't worry about fleshing them out fully. Instead, take Chris's feedback and Thomas's feedback (shown below) to create a set of test scenarios at the top of the file and run through those. We will leave the creation of exhaustive test cases to later when we have more QA resources.
- Review the unit tests. See if they exercise the functionality. Run them through Emma to check for code coverage
- Run the test scenarios and file any bugs found.
Checkout the "participant" branch of studio3, studio3-ruby and studio3-php
- All tests are marked as full. The point of a test plan is to break functionality into smoke, acceptance and full levels of test. If the tester can't figure out these coverage levels, they need to work with dev to figure it out.
- Resource requirements are not filled out - now way of knowing from this plan what Os are being tested or targeted
- An actual description of what "Build Participant functionality" is would be helpful if the test plan is going to be run by anyone other than the author. There seems to be multiple features in the test cases
- No risks or assumptions are filled out
- Often missing set up steps - For the PHP and Ruby specific test cases, how do we setup our environment to test those test cases?
- Under genericized tests, it says "Applicable to all languages." Which programming languages is being tested: JS, Java, Objective C?
- Unclear if it applies to Aptana or Tistudio.
- Not seeing links to any dev specs: http://wiki.appcelerator.org/display/tis/Participating+in+the+build+and+reconcile+process
Ignore for now:
- Overall, the test cases should be more detailed since the test plan seems to be dealing with different programatic syntax and the test plan needs to clearly define what it is testing. (ignore for now)
- Not sure of why the tests at the top are repeated from 1012 on with a "syntactically sound" js file. Is there a target for this specific set of tests?
- In the "Declarations and documentation" tests, need specific examples for the programatic syntax (e.g. Document a class, declare a class, declare a function etc.)
- 1112 - 1115 refer to Aptana Studio. Also the location is Mac-only for the preferences. Unless the testing is Mac only either leave out the menu path or put Linux and Win info too.