Today, 06/03/2021, the QA team (@IanM @Ralkage and I) got together to conduct our first ever team meeting!
I am pleased to report that this meeting was very successful, and we discussed a range of topics to ensure that we continue to deliver the highest quality product that we possibly can; with this in mind, a range of topics were discussed.
1) Our automated tests will be split out to individual test cases, allowing us to ensure that core functionality of Flarum continues to work with every commit – this allows us to quickly capture any issues, and create fixes where necessary, or revert commits where required.
2) We will be introducing a change to our QA suite to ensure that we test against a ‘golden image’ that we revert to prior to any test cases. This will help us to ensure that outside variables are eradicated and allows a fair baseline for comparing performance metrics between tests, allowing us to identify any performance impacts following a commit.
3) With thanks to @IanM we will be introducing Docker images (😮) to the suite. This will allow our QA members to quickly spin up local test suites on which they can test a range of PHP versions, Server types (NginX / Apache) and also to ensure a standardised testing environment across the team.
4) We will once again be reaching out to Cypress’ staff team to ascertain an open source license so as to gain access to priority support, better data retention periods for our test results, video recordings of tests, and more!
5) We will be creating a console job to clear the database and revert to a golden image prior to any test – again to standardise our test suite.
6) We will also be creating a QA lab, allowing our members to test and create any QA tool on an isolated instance without risk of ‘spill-out’
7) We have discussed our current testing checklist, and are pleased to announce that we will be adding additional tests for multi-browser, and multi-device (and OS) checks.