Review Board 1.7.16


Add an option to the test suite that makes it believe that the version of Asterisk under test is some other version

Review Request #2202 - Created Nov. 20, 2012 and submitted

Matt Jordan
testsuite
Reviewers
asterisk-dev
testsuite
At first glance, this patch sounds stupid at best, harmful at worst.  However, there is a purpose to it...

The Asterisk Test Suite has to "understand" Asterisk version numbers.  It uses the version reported by the installed Asterisk to determine if a test can run.  In general, this is used so that tests written against Asterisk 10 don't execute when the version of Asterisk is 1.8 (and in other similar cases).  This mechanism works great for Asterisk version branches and tags (minus all of the usual shenanigans that ensue given all of the ways in which we've changed the Asterisk versioning scheme).

Trunk gets a little funky.  We treat trunk as being greater then, well, everything.  Fair enough.

Team branches are worse.  What is mjordan-cool-branch?  How do you know that branch was based off of 1.8, 10, 11, or trunk?  Inspecting subversion properties for the information gets complicated, and puts some constraints on the Test Suite that we're trying to avoid.  Currently, we treat team branches as being equivalent to trunk.  That works for any team branch that is being used to develop new features against trunk.

This doesn't work if the team branch is actually related to 1.8, or some other version of Asterisk.  If you install a team branch that is based on 1.8, all tests will be executed - including those that require features developed in later versions of Asterisk.  Those tests will fail spectacularly, making it very difficult to tell if your branch has legitimate issues.

Rather then trying to manipulate the version parsing schemes (again), this patch adds an option to the test configurations that allow them to override the Asterisk version under test.  In the top level test-config.yaml, define:

properties:
  forced-version: 1.8.11.1

Or some other version you'd like to make the Test Suite think its running against.
Used the forced-version to make the Test Suite think it was running against 1.8.0.0.  Checked out a team branch based on trunk, compiled, installed, and attempted to run a ConfBridge test (confbridge_marked).  Test refused to run, because it thought it was running against 1.8.  Removed the forced-version; test ran.

Changes between revision 2 and 3

1 2 3
1 2 3

  1. /asterisk/trunk/runtests.py: Loading...
/asterisk/trunk/runtests.py
Diff Revision 2 Diff Revision 3
[20] 169 lines
[+20] [+] def __parse_run_output(self, output):
170
class TestSuite:
170
class TestSuite:
171
    def __init__(self, ast_version, options):
171
    def __init__(self, ast_version, options):
172
        self.options = options
172
        self.options = options
173

    
   
173

   
174
        self.tests = []
174
        self.tests = []
175
        self.tests = self._parse_test_yaml("tests", ast_version)

   
176
        self.global_config = self._parse_global_config()
175
        self.global_config = self._parse_global_config()

    
   
176
        self.tests = self._parse_test_yaml("tests", ast_version)
177
        self.total_time = 0.0
177
        self.total_time = 0.0
178
        self.total_count = 0
178
        self.total_count = 0
179
        self.total_failures = 0
179
        self.total_failures = 0
180

    
   
180

   
181
    def _parse_global_config(self):
181
    def _parse_global_config(self):
[+20] [20] 301 lines
  1. /asterisk/trunk/runtests.py: Loading...

https://reviewboard.asterisk.org/ runs on a server provided by Digium, Inc. and uses bandwidth donated to the open source Asterisk community by API Digital Communications in Huntsville, AL USA.
Please report problems with this site to asteriskteam@digium.com.