Dynamically generating FlexUnit test suite

I have been working with Flex
recently, which is a framework for building Flash applications. The
standard testing framework for Flex is
FlexUnit. A good example of FlexUnit can be found at How to use FlexUnit with FlexBuilder
2

One limitation of FlexUnit is that it does not have a way to dynamically
build a test suite at runtime. All of the examples online have a
manually created and maintained test suite that looks like:

package {  
  import flexunit.framework.TestSuite;
  import com.foo.FooTest;
  import com.bar.BarTest;

  public class Suite {

    public static function createTestSuite() : TestSuite {

      var ts : TestSuite = new TestSuite();
      ts.addTestSuite(FooTest);
      ts.addTestSuite(BarTest);
      // Many more additions
      return ts;
    }
  }
}

We believe in constantly writing and refactoring tests, and we did not
want to manually maintain this test suite. We first investigated reading
the filesystem to get the list of available tests at runtime. However,
since the tests run inside of flash (in a browser or standalone player),
there is no access to the filesystem. Kent
Spillner
and
I brainstormed and came up with a build time solution instead.

First, we wrote a ruby script that would read the filesystem and
generate the test suite file. We called the script
test_suite_generator.rb, and it looks something like:

#!/usr/bin/env ruby

require 'find'

search_path = File.dirname(__FILE__) + '/test/'

test_cases = []  
Find.find search_path do |path|  
  filename = File.basename(path)
  Find.prune if filename =~ /^\./
  test_cases << path.gsub("#{search_path}", '').gsub('.as', '').gsub('/', '.') if filename =~ /Test\.as$/
end

test_cases.sort!

File.open('test/Suite.as', 'w') do |file|  
  file.puts <<EOF
// This file is generated by #{File.basename(__FILE__)}.
package {  
  import flexunit.framework.TestSuite;
EOF  
  test_cases.each {|tc| file.puts "  import #{tc};"}
  file.puts <<EOF

  public class Suite {

    public static function createTestSuite() : TestSuite {

      var ts : TestSuite = new TestSuite();
EOF  
  test_cases.each {|tc| file.puts "      ts.addTestSuite(#{tc});"}
  file.puts <<EOF
      return ts;
    }
  }
}
EOF  
end  

This script uses Find to get all of the test files in the test folder
(files that end in Test.as). For each file, it turns the path into a package (com/foo/HelloWorldTest.as -> com.foo.HelloWorldTest). Then,
it writes a file called Suite.as into the test directory which imports
each of these packages and then adds them to a TestSuite object.

We use Flex
Builder
for
flex development, which is based on eclipse. This allows us to add a
custom builder which will run our ruby script.

In Project -> Properties -> Builders, choose New -> Program.
Type TestSuiteGenerator for the name, and browse for the ruby script in
the Location field:

{width="601px" height="479px"}

Then, on the “Build Options” tab, select “During auto builds” and
“Specify working set of relevant resources:”

{width="601px" height="479px"}

Finally, click on “Specify Resources…” and choose the folder where your
tests live. Make sure that you don’t select the folder that includes
Suite.as, or the generation of the file will kick off the builder again.
If your tests are nested inside a “com” folder, then choose “com.”

Now, whenever we add or rename a test, Flex Builder runs
test_suite_generator.rb and regenerates the Suite.as file. We don’t
have to manually maintain anything.