Industry Solution- Ethernet Product Automation Test
A global leading Ethernet 10GE technologies and product vendor – the test on its network adaptor product line is to cover dozens of existing OS and their combinations, including Windows 2000 Home/Professional/Server, WindowsXP (Home/Professional), Windows 2003 Server, Linux RHEL3, RHEL4, RHEL5, SUSE. About 100 combinations of 32/64 bit versions, packs, and the Intel/AMD CPUs add even more technical complexity to the test jobs.
Conventionally, test engineers will set out with a series of manual processes, such as building network topology, installing operation systems of different versions, installing network adaptor drivers, test traffic generation, data analysis and examination. Because the coordination between multiple devices involves such factors as topology connection, installation of different operation systems and data statistics/analysis, the processes other than a few traffic generation steps are difficult to automate. To secure the product delivery schedule, the customer had to invest huge human and equipment resources for work coordination, setting up a gigantic testbed for different operation systems and the critical OS version-related device drivers, which requires over a hundred PCs, driving up cost.
The customer could consider virtual machines like VMware to cut back on hardware investment. For the 64-bit system’s downward compatibility, the hardware may consist of several 64-bit Intel and AMD PCs with enough memories, in addition to the installment of all required OS versions on the virtual machine. However, the release of each product driver is always accompanied with backlog of regression test; it is repetitive, error-prone and low effective.
Sigma provided an automation test solution to the product line: orchestrate the testbed equipment with our full-IP automation test framework - SigmationTF, and fulfill the automation missions from automated network topology connection, OS (with varied versions) installation, network adaptor driver installation, traffic generation, test data analysis/examination, log generation and reporting.
Testbed configuration of the SigmationTF solution is shown in the following chart. Except for the two PCs acting as master controller (the MC) and mapping file server, a testbed, scalable, only requires one test switch, two PCs, one power controller and one terminal server, cutting down cost.
The MC, designed in the SigmationTF automation test solution, assigns tasks /equipment resources and the priority of tester-submitted Jobs, controls APC (automation power controller) to power on the test PCs, KickStart PCs in the control network, and then downloads and installs specific operation systems from the mapping file sever with certain task parameters before unattended installation. The PCs automatically reboot after OS installation, and download target network adaptor drivers and test programs from the mapping file server for compiling and installation.
The MC controls PCs to run automation test processes designed in the test cases, check test result (pass/failed) and collects test data/logs with test reporting and analysis. The entire test process will be fully automated without manual intervention. The entire test running process is under real-time monitoring of a logging system.
tests running, dramatically improving test efficiency. The two testbeds, respectively equipped with a 64-bit Intel PC and a 64-bit AMD PC, can cover all driver tests under these two circumstances (Intel and AMD), as well as test jobs on 32-bit operation systems.
The SigmationTF solution requires minimum two testbeds with four PCs for the test job fulfillment. The test efficiency can be multiplied by simple testbed expansion and meet schedules of varied test plans.
Obviously, the SigmationTF solution greatly saves human resources and hardware investment compared to traditional measures. A complicated test job involving dozens of, or even hundreds of testers and huge equipment resource now can be precisely conducted by only a few engineers and much less device investment.