Today, the home has become a mini enterprise that pushes the performance limits of an average residential access point. As a result, with the need to improve the Wi-Fi quality at home, extensive testing before deployments is inevitable. As part of our Wi-Fi testing services, ThinkPalm offers TR-398 test cases to bring in widely accepted, repeatable, and well-defined test cases.
TR-398 test cases measure and compare the real-world performance of access points (APs) by operators and their OEM/ODM vendors. In this article, we will discuss the TR-398 test cases and the results to showcase how we make efforts to ensure the best home Wi-Fi experience for our customers. Before we go into these details, we shall briefly discuss the TR-398 standard.
What is TR-398?
The TR-398 is an industry-standard test plan defined by the Broadband Forum that helps benchmark and compare the capabilities of APs to ensure the best Quality of Experience (QoE). TR-398 issue 2 covers the latest in Wi-Fi technology (Wi-Fi 6). Additionally, it provides a set of performance test cases with pass/fail requirements for 802.11n/ac/ax implementations to assist operators in the selection of Wi-Fi-capable devices.
TR-398 – Wi-Fi in-Premises Access Points Performance Testing
At ThinkPalm, with our TR-398 Issue 2 Wi-Fi testing plan, we provide a comprehensive set of tests to qualify the performance of Wi-Fi access points. For that reason, the primary goal of TR-398 is to provide a set of test cases and framework to verify the performance between Access Point (AP) and one or more Station (STA). It defines each test case as a scenario running between one or more Wi-Fi Stations (STA) and a Device Under Test (DUT).
TR-398 Issue 2 Performance Test Suite
We at ThinkPalm, offer a fully automated TR-398 Issue 2 performance test suite. All the required test hardware and fully automated test software, along with PASS/FAIL, feature a fully packaged, easy-to-use, and affordable solution.
The key features of the Wi-Fi testing solution are:
- Isolated from any outside interference for deriving reliable and repeatable results.
- Automated test scripts that give detailed reports
Key Tests from TR-398 Issue 2
In the real world, the AP needs to handle different types of clients at various distances, transmitting at different data rates and power values. The APs receiver should be able to handle all these scenarios to communicate successfully with clients and achieve good throughput.
Radio Frequency (RF) capability
Receiver Sensitivity Test
This Wi-Fi testing solution measures the receiver’s ability to receive and correctly demodulate weak signals. Further, it provides a simplified measurement of the receiver’s sensitivity relative to the total attenuation inserted between the DUT and the STA
Test Procedure
- Initialize attenuation to the starting value
- Generate UDP traffic by configuring the Traffic Generator
- Configure the attenuators to emulate a 2-meter distance
- Increment attenuation by 1 dB each step and run traffic for 20 seconds. Record the packet error rate (PER). If the PER is greater than 10%, then consider that as the sensitivity cutoff and move to the next iteration
- These test steps are run for each 45-degree spatial rotation
Baseline Performance
Maximum Connection Test
The test intends to verify that the Wi-Fi AP can support 32 STAs simultaneously with minimal packet loss and no disassociations taking place.
Test Procedure
- Establish the LAN connection and create and allow 32 stations to associate with the DUT
- Measure the downlink UDP packet loss using a test time of 120 seconds and a traffic rate of 2Mbps for 802.11n, 8Mbps for 802.11ac and 3Mbps for 802.11ax on 2.4GHz, 10Mbps for 802.11ax on 5GHz or 20Mbps for 802.11ax on 6GHz through each STA concurrently. Record the number of transmitted and received packets to calculate the packet error rate
- Repeat the test in the upload direction as well
Maximum Throughput Test
This Wi-Fi testing plan intends to measure the maximum throughput performance of the DUT. Also, it indicates the capability of a device to transmit data.
Airtime Fairness Test
The test verifies the capability of the Wi-Fi device to guarantee the fairness of airtime usage when handling a mix of clients using new and legacy (802.11a/b/g) standards.
Typically, 3 stations are used in this test
- Allow STA 1 and 2 to associate with DUT and measure traffic for 120 seconds
- Move STA 2 to a medium distance and measure the throughput again
- Replace STA2 with STA 3 (legacy device in 5GHz is 802.11a and in 2.4GHz is 802.11b/g) at a short distance and measure traffic in both STA 1 and 3 for 120 seconds
- Configure STA 3 for the 5GHz operating band and measure traffic for 120 seconds
Dual-band Throughput Test
The test is intended to measure the throughput the DUT can support when concurrently connected to multiple stations on both 2.4GHz and 5 or 6 GHz bands. The purpose of this test is to detect cross-band interference.
Bidirectional Throughput Test
In bidirectional throughput Wi-Fi testing, the maximum throughput performance of the DUT is measured when both uplink and downlink traffic is running. Also, it checks that the packet loss ratio will not exceed the specified value when AP performs uplink and downlink transmissions simultaneously.
Test Procedure
- Establish a LAN connection and allow STA to associate with the DUT.
- Measure the downlink TCP throughput for 120 seconds
- Measure the uplink TCP throughput for 120 seconds
- Create bidirectional UDP connection
- Record packet loss in each direction
Coverage
Range versus Rate Test
The test intends to measure the rate control, baseband and RF chain performance of Wi-Fi devices at different signal levels. The attenuation of signals due to range increase is provided by attenuators.
Rate versus Range versus Orientation Test
The test intends to measure the performance over the distance of the DUT. Also, it measures the performance of the AP at various antenna orientations using a programmable turn table with respect to the stations and at different emulated distances provided by attenuators. The test provides throughput and RSSI values related to signal and rotation plotted on a chart.
Spatial Consistency Test
The test is to verify the Wi-Fi signal consistency in the spatial domain. Further, this checks if there is any single dead zone when the AP is at a specific angle that leads to low throughput. In this test, AP is examined in terms of performance and stability at predefined attenuation levels and angles as per the TR-398 standard.
802.11ax Peak Performance Test
The test measures the maximum throughput performance of the DUT with a single station active. The AP should be configured for 160MHz on 5 GHz and 40 MHz on 2.4GHz bands. This Wi-Fi testing plan confirms the highest transmission performance in the 802.11ax mode.
Capacity
Multiple Stations Performance Test
This Wi-Fi testing plan focuses on measuring the performance of the AP with multiple stations at different distances to emulate real-world behaviour. There are three sets of three stations, with each group at a different emulated distance. Thus, in this case, the stability and performance of the connections are relatively important.
Multiple Association/ Disassociation Stability Test
The test intends to measure the stability of Wi-Fi devices under a dynamic environment with frequent changes in connection status. There are 16 stations associated with the DUT; 8 stations are picked for sending/ receiving packets, while the other 8 stations are picked to do a disassociation/ re-association process during the test.
Stability/ Robustness
Long-term Stability Wi-Fi Testing
The test intends to measure the stability performance of a Wi-Fi device under stress. It makes sure the AP can consistently achieve high throughput and stability over a long test duration.
AP Coexistence Test
The Wi-Fi testing plan verifies the performance of DUT with existence of alien AP. The alien AP shall support the same Wi-Fi standard (802.11n/ 802.11ac/ 802.11ax). The test intends to simulate AP resistance in case of external interference, such as co-channel interference and adjacent channel interference, to check whether it is possible to maintain stability and performance when disturbed.
Why Should You Rely on ThinkPalm’s TR-398 Wi-Fi Testing?
After analyzing this test plan in detail and our experience in Wi-Fi testing, we assure you that any access point that passes all these tests has a very good chance of providing customers with an excellent overall Wi-Fi experience when deployed.
At ThinkPalm, with the help of the TR-398 standard, we help test indoor Wi-Fi performance and build home networks to enhance the broadband experience for our customers. We verify multiple performance, stability and sensitivity factors related to Wi-Fi chipsets.
Experience the benefits of this Wi-Fi testing plan and ensure the quality and performance of your home Wi-Fi devices!
Author Bio
Jincymol P is an Embedded and Wi-Fi system professional working at ThinkPalm Technologies, and is interested in researching about recent developments in technology. She is an MS Scholar in Embedded Systems from University of Greenwich, UK. Her area of expertise includes embedded systems, wireless networks and Internet of Things.
