Automation Academy
Frameworks & Patterns
TestNG Complete Guide

TestNG Complete Guide: Advanced Java Testing Framework

Parul Dhingra - Senior Quality Analyst
Parul Dhingra13+ Years ExperienceHire Me

Senior Quality Analyst

Updated: 1/23/2026

TestNG (Test Next Generation) has been a cornerstone of Java testing since 2004, offering features that JUnit lacked at the time - annotations, parallel execution, data-driven testing, and flexible configuration. While JUnit 5 has caught up in many areas, TestNG remains popular in enterprise environments and is the framework of choice for Selenium test automation.

This guide covers TestNG from basic concepts to advanced features that make it powerful for complex testing scenarios.

What is TestNG?

TestNG is a testing framework for Java inspired by JUnit and NUnit but with additional capabilities:

  • Annotations: Flexible test configuration
  • Data Providers: Built-in parameterized testing
  • Groups: Organize and filter tests
  • Parallel execution: Run tests concurrently
  • Dependencies: Define test execution order
  • XML configuration: External test suite definition
  • Listeners: Customize test behavior and reporting

TestNG is widely used for:

  • Unit testing
  • Integration testing
  • End-to-end testing (especially with Selenium)
  • API testing (with REST Assured)

Setup and Configuration

Maven Dependency

<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.9.0</version>
    <scope>test</scope>
</dependency>

Gradle Dependency

dependencies {
    testImplementation 'org.testng:testng:7.9.0'
}
 
test {
    useTestNG()
}

IDE Integration

  • IntelliJ IDEA: Built-in TestNG support
  • Eclipse: Install TestNG plugin from Marketplace
  • VS Code: Use Java Test Runner extension

Annotations

Core Test Annotations

import org.testng.annotations.*;
 
public class AnnotationsDemo {
 
    @Test
    public void testMethod() {
        // Test code
    }
 
    @Test(description = "Verifies user login")
    public void testWithDescription() {
        // Test code
    }
 
    @Test(enabled = false)
    public void disabledTest() {
        // Skipped during execution
    }
 
    @Test(timeOut = 5000)
    public void testWithTimeout() {
        // Fails if takes longer than 5 seconds
    }
 
    @Test(expectedExceptions = IllegalArgumentException.class)
    public void testExpectedException() {
        throw new IllegalArgumentException("Expected");
    }
}

Configuration Annotations

public class ConfigurationDemo {
 
    @BeforeSuite
    public void beforeSuite() {
        // Runs once before all tests in suite
    }
 
    @AfterSuite
    public void afterSuite() {
        // Runs once after all tests in suite
    }
 
    @BeforeTest
    public void beforeTest() {
        // Runs before each <test> in XML
    }
 
    @AfterTest
    public void afterTest() {
        // Runs after each <test> in XML
    }
 
    @BeforeClass
    public void beforeClass() {
        // Runs once before first test in class
    }
 
    @AfterClass
    public void afterClass() {
        // Runs once after last test in class
    }
 
    @BeforeMethod
    public void beforeMethod() {
        // Runs before each @Test method
    }
 
    @AfterMethod
    public void afterMethod() {
        // Runs after each @Test method
    }
}

Execution Order

@BeforeSuite
  @BeforeTest
    @BeforeClass
      @BeforeMethod
        @Test
      @AfterMethod
      @BeforeMethod
        @Test
      @AfterMethod
    @AfterClass
  @AfterTest
@AfterSuite

@BeforeSuite and @AfterSuite run once per XML suite file, regardless of how many test classes are included.

Test Configuration

testng.xml Suite File

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression Suite" verbose="1">
 
    <test name="Login Tests">
        <classes>
            <class name="com.example.LoginTest"/>
            <class name="com.example.LogoutTest"/>
        </classes>
    </test>
 
    <test name="User Tests">
        <packages>
            <package name="com.example.users.*"/>
        </packages>
    </test>
 
</suite>

Suite Parameters

<suite name="Suite" parallel="tests" thread-count="3">
    <parameter name="browser" value="chrome"/>
    <parameter name="environment" value="staging"/>
 
    <test name="Chrome Tests">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="com.example.BrowserTest"/>
        </classes>
    </test>
</suite>

Using parameters in tests:

public class BrowserTest {
 
    @Parameters({"browser", "environment"})
    @BeforeClass
    public void setup(String browser, String environment) {
        System.out.println("Browser: " + browser);
        System.out.println("Environment: " + environment);
    }
 
    @Test
    public void testHomePage() {
        // Test code
    }
}

Data Providers

Data Providers enable data-driven testing by supplying different data sets to test methods.

Basic Data Provider

public class DataProviderDemo {
 
    @DataProvider(name = "loginData")
    public Object[][] loginDataProvider() {
        return new Object[][] {
            {"user1", "pass1", true},
            {"user2", "pass2", true},
            {"invalid", "wrong", false}
        };
    }
 
    @Test(dataProvider = "loginData")
    public void testLogin(String username, String password, boolean expectedResult) {
        boolean result = performLogin(username, password);
        Assert.assertEquals(result, expectedResult);
    }
}

Data Provider in Separate Class

public class TestDataProviders {
 
    @DataProvider(name = "userData")
    public static Object[][] userData() {
        return new Object[][] {
            {"John", "john@test.com"},
            {"Jane", "jane@test.com"}
        };
    }
}
 
public class UserTest {
 
    @Test(dataProvider = "userData", dataProviderClass = TestDataProviders.class)
    public void testCreateUser(String name, String email) {
        // Test code
    }
}

Data Provider with Method Reflection

@DataProvider(name = "testData")
public Object[][] getData(Method method) {
    if (method.getName().equals("testLogin")) {
        return new Object[][] {{"user1", "pass1"}};
    } else if (method.getName().equals("testSearch")) {
        return new Object[][] {{"query1"}, {"query2"}};
    }
    return new Object[][] {{}};
}

Parallel Data Provider

@DataProvider(name = "parallelData", parallel = true)
public Object[][] parallelDataProvider() {
    return new Object[][] {
        {"data1"},
        {"data2"},
        {"data3"},
        {"data4"}
    };
}
 
@Test(dataProvider = "parallelData")
public void testParallelExecution(String data) {
    // Each data set runs in parallel
}

Test Groups

Groups organize tests for selective execution.

Defining Groups

public class GroupsDemo {
 
    @Test(groups = {"smoke"})
    public void smokeTest1() { }
 
    @Test(groups = {"smoke", "regression"})
    public void smokeAndRegressionTest() { }
 
    @Test(groups = {"regression"})
    public void regressionTest() { }
 
    @Test(groups = {"integration", "slow"})
    public void integrationTest() { }
}

Running Groups via XML

<suite name="Suite">
    <test name="Smoke Tests">
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="com.example.GroupsDemo"/>
        </classes>
    </test>
 
    <test name="All Except Slow">
        <groups>
            <run>
                <include name=".*"/>
                <exclude name="slow"/>
            </run>
        </groups>
        <packages>
            <package name="com.example.*"/>
        </packages>
    </test>
</suite>

Class-Level Groups

@Test(groups = {"functional"})
public class UserFunctionalTests {
 
    public void testCreateUser() { }  // Inherits 'functional' group
 
    public void testUpdateUser() { }  // Inherits 'functional' group
 
    @Test(groups = {"smoke"})  // Both 'functional' and 'smoke'
    public void testDeleteUser() { }
}

Parallel Execution

TestNG supports parallel execution at multiple levels.

Parallel Methods

<suite name="Suite" parallel="methods" thread-count="5">
    <test name="Test">
        <classes>
            <class name="com.example.ParallelTest"/>
        </classes>
    </test>
</suite>

Parallel Classes

<suite name="Suite" parallel="classes" thread-count="3">
    <test name="Test">
        <classes>
            <class name="com.example.Test1"/>
            <class name="com.example.Test2"/>
            <class name="com.example.Test3"/>
        </classes>
    </test>
</suite>

Parallel Tests

<suite name="Suite" parallel="tests" thread-count="2">
    <test name="Chrome Test">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="com.example.BrowserTest"/>
        </classes>
    </test>
    <test name="Firefox Test">
        <parameter name="browser" value="firefox"/>
        <classes>
            <class name="com.example.BrowserTest"/>
        </classes>
    </test>
</suite>

Thread-Safe Test Design

public class ThreadSafeTest {
    // Thread-local driver for parallel execution
    private static ThreadLocal<WebDriver> driver = new ThreadLocal<>();
 
    @BeforeMethod
    public void setup() {
        driver.set(new ChromeDriver());
    }
 
    @AfterMethod
    public void teardown() {
        driver.get().quit();
        driver.remove();
    }
 
    @Test
    public void testParallel() {
        driver.get().get("https://example.com");
        // Thread-safe access to driver
    }
}

Dependencies

Method Dependencies

public class DependencyDemo {
 
    @Test
    public void login() {
        // Login first
    }
 
    @Test(dependsOnMethods = {"login"})
    public void navigateToDashboard() {
        // Runs after login passes
    }
 
    @Test(dependsOnMethods = {"navigateToDashboard"})
    public void performAction() {
        // Runs after navigateToDashboard passes
    }
}

Group Dependencies

@Test(groups = {"setup"})
public void createTestData() { }
 
@Test(groups = {"setup"})
public void configureEnvironment() { }
 
@Test(dependsOnGroups = {"setup"})
public void runActualTest() {
    // Runs after all 'setup' group tests pass
}

Soft Dependencies

// alwaysRun=true runs even if dependency fails
@Test(dependsOnMethods = {"login"}, alwaysRun = true)
public void testLogout() {
    // Cleanup code that should run regardless
}
⚠️

Avoid creating circular dependencies. TestNG will throw an exception if it detects dependency cycles.

Listeners and Reporting

ITestListener

public class TestListener implements ITestListener {
 
    @Override
    public void onTestStart(ITestResult result) {
        System.out.println("Starting: " + result.getName());
    }
 
    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println("Passed: " + result.getName());
    }
 
    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("Failed: " + result.getName());
        // Take screenshot, log details, etc.
    }
 
    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println("Skipped: " + result.getName());
    }
}

Registering Listeners

Via annotation:

@Listeners(TestListener.class)
public class MyTest {
    @Test
    public void testMethod() { }
}

Via XML:

<suite name="Suite">
    <listeners>
        <listener class-name="com.example.TestListener"/>
    </listeners>
    <test name="Test">
        <classes>
            <class name="com.example.MyTest"/>
        </classes>
    </test>
</suite>

Custom Reporter

public class CustomReporter implements IReporter {
 
    @Override
    public void generateReport(List<XmlSuite> xmlSuites,
                               List<ISuite> suites,
                               String outputDirectory) {
        for (ISuite suite : suites) {
            Map<String, ISuiteResult> results = suite.getResults();
            for (ISuiteResult result : results.values()) {
                ITestContext context = result.getTestContext();
                // Generate custom report
                System.out.println("Passed: " + context.getPassedTests().size());
                System.out.println("Failed: " + context.getFailedTests().size());
            }
        }
    }
}

Best Practices

Organize Tests Logically

src/test/java/
├── com/example/
│   ├── base/
│   │   └── BaseTest.java
│   ├── listeners/
│   │   └── TestListener.java
│   ├── dataproviders/
│   │   └── TestData.java
│   ├── login/
│   │   ├── LoginTest.java
│   │   └── LogoutTest.java
│   └── users/
│       ├── CreateUserTest.java
│       └── UpdateUserTest.java
└── resources/
    └── testng.xml

Use Base Test Classes

public abstract class BaseTest {
 
    @BeforeClass
    public void baseSetup() {
        // Common setup
    }
 
    @AfterClass
    public void baseTeardown() {
        // Common cleanup
    }
 
    protected void commonHelper() {
        // Shared utility methods
    }
}
 
public class LoginTest extends BaseTest {
 
    @Test
    public void testLogin() {
        commonHelper();
        // Test code
    }
}

Meaningful Test Names

// Good
@Test
public void login_WithValidCredentials_ShouldRedirectToDashboard() { }
 
@Test
public void createUser_WithDuplicateEmail_ShouldReturn409() { }
 
// Avoid
@Test
public void test1() { }

Assertions Best Practices

import org.testng.Assert;
import static org.testng.Assert.*;
 
@Test
public void testWithAssertions() {
    // Include meaningful messages
    assertEquals(actual, expected, "User name should match");
    assertTrue(isLoggedIn(), "User should be logged in after login");
    assertNotNull(response, "Response should not be null");
 
    // Soft assertions for multiple checks
    SoftAssert softAssert = new SoftAssert();
    softAssert.assertEquals(name, "John");
    softAssert.assertEquals(email, "john@test.com");
    softAssert.assertAll(); // Fails if any assertion failed
}

TestNG's rich feature set makes it ideal for complex test scenarios requiring parallel execution, data-driven testing, and flexible configuration. Its XML-based suite definition allows running different test configurations without changing code, making it particularly valuable in enterprise testing environments.

Quiz on TestNG

Your Score: 0/10

Question: What is the correct order of TestNG configuration annotation execution?

Continue Reading

The Software Testing Lifecycle: An OverviewDive into the crucial phase of Test Requirement Analysis in the Software Testing Lifecycle, understanding its purpose, activities, deliverables, and best practices to ensure a successful software testing process.Types of Software TestingThis article provides a comprehensive overview of the different types of software testing.Accessibility TestingLearn about accessibility testing, its importance, types, best practices, and tools.Unit Testing in SoftwareLearn the fundamentals of unit testing in software, its importance in functional testing, and how to ensure early bug detection, improved code quality, and seamless collaboration among team members.Integration TestingLearn the essentials of integration testing, its importance, types, best practices, and tools.System TestingLearn about system testing, its importance, types, techniques, process, best practices, and tools to effectively validate software systems.Performance TestingLearn about performance testing, its importance, types, techniques, process, best practices, and tools to effectively validate software systems.Security TestingLearn about security testing, its importance, types, techniques, process, best practices, and tools to effectively validate software systems.User Acceptance TestingLearn about user acceptance testing, its importance, types, techniques, process, best practices, and tools to effectively validate software systems.

Frequently Asked Questions (FAQs) / People Also Ask (PAA)

What's the difference between TestNG and JUnit?

How do I run TestNG tests from the command line?

Can I run the same test with different data without DataProvider?

How do I retry failed tests automatically?

What's the difference between @BeforeTest and @BeforeClass?

How do I exclude certain tests from running?

Can I use both TestNG and JUnit in the same project?

How do I generate HTML reports with TestNG?

Mobile Testing Best PracticesJUnit 5 Complete Guide