Pest 3: Mutation Testing

This is a new feature in the Pest 3 version.

Mutation testing makes small changes (mutations) to your code and then runs the tests to check if they are still passing. It's a great way to identify weaknesses in your test suite.

Notice: this feature requires XDebug 3.0+ or PCOV.

Running Mutation Test

To start mutation testing, first, you must specify which part you are testing using the covers() method in the test.

For example, Laravel Breeze comes with some Pest tests. Let's check the registration testing.

tests/Feature/Auth/RegistrationTest.php:

covers(\App\Http\Controllers\Auth\RegisteredUserController::class); 
 
test('registration screen can be rendered', function () {
    $response = $this->get('/register');
 
    $response->assertStatus(200);
});
 
test('new users can register', function () {
    $response = $this->post('/register', [
        'name' => 'Test User',
        'email' => '[email protected]',
        'password' => 'password',
        'password_confirmation' => 'password',
    ]);
 
    $this->assertAuthenticated();
    $response->assertRedirect(route('dashboard', absolute: false));
});

Now, we can run the test suite with mutation testing.

php artisan test --mutate

We can see that the mutation shows 18 mutations untested, and the score is 28%.

Adding One Mutation

One of the untested cases is about the Event. And if we look at the tests/Feature/Auth/RegistrationTest.php test, there are no tests that the Event is fired.

Let's add a quick test to check if the Event is fired.

tests/Feature/Auth/RegistrationTest.php:

use Illuminate\Support\Facades\Event;
use Illuminate\Auth\Events\Registered;
 
covers(\App\Http\Controllers\Auth\RegisteredUserController::class);
 
test('registration screen can be rendered', function () {
    $response = $this->get('/register');
 
    $response->assertStatus(200);
});
 
test('new users can register', function () {
    $response = $this->post('/register', [
        'name' => 'Test User',
        'email' => '[email protected]',
        'password' => 'password',
        'password_confirmation' => 'password',
    ]);
 
    $this->assertAuthenticated();
    $response->assertRedirect(route('dashboard', absolute: false));
});
 
test('even is fired after user registers', function () { 
    Event::fake();
 
    $response = $this->post('/register', [
        'name' => 'Test User',
        'email' => '[email protected]',
        'password' => 'password',
        'password_confirmation' => 'password',
    ]);
 
    Event::assertDispatched(Registered::class);
}); 

After rerunning mutation testing, we can see that the number of untested mutations has dropped from 18 to 17, and the score has increased from 28% to 32%. So, we improved the tests.

Ignore Some Mutations

Another case is that you would need to ignore some lines of the code. For example, many untested registration mutations come from the validation rules. We can ignore those lines by adding // @pest-mutate-ignore.

app/Http/Controllers/Auth/RegisteredUserController.php:

class RegisteredUserController extends Controller
{
    // ...
 
    public function store(Request $request): RedirectResponse
    {
        $request->validate([ 
            'name' => ['required', 'string', 'max:255'], // @pest-mutate-ignore
            'email' => ['required', 'string', 'lowercase', 'email', 'max:255', 'unique:'.User::class], // @pest-mutate-ignore
            'password' => ['required', 'confirmed', Rules\Password::defaults()], // @pest-mutate-ignore
        ]);
 
        // ...
    }
}

Now, only two untested mutations are left after running the mutation testing.

This way, you would check every untested mutation line by line and add tests for them.

100% Score is Not a Strict Requirement

Of course, your goal should be to score as high as possible, but that's not a requirement. However, a higher score would make upgrades, refactors, or adding new features easier, as you would feel more secure with fewer bugs.

You should read the official documentation to find out more.