cult3

Setting up Mandrill Webhooks with Laravel

Feb 15, 2016

Table of contents:

  1. The problem we are looking to resolve
  2. How this is going to work
  3. Setting up the Route
  4. Adding the Controller
  5. Writing an Acceptance Test
  6. Authenticating the Webhook
  7. Accepting the Webhook
  8. The Event Value Object
  9. Creating the abstract Webhook class
  10. Creating the first Webhook implementation
  11. Conclusion

I really love the simplicity of transactional email API services such as Mandrill.

It’s so much easier to send application emails when all you need to do is to make an API request, rather than dealing with the headache of an email server.

However, when sending emails, there is a lot of things that can go wrong.

Emails can be rejected, bounced, or marked as spam, leaving your users wondering why they did not receive their confirmation, invitation, or notification from a friend.

One of the benefits of using a service such as Mandrill is that they also provide Webhooks (What are Webhooks?) on events such as when the email was opened, a link was clicked, or when the email was rejected, or bounced.

This means we can listen for these Webhooks and take the appropriate action to notify the user of the problem, so they are not left in the dark.

In today’s tutorial we are going to be looking at setting up and using Mandrill Webhooks in a Laravel application.

The problem we are looking to resolve

Imagine we’re building a Project Management application that allows teams of users to collaboratively work together to deliver a project.

An important part of the on-boarding process is inviting new team members to the application so they can also work on the project.

However, if the invitation email is rejected or it is bounced, the invited user will be left in limbo and unable to access the project.

In order to solve this problem we need a way of notifying the user who sent the invitation of the rejection so they can double check to make sure the email address they entered was correct.

And lo and behold, this is the perfect situation to use Mandrill’s Webhooks!

We need to set up a Mandrill Webhook that will be triggered whenever an email is rejected or bounced.

How this is going to work

Mandrill will send us a POST request that will contain a body of details about the rejected email so we can then take an action to resolve the issue.

This POST request will also contain a hashed header that we need to check against a secret key to make sure the request is actually coming from Mandrill.

At this point I’m going to assume you’ve set up this Webhook in Mandrill and you’re ready to do the work on the Laravel side of the equation.

When you set up the Webhook you will be given a secret key. So the first thing we need to do is to create a new config file called webhooks.php under the config directory:

return [
    "mandrill" => [
        "invitation_was_rejected" => env("MANDRILL_INVITATION_WAS_REJECTED"),
    ],
];

Each Webhook will have a different secret key, and so creating a config file is a convenient place to store them. I’m also storing my secret keys as environment variables to keep them out of the git repository.

Setting up the Route

The next thing I will do will be to set up the route to accept the Webhook from Mandrill:

use Illuminate\Contracts\Routing\Registrar;

class WebhooksRoutes
{
    /**
     * Define the routes
     *
     * @param Registrar $router
     * @return void
     */
    public function map(Registrar $router)
    {
        $router->post("webhooks/mandrill/{webhook}", [
            "as" => "webhooks.mandrill",
            "uses" => "WebhooksController@mandrill",
        ]);
    }
}

I’m going to be mapping the Webhook request to a specific handler based upon the dynamic segment of the URL. This means we can set up different Webhooks for the different events that occur when an email is sent.

I’m also sending this request to the mandrill() method on the WebhooksController. I usually end up listening to Webhooks from multiple services and so it makes sense to just deal with them all in the same Controller.

Adding the Controller

Next I will add the Controller that will accept the request and return the response to Mandrill.

use Illuminate\Http\Request;

class WebhooksController extends Controller
{
    /**
     * Handle a Mandrill Webhook
     *
     * @param string $webhook
     * @param Request $request
     * @return JsonResponse
     */
    public function mandrill($webhook, Request $request)
    {
    }
}

In order to prove that the Webhook request is coming from Mandrill, the request will include a X-Mandrill-Signature with a hash of the full URL of the request, the body of the request and a secret key that is generated when you create the Webhook in Mandrill’s dashboard.

So first we need to grab those items from the Request:

$url = $request->fullUrl();
$signature = $request->header("X-Mandrill-Signature");
$body = $request->all();

Next we need to get the correct handler for the given Webhook. Here is what that API will look like:

$webhook = MandrillWebhook::accept($webhook, $url, $signature, $body);

$webhook->handle();

Here I’m passing the details into the accept() method and I will automatically be returned the correct implementation.

Finally we can provide a response so Mandrill knows everything went smoothly:

return response()->json([]);

Here is the Controller in full:

use Illuminate\Http\Request;
use Acme\Webhooks\MandrillWebhook;

class WebhooksController extends Controller
{
    /**
     * Handle a Mandrill Webhook
     *
     * @param string $webhook
     * @param Request $request
     * @return JsonResponse
     */
    public function mandrill($webhook, Request $request)
    {
        $url = $request->fullUrl();
        $signature = $request->header("X-Mandrill-Signature");
        $body = $request->all();

        $webhook = MandrillWebhook::accept($webhook, $url, $signature, $body);
        $webhook->handle();

        return response()->json([]);
    }
}

Writing an Acceptance Test

At this point I’m going to write an Acceptance Test that will test this process in full. The Acceptance Test will accept the POST Request and will run through the application and make sure we are handling these Webhooks correctly.

Of course, at the minute this test won’t pass, but I think it’s worth writing just because it allows us to take a step back and think about how the process should run in full.

class MandrillMethodTest extends \TestCase
{
    /** @test */
    public function should_mark_invitation_as_rejected()
    {
    }
}

The first thing I will do will be to set the secret in the config to a known value:

config([
    "webhooks.mandrill.invitation_was_rejected" => ($secret = str_random(40)),
]);

Next I’m going to create a new Invitation model object from the factory. I’m missing out the part where I create the migration and the model as that’s not really the important part of this particular application:

$invitation = factory(Invitation::class)->create();

Next I need to create the body of the email that will be sent as part of the request:

$metadata = [
    "email" => "member_invitation",
    "uuid" => $invitation->uuid,
    "env" => "testing",
];

Each request will contain a metadata object that contains the above parameters. This will allow me to know which email was sent, the uuid of the invitation that was sent, and what the environment of the application that sent the message.

Next I roll the metadata into a msg object and add a state parameter:

$msg = [
    "metadata" => $metadata,
    "state" => "rejected",
];

Finally I json_encode and then urlencode the whole thing to form the $body:

$body = urlencode(json_encode([compact("msg")]));

As a side note, look how gross it is that json_encode has an underscore, but urlencode doesn’t.

Next we need to generate the header for the request that will be checked against during the application request:

$host = "https://localhost/";
$url = "webhooks/mandrill/invitation_was_rejected";
$signed = sprintf("%s%smandrill_events%s", $host, $url, $body);
$signature = base64_encode(hash_hmac("sha1", $signed, $secret, true));

The signature is comprised of the full URL of the request as well as the body as a string of events. This is then hashed using sha1 with the $secret, and then base64 encoded.

Next we can perform the POST request to the application by including the body and the Mandrill header:

$this->call('POST', $url,
    ['mandrill_events' => $body], [], [],
    ['HTTP_X_Mandrill_Signature' => $signature], "");

Finally we can assert that the response was 200 OK and that the Invitation was marked as rejected:

$this->assertResponseOk();
$this->assertTrue($invitation->fresh()->isRejected());

Authenticating the Webhook

Whenever we receive a request we need to check that the header signature is valid to ensure that the request is coming from Mandrill.

Each Webhook is going to have it’s own secret key, and so in order to encapsulate this whole process in a single class, I’m going to deal with the authentication within the implementation.

However, the process of authenticating is going to be the same for each implementation, and when testing each implementation, I don’t want to have to set up each test with valid authentication details just to test the functionality of the implementation.

So instead of repeating myself, I can abstract this process into it’s own class!

So the first thing to do is to write the tests for this class:

class AuthTest extends \TestCase
{
}

The first test will assert that the correct Exception is thrown when the signature is incorrect:

/** @test */
public function should_throw_exception_on_invalid_signature()
{
    $this->setExpectedException(InvalidWebhookSignature::class);

    (new Auth)->check('key', 'url', 'signature', []);
}

I’m throwing an Exception because if the signature is not valid I want to immediately halt the execution and jump out of the request.

In this example the InvalidWebhookSignature will bubble up to the surface of the application and automatically return the correct HTTP status code.

You can read more about these two concepts here When should you use an Exception? Dealing with Exceptions in a Laravel API application.

Here is the second test to assert that true is returned when the signature is valid:

/** @test */
public function should_mark_invitation_as_rejected()
{
    $key = '1cQNNoyNjFj4jj32o3C4rt9';
    $url = 'https://requestb.in/cf2q8nmk';
    $signature = 'srtNHRaBpXl0y+2dccn3j4f5b5r=';
    $body = ['mandrill_events' => urldecode(trim(file_get_contents(__DIR__.'/request.txt')))];

    $this->assertTrue((new Auth)->check($key, $url, $signature, $body));
}

To get the values for this test sent I actually sent a Mandrill Webhook to Request Bin and captured the results. I’m then using these values and the body of the request to assert that my authentication class is working correctly.

With the tests in place we can now look at creating the implementation.

This class is going to be really simple as we only require a single method:

class Auth
{
/**
 * Check the signature
 *
 * @param string $key
 * @param string $url
 * @param string $signature
 * @param array $body
 * @return bool
 */
public function check($key, $url, $signature, array $body)
{

}

The check() method should receive the secret key for this Webhook, the full URL of the request, the signature from the header of the request and the array of body parameters.

First we need to ensure that the $body parameters are sorted correctly:

ksort($body);

Next we need to create the $signed_url by looping over the $body parameters and adding them to the $url of the request:

$signed_url = $url;

foreach ($body as $k => $v) {
    $signed_url = sprintf("%s%s%s", $signed_url, $k, $v);
}

Next we can compute the hash for the given request:

$generated = base64_encode(hash_hmac("sha1", $signed_url, $key, true));

Next we can check to see if the signature is valid and return true if it is:

if ($generated === $signature) {
    return true;
}

And finally, if the signature is not valid we can throw an Exception:

throw new InvalidWebhookSignature("invalid_webhook_signature");

Accepting the Webhook

The next thing I’m going to create will be the class that accepts the request and returns the correct implementation:

class MandrillWebhook
{
    /**
     * Accept a Webhook request
     *
     * @param string $webhook
     * @param string $url
     * @param string $signature
     * @param array $body
     * @return Webhook
     */
    public static function accept($webhook, $url, $signature, array $body)
    {
    }
}

As you can see, this is again a simple class that only requires a single accept() method.

The accept() method should receive the name of the Webhook (from the URL), the full URL, the signature header, and the array of body parameters.

First we form the namespace of the implementation class using the name we have been given from the URL:

$hook = sprintf("Acme\Webhooks\Mandrill\%s", studly_case($webhook));

Next we check to see if the class exists. If the class does exist we instantiate a new instance and return it from the method:

if (class_exists($hook)) {
    return new $hook(new Auth(), $url, $signature, $body);
}

As you can see, we are injecting the Auth class from earlier and passing in the details of the request.

If the given Webhook name is not a valid Webhook, we can throw an Exception:

throw new WebhookException("invalid_webhook", [$webhook]);

Next we can write a couple of tests to ensure this is working correctly:

class MandrillWebhookTest extends \TestCase
{
}

Firstly, we assert that the correct WebhookException is thrown when the Webhook does not exist:

/** @test */
public function should_throw_exception_on_invalid_webhook()
{
    $this->setExpectedException(WebhookException::class);

    MandrillWebhook::accept('invalid', ", ", []);
}

Secondly, we assert that the correct implementation is returned when we provide the name of a valid Webhook:

/** @test */
public function should_return_webhook()
{
    $webhook = MandrillWebhook::accept('invitation_was_rejected', ", ", []);

    $this->assertInstanceOf(InvitationWasRejected::class, $webhook);
}

The Event Value Object

When Mandrill sends us a Webhook request, the body of the request is going to be a big object of parameters.

We’re probably only going to need a couple of parameters from the request, but traversing through a big object is ugly and prone to errors.

Instead, we can create a simple Value Object that will make working with this big object much nicer.

use StdClass;

class Event
{
    /**
     * @var array
     */
    private $data;

    /**
     * @param StdClass $data
     * @return void
     */
    public function __construct(StdClass $data)
    {
        $this->data = $data;
    }

    /**
     * Catch property requests
     *
     * @param string $property
     * @return mixed
     */
    public function __get($property)
    {
        return $this->data->$property;
    }
}

This Event class accepts a StdClass object and then implements the __get() magic method (What are PHP Magic Methods?).

We can then provide helper methods for the common attributes that we need from the data object:

/**
 * Get the email name
 *
 * @return string
 */
public function email()
{
    if (isset($this->msg->metadata->email)) {
        return $this->msg->metadata->email;
    }
}

/**
 * Get the uuid
 *
 * @return string
 */
public function uuid()
{
    if (isset($this->msg->metadata->uuid)) {
        return $this->msg->metadata->uuid;
    }
}

/**
 * Get the env
 *
 * @return string
 */
public function env()
{
    if (isset($this->msg->metadata->env)) {
        return $this->msg->metadata->env;
    }
}

/**
 * Get the state
 *
 * @return string
 */
public function state()
{
    return $this->msg->state;
}

The tests for this class simply need to assert that you can access the properties of the data object correctly:

class EventTest extends \TestCase
{
    /** @test */
    public function should_get_event_property()
    {
        $event = new Event(obj(["event" => "hard_bounce"]));

        $this->assertEquals("hard_bounce", $event->event);
    }

    /** @test */
    public function should_get_nested_event_property()
    {
        $event = new Event(
            obj(["msg" => ["metadata" => ["email" => "member_invitation"]]])
        );

        $this->assertEquals("member_invitation", $event->msg->metadata->email);
    }

    /** @test */
    public function should_get_email_name()
    {
        $event = new Event(
            obj(["msg" => ["metadata" => ["email" => "member_invitation"]]])
        );

        $this->assertEquals("member_invitation", $event->email());
    }

    /** @test */
    public function should_get_uuid()
    {
        $event = new Event(obj(["msg" => ["metadata" => ["uuid" => "abc"]]]));

        $this->assertEquals("abc", $event->uuid());
    }

    /** @test */
    public function should_get_state()
    {
        $event = new Event(obj(["msg" => ["state" => "rejected"]]));

        $this->assertEquals("rejected", $event->state());
    }

    /** @test */
    public function should_return_null_when_metadata_is_not_set()
    {
        $event = new Event(obj(["msg" => ["state" => "rejected"]]));

        $this->assertNull($event->email());
        $this->assertNull($event->uuid());
    }
}

In these tests I’m using a helper method called obj() which converts an array into a StdClasS object:

if (!function_exists("obj")) {
    /**
     * Convert an array into a StdClass
     *
     * @param array $data
     * @return StdClass
     */
    function obj(array $data)
    {
        return json_decode(json_encode($data));
    }
}

Creating the abstract Webhook class

The final thing to do is to create the first Webhook implementation class that will accept the request, check the signature, and then carry out the work that is required.

In each implementation, we’re going to need to convert the body of events into Event objects and filter out any that aren’t relevant to this implementation.

So instead of repeating ourselves in each implementation, we can create an abstract class that each implementation can inherit from:

abstract class Webhook
{
}

First I will define the handle() method which must be implemented by the child class:

/**
 * Handle the Webhook
 *
 * @return void
 */
abstract public function handle();

Each implementation can potentially serve more than one “type” of email. For example, we might have member emails and admin emails where the inviter needs to be notified that the email was rejected. So each implementation will need an array of email names that this implementation is relevant for:

/**
 * @var array
 */
protected $emails = [];

Next we need to convert the array of event StdClass objects from the request into Event objects and then filter out any that aren’t relevant to this implementation:

/**
 * Get the Events from the body
 *
 * @param array $body
 * @return Collection
 */
public function events($body)
{

}

The first thing we need to do in this method is convert the $body into something useful:

$body = collect(json_decode(urldecode(array_get($body, "mandrill_events"))));

First we need to get the mandrill_events from the array of $body parameters. Next we need to urldecode the string of events, and then json_decode them into an array of StdClass objects. Finally I’m going to convert the array into a Laravel Collection using the collect() helper function in order to make it nicer to work with.

Next we need to convert the collection of StdClasS objects into Event objects, filter out the ones that aren’t relevant and then return them from the method:

return $body
    ->map(function ($event) {
        return new Event($event);
    })
    ->filter(function ($event) {
        return in_array($event->email(), $this->emails);
    })
    ->filter(function ($event) {
        return $event->env() == app()->env;
    });

So first we map over the collection and convert each into an Event object.

Next we filter each $event by checking to see if is is one of the valid emails for this implementation.

And finally we check to make sure the event was sent from the same environment as the current environment.

With the abstract class ready to go, we can now create the first implementation!

Creating the first Webhook implementation

Next we can create the InvitationWasRejected implementation class to handle this Webhook:

class InvitationWasRejected extends Webhook
{
}

First I will define the class properties of this class that will be injected through the __construct() method:

/**
 * @var Auth
 */
private $auth;

/**
 * @var string
 */
private $url;

/**
 * @var string
 */
private $signature;

/**
 * @var array
 */
private $body;

By now you should know what these properties are so they don’t need any further explanation.

Next I will define the emails that this implementation class is relevant for:

/**
 * array
 */
protected $emails = [
    'member_invitation',
    'admin_invitation'
];

Next I will define the __construct() method:

/**
 * @param Auth $auth
 * @param string $url
 * @param string $signature
 * @param array $body
 * @return void
 */
public function __construct(Auth $auth, $url, $signature, array $body)
{
    $this->auth = $auth;
    $this->url = $url;
    $this->signature = $signature;
    $this->body = $body;
}

Next we can implement the handle() method:

/**
 * Handle the Webhook
 *
 * @return void
 */
public function handle()
{

}

First I will grab the key for this Webhook from the config:

$key = config("webhooks.mandrill.invitation_was_rejected");

Next I will use the injected Auth class to make sure the signature is valid:

$this->auth->check($key, $this->url, $this->signature, $this->body);

Next I will convert the array of StdClass objects from the $body into a Collection of Event objects using the method on the abstract class from earlier:

$events = $this->events($this->body);

Finally I will deal with the business logic of this Webhook:

$events->each(function ($event) {
    $invitation = Invitation::findByUuid($event->uuid());

    $invitation->markAsRejected($event->state());
});

Next we can write a test to ensure this is working correctly:

class InvitationWasRejectedTest extends \TestCase
{
    use DatabaseMigrations;
}

I’m only going to show the happy path in this example, but you should test the other paths too:

/** @test */
public function should_mark_invitation_as_rejected()
{

}

First I will create a new Invitation model object using the factory:

$invitation = factory(Invitation::class)->create();

Next I will mock the Auth class to return true so I don’t have to generate the signature:

$auth = m::mock(Auth::class);
$auth
    ->shouldReceive("check")
    ->once()
    ->andReturn(true);

Next I will instantiate the InvitationWasRejected class and call the handle() method:

(new InvitationWasRejected($auth, ", ", [
    "mandrill_events" => urlencode(
        json_encode([
            "msg" => [
                "metadata" => [
                    "email" => "member_invitation",
                    "uuid" => $invitation->uuid,
                    "env" => "testing",
                ],
                "state" => "rejected",
            ],
        ])
    ),
]))->handle();

Finally I can ensure that the Invitation has been marked as rejected:

$invitation = $invitation->fresh();

$this->assertTrue($invitation->isRejected());
$this->assertEquals("rejected", $invitation->rejected_state);

If you run the Acceptance Test from earlier you should also see it pass now too.

Conclusion

We’ve covered quite a bit of ground in today’s tutorial for implementing Mandrill Webhooks.

Despite the fact that I’ve written this from the perspective of implementing Mandrill Webhooks, I think a lot of the material is generally applicable to all types of situations.

We’ve looked at writing Acceptance Tests as a broad overview of the functionality we need to build, and then writing Unit Tests to drop into the specific details of making that functionality work.

We’ve also looked at separating the components to effectively reduce duplication and keep things simple.

So hopefully even if you aren’t planning to use Mandrill Webhooks in the near future, you can still take something away from today’s tutorial.

Philip Brown

@philipbrown

© Yellow Flag Ltd 2024.