Returning secure files from an API with temporary URLs

Dec 14, 2015

Table of contents:

Understanding the problem
An overview of the solution
Creating the Token class
Generating the URLs
Authenticating the request
Conclusion

A common requirement for many types of web application is the ability for users to upload files.

For example, a project management application might allow the user to upload documents, images, or videos that should be stored with the project.

Once the files are uploaded, you need to provide a way for the client to return them to be displayed or allow for download.

Many APIs will simply return a direct URL to the file that can be used in the client.

This makes it easy to grab the file because it’s just another HTTP request.

For example, it’s easy to just pop a URL to an image in a img tag.

However, if the file should be secured behind authentication, this is not a suitable solution because the URL is public to the world.

A better solution is to generate temporary URLs that can be used to return the file.

These URLs can be used as simple HTTP requests, but they expire and are unique to the user who has access to the file.

In today’s tutorial we will look at generating temporary URLs for returning secure files from an API.

Understanding the problem

Before we jump into the solution, first lets take a look at the problem we face.

A user uploads an attachment to a project she is working on in our project management web application.

The file is very sensitive and so only the project leaders should be able to view the file.

However, when you make a request to the API for the project and it’s attachments, the URL to the file on S3 is exposed.

If anyone was to get access to this URL, via any number of different methods, it would mean the secure files in the application aren’t really secure at all!

We need to find a better way to return the URLs to files in our application that won’t expose them to the world.

An overview of the solution

To avoid this problem, instead of exposing the direct URL to the file in our storage, we need to return a temporary URL that will allow the user to get the file.

This temporary url will not need any authentication or additional HTTP headers so it can be used as a normal URL for use in img tags or whatever the client needs the file for.

The temporary url will expire after a short period of time after which the client will need to generate a new temporary URL.

And the URL will be unique to the authenticated user who generated it.

Finally, the API endpoint that accepts the temporary URL will allow us to hide the implementation details of where the file is stored. So if we switch from S3 to another storage service, the client will never know.

Creating the Token class

The first thing we need to do is to create a Token class that will encapsulate generating the token that will secure the file:

class Token
{
    /**
     * Generate a token
     *
     * @param string $id
     * @param string $filename
     * @param string $extension
     * @param int $expires
     * @return string
     */
    public static function generate($id, $filename, $extension, $expires)
    {
        $payload = ["name" => $filename, "extension" => $extension];
        $payload = urldecode(http_build_query($payload));
        $payload = implode("\n", [$id, $expires, $payload]);
        return hash_hmac("sha256", $payload, config("app.key"));
    }

    /**
     * Check a token
     *
     * @param string $token
     * @param string $id
     * @param string $filename
     * @param string $extension
     * @param int $expires
     * @return bool
     */
    public static function check($token, $id, $filename, $extension, $expires)
    {
        if (time() > $expires) {
            return false;
        }

        return $token == static::generate($id, $filename, $extension, $expires);
    }
}

First we will define a generate method that will create a hmac hash of the user id, the name of the file, the extension of the file, and the time in which the token should expire as a unix timestamp.

Next we will define a method for checking a token.

First we will check to make sure the expires timestamp is still valid, and return false if it is not.

Secondly we will create a hash of the given values and compare it to the given token.

Generating the URLs

Next we need a way to use the Token class to generate the temporary urls.

A good place to deal with this is to put it in the layer of your application that deals with transforming data to be returned from the api, using something like Fractal

/**
 * Generate the action URL
 *
 * @param User $user
 * @param Attachment $model
 * @param string $action
 * @return string
 */
private function action(User $user, Attachment $model, $action)
    {
    $expires = time() + 60;
    $token = Token::generate($user->uuid, $model->filename(), $model->extension(), $expires);

    return URL::route(sprintf('attachments.actions.%s', $action), [
        'attachment_id' => $model->uuid,
        'filename' => base64_encode($model->name),
        'token' => $token,
        'expires' => $expires
    ]);
}

Here I’m generating a URL using Laravel’s URL::route() method

I’m passing the id of the attachment and a base encoded version of the file name to form the url, and I’m passing the token and the expires timestamp to be query parameters.

Authenticating the request

Finally in the Controller method we can authenticate the temporary URL and return the file from our cloud storage.

First I base decode the filename from the url:

/**
 * View an Attachment
 *
 * @return Response
 */
public function view(Request $request, $attachment_id, $filename)
{
    $user = Context::get('User')->model();

    $filename = base64_decode($filename);

    $this->checkForValidToken($user, $request, $filename);
}

Next I can check to make sure the token is valid:

/**
 * Check for a valid token
 *
 * @param User $user
 * @param Request $request
 * @param string $filename
 * @return void
 */
private function checkForValidToken(User $user, Request $request, $filename)
{
    $token = $request->input('token');
    $expires = $request->input('expires');
    $name = pathinfo($filename, PATHINFO_FILENAME);
    $extension = pathinfo($filename, PATHINFO_EXTENSION);

    if (! Token::check($token, $user->uuid, $name, $extension, $expires)) {
        throw new InvalidAttachmentToken('invalid_attachment_token', [$token]);
    }
}

If the token is not valid I will throw an exception that will automatically bubble up to the surface and return the correct HTTP response as we saw in Dealing with Exceptions in a Laravel API application

Otherwise we can grab the file and return it as a response to the request.

Conclusion

Returning files form an API can be tricky because you need to make it easy enough for the client to actually use the file, but still protect access to the file.

Public URLs are the best way of doing this because it allows clients or browsers to use the file in the way it was intended.

But your user’s files should not be exposed to the public internet!

A better solution is to generate temporary urls that can be used as public URLs but don’t totally expose your user’s files to the world.

These urls expire and are unique to each user of the application.

And as an added benefit they hide the implementation details of where your files are stored allowing you to switch things behind the scenes without breaking URLs that your users are relying on.