Testing Helpers

The treq.testing module provides some tools for testing both HTTP clients which use the treq API and implementations of the Twisted Web resource model.

Writing tests for HTTP clients

The StubTreq class implements the treq module interface (treq.get(), treq.post(), etc.) but runs all I/O via a MemoryReactor. It wraps a twisted.web.resource.IResource provider which handles each request.

You can wrap a pre-existing IResource provider, or write your own. For example, the twisted.web.resource.ErrorPage resource can produce an arbitrary HTTP status code. twisted.web.static.File can serve files or directories. And you can easily achieve custom responses by writing trivial resources yourself:

@implementer(IResource)
class JsonResource(object):
    isLeaf = True  # NB: means getChildWithDefault will not be called

    def __init__(self, data):
        self.data = data

    def render(self, request):
        request.setHeader(b'Content-Type', b'application/json')
        return json.dumps(self.data).encode('utf-8')

However, those resources don’t assert anything about the request. The RequestSequence and StringStubbingResource classes make it easy to construct a resource which encodes the expected request and response pairs. Do note that most parameters to these functions must be bytes—it’s safest to use the b'' string syntax, which works on both Python 2 and 3.

For example:

from twisted.internet import defer
from twisted.trial.unittest import SynchronousTestCase
from twisted.web import http

from treq.testing import StubTreq, HasHeaders
from treq.testing import RequestSequence, StringStubbingResource


@defer.inlineCallbacks
def make_a_request(treq):
    """
    Make a request using treq.
    """
    response = yield treq.get('http://an.example/foo', params={'a': 'b'},
                              headers={b'Accept': b'application/json'})
    if response.code == http.OK:
        result = yield response.json()
    else:
        message = yield response.text()
        raise Exception("Got an error from the server: {}".format(message))
    defer.returnValue(result)


class MakeARequestTests(SynchronousTestCase):
    """
    Test :func:`make_a_request()` using :mod:`treq.testing.RequestSequence`.
    """

    def test_200_ok(self):
        """On a 200 response, return the response's JSON."""
        req_seq = RequestSequence([
            ((b'get', 'http://an.example/foo', {b'a': [b'b']},
              HasHeaders({'Accept': ['application/json']}), b''),
             (http.OK, {b'Content-Type': b'application/json'}, b'{"status": "ok"}'))
        ])
        treq = StubTreq(StringStubbingResource(req_seq))

        with req_seq.consume(self.fail):
            result = self.successResultOf(make_a_request(treq))

        self.assertEqual({"status": "ok"}, result)

    def test_418_teapot(self):
        """On an unexpected response code, raise an exception"""
        req_seq = RequestSequence([
            ((b'get', 'http://an.example/foo', {b'a': [b'b']},
              HasHeaders({'Accept': ['application/json']}), b''),
             (418, {b'Content-Type': b'text/plain'}, b"I'm a teapot!"))
        ])
        treq = StubTreq(StringStubbingResource(req_seq))

        with req_seq.consume(self.fail):
            failure = self.failureResultOf(make_a_request(treq))

        self.assertEqual(u"Got an error from the server: I'm a teapot!",
                         failure.getErrorMessage())

This may be run with trial testing_seq.py. Download: testing_seq.py.

Loosely matching the request

If you don’t care about certain parts of the request, you can pass unittest.mock.ANY, which compares equal to anything. This sequence matches a single GET request with any parameters or headers:

from unittest.mock import ANY

RequestSequence([
    ((b'get', ANY, ANY, b''), (200, {}, b'ok'))
])

If you care about headers, use HasHeaders to make assertions about the headers present in the request. It compares equal to a superset of the headers specified, which helps make your test robust to changes in treq or Agent. Right now treq adds the Accept-Encoding: gzip header, but as support for additional compression methods is added, this may change.

Writing tests for Twisted Web resources

Since StubTreq wraps any resource, you can use it to test your server-side code as well. This is superior to calling your resource’s methods directly or passing mock objects, since it uses a real Agent to generate the request and a real Site to process the response. Thus, the request object your code interacts with is a real twisted.web.server.Request and behaves the same as it would in production.

Note that if your resource returns NOT_DONE_YET you must keep a reference to the RequestTraversalAgent and call its flush() method to spin the memory reactor once the server writes additional data before the client will receive it.