Unit Testing Your Service Layer is a Waste of Time

Writing unit tests for your application’s service layer is a waste of your time and won’t strengthen your application any better than functional tests would. Writing functional tests that thoroughly test your application does what it says it will do correctly is a much better use of your resources.

I first learned about test driven development (TDD) and automated testing in 2009 and was blown away. I immediately purchased the book xUnit Test Patterns and devoured it. I learned about code smells and fixtures and stubs and mocks and everything in-between. It was amazing that I could actually verify that my application works like I intended it to. Throw TDD in the mix where I could guarantee that I only write as much code as necessary to pass my tests and I was hooked.

I immediately took an academic approach to testing. Invert every components control, mock every depended on component (DoC), completely isolate and unit test everything: the database, other objects, even the file system.

For trivial applications like a project hacked together on a weekend, this practice was fine. I felt like I spent an equal amount of time writing tests as I did writing the code itself.

Fast forward a few years and I have a successful consultancy and am working full time writing web and network applications. I’m still trying to write unit tests and follow TDD principles, but it just isn’t working.

For example, I would write both unit and functional tests for my application’s controllers. For a Symfony application, this is a non-trivial task as the number of depended on components in each controller can easily force your tests to be longer than the controller code itself. Breaking controllers into very small units is difficult as they often deal with handling UI interactions and must communicate to multiple components of the application.

Application Development Has Evolved

When TDD and automated testing became integrated with object oriented programming, databases were big behemoths that required a full time team to manage. File systems were very slow and not reliable. It made sense that your tests should be comprised of small units that were tested in isolation: your components were unreliable!

Today, enterprise applications are contained in a single repository. You can mirror a production server on your insanely powerful laptop with a single Vagrantfile. Working with databases in your tests is a cinch (especially with tools to build out your fixtures in YAML). In all but a few instances, you generally don’t need to worry about a network device or disk drive failing during test suite execution. And modern frameworks employ multiple environments with a dependency container that makes it very simple to mock complex components in your test suite.

Application development has evolved, and the time spent writing unit tests that mock every component in your system does not offer the same dividends as writing a simple functional or integration test does.

Motivating Example

Lets look at a small example of how writing a simple functional test can rapidly increase development and still provide the same guarantees your code works.

If you recall from my previous article on using the ORM, I am in the process of building an importer for an online store. I have the code written, and now I want to verify it is correct (I’ve thrown any notion of TDD out of the window).

In the first example, all depended on components of the system under test (SUT) are mocked.


use MyApp\Library\Feeds\ProductImporter;

class ProductImporterTest extends PHPUnit_Framework_TestCase

    public function testImportingProducts()
        // Imagine this was a sample array of data.
        $results = [];

        // Spend the next N lines mocking all of your DOCs.
        $db = Mockery::mock('Doctrine\DBAL\Connection')

        $importer = new ProductImporter($db);

        $this->assertEquals(4, $importer->getRecordCount());


In the second example, I take actual JSON files delivered by clients, throw them into a data provider, and set up some expected versus actual values to test against.


use MyApp\Tests\TestCase;

class ProductImporterTest extends TestCase

     * @dataProvider providerProductFile
    public function testImportingProducts($file, $recordCount, $hasError)
        $fileContents = file_get_contents(__DIR__ . '/' . $file);

        $importer = $this->getContainer()

        $this->assertEquals($recordCount, $importer->getRecordCount());
        $this->assertEquals($hasError, $importer->hasError());

    public function providerProductFile()
        $provider = [
            ['products.invalid.json', 0, true],
            ['products.small.json', 1, false],
            ['products.large.json', 1000, false]

        return $provider;


The first example is academic; it’s pure. It proves my code works, it’s fast, and it tests individual units. The second example is functional. I’m taking actual files, running them through my code, and seeing how the code responds.

Another benefit of the second example is that the data provider can grow over time. Client deliver a malformed file and you want to see how the code responds? Throw it in the provider. Client thinks there’s an issue with your importer because their file matches your spec? Throw it in the provider. It makes finding actual bugs that a unit test would completely ignore.

When to Write Unit Tests

There are, of course, times when writing unit tests is necessary and writing functional tests may be impossible.

If you are writing a library or framework, it is wise to write unit tests. You don’t know how your code will actually be used, so having formal verification that your library classes do what they say they will is a must. It also adds a great deal of built in documentation to your codebase because a passing test, by definition, accurately documents your code.

Another time to write unit tests would be for a legacy codebase right before a refactoring. If the codebase is old enough, your DoC’s may be very difficult to work with and thus writing a unit test will accurately capture the functionality of your application.

From the moment I started writing tests, I’ve attempted to get every developer at every company I’ve worked for to start writing tests without much luck. In retrospect, I feel if I had started with a “functional first” approach, I would have been more successful in my efforts. Introducing a developer to testing by way of writing simple functional tests may be the best bet to get all developers writing tests, period.

Application Architecture Without an ORM


I’m building a large, enterprise Symfony application and was having trouble with how I should design an initial, core feature of the application: feed processing.

Unlike many other web applications where the source of data is entered through a web interface, the source of data for this application comes in the form of file feeds from its users. Feeds are files that could contain tens or hundreds of thousands of records (a list of products or items in a warehouse, for example).

Building the rest of the application without sample data would be difficult and thus it was important to write the feed processing code first. The overall design was simple: through a basic API endpoint, a client would upload a feed file to be processed. The feed would be saved to the server, and a background worker would be enqueued to process it.

Initial Design

Like many other Symfony applications, I wrote my database migrations, defined my entity configuration files, generated the entity files themselves, and wrote validation settings for each entity. I then wrote the initial version of the feed processor using Doctrine and the Symfony Validator library.

For trivial data sizes that I mocked up, the code worked fine. It was fast enough and worked well. Things started to break down when I began working with real client data.

Real data is messy. It’s likely exported directly from a client’s system. Their unique keys may not be the same as our unique keys. Their data set is large, my mocked-up data was tiny.

Doctrine follows a Unit-of-Work pattern, which means you persist entities to the entity manager, and it determines what needs to be done with them. The unit-of-work algorithms are complex, so you want to invoke them as infrequently as possible. This works fine when you’re updating a single record through a web interface; they may take a couple of milliseconds. However, when you’re attempting to flush tens of thousands of records, that time adds up, and quickly.

Doctrine can also use quite a bit of memory when tracking all of the entities that you have persisted.


Lets begin by looking at a real life example of where using an ORM can lead to not only slow performance but also incorrect results.

File Feeds

Imagine a client sent your application a list of SKUs. Your application has a unique key on a SKU Code value, however, their system has a unique key on the barcode value. They might send you a file that looks like this:

    "name": "Product 1",
    "skucode": "ABC123",
    "barcode": "199300199302993"
    "name": "Product 2",
    "skucode": "ABC123",
    "barcode": "301930199302443"

Note that the skucode in each record is the same, while the barcode is different. The feed processor would iterate over each record, hydrate it a corresponding Doctrine entity, and then persist it. This is called the “ORM approach”.

ORM Approach

foreach ($skus as $_sku) {
    $sku = $entityManager->getRepository('MyApp:Sku')

    if (!$sku) {
        $sku = new Sku;


    // Executes a SELECT NEXTVAL('sku_sku_id_seq')
    // query but not the actual INSERT.

// The INSERT queries are executed here.

Even if the Sku entity had a validation constraint that required the skucode had to be unique, it would not catch it in this case because no records have been written to the database until the loop ends. The call to flush() is what generates all of the INSERT queries, not the call to persist().

The first INSERT statement would work fine; the second one would cause a unique index violation in the database and cause the entity manager to close itself.

This problem can be solved by manually handling the database transaction and calling flush() after each persist(), but that solution completely wrecks performance. Each call to flush() requires another unit-of-work calculation which is slow.

A better, much more performant solution is to ditch the ORM altogether and write directly to the database. You can still take advantage of Symfony’s Validation library, and you can more easily manage your own transactions. This is called the “explicit approach”.

Explicit Approach

$conn = $entityManager->getConnection();

foreach ($skus as $_sku) {
    $skuId = $conn->fetchColumn("SELECT s.sku_id FROM sku s WHERE s.skucode = ?", [$_sku->skucode]);

    $sku = [
        'name' => $_sku->name,
        'skucode' => $_sku->skucode,
        'barcode' => $_sku->barcode

    if (!$skuId) {
        $skuId = $conn->fetchColumn("SELECT nextval('sku_sku_id_seq')");
        $sku['sku_id'] = $skuId;
        // Immediately calls INSERT so the SELECT above
        // will find the new record on a subsequent call.
        $conn->insert('sku', $sku);
    } else {
        $conn->update('sku', $sku, ['sku_id' => $skuId]);


This example is definitely longer, but a lot of the boilerplate could easily be abstracted out into a parent class or trait. Memory usage is lower, and execution time is much faster in this example as well. Some basic (though unscientific) benchmarks showed the Doctrine example took around 4 minutes to process 40,000 records, while the explicit example took around 20 seconds for the same dataset.

The second example can be sped up even more by a process called memoization that records a list of skucode values to sku_id values and reduces the need to call the SELECT on each loop iteration (it would be more difficult memoizing an array of objects because the array would grow very large).


I dislike articles that spend several thousand words discussing a tool, and then wind up saying “use the best tool for the job!” It seems like such a cop out – it’s obvious you should use the best tool for the job (which is a nebulous idea to begin with).

There are a lot of areas where an ORM excels. Updating small record sets (or a single record) is a great example of something you should use an ORM for, especially if one is tightly integrated into your development stack. Building application interfaces that follow the basic CRUD data cycle are perfect for an ORM.

When working with code that needs to take advantage of a database specific extension, requires performance over anything else, or needs to tightly control your transactions, it is best to write the SQL yourself.

It can feel uncomfortable to move away from the ORM for one specific piece of your application, especially when it is used elsewhere; it’s a feeling I’m completely familiar with. Understanding the tradeoffs that come with an ORM is a great step to building more powerful and robust applications.