J

Lightweight RESTful API's in Drupal

22 Oct 2014

Building RESTful services within Drupal isn’t a new concept and there are already a lot of existing projects out there that will allow you to build a simple REST service. However, one of the big problems with this approach is that because they are based on Drupal modules they require a full bootstrap process and in many cases, causes the application to pull in functionality that will never be used.

After digging through articles online and finding no decent options I set out in search of a solution with the following requirements:

  • Must be fast. The services on these endpoints will need to respond to thousands of requests per minute without skipping a beat.
  • Be somewhere within Drupal. Drupal is great at powering our existing sites and we don’t want to remove the opportunity to integrate with it later down the track.
  • Needs to be flexible and highly extendable. Like ctools but with less of the loading and registry problems we’ve experienced in the past.

The decision

A solid week of research and tinkering later, I had come to the conclusion that a custom solution with it’s own Drupal bootstrap process would be the best fit for our requirements. While this did take away some of the niceties of working with Drupal, it would give us the power and flexibility we needed to scale these services.

Benchmarks

During the decision making process there was a strong push to ensure this solution (whatever it ended up being) would be the fastest option available - so naturally, I benchmarked the shit out of it.

All traffic was sent as an anonymous user and profiled using wbench.

Basic Drupal

  • Fresh installation with no additional modules enabled.
  • No caching or aggregation enabled.
$ wbench http://localdev.drupal

Testing http://localdev.drupal
At Tue Oct 21 09:52:18 2014
10 loops
                                   Fastest   Median    Slowest   Std Dev
---------------------------------------------------------------------------

Host latency:
localdev.drupal:80                 0ms       0ms       0ms       0ms

Browser performance:
Navigation Start:                  0ms       0ms       0ms       0ms
Fetch Start:                       280ms     309ms     334ms     13ms
Domain Lookup Start:               291ms     320ms     344ms     13ms
Connect Start:                     293ms     321ms     346ms     13ms
Domain Lookup End:                 293ms     321ms     346ms     13ms
Connect End:                       293ms     321ms     346ms     13ms
Request Start:                     293ms     321ms     346ms     13ms
Response Start:                    666ms     710ms     771ms     34ms
Response End:                      667ms     711ms     773ms     34ms
DOM Loading:                       667ms     712ms     773ms     34ms
DOM Interactive:                   718ms     761ms     3360ms    780ms
DOM Content Loaded Event Start:    718ms     761ms     3360ms    780ms
DOM Content Loaded Event End:      774ms     819ms     3411ms    777ms
Load Event End:                    791ms     837ms     3428ms    777ms
DOM Complete:                      791ms     837ms     3428ms    777ms
Load Event Start:                  791ms     837ms     3428ms    777ms

Custom bootstrapped application just rendering a 200 status

  • No caching enabled.
$ wbench http://localdev.drupal/api/example

Testing http://localdev.drupal/api/example
At Tue Oct 21 09:50:20 2014
10 loops
                                   Fastest   Median    Slowest   Std Dev
---------------------------------------------------------------------------

Host latency:
localdev.drupal:80                 0ms       0ms       0ms       0ms

Browser performance:
Navigation Start:                  0ms       0ms       0ms       0ms
Fetch Start:                       278ms     297ms     341ms     16ms
Domain Lookup Start:               290ms     309ms     352ms     16ms
Connect Start:                     295ms     310ms     353ms     16ms
Domain Lookup End:                 295ms     310ms     353ms     16ms
Connect End:                       296ms     310ms     353ms     15ms
Request Start:                     296ms     310ms     353ms     15ms
Response Start:                    304ms     317ms     362ms     16ms
Response End:                      533ms     559ms     570ms     10ms
DOM Loading:                       544ms     562ms     579ms     10ms
DOM Interactive:                   545ms     572ms     584ms     11ms
DOM Content Loaded Event End:      545ms     572ms     584ms     11ms
DOM Complete:                      546ms     572ms     584ms     10ms
DOM Content Loaded Event Start:    545ms     572ms     584ms     11ms
Load Event End:                    546ms     572ms     584ms     10ms
Load Event Start:                  546ms     572ms     584ms     10ms

Views datasource returning 10 items from a database

  • Pulled 10 items with 2 string fields.
$ wbench http://localdev.drupal/views-json/10-items

Testing http://localdev.drupal/views-json/10-items
At Tue Oct 21 10:13:12 2014
10 loops
                                   Fastest   Median    Slowest   Std Dev
---------------------------------------------------------------------------

Host latency:
localdev.drupal:80                 0ms       0ms       0ms       0ms

Browser performance:
Navigation Start:                  0ms       0ms       0ms       0ms
Fetch Start:                       281ms     296ms     335ms     17ms
Domain Lookup Start:               292ms     310ms     346ms     16ms
Connect Start:                     293ms     312ms     348ms     16ms
Domain Lookup End:                 293ms     312ms     348ms     16ms
Connect End:                       293ms     312ms     348ms     16ms
Request Start:                     293ms     312ms     348ms     16ms
Response Start:                    856ms     922ms     1002ms    51ms
Response End:                      867ms     923ms     1004ms    50ms
DOM Loading:                       868ms     924ms     1005ms    50ms
Load Event End:                    870ms     926ms     1007ms    50ms
DOM Content Loaded Event End:      870ms     926ms     1007ms    50ms
DOM Complete:                      870ms     926ms     1007ms    50ms
DOM Interactive:                   870ms     926ms     1007ms    50ms
DOM Content Loaded Event Start:    870ms     926ms     1007ms    50ms
Load Event Start:                  870ms     926ms     1007ms    50ms

Custom bootstrapped application returning 10 items from a database

  • Pulled 10 items with 2 string fields.
$ wbench http://localdev.drupal/api/10-items

Testing http://localdev.drupal/api/10-items
At Tue Oct 21 10:18:15 2014
10 loops
                                   Fastest   Median    Slowest   Std Dev
---------------------------------------------------------------------------

Host latency:
localdev.drupal:80                 0ms       0ms       0ms       0ms

Browser performance:
Navigation Start:                  0ms       0ms       0ms       0ms
Fetch Start:                       270ms     287ms     311ms     13ms
Domain Lookup Start:               282ms     298ms     327ms     14ms
Connect End:                       284ms     300ms     329ms     14ms
Connect Start:                     284ms     299ms     329ms     13ms
Request Start:                     284ms     300ms     329ms     14ms
Domain Lookup End:                 284ms     299ms     329ms     13ms
Response Start:                    298ms     310ms     343ms     14ms
Response End:                      542ms     568ms     632ms     27ms
DOM Loading:                       543ms     573ms     633ms     25ms
Load Event End:                    555ms     583ms     645ms     25ms
Load Event Start:                  555ms     583ms     645ms     25ms
DOM Complete:                      555ms     583ms     645ms     25ms
DOM Content Loaded Event End:      555ms     583ms     644ms     25ms
DOM Interactive:                   555ms     583ms     644ms     25ms
DOM Content Loaded Event Start:    555ms     583ms     644ms     25ms

From the results above, you can see that the custom solution has come out with response times almost half that of the alternatives.

The code

Now that you can see it’s fast, let’s dive into what the code behind this looks like and how it all fits together.

Directory structure

To ensure this was (somewhat) isolated from the rest of the Drupal sites, I have put all the files required inside of a sub-directory called api. This does two things:

  • Create a namespace within the URI that only these endpoints will use. I.e. http://example.com/api/<service_name>. This helps with cache negotiation and preventing Drupal from attempting to route everything as it normally would.
  • Allow me to pick and choose what the service endpoints have file access to.
api
├── .htaccess
├── README.md
├── includes
|   ├── core.inc
│   └── route.inc
├── index.php
└── services
    └── example_service.inc

File overview

  • .htaccess: Handles pretty URL’s so you don’t end up with that ugly index.php in all of your requests.
  • includes/core.inc: Contains the functionality that all services have access to. Includes methods like outputting JSON and XML responses in a standardised manner.
  • includes/route.inc: Is for creating routes to the service endpoints with additional parameters like callbacks and arguments to pass through.
  • index.php: This is where the leg work to bring it all together and bootstrap Drupal to a low level is done.
  • services/example_service.inc: A demonstration on how to setup the service and what can do.

I’m not going to break this down line by line but the full code is available online for your viewing pleasure at jacobbednarz/lightweight-drupal-rest-services.

Caution!

This approach does have some caveats and before anyone embarks down this route I would recommend you at least be aware of them as your mileage may vary.

  • Firstly, and probably most importantly, this is outside the realm of just Drupal so nothing is given for free.
  • You will need to build up your own security around Apache and your custom .htaccess file.