Note: The Twig conversion acts here more as a pretext to showcase the power and flexibility of using Apache JMeter™ as a benchmarking tool for Drupal.

Staring with release 8, Twig was adopted as theme engine for Drupal. The conversion of PHPTemplate theme_*() functions and *.tpl.php template require also benchmarking.

Here's a test plan that uses Apache JMeter to test a Twig conversion patch. The Apache JMeter™ desktop application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance.

What to test?

Some of image style themes were converted to Twig in the issue: image.module - Convert theme_ functions to Twig. First I'm running my test plan against a clean checkout from 8.x Drupal branch. Then I will apply the patch from #93 and will run again the same test.

The environment

I will run tests on a MacBook Pro 15" with SSD and 16GB of RAM on MAMP with PHP 5.4.10 and APC installed.

I'm checking out a fresh Drupal codebase from 8.x branch:

$ git clone --branch 8.x http://git.drupal.org/project/drupal.git

After installing Drupal with Standard profile, I'm enabling the Stark theme:

$ drush en stark -y
$ drush cset -y system.theme enabled.stark 1
$ drush cset -y system.theme default stark

Let's enable also the APC class loader by uncommenting this line in settings.php:

$settings['class_loader'] = 'apc';

Content to be tested

It's time now to take care a little bit about the content. As the patch is converting also "effect summary" themes (e.g. theme_image_resize_summary) I will add one effect of each. I'm adding next effects to the image style Large at admin/config/media/image-styles/manage/large:

• Scale 480x480 (upscaling allowed)
• Crop 200x200
• Resize 300x300
• Rotate 7°

This will allow me to test all next themes at admin/config/media/image-styles/manage/large URL: image_style_preview, image_resize_summary, image_scale_summary, image_crop_summary.

[[{
"type" : "media",
"view_mode" : "teaser",
"fid" : "41",
"attributes" : {
"class" : "media-image",
"typeof" : "foaf:Image"
}
}]]

There are also image_formatter and image_widget that need benchmarking. So let's add also an article node. The article node-type installed with the Drupal Standard profile have an image profile with the image display formatter configured to use the Large (large) image style. The formatter will be tested at node/1 as anonymous and the field widget at node/1/edit.

The Test Plan

You can download the test plan here: twig.jmx.

[[{
"type" : "media",
"view_mode" : "teaser",
"fid" : "49",
"attributes" : {
"class" : "media-image",
"typeof" : "foaf:Image"
}
}]]

Few words about main points of the test plan.

General and Drupal 8 are sections where we are storing variable to be used later in tests. You may notice that some variables like host, for example, are using the __P() function: ${__P(host,localhost)}.

[[{
"type" : "media",
"view_mode" : "teaser",
"fid" : "50",
"attributes" : {
"class" : "media-image",
"typeof" : "foaf:Image"
}
}]]

This is a parametrized way to pass a variable to a test in the case you are invoking JMeter from the command line prompt. Using the -J option you can pass parameters to testing environment:

$ jmeter.sh -n -t twig.jmx -Jhost=127.0.0.1 -Jport=80 -Jpath=/drupal8 -Juser=admin -Jpassword=mypass

HTTP Request Defaults is setting the common request info, like protocol, host, port and path. HTTP Header Manager is defining a list of default HTTP headers that makes the test simulate a real life browser behavior.

Effective testing groups are following:

The image_formatter (anonymous) thread group is testing the image_formatter theme as anonymous.

The next tests will run under admin account. Here, there are some additional nice tasks that we need to perform in order to authenticate. Let's focus a little bit on how to test as authenticated user because this should be something that you may want to use for other Drupal tests as well. This where the JMeter power comes in :)

Under the thread group I'm adding a controller. I used Runtime Controller here but you are free to take into consideration other controllers that suits your needs. Runtime Controller is time-based, it runs tests until reaching a defined period of time (stored in ${runtime} variable in our test case). You can use, for example, the Loop Controller if you want to run the test for a specified number of times instead of period.

Under Runtime Controller I'm adding the HTTP Cookie Manager. This is responsible to store the session cookie after authentication and send that cookie for all requests in this controller. Next, we need to simulate the authentication. We'll do that by adding needed tasks under an Once Only Controller.

Drupal authentication needs to be done only once so we'll put all under Once Only Controller. Drupal uses a form token to protect forms for spam. That's why we need first to obtain the token. We visit the login page (user/login) and copy the token into a variable (form_build_id) by using a JMeter Regular Expression Extractor. Now let's login by POSTing to the same URL with all parameters:

• name=${user.name}
• pass=${user.pass}
• form_id=user_login_form
• op=Log+in
• form_build_id=${form_build_id}

We're in! But we want to make sure that the login succeed. JMeter is providing an assertion mechanism similar to Simpletest. Under this, last, HTTP Request, I'm adding an assertion and I'm checking for "Log out" text in the page loaded, assuming that a standard Drupal installation cannot have such a text on an anonymous visit. We're done with the authentication mechanism let's hammer the site: Directly under Runtime Controller I'm adding the test itself which is basically a HTTP GET on the Large (large) image style form: ${path}/admin/config/media/image-styles/manage/large.

As you can see in the test plan we are repeating all the steps to test also: image_anchor and image_widget.

Finally, we need reports. This is achieved by adding various listeners to out test plan.

Testing 8.x

First I'm testing the clean 8.x clone. After a cache clear, I'm running a test only for warming up. The the next 3 tests will be recorded for in order to be compared. Here are the results of one from 3 tests. We are looking only for relevant tests

[[{
"type" : "media",
"view_mode" : "teaser",
"fid" : "51",
"attributes" : {
"class" : "media-image",
"typeof" : "foaf:Image"
}
}]]

Testing Twig conversion

Let's apply the patch from https://drupal.org/node/1898420#comment-7635445 and clear the cache:

$ curl https://drupal.org/files/twig-1898420-89.patch | git apply
$ drush cache-clear all

Now I'm running again the test once for warming caches then three times for measuring results.

Comparing results

Everything seems OK, but how we compare results between 8.x and Twig conversion? This is easy because of Loadosophia.org project. In fact you can load here your tests over time and compare them. Creatingg an account it's easy because it uses Google Authentication, so I only have to link my Google account. Then you need to configure JMeter to upload results there. There is a JMeter plugin that links with Loadosophia.org by implementing a listener. The listener is included in the Standard Set package from JMeter Plugins. Install the plugin and add the Loadosophia.org listener.

Configure the uploader:

• Add the project name. This must be the same name as a previous Loadosophia.org defined project.
• Add a test title and a color flag. I used 8.x with green when tested the existing Drupal 8 install and twig conversion with red when tested the patched version.
• Add the upload token. You can get the token from here https://loadosophia.org/gui/upload/token/.

[[{
"type" : "media",
"view_mode" : "teaser",
"fid" : "52",
"attributes" : {
"class" : "media-image",
"typeof" : "foaf:Image"
}
}]]

Don't forget to change the test title and color when you start testing the Twig stuff!

After running tests we're able to compare on Loadosophia.org.

Conclusion

There's 2-3 milliseconds variation between legacy Drupal theme system and new Twig theme engine. That may not be necessary because of Twig engine — there are a lot of "testing environment" possible influences.

To be honest, I wrote this article mainly to reveal what a great tool is Apache JMeter™ and how nice is playing with Drupal. The "Twig stuff" was more a pretext to show that we can write complex test plans and do benchmarks for various scenarios on a Drupal website.

Happy benchmarking!

Resources:

• Apache JMeter™ — official website
• A JMeter test plan for Drupal