10

How to Drastically Speed Up WordPress with Redis

Posted (Updated ) in Database, PHP

I recently came across a tutorial on sitting Redis infront of WordPress allowing for insanely fast page generation. I gave it a try and it really works, in fact I’m now using it on this very site! The best part however is the fact that the script requires absolutely no modification to your existing WordPress site save for 1 line of htaccess. Truly amazing.

Below I’ll detail my slightly modified version of Jim’s script along with some metrics.

 

Firstly, What is Redis and what will it do for me?

The Redis website describes Redis as

… an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain stringshasheslistssets and sorted sets.

What does this mean? Essentially it’s Memcached but more useful. Redis stores key-value pairs in memory and spits them out when requested. Unlike Memcached it has built in persistence but what’s most important to us is that it’s fast – very fast.

We’ll be using Redis to speed up our site by loading cached pages from it directly without even booting up WordPress. This will save a large amount of page generation time and get out site infront of our users’ eyeballs faster.

 

Exactly how much faster are we talking?

In my very unscientific tests, loading www.flynsarmy.com a bunch of times resulted in the following:

Before (Secs) After (Secs)
1.556
0.468
0.494
0.498
0.492
0.514
0.499
0.511
0.499
0.02001
0.00896
0.00883
0.00959
0.01472
0.00916
0.00915
0.00756
0.01989

As you can see from the table above this equates to a 20x to 50x speed increase and that was WITH W3 Total Cache installed! Results of course may vary but I think you get the picture.

 

Installation

Here’s how our script will work. When a visitor hits our site, the script will grab the URL the visitor is loading  and check if we have a cached copy in Redis. If we do, serve it. Otherwise generate the page and cache a copy for future serving.

There are a few details here to note:

  • cached pages do not expire not unless explicitly deleted or reset
  • appending a ?c=y to a url deletes the entire cache of the domain, only works when you are logged in
  • appending a ?r=y to a url deletes the cache of that url
  • submitting a comment deletes the cache of that page
  • refreshing (f5) a page deletes the cache of that page
  • includes a debug mode, stats are displayed at the bottom most part after </html>

OK with all that out of the way, let’s get started.

Install Redis

On Ubuntu this is very easy. Firstly add a PPA to make sure you can get a recent version of Redis:

sudo apt-add-repository ppa:rwky/redis

Then install:

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install redis-server

Note: Older versions of ubuntu may need to install python-software-properties before calling apt-add-repository:

sudo apt-get install python-software-properties

Get the latest copy of Predis

Predis is a library that lets PHP communicate with Redis. Drop it into your sites root folder next to index.php.

cd /to/site/root
git clone https://github.com/nrk/predis.git

Add the script

Our script will communicate to Predis which in turn talks to Redis. Create a file also next to index.php in your sites root folder called index-with-redis.php and drop the following into it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
<?php
 
/*
    Author: Jim Westergren, Jeedo Aquino & Flynsarmy
    File: index-with-redis.php
    Updated: 2012-10-25
 
    This is a redis caching system for wordpress.
    see more here: www.jimwestergren.com/wordpress-with-redis-as-a-frontend-cache/
 
    Originally written by Jim Westergren but improved by Jeedo Aquino.
 
    some caching mechanics are different from jim's script which is summarized below:
 
    - cached pages do not expire not unless explicitly deleted or reset
    - appending a ?c=y to a url deletes the entire cache of the domain, only works when you are logged in
    - appending a ?r=y to a url deletes the cache of that url
    - submitting a comment deletes the cache of that page
    - refreshing (f5) a page deletes the cache of that page
    - includes a debug mode, stats are displayed at the bottom most part after </html>
 
    for setup and configuration see more here:
 
    www.jeedo.net/lightning-fast-wordpress-with-nginx-redis/
 
    use this script at your own risk. i currently use this albeit a slightly modified version
    to display a redis badge whenever a cache is displayed.
 
*/
 
// change vars here
$debug = 0;			// set to 1 if you wish to see execution time and cache actions$display_powered_by_redis = 1;  // set to 1 if you want to display a powered by redis message with execution time, see below 
$start = microtime();   // start timing page exec
 
// if cloudflare is enabled
if ( isset($_SERVER['HTTP_CF_CONNECTING_IP']) )
    $_SERVER['REMOTE_ADDR'] = $_SERVER['HTTP_CF_CONNECTING_IP'];
 
// from wp
define('WP_USE_THEMES', true);
 
// init predis
require 'predis/lib/Predis/Autoloader.php';
Predis\Autoloader::register();
$redis = new Predis\Client();
 
// init vars
$cached = 0;
$domain = $_SERVER['HTTP_HOST'];
$url = "http://".$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
$url = str_replace('?r=y', '', $url);
$url = str_replace('?c=y', '', $url);
$dkey = md5($domain);
$ukey = md5($url);
 
// check if page isn't a comment submission
$submit = isset($_SERVER['HTTP_CACHE_CONTROL']) && $_SERVER['HTTP_CACHE_CONTROL'] == 'max-age=0';
 
// check if logged in to wp
$cookie = var_export($_COOKIE, true);
$loggedin = preg_match("/wordpress_logged_in/", $cookie);
 
// check if a cache of the page exists
if (!$loggedin && !$submit && $redis->hexists($dkey, $ukey) && !strpos($url, '/feed/')) {
 
    echo $redis->hget($dkey, $ukey);
    $cached = 1;
    $msg = 'this is a cache';
 
// if a comment was submitted or clear page cache request was made delete cache of page
} else if ($submit || substr($_SERVER['REQUEST_URI'], -4) == '?r=y') {
 
    require('./wp-blog-header.php');
    $redis->hdel($dkey, $ukey);
    $msg = 'cache of page deleted';
 
// delete entire cache, works only if logged in
} else if ($loggedin && substr($_SERVER['REQUEST_URI'], -4) == '?c=y') {
 
    require('./wp-blog-header.php');
    if ($redis->exists($dkey)) {
        $redis->del($dkey);
        $msg = 'domain cache flushed';
    } else {
        $msg = 'no cache to flush';
    }
 
// if logged in don't cache anything
} else if ($loggedin) {
 
    require('./wp-blog-header.php');
    $msg = 'not cached';
 
// cache the page
} else {
 
    // turn on output buffering
    ob_start();
 
    require('./wp-blog-header.php');
 
    // get contents of output buffer
    $html = ob_get_contents();
 
    // clean output buffer
    ob_end_clean();
    echo $html;
 
    // Store to cache only if the page exist and is not a search result.
    if (!is_404() && !is_search()) {
        // store html contents to redis cache
        $redis->hset($dkey, $ukey, $html);
        $msg = 'cache is set';
    }
}
 
$end = microtime(); // get end execution time
 
// show messages if debug is enabled
if ($debug) {
    echo "REDIS DEBUG ";
    echo $msg.': ';
    echo t_exec($start, $end);
}
 
if ($cached && $display_powered_by_redis) {
    // You should move this CSS to your CSS file and change the: float:right;margin:20px 0;    echo "<style>#redis_powered{float:right;margin:20px 0;background:url(http://images.staticjw.com/jim/3959/redis.png) 10px no-repeat #fff;border:1px solid #D7D8DF;padding:10px;width:190px;}#redis_powered div{width:190px;text-align:right;font:10px/11px arial,sans-serif;color:#000;}</style>";    echo "<a href='http://www.jimwestergren.com/wordpress-with-redis-as-a-frontend-cache/' style='text-decoration:none;'><div id='redis_powered'><div>Page generated in<br/> ".t_exec($start, $end)." sec</div></div></a>";}
 
// time diff
function t_exec($start, $end) {
    $t = (getmicrotime($end) - getmicrotime($start));
    return round($t,5);
}
 
// get time
function getmicrotime($t) {
    list($usec, $sec) = explode(" ",$t);
    return ((float)$usec + (float)$sec);
}
 
?>

This is a slightly modified version of Jim’s script with support for the newer version of Predis. I’ve highlighted a few lines that it’s safe to play around with.

Turn it on and off

Everything is now in place. It’s time to turn it on and off. This is the only change you’ll need to make to your actual site. Open .htaccess in your sites root folder and add the following line at the top:

DirectoryIndex index.html index-with-redis.php index.php

That’s it! Open an incognito/private browsing window and load your site. Load it a second time and scroll to the very bottom of your page. If all went well you should see an image something like the following:

Page generated in 0.0041 seconds

Extremely fast page generation time thanks to Redis Cache

 

Cache Invalidation

This Redis script never invalidates its cache unless you hit F5 of use one of the query strings mentioned above. This can be a problem when creating, editing or deleting posts, making comments etc.

Take creating a new post for example. That will push the last post currently on your homepage onto page 2, the last page on page 2 to page 3 and so on. The cache on those pages aren’t getting invalidated essentially hiding the overflowing posts.

There are two modifications I came up with to handle this.

Only cache home, individual post and page pages

Change line 66 of index-with-redis.php from

if (!$loggedin && !$submit && $redis->hexists($dkey, $ukey) && !strpos($url, '/feed/')) {

to

if (!$loggedin && !$submit && !strpos($url, '/feed/') && !strpos($url, '/page/') && !strpos($url, '/category/') && $redis->hexists($dkey, $ukey)) {

Invalidate when updating a post, comment or category

If we’re updating a post or comment we want that change to reflect instantly to site visitors. Adding or removing a comment or changing your excerpt could also affect the homepage so clear that too.

In your themes functions.php file add the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
/**
 *  Redis Cache invalidation
 */
function get_redis()
{
	require_once ABSPATH . 'predis/lib/Predis/Autoloader.php';
	Predis\Autoloader::register();
	return new Predis\Client();
}
 
// Chances are you'll have a list of categories on every page.
// So delete all site cache if categories are messed with
add_action('add_category', 'redis_invalidate_all');
add_action('delete_category', 'redis_invalidate_all');
add_action('edit_category', 'redis_invalidate_all');
// Delete all site cache for the current domain
function redis_invalidate_all()
{
	$redis = get_redis();
 
	$domain = $_SERVER['HTTP_HOST'];
	$dkey = md5($domain);
	if ( $redis->exists($dkey) )
		$redis->del($dkey);
}
 
// When adding/editing/deleting a post, invalidate post and home pages
add_action('trashed_post', 'redis_invalidate_post');
add_action('save_post', 'redis_invalidate_post');
function redis_invalidate_post( $post_id )
{
	// Don't delete cache on auto-save
	if ( isset($_POST['action']) && $_POST['action'] == 'autosave' )
		return;
 
	// Don't delete cache if we're saving as draft or pending
	if ( isset($_POST['save']) && in_array($_POST['save'], array('Save Draft', 'Save as Pending')) )
		return;
 
	$redis = get_redis();
 
	$domain = $_SERVER['HTTP_HOST'];
	$dkey = md5($domain);
 
	// Invalidate homepage
	$ukey = md5("http://".$_SERVER['HTTP_HOST'].'/');
	if ( $redis->hexists($dkey, $ukey) )
		$redis->hdel($dkey, $ukey);
 
	// Invalidate post page
	$ukey = md5($permalink = get_permalink( $post_id ));
	if ( $redis->hexists($dkey, $ukey) )
		$redis->hdel($dkey, $ukey);
}
 
// When adding/editing/deleting a comment, invalidate post and home pages
add_action('comment_closed', 'redis_invalidate_comment');
add_action('comment_post', 'redis_invalidate_comment');
add_action('edit_comment', 'redis_invalidate_comment');
add_action('delete_comment', 'redis_invalidate_comment');
add_action('wp_set_comment_status', 'redis_invalidate_comment');
function redis_invalidate_comment( $comment_id )
{
	$comment = get_comment( $comment_id );
	redis_invalidate_post( $comment->comment_post_ID );
}
  • Pooya

    Hi,

    Is it possible to exclude some pages from caching? I’m using “contact form 7” on some pages but it won’t work with redis caching!
    Do you know how can I exclude a page from being cached?
    thanks

    • flynsarmy

      Yes. On line 65 of index-with-redis.php you can see I’m excluding the /feed page. You can do the same for any other page. For instance here’s the line I’m currently using on flynsarmy.com:


      if (!$loggedin && !$submit && !strpos($url, '/feed/') && !strpos($url, '/contact/') && !strpos($url, '/page/') && !strpos($url, '/category/') && $redis->hexists($dkey, $ukey)) {

  • DataBoy

    i am using a cloud-based Redis service (Redis Cloud) and I have a limited amount of memory (100 mb). Redis Data Eviction Policy settings were set to volatile-lru and the memory filled up very quickly; when it was full, the index page started giving 500 errors, so I had to flush it every few hours. I changed the Redis settings to allkeys-lru and now the memory never fills up, but I don’t think I am getting very good performance. What do you think are good settings for Predis Data Eviction Policy?

    • flynsarmy

      Unfortunately I only wrote and fiddled around with this script as a pet project for this low-post count blog and I’m not familiar with the more intricate workings of Redis. Hopefully someone else will comment and be able to help 🙂

  • when I’m following the instructions of your blog I get an error indicating that PredisAutoloader cannot be found.
    When I open predis/lib/Predis/Autoloader.php I notice a namespace Predis on line 12 and a class Autoloader on line 20.

    When I open index-with-redis.php I notice PredisAutoloader::register(); on line 46.
    Changing PredisAutoloader into PredisAutoloader results in a working implementation.

  • Thank you for this great update to the already great post by Jim Westergren.
    The Redis implementation on the shared hosting of my hosting provider doesn’t listen to the default server nor port. It also uses a password to connect.

    I would like to share my implementation with you:

    OPEN index-with-redis.php
    FIND line46
    $redis = new PredisClient();

    CHANGE BY
    $redis = new PredisClient(array(
    ‘host’ => ‘provided-hostname’,
    ‘port’ => ‘provided-portnumber’,
    ‘password’ => ‘provided-password’,
    ));
    SAVE&CLOSE

  • John

    On line 59 it states that the $submit variable stores whether the user did a comment submission, but isn’t this actually just checking if the user did an F5 refresh? Your modification to the functions.php file already handles a comment post and comment delete…

  • Holger Freier

    i use your script – when a user leave a comment then the page comment box shows the email and name of the commenter

    when i use the function script is no error but after make a comment it shows a blank screen and in the admin section the same error in the moderate comments menu

  • Hello,

    i must change the line 44 of the index-with-predis.php

    from

    // init predis
    require ‘predis/lib/Predis/Autoloader.php’;
    to

    // init predis
    require ‘predis/autoload.php’;

  • Émerson Felinto

    Exactly how much faster are we talking?

    In my very unscientific tests, loading http://www.flynsarmy.com a bunch of times resulted in the following:

    [Table with results]

    How did you analyze server response time? What tool did you use?