Nov 4, 2010

Compiling Google mod_pagespeed on Archlinux

Google released its awesome apache module mod_pagespeed recently. Unfortunately only .deb/.rpm packages are provided officially. For arch user, we have to compile it by ourselves.

Based on google's howto, we need depot_tools to compile mod_pagespeed. There is an aur package depot_tools-svn, but seems not work now because arch switched to python 3.

So I downloaded depot_tools myself and put it in my ~/scripts/depot_tools.

mkdir -p ~/scripts
cd ~/scripts
svn co http://src.chromium.org/svn/trunk/tools/depot_tools


To make depot_tools work, you need to switch to python2:

sudo rm /usr/bin/python
sudo ln -s /usr/bin/python2 /usr/bin/python
sudo rm /usr/bin/python-config
sudo ln -s /usr/bin/python2-config /usr/bin/python-config


Next step we download mod_pagespeed source code use depot_tools:

mkdir ~/mod_pagespeed # any directory is fine
cd ~/mod_pagespeed
gclient config http://modpagespeed.googlecode.com/svn/trunk/src
gclient sync --force # this will download all source code


You're ready to compile now.

cd ~/mod_pagespeed/src
make BUILDTYPE=Release # BUILDTYPE defaults to 'Debug'


I got an error when I compile it on my box:

In file included from /usr/lib/gcc/x86_64-unknown-linux-gnu/4.5.1/../../../../include/c++/4.5.1/utility:71:0,
from /usr/lib/gcc/x86_64-unknown-linux-gnu/4.5.1/../../../../include/c++/4.5.1/algorithm:61,
from ./net/instaweb/util/fetcher_test.h:24,
from ./net/instaweb/util/cache_fetcher_test.h:27,
from net/instaweb/util/cache_fetcher_test.cc:19:
...
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.5.1/../../../../include/c++/4.5.1/bits/stl_map.h:87:5: instantiated from here
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.5.1/../../../../include/c++/4.5.1/bits/stl_pair.h:77:11: error: ?std::pair<_t1,>::second? has incomplete type
./net/instaweb/util/public/cache_interface.h:28:7: error: forward declaration of ?struct net_instaweb::SharedString?


After digging a while, I made this patch to fix the error:

Index: net/instaweb/util/public/cache_interface.h
===================================================================
--- net/instaweb/util/public/cache_interface.h (revision 137)
+++ net/instaweb/util/public/cache_interface.h (working copy)
@@ -21,6 +21,7 @@

#include <string>
#include "net/instaweb/util/public/string_util.h"
+#include "net/instaweb/util/public/shared_string.h"

namespace net_instaweb {


ps: I also deployed mod_pagespeed to our centos staging server, the whole process is very smooth. The result is also very impressive: gzip, cache-control, inline assets, etc. etc. all suddenly work like a charm. The only problem is with extend_cache filter: it modified asset name correctly, but when brower send request to those assets, the server reponds with 404 error. I have to disable rewrite_javascript and extend_cache filter to make our sites work correctly:

ModPagespeedDisableFilters rewrite_javascript
ModPagespeedDisableFilters extend_cache

3 comments:

  1. Great to see you trying our module out, congratulations on your success at porting! If you haven't already, please mention some of the problems your getting at mod-pagespeed-discuss@googlegroups.com and post your patch to http://code.google.com/p/modpagespeed/issues/list, thanks for the contribution!

    ReplyDelete
  2. Hi -- as far as I can recall this is the first issue with extend_cache reported. I'm hoping that you can type 'gclient update --force' one more time; there was a system instability that has been resolved since your post that might account for a random failure in extend_cache.

    In general this is an important and, we think, robust filter.

    On the other hand, like you, we've found some rewrite_javascript issues. We've turned it off by default in the default set of filters. So after you update, you won't need to turn it off explicitly in your config.

    We have a fix baking in the oven and when it's fully cooked -- meaning we have validated it over a large corpus of web sites -- we'll turn it back on in the core filters and hopefully it will work better for you.

    But in the meantime, do you think you could repro your rewrite_javascript and add it as an issue on http://code.google.com/p/modpagespeed/issues/list and we'll test it to ensure it's fixed?

    ReplyDelete
  3. The extend_cache problem does no longer exist in latest revision, great work!

    However our sites still has problem with rewrite_javascript, I've opened ticket on issue tracker.

    Thanks.

    ReplyDelete