Monday, March 30, 2009

How I would like to use Propel and Memcache

In this article I will like to share some ideas that are wandering in my mind but I haven't implemented yet. So beware!

I was thinking of a way to reduce database load by caching some results inside Memcache -idea which has nothing special or revolutionary this days-. This article gave me some ideas that I would like to see implemented in some of the projects I work for.

The code is for using mostly inside a Symfony/Propel project, but could be adapted to a different one with ease.

Propel Peer classes come packed with the following method.

BaseUserPeer::retrieveByPK($pk, $con=null);

I would like to override it in the following way:

public static function retrieveByPK($pk, $con = null)
  $cacheKey = sprintf('user:id:%d', $pk);
  $asArray = $memcache->get($cacheKey);
  if($asArray === null)
    $obj = parent::retrieveByPK($pk, $con);
    if($obj !== null)
      $memcache->set($cacheKey, $obj->toArray(BasePeer::TYPE_FIELDNAME));
    $obj = new User();
    $obj->fromArray($asArray, BasePeer::TYPE_FIELDNAME);
  return $obj;

Note that I've avoided the memcache connection code. I assume that there is a memcache class that abstracts the process. I'm also avoided the details of this class instantiation.

As you can see from the code, I store the object in memcache as an associative array. I like to do so because I don't want to store a serialized version of the Propel object which will take more space. Also by using a native data type I can warm up the cache from -let's say- a batch script without the need of using Propel at all. Also, if a I have code that don't require symfony or Propel, I can still use the cached data. 

The fromArray and toArray methods are built inside propel objects as a convenient way of populating them, so there's no extra effort in our side to get their benefits. 

As key for the cache I'm using as prefix the name of the table, followed by a colon and then the primary key column name, a colon and the primary value. i. e.: table_name:primary_key:value

Then what is left for this to actually work is to override the User::save($con = null) method, so every time the database row is updated the changes will be reflected in the cache.

I came up with the following code:

public function save($con = null)
  $affectedRows = parent::save();
  $memcache->set(sprintf('user:id:%d', $this->getId()), $this->toArray(BasePeer::TYPE_FIELDNAME));
  return $affectedRows;

There by updating the entry after it's saved to the database I used a traditional pattern with memcache to warm up the cache.

So lets say that in a normal login form, the user will submit his nickname and password to be checked against the database. In this case we have to do a SELECT query
using the nickname and password as WHERE parameters. Instead of issuing a query, I would like to do the following inside UserPeer::retrieveByNickname($nickname).

public static function retrieveByNickname($nickname)
  $nicknameCacheKey = sprintf('user:nickname:%s', $nickname);
  $userId = $memcache->get($nicknameCacheKey);
  if($userId !== null)
    return UserPeer::retrieveByPK($userId);
    $c = new Criteria();
    $c->add(UserPeer::NICKNAME, $nickname);
    $user = UserPeer::doSelectOne($c);
    if($user !== null)
      $memcache->set(sprintf('user:id:%d', $user->getId()), $user->toArray(BasePeer::TYPE_FIELDNAME));
      $memcache->set($nicknameCacheKey, $user->getId());
    return $user;

In the last example first I check if there is a memcache entry with the following key: user:nickname:somenickname. The value stored will be the user id which I assume
is the primary key of the table. If the user id is not null then I delegate the call to UserPeer::retrieveByPK to do the job. In the other case I fetch the user from
the database using the nickname as Criteria condition. If the record exists I store inside memcache the user data as an array and also I store the id using the
user:nickname:somenickname key. Now should be clear why in the previous example I used used:id:somenid as key.

Some improvements to do in the retrieveByPK method and in the save method will be to also store the respective values for the user:nickname:somenickname key once
the object has been populated. In this way we increase the chances that retrieveByNickname will successfully hit the memcache.

I hope this article results useful for you and thanks for reading.

NOTE: I know that the code is pretty ugly, some code needs to be refactored out to it's own methods and maybe Propel classes are not the best place for
this caching logic to reside, but I think is a nice example to build upon.

Tuesday, March 3, 2009

Symfony Speed and Hello World Benchmarks

After reading some posts showing that my blah blah framework is way more fast than symfony for a Hello World application I decided to explain why: because symfony is extensible and can adapt to your needs. That’s easy to say you may think, in fact, every framework out there claims that. So what makes symfony so special?

The following list names some of the features provided by symfony.

  • Factories
  • The Filter Chain 
  • The Configuration Cascade
  • The Plugins System
  • Controller adaptability
  • View adaptability


Since version 1.0 symfony provides a configuration file called factories.yml. This file affects the application core classes configuration. There you can override symfony default classes by your own ones. This means you can set up a custom Front Controller, Web Request, Cache classes, Session storage, etc. 

But this come with an extra price: when a symfony application bootstraps, it reads the configuration file from the filesystem. If the yml file was parsed before, then it loads a PHP file -which can also be cached with APC-, if not, then it parses the yml file, stores the parsed file on the cache folder and loads the configuration from there.

Why should I need that flexibility you may ask? In a project I’m involved with we needed Memcached. This means that we overrode all of the symfony default cache mechanism by our own custom classes. How? Setting up our classes in an easy to read yml file. So for sure when you benchmark your Hello World application symfony will be slower.

Filter Chain

One of the patterns from the Core J2EE Patterns book that impressed me the most is the Intercepting Filter. This patterns teach how to modify a request processing without the need to change Controllers or Model code. The idea is that in a configuration file you plug a class that will take care of pre or post processing the request. This classes are called filters. As an example, you can add a filter that checks if the user has the proper credentials to execute the action she wants. Another filter can cache the response, etc. 

Symfony has the filters.yml file which can be specified by application or by module. This means that we can set filters to be executed for the whole application, and then for specific modules -let’s say for Ajax actions-, we disable them. Does your framework provides this flexibility without resorting to some kind of monkey patching techniques? No? Well symfony does. Say hello to the Filter Chain. So for sure when you benchmark your Hello World application symfony will be slower. Because it adds flexibility to the process. You want to get rid of this behavior? Sure, set up a custom controller in the factories.yml file, and in your new class override the loadFilters method.

Configuration Cascade

As explained in the configuration chapter of the symfony book, symfony allows to modify it’s behavior through some yml files. So for example we have the view.yml that tells symfony which css and javascript files it should load for the current request. We can have an application view.yml configuration and then override the settings per module. When symfony process the request it checks all this files, that’s why in the Hello World benchmark it‘s slow.

Plugins System

Symfony has a very powerful and easy to use Plugin System. It’s more than 400 plugins with a set of 200+ developers speaks by itself of it success. 

The plugins can contain modules of their own and also a config.php file, similar to the project config.php file or the module config.php. When symfony process a request it checks for the settings in those files, this means that they will be read from disk. So in a plugin we can provide a custom logger that is fired up when bootstrapping the application. The Plugin user doesn’t need to care how the logger will be activated, she just now that it will work. 

The same applies for plugin modules. How do you think that symfony knows that certain module/action should be called from a plugin? If you enable the plugin module on the settings.yml file symfony will check inside the plugin module to see if the requested action should be executed there. 

Controller adaptability

For each action that the user call symfony will execute a page controller. Inside our modules symfony let us use a generic myModuleActions class that will extend sfActions or one specific to the action requested by the user, that as an example could be called indexAction and will extend the sfAction class. When a request is processed symfony first checks for the existence of the later. If it doesn’t exists then it tries to load the generic action for that module. Of course you don’t need this kind of flexibility for Hello World apps.

View Adaptability

For rendering the response symfony uses by default the sfPHPView class. If certain module in your application requires a different view, then there are at least three ways to accomplish this as explained here.


Symfony is a Professional Web Application Framework built to cope with real world needs. In a large project with more than a simple salutation feature sooner or later you will need the flexibility provided by the framework. This will save you time and will prevent headaches, because when you have built a whole system with a framework and the business needs start to push in a direction where you have to extend the framework you will thank yourself for having choose symfony at first.

In case that your client requires a Hello World! application, then you can use the following hyper fast framework code: die(“Hello World!”) ;-)