Thursday, July 2, 2009
Benchmarking MongoDB VS. Mysql
Thursday, June 4, 2009
New Firesymfony Release
I'm pleased to announce the release of version 1.1 of Firesymfony. This time it has a new design that we believe improves the user experience. The design was made by my colleague Jacqueline Wan and the logo by Olaf Horstmann.
Bellow you can see some screenshots and here you have the urls to update the symfony plugins and the Firebug extension:About Panel:
Configuration and Variables Panel:
Logs Panel:
Cache Panel:
Database Panel:
Timers Panel:
Information Panel:
Monday, March 30, 2009
How I would like to use Propel and Memcache
BaseUserPeer::retrieveByPK($pk, $con=null);public static function retrieveByPK($pk, $con = null){ $cacheKey = sprintf('user:id:%d', $pk); $asArray = $memcache->get($cacheKey); if($asArray === null) { $obj = parent::retrieveByPK($pk, $con); if($obj !== null) { $memcache->set($cacheKey, $obj->toArray(BasePeer::TYPE_FIELDNAME)); } } else { $obj = new User(); $obj->fromArray($asArray, BasePeer::TYPE_FIELDNAME); } return $obj;} public function save($con = null){ $affectedRows = parent::save(); $memcache->set(sprintf('user:id:%d', $this->getId()), $this->toArray(BasePeer::TYPE_FIELDNAME)); return $affectedRows;}public static function retrieveByNickname($nickname){ $nicknameCacheKey = sprintf('user:nickname:%s', $nickname); $userId = $memcache->get($nicknameCacheKey); if($userId !== null) { return UserPeer::retrieveByPK($userId); } else { $c = new Criteria(); $c->add(UserPeer::NICKNAME, $nickname); $user = UserPeer::doSelectOne($c); if($user !== null) { $memcache->set(sprintf('user:id:%d', $user->getId()), $user->toArray(BasePeer::TYPE_FIELDNAME)); $memcache->set($nicknameCacheKey, $user->getId()); } return $user; }}Tuesday, March 3, 2009
Symfony Speed and Hello World Benchmarks
After reading some posts showing that my blah blah framework is way more fast than symfony for a Hello World application I decided to explain why: because symfony is extensible and can adapt to your needs. That’s easy to say you may think, in fact, every framework out there claims that. So what makes symfony so special?
The following list names some of the features provided by symfony.
- Factories
- The Filter Chain
- The Configuration Cascade
- The Plugins System
- Controller adaptability
- View adaptability
Factories
Since version 1.0 symfony provides a configuration file called factories.yml. This file affects the application core classes configuration. There you can override symfony default classes by your own ones. This means you can set up a custom Front Controller, Web Request, Cache classes, Session storage, etc.
But this come with an extra price: when a symfony application bootstraps, it reads the configuration file from the filesystem. If the yml file was parsed before, then it loads a PHP file -which can also be cached with APC-, if not, then it parses the yml file, stores the parsed file on the cache folder and loads the configuration from there.
Why should I need that flexibility you may ask? In a project I’m involved with we needed Memcached. This means that we overrode all of the symfony default cache mechanism by our own custom classes. How? Setting up our classes in an easy to read yml file. So for sure when you benchmark your Hello World application symfony will be slower.
Filter Chain
One of the patterns from the Core J2EE Patterns book that impressed me the most is the Intercepting Filter. This patterns teach how to modify a request processing without the need to change Controllers or Model code. The idea is that in a configuration file you plug a class that will take care of pre or post processing the request. This classes are called filters. As an example, you can add a filter that checks if the user has the proper credentials to execute the action she wants. Another filter can cache the response, etc.
Symfony has the filters.yml file which can be specified by application or by module. This means that we can set filters to be executed for the whole application, and then for specific modules -let’s say for Ajax actions-, we disable them. Does your framework provides this flexibility without resorting to some kind of monkey patching techniques? No? Well symfony does. Say hello to the Filter Chain. So for sure when you benchmark your Hello World application symfony will be slower. Because it adds flexibility to the process. You want to get rid of this behavior? Sure, set up a custom controller in the factories.yml file, and in your new class override the loadFilters method.
Configuration Cascade
As explained in the configuration chapter of the symfony book, symfony allows to modify it’s behavior through some yml files. So for example we have the view.yml that tells symfony which css and javascript files it should load for the current request. We can have an application view.yml configuration and then override the settings per module. When symfony process the request it checks all this files, that’s why in the Hello World benchmark it‘s slow.
Plugins System
Symfony has a very powerful and easy to use Plugin System. It’s more than 400 plugins with a set of 200+ developers speaks by itself of it success.
The plugins can contain modules of their own and also a config.php file, similar to the project config.php file or the module config.php. When symfony process a request it checks for the settings in those files, this means that they will be read from disk. So in a plugin we can provide a custom logger that is fired up when bootstrapping the application. The Plugin user doesn’t need to care how the logger will be activated, she just now that it will work.
The same applies for plugin modules. How do you think that symfony knows that certain module/action should be called from a plugin? If you enable the plugin module on the settings.yml file symfony will check inside the plugin module to see if the requested action should be executed there.
Controller adaptability
For each action that the user call symfony will execute a page controller. Inside our modules symfony let us use a generic myModuleActions class that will extend sfActions or one specific to the action requested by the user, that as an example could be called indexAction and will extend the sfAction class. When a request is processed symfony first checks for the existence of the later. If it doesn’t exists then it tries to load the generic action for that module. Of course you don’t need this kind of flexibility for Hello World apps.
View Adaptability
For rendering the response symfony uses by default the sfPHPView class. If certain module in your application requires a different view, then there are at least three ways to accomplish this as explained here.
Conclusion
Symfony is a Professional Web Application Framework built to cope with real world needs. In a large project with more than a simple salutation feature sooner or later you will need the flexibility provided by the framework. This will save you time and will prevent headaches, because when you have built a whole system with a framework and the business needs start to push in a direction where you have to extend the framework you will thank yourself for having choose symfony at first.
In case that your client requires a Hello World! application, then you can use the following hyper fast framework code: die(“Hello World!”) ;-)
Wednesday, January 28, 2009
Integrating Facebook Hive with Symfony
A symfony feature that has been really helpful while working on projects with it is the debugging capabilities of the framework. Almost every day I see my self going to the command line an doing a tail -f log/frontend_dev.log in my symfony project. This helps me see what is going on behind the scenes, spot bugs, find places where I can improve the code, etc. If something goes wrong there is always a developer on the team that shouts “check the symfony logs”, showing that they had became an essential tool for development.
But not everything shines under the sun. Sometimes it happens that we would like to have a tool that allows us to filter the logs according to specific criteria. A tool that goes beyond a simple cat logs/frontend_dev.log | grep SELECT. We wanted to perform some analysis on the website usage, basically to help us improve it performance.
A colleague talked about Facebook Scribe, that was not actually related with this dream tool but later lead me to learn about the existence of the Apache Hive project (which was started by Facebook).
The Hive project allows to load a logs file and then filter it with SQL like commands. -Hive is more than this raw description, you can read more about it here.
My idea was to adapt the file logging format of symfony to make easy to import those files inside a Hive database. Because Hive support table partitioning by date, it should be easier to load the data from the logs and then perform the analysis with the SQL like syntax provided by Hive.
After some fights with Ant and Java 1.6 It was possible for me to get Hive running in my Mac. Then I just created a shameless copy of the sfFileLogger and renamed it to sfHiveLogger. I did small changes here and there and got it ready to log in a format that it’s easy to load later into Hive. I browsed through my testing symfony project to generate some logs and then I moved to the command line to start the fun with Hive.
There I created a table to hold the logs with the following command:
CREATE TABLE sflogs(
logTime STRING,
priorityName STRING,
message STRING
)
COMMENT 'This is the sflogs table'
PARTITIONED BY(dt STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\011'
LINES TERMINATED BY '\012';
Everything was working smoothly. The next step was to load the data from the logs file into the sflogs table to start issuing queries to it. I did it with this command:
LOAD DATA LOCAL INPATH '/path/to/myproject/log/frontend_dev.log.2009-01-28' INTO TABLE sflogs PARTITION (dt='2009-01-28');
With the data loaded I started to issue some commands against the Hive console like:
SELECT * FROM sflogs WHERE message LIKE "{sfRequest}%";
SELECT DISTINCT priorityname FROM sflogs;
SELECT COUNT(1) FROM sflogs;
The results were similar as when we work on a mysql client which was awesome. What amazed me the most was how with some easy changes it's possible to adapt symfony to our needs. But which were this changes? Here they are:
First we need to enable the sfHiveLogger in the logging.yml file of your symfony project under the sf_file_debug entry
Then do the shameless copy of the symfony class to your lib folder and rename it to sfHiveLogger.
Then change the code of the initialize method to look like this:
if (!isset($options['file']))
{
throw new sfConfigurationException('File option is mandatory for a file logger');
}
$dir = dirname($options['file']);
if (!is_dir($dir))
{
mkdir($dir, 0777, 1);
}
$logFileName = $options['file'] . '.' . date('Y-m-d');
$fileExists = file_exists($logFileName);
if (!is_writable($dir) || ($fileExists && !is_writable($logFileName)))
{
throw new sfFileException(sprintf('Unable to open the log file "%s" for writing', $logFileName));
}
$this->fp = fopen($logFileName, 'a');
if (!$fileExists)
{
chmod($logFileName, 0666);
}
Basically the change there is to append the current date in “Y-m-d-” format at the end of the logs file name. This will make easier to import the logs into Hive -only if you want them partitioned by date-.
Then on the log method change the line with:
$line = sprintf("%s %s [%s] %s%s", strftime('%b %d %H:%M:%S'), 'symfony', $priorityName, $message, DIRECTORY_SEPARATOR == '\\' ? "\r\n" : "\n");
to:
$line = sprintf("%s\t%s\t%s%s", strftime('%b %d %H:%M:%S'), $priorityName, $message, "\n");
Here what we do is to apply a little formating there. The most important part is to have the tab and then new line characters as delimiters because this was what we specified in the CREATE TABLE command above.
With this easy steps we can have a Hive enabled log file. If you want to learn more about Hive and the supported commands and the theory behind it please refer to the wiki.
Conclusion:
Besides that I’m still comparing Hive with other solutions to parse and analyze the logs, I think that this tool has a lot of potential to help debugging and profiling symfony applications. If we polish the log format and refine the table structure, then it’s shouldn’t be hard to setup some cronjobs that generate reports of the website usage, improving the usability of the symfony logs.
Monday, January 26, 2009
Custom Views in Symfony 1.0
public function executeFirstExample() {}class firstExampleSuccessView extends sfPHPView{ }protected function getGlobalVars(){ $context = $this->getContext();
$shortcuts = array(
'sf_context' => $context, 'sf_params' => $context->getRequest()-> getParameterHolder(), 'sf_request' => $context->getRequest(), 'sf_user' => $context->getUser(), 'sf_view' => $this, 'my_view_data' => array('foo' => 'this data came from firstExampleSuccessView') );
if (sfConfig::get('sf_use_flash')) { $sf_flash = new sfParameterHolder(); $sf_flash->add($context->getUser()-> getAttributeHolder()-> getAll('symfony/flash')); $shortcuts['sf_flash'] = $sf_flash; }
return $shortcuts;}<h1>First Example View:</h1><?php var_dump($my_view_data); ?>'my_view_data' => array('foo' => 'this data came from secondExampleView')public function executeSecondExample(){}<h1>Second Example View:</h1><?php var_dump($my_view_data); ?>all: view_class: secondExamplearray('foo' => 'this data came from thirdExampleView')<h1>First Example View:</h1><?php var_dump($my_view_data); ?>public function executeThirdExample(){ $module = $this->getRequestParameter('module'); $action = $this->getRequestParameter('action'); $this->getRequest()->setAttribute($module.'_'.$action.'_view_name', 'thirdExample', 'symfony/action/view');}- It checks inside the module/views folder to see if it can find our custom class as explained on the first example.
- If there is no user class there, it will check if we have set up one in the sfRequest as explained on example three.
- Then it will try to see if we defined a view in the module.yml file.
- As last resort it will load the default sfPHPView class.
Thursday, January 15, 2009
Firebug Framework: CSS helper functions
lib.js source file:hasClass = function(node, name)setClass = function(node, name)name to the noderemoveClass = function(node, name)name from nodetoggleClass = function(elt, name)elt has class name then it will remove it from the element, otherwise it will add the class to the element.setClassTimed = function(elt, name, context, timeout)name to the element. When the milliseconds timeout has expired, the class will be removed from the element and the timeout will be cleared. The timeout is added to the provided Firebug contextcancelClassTimed = function(elt, name, context) FBL.methodName(...)





