Programmer's Notes » production

Building high-loaded portal – InnoDB vs MyISAM

Konstantin Mirin — Wed, 20 May 2009 07:00:55 +0000

Preface

In November 2008 I’ve got a nice project, that resulted in a complex search engine, that specializes in clothing. I can’t give the link now, but will do when site officially launches. Key features of such service are:

Large number of users, which search with various parameters, rate items, leave comments, create articles etc.
Large database, that is updated frequently

Sure, the site is running on a dedicated server, sure I use caching, but optimal DB is a key to success.

MyISAM or InnoDB?

Since we need reliability, I created different tests, search the internet etc. My main question was – what to use MyISAM or InnoDB? I found the post on the MySQL Perfomance Blog (also this and this) and came to the conclusion, that when SELECT’ing, InnoDB is definitely faster (or, at least, is not slower). In addition, it provides constraints mechanism and allows transactions. The only thing it doesn’t have is FULLTEXT search. One more disadvantage is slower inserts. However, 80% of DB usage in this project are SELECT queries without FULLTEXT. For the fulltext search I created MyISAM table with fields I need and it works fine. The project is finished, but there will be phase 2 and I’m thinking of introducing Sphinx there. Anybody has experience with it?
So I came up with InnoDB storage engine.

Optimizing structure

Next important step is DB structure. I’ll give some background info about the project so you could better understand what I’m talking about. The site collects info from different UK shops. Then users can rate them, add some tags, create articles (called “looks”) and link that articles to the products they’re writing about. And the same (tags, comments, rating) applies to the looks. Here we come across with an interesting question. We can create structure of different types for this:

Create 2 separate cross-tables for linking Rating to Look or Product
Create one cross-table and add additional field “type” there to differentiate between Product and Look entities

See both variants in the following images:

Normalized variant

Denormalized version

Initially I thought that the second variant would perform better, but after creating and running tests, I saw, that:

If there are 0-2 ratings linked to entity, then normalized variant is better
If there are 3-5 ratings linked to entity, then denormalized variant is slightly (less than 1%) better
If there are 6-8 ratings linked to the entity, the normalized variant wins again

I did these tests for different situations similar to this one, in each test I generated random records and randomly linked them with each other. Number of record in Product, Look tables was 10000, in Rating – 20000. Number of records in cross tables varies because the linkings are random.
So I came up with normalizing everything like it should be done according to the DB theory.

The only exclusion of this rule is the following situation. In this project you can add different alternatives to the product. Each alternative is “better” or “cheaper” or “different” etc. And since the only case when these alternatives are pulled from DB is when user views the product, they are pulled altogether, so it is better to keep them in the same table.

Conclusion

InnoDB turns out to behave better under the high load, it supports transactions and maintains constraints. All that helps to develop better and never leave orphan records.
Using normalization techniques you usually get better performance
Create tests and test everything if you are going to do something great

Speeding up Yii or why should you use DB sessions

Konstantin Mirin — Thu, 05 Mar 2009 04:02:35 +0000

Yesterday I’ve read a post on The Developer’s Day blog. Then tested this thing in Yii. There is a jQuery in my application, so I created test controller and test action, that only outputted “Hello, world” and called it from the firebug console like this (I have url rewrite set up):

$.ajax({url:‘/test/test.html’});
$.ajax({url:‘/test/test.html’});
$.ajax({url:‘/test/test.html’});
$.ajax({url:‘/test/test.html’});
$.ajax({url:‘/test/test.html’});
$.ajax({url:‘/test/test.html’});

Yes, really it matters. Each call starts after the previous one ends and it takes long to finish them all. But each web application handles the concurrent requests by nature. So I started a forum thread to clarify this. And sebas recommended me to use CDbHttpSession instead of standard one. Yes, I knew, that storing sessions in DB speeds up the application, but I thought that is because of more effective search. And it turns out, that besides more effective search it gives one more DB benefit – concurrent access to the session storage. That’s actually the reason #1 why we usually use databases instead of files
So here is a quick trick, that solves the problem:

// application components
‘components’=>array(
……….
‘session’ => array(
‘class’ => ‘system.web.CDbHttpSession’,
‘connectionID’ => ‘db’,
),
),

It creates a new table YiiSession and uses it for session data. This time the JS test above runs much faster. Try it yourself!

Liked the post? Bookmark it

Hide Sites

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Yii framework – tips when going production

Konstantin Mirin — Fri, 20 Feb 2009 07:17:43 +0000

Yii is a great framework. It gives excellent built-in functionality, but, from my experience, is quite sensitive to the server configuration. Mostly because it relies on the newest technologies.

So here is a list of things to check when you experience strange things (blank screen without any messages is the worst one that may happen):

Check PHP version. Yii needs 5.1 at least. When using some specific PDO or other features in your application, ensure they’re available in the server’s PHP installation.
Check MySQL version. Should be at least 5. If it differs from development one (e.g. server has 5.0.21 and you have 5.0.51), then pay attention to the CDbConnection::emulatePrepare. Actually if you’re using MySQL, you’d better turn it on because MySQL doesn’t cache prepared statements and has buggy support for some types of qeries (e.g. SHOW)
If you’re using automated timestamps insertion like described in Yii Cookbook, you may encounter problems, when using different PDO or MySQL version. If you see your queries fail, try using
1. date(‘Y-m-d H:i:s’)
instead of
1. new CDbCommand(‘NOW()’)
When you can’t see any output, configure logging. If log is emply also, try die(‘message’) in all possible places. Starting from the entry script, continuing to your controller’s action. In some cases PHP crash is possible, which leads to immediate application termination, which makes both Yii logging and PHP’s echo ineffective and only die() workds fine
When using different PHP version, PDO versin may be different also. If you have application crashes when using bindParam(), try using bindValue() instead. This helps sometimes
Don’t forget to switch off YII_DEBUG in the index.php (entry script). However, if you encounter any problems, switch it back to help debugging