Programmer's Notes » tip http://programmersnotes.info Notes on the web-development and artificial intelligence. Mon, 18 Jul 2011 13:20:46 +0000 en hourly 1 http://wordpress.org/?v=3.1 Concurrent process management in Yii http://programmersnotes.info/2010/03/25/concurrent-process-management-in-yii/ http://programmersnotes.info/2010/03/25/concurrent-process-management-in-yii/#comments Thu, 25 Mar 2010 07:31:23 +0000 Konstantin Mirin http://programmersnotes.info/?p=372 Yet Another Related Posts Plugin.]]> Introduction

In my recent project there are quite many tasks that run in the background – generate thumbnails, detect colours on the pictures, run DB updates etc. Sure, all that is handled using cron and Yii commands. However there is a little problem. Consider we have DB update routine that should import new stock data from the datafeed. Datafeed is uploaded hourly, but upload can’t be scheduled to minutes – connection speed, different errors may interrupt the upload. On the other hand, sometimes processing takes 5 mins and sometimes – a few hours because in the first variant we just replace records and in the latter – download images for the new products.

The problem

So if we schedule to run every 5 mins (to check if update is ready), then we get the following:

  1. 19:00:00 started update
  2. 19:00:01 finished. No files arrived
  3. 19:05:00 started update
  4. 19:05:02 started DB update (1)
  5. 19:10:00 started update
  6. 19:10:02 started DB update (2)
  7. 19:11:48 finished DB update. Source file removed. (1)
  8. 19:11:50 error updating. File is not readable. Terminating (2)

As a result, DB will have wrong data because records were inserted twice. Sure, we can control this with DB constraints, but duplicates is not the only possible error.

Solution overview

The only variant to fix this – go Unix way and create lock file when some process starts. Actually that is a good variant, but I decided to use one file for all processes. Sure, I have to lock it when I read/write from it and other processes have to wait when I’m done with this. It is not brilliant, but I decided to go this way and I’ll show how I did this in this post.

First of all, I didn’t want to put the same code for locking/checking/unlocking in every command. So I created the taskmanager extension. It is not yet finished, so I don’t publish it.

Technical details

This is an application component which is extended from CApplicationComponent class and implements TaskManger interface:

  1. interface TaskManager
  2. {
  3.         public function isRunning($id);
  4.         public function logStarted($id);
  5.         public function logFinished($id);
  6.         public function logTerminated($id);
  7.         public function sendSignal($id);
  8.         public function getStatus($id);
  9.         public function getLog($id);
  10. }

Each command/process has it’s ID. ID is a plain text without spaces. Although spaces are not restricted and it will work with them, I decided to apply the same naming conventions as for class properties in Yii.
Let’s go through the methods:

  • isRunning($id). Takes process ID as a parameter and return true if this process is running.
  • logStarted($id). This checks if process is not already running (or if duplicates are allowed) and adds records to the pids file. Actually this interface may be implemented using the DB, so no locking or other things will be needed.
  • logFinished($id). Removes record from the pids file (or from DB if implemented using it).
  • logTerminated($id). Nearly the same as previous, but it doesn’t check if process is running, it tries to remove the record anyway.
  • sendSignal($id, $signal). Sends signal to the process. Process should check it’s buffer file when it is running and react to signals. It is not implemented because requires changes in the command code.
  • getStatus($id). Returns current status of a given process. It is not implemented either for the same reason.
  • getLog($id). Should return log of a process execution. Not implemented either.

And now let’s take a look at the class implementing it.
First of all, it’s initialization:

  1. class FileTaskManager extends CApplicationComponent implements TaskManager
  2. {
  3.     public $pidFile;
  4.     public $pidLogFile;
  5.     public $processDir;
  6.  
  7.         const TRIES_TIMEOUT = 1000;
  8.  
  9.         public function init()
  10.         {
  11.                 if (empty($this->pidFile)) $this->pidFile = Yii::app()->params[‘cmdPidFile’];
  12.                 if (empty($this->pidLogFile)) $this->pidLogFile = Yii::app()->params[‘cmdPidLogFile’];
  13.                 if (empty($this->processDir)) $this->processDir = Yii::app()->params[‘cmdProcDir’];
  14.                 $this->pidFile = Yii::app()->basePath.DIRECTORY_SEPARATOR.$this->pidFile;
  15.                 $this->pidLogFile = Yii::app()->basePath.DIRECTORY_SEPARATOR.$this->pidLogFile;
  16.                 $this->processDir = Yii::app()->basePath.DIRECTORY_SEPARATOR.$this->processDir;
  17.                 if (!file_exists($this->pidFile))
  18.                         touch($this->pidFile);
  19.                 if (!file_exists($this->pidLogFile))
  20.                         touch($this->pidLogFile);
  21.                 parent::init();
  22.         }
  23. ……
  24. }
  • $pidFile is where all currently running processes are stored
  • $pidLogFile is where we write “DD.MM.YYY HH:MM:SS process XXX started” and “ DD.MM.YYY HH:MM:SS process XXX finished”
  • $processDir is where log files and exchange files should reside (methods sendSignal, getStatus, getLog)

We get these variables from the config file and update to fit current application path. Also, since we extend the class from CApplicationComponent, we should perform standard initialization, an we do this by calling parent::init() in the last line.

Since we’re using file, it should be locked from other processes when we’re working with it. So we need a method that will open it and wait if it is busy now:

  1. protected function openLocked($fileName, $lockMode = LOCK_EX)
  2. {
  3.         if ($lockMode == LOCK_SH) $openMode = ‘r’;
  4.         else $openMode = ‘a’;
  5.         if ($fp = fopen($fileName, $openMode))
  6.         {
  7.                 $startTime = microtime(true);
  8.                 do
  9.                 {
  10.                         $canWrite = flock($fp, $lockMode);
  11.                         // If lock not obtained sleep for 0 – 100 milliseconds, to avoid collision and CPU load
  12.                         if(!$canWrite) usleep(round(rand(0, 100)*1000));
  13.                 }
  14.                 while ((!$canWrite)and((microtime()-$startTime) < self::TRIES_TIMEOUT));
  15.                 //file was locked so now we can store information
  16.                 if ($canWrite)
  17.                 {
  18.                         return $fp;
  19.                 }
  20.                 else
  21.                 {
  22.                         fclose($fp);
  23.                         throw new CException(‘File is locked’, 1);
  24.                 }
  25.         }
  26. }

Idea is pretty simple – we try to lock the file. If we succeed, we return the file pointer and we throw exception otherwise. We try it several times until we reach timeout. Intervals are random to ensure that different processes started simultaneously will not try to lock it at the same time.
And now we can implement our methods:

  1. public function isRunning($id)
  2. {
  3. $fl = $this->openLocked($this->pidFile, LOCK_SH);
  4.         $running = false;
  5.         while (!feof($fl))
  6.         {
  7.                 $process = fgets($fl);
  8.                 if (stristr($process, $id."\t"))
  9.                 {
  10.                         $running = true;
  11.                         break;
  12.                 }
  13.         }
  14.         fclose($fl);
  15.         return $running;
  16. }
  17.  
  18. public function logStarted($id, $allowConcurrent = false)
  19. {
  20.         if ($this->isRunning($id) && !$allowConcurrent)
  21.         throw new CException(‘Process is running’, 2);
  22.         $fl = $this->openLocked($this->pidFile, LOCK_EX);
  23.         $dtStarted = date(‘Y-m-d H:i:s’);
  24.         $pid = getmypid();
  25.         fwrite($fl, $dtStarted."\t".$id."\t".$pid."\n");
  26.         fclose($fl);
  27.         $fl = $this->openLocked($this->pidLogFile, LOCK_EX);
  28.         fwrite($fl, $dtStarted."\t".$id."\t"."started"."\n");
  29.         fclose($fl);
  30.         return $dtStarted;
  31. }
  32.  
  33. public function logFinished($id, $time = ”)
  34. {
  35.                 if (!$this->isRunning($id))
  36.                 throw new CException(‘Process is not running!’, 3);
  37.         $fl = $this->openLocked($this->pidFile, LOCK_SH);
  38.         $cont = ”;
  39.         $pid = getmypid();
  40.         $match = $time ? $time."\t".$id."\t".$pid : $id."\t".$pid;
  41.         fseek($fl, 0);
  42.         while (!feof($fl))
  43.         {
  44.                 $process = fgets($fl);
  45.                 if (stristr($process, $match) === false)
  46.                 {
  47.                         if (!empty($process) && $process != "\n")
  48.                                 $cont .= $process;
  49.                 }
  50.         }
  51.         fclose($fl);
  52.         $fl = $this->openLocked($this->pidFile, LOCK_EX);
  53.         ftruncate($fl, 0);
  54.         fwrite($fl, $cont);
  55.         fclose($fl);
  56.         $fl = $this->openLocked($this->pidLogFile, LOCK_EX);
  57.         fwrite($fl, date(‘Y-m-d H:i:s’)."\t".$id."\t"."finished"."\n");
  58.         fclose($fl);
  59. }
  60.  
  61. public function logTerminated($id, $time = ”)
  62. {
  63.         $fl = $this->openLocked($this->pidFile, LOCK_SH);
  64.         $cont = ”;
  65.         $pid = getmypid();
  66.         $match = $time ? $time."\t".$id."\t".$pid : $id."\t".$pid;
  67.         fseek($fl, 0);
  68.         while (!feof($fl))
  69.         {
  70.                 $process = fgets($fl);
  71.                 if (stristr($process, $match) === false)
  72.                 {
  73.                         if (!empty($process) && $process != "\n")
  74.                                 $cont .= $process;
  75.                 }
  76.         }
  77.         fclose($fl);
  78.         $fl = $this->openLocked($this->pidFile, LOCK_EX);
  79.         ftruncate($fl, 0);
  80.         fwrite($fl, $cont);
  81.         fclose($fl);
  82.         $fl = $this->openLocked($this->pidLogFile, LOCK_EX);
  83.         fwrite($fl, date(‘Y-m-d H:i:s’)."\t".$id."\t"."finished"."\n");
  84.         fclose($fl);
  85. }

So now we just need to call these methods from our command. But you know, I’m very lazy, so I created a base class for all my commands:

  1. abstract class GenericCommand extends CConsoleCommand
  2. {
  3.         protected $pid = ‘gen-cmd’;
  4.         protected $allowConcurrent = false;
  5.  
  6.         public function __construct()
  7.         {
  8.                 $this->pid = str_replace(‘Command’, ”, get_class($this));
  9.         }
  10.  
  11.         public function run($params)
  12.         {
  13.                 try
  14.                 {
  15.                         $time = Yii::app()->tm->logStarted($this->pid, $this->allowConcurrent);
  16.                         $this->runCmd($params);
  17.                         Yii::app()->tm->logFinished($this->pid, $time);
  18.                 }
  19.                 catch (Exception $e)
  20.                 {
  21.                         Yii::log(‘Process ‘.$this->pid.‘ error. ‘.$e->getMessage());
  22.                         echo ‘Process ‘.$this->pid.‘ error. ‘.$e->getMessage();
  23.                         if (!empty($time))
  24.                         {
  25.                                 try
  26.                                 {
  27.                                         Yii::app()->tm->logFinished($this->pid, $time);
  28.                                 }
  29.                                 catch (Exception $e2)
  30.                                 {
  31.                                         Yii::log(‘Process ‘.$this->pid.‘ can\’t be stopped. ‘.$e->getMessage());
  32.                                         echo ‘Process ‘.$this->pid.‘ can\’t be stopped. ‘.$e->getMessage();
  33.                                         Yii::app()->tm->logTerminated($this->pid, $time);
  34.                                 }
  35.                         }
  36.                 }
  37.         }
  38. }

It’s constructor created the process ID from the class name. It is quite convenient feature. It also handles start and stop so all we need to do is just to implement runCmd() method in the derived classes like this:

  1. class ChangeDBCommand extends GenericCommand
  2. {
  3.         public function runCmd($args)
  4.         {
  5.                 $shops = Shop::model()->findAll();
  6.                 foreach ($shops as $shop)
  7.                 {
  8.                         $shop->save();
  9.                 }
  10.                 $categories = Category::model()->findAll();
  11.                 foreach ($categories as $cat)
  12.                 {
  13.                         $cat->save();
  14.                 }
  15.         }
  16. }

Then you just type:

  1. $ php console.php changeDB &
  2. $ php console.php changeDB

Can’t start process! It is already running!

First command starts routine in background. Second command starts the same command but in the terminal. So you’ll see the output for it. It says that it can’t be started because previous run is not finished.

There is one thing you should do in order to make that work. You should add a component to the application configuration. Open the protected/config/console.php file and add into the components list:

  1. ‘tm’=>array(
  2.                         ‘class’ => ‘FileTaskManager’
  3.         )

And into the import section:

  1. ‘import’=>array(
  2.                 ‘application.models.*’,
  3.                 ‘application.components.*’,
  4.                 ‘application.helpers.*’,
  5.                 ‘application.extensions.taskmanager.*’, //this is added!
  6.         ),

Other problems

I’ve been using this solution for 5 months. Practice shows that sometimes process may run out of memory or some other error occurs that is not caught by out exception handling and it is terminated without our $tm->logFinished() call. This leads to the situation when process is not actually running, but task manager thinks it is because it’s record is still present in the pids file. In order to prevent such situations, I modified isRunning routine like this:

  1. public function isRunning($id)
  2. {
  3.         $fl = $this->openLocked($this->pidFile, LOCK_SH);
  4.         $running = false;
  5.         while (!feof($fl))
  6.         {
  7.                 $process = fgets($fl);
  8.                 if (stristr($process, $id."\t"))
  9.                 {
  10.                         if ($this->isReallyRunning($process))
  11.                         {
  12.                                 $running = true;
  13.                         }
  14.                         else
  15.                         {
  16.                                 $term = true;
  17.                         }
  18.                         break;
  19.                 }
  20.         }
  21.         fclose($fl);
  22.         if ($term) {
  23.                 $this->logTerminated($id);
  24.         }
  25.         return $running;
  26. }
  27.  
  28. protected function isReallyRunning($proc)
  29. {
  30.         list($_, $_, $pid) = explode("\t",$proc);
  31.         $pid = trim($pid);
  32.         $res = exec(‘ps -A|grep ‘.$pid);
  33.         if (empty($res))
  34.                 return false;
  35.         return true;
  36. }

This performs additional check by trying to find the selected PID in the system process list by listing all processes ps -A and checking if our pid is there (grep [pid]). If result is empty, then process is not running.

There is a potential pitfall in the isRunning method. If you try to do something with pids file before it is closed in line 21, you’ll get into the deadlock because file is locked for modifications. That’s why I call logTerminated after it is closed. Be attentive!

I’d be thankful if someone takes the code and adds signal processing there and maybe implements DBTaskManager in addition to FileTaskManager. Let me know if someone is interested, we can collaborate on this and deliver a nice ready-made extension.

As usual, any comments are welcome!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2010/03/25/concurrent-process-management-in-yii/feed/ 0 8 Firefox plugins I use every day http://programmersnotes.info/2010/03/16/8-firefox-plugins-i-use-every-day/ http://programmersnotes.info/2010/03/16/8-firefox-plugins-i-use-every-day/#comments Tue, 16 Mar 2010 06:00:39 +0000 Konstantin Mirin http://programmersnotes.info/?p=325 Yet Another Related Posts Plugin.]]> Firefox is great for web development mainly because of the large number of plug-ins available. Here’s my list:

  1. Web developer toolbar – great thing for the web development. Personally I use resizing features, password revealing and elements highlighting.
  2. Firebug – absolute winner. This is actually the main tool for development I use. I’m going to write a more detailed post about it later. For now, you can enjoy this video
  3. Cache status shows cached size in the status bar. But the main use is it’s ability to clear cache in a few clicks. This saves lots of time clicking here and there when developing complex JS application.
  4. Live HTTP Headers allows to view all HTTP headers that go here and there while you’re requesting the page. It’s a great thing when you need to grab some content from the password-protected area or inspect what’s going on when you request page.
  5. YSlow is a firebug plug-in from the Yahoo team which analyzes your content and shows you the ways of the client-size optimization of your page. It’s recommendations are really valuable. Try it and you’ll never uninstall it :)
  6. SitePoint HTML reference is a firebug plug-in too, that adds HTML help to the every HTML tag and CSS property. It also provides examples and best practices. I don’t think this is great value for gurus, but definitely useful for all others :)
  7. Fasterfox is a plug-in that boost Firefox performance. It is in the “experimental” state, but quite stable. I haven’t ever experience any problems with it.
  8. Selenium IDE is designed as a helper tool for creating selenium tests. It is really great and handy tool if you use automated testing. I’m going to cover this in my blog later.

And what are your favourite plug-ins? How do you use them?

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2010/03/16/8-firefox-plugins-i-use-every-day/feed/ 1 Grabbing password-protected content with cURL http://programmersnotes.info/2009/05/24/grabbing-password-protected-content-with-curl/ http://programmersnotes.info/2009/05/24/grabbing-password-protected-content-with-curl/#comments Sun, 24 May 2009 07:00:55 +0000 Konstantin Mirin http://programmersnotes.info/?p=159 Yet Another Related Posts Plugin.]]> Introduction

Websites, that aggregate something become more and more popular because we need all information available in one place and accessed fast and easily. From time to time, you come across with tasks, that require retrieving some data from the password-protected area. For example, I use Guru.com for my job search. They post lots of the projects there, but they don’t offer convenient listing and filtering. So I developed my own tool, that grabs everything from there and ranks it all in the way I need. So, we’ll take a look at different types of password-protected areas and see how to deal with any of them

Cookie-based authorization

We’ll start from the most popular type of authorization :) First, let’s take a look, how data goes between the user and the server:

Cookie-based authentication

Cookie-based authentication


From the process it becomes evident, that to pull the data from the pass-protected area, we should basically “catch” the cookie and then perform all requests passing it with every request. It seems to be complex, but luckily we have cURL PHP extension, that makes all the dirty job.

So all we have to do is:

  1. Send login request
  2. Catch cookie
  3. Send all data requests we need

The second part is done automatically, so we can just focus on authentication and data retrieval.

Implementation

We’ll basically need 3 functions:

  • requestContent – this will make actual HTTP request to the URL we provide with parameters we provide (POST, GET variables, HTTP_REFERRER etc)
  • authenticate – this will send authentication info to the server using the requestContent function
  • getPage – this will actually get data we need using the same requestContent function. Actually, you may use requestContent instead, but you’ll definitely want some post-processing the result, for example, fetch some info. So it’s better to define a separate function, that does it.

So let’s go! I’ve created 3 files: config.php where we’ll store config data, functions.php where we’ll define out functions and index.php, that actually carries out the job.
config.php follows:

  1. <?php
  2. /**
  3.  * Specifies user agent. YOu can put anything you want here, I’m just showing,
  4.  * that with cUrl you can fake anything. I just copied my UserAgent string. Make ]
  5.  * sure you don’t have line breaks in it!!!
  6.  */
  7. define(‘GR_USER_AGENT’, ‘Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 FirePHP/0.2.4′);
  8. /**
  9.  * Path to the file where cUrl will store cookies. cUrl behaves like a browser -
  10.  * it stores all cookies it gets somewhere and then submits them with each HTTP
  11.  * request.
  12.  */
  13. define(‘GR_COOKIE_FILE’, ‘cookies.txt’);
  14. /**
  15.  * URL where you log in to your service. We’ll be logging in to Wikipedia.
  16.  * I took this path from the action parameter of the <form> tag on this page:
  17.  * http://en.wikipedia.org/w/index.php?title=Special:UserLogin&returnto=Special:UserLogout
  18.  *
  19.  * You should just take a look at the source code of the page you’re going to
  20.  * login from, finf the login form and see action parameter there.
  21.  */
  22. define(‘GR_LOGIN_URL’, ‘http://en.wikipedia.org/w/index.php?title=Special:UserLogin&action=submitlogin&type=login’);
  23. /**
  24.  * Your username
  25.  */
  26. define(‘GR_USERNAME’, ‘KJedi’);
  27. /**
  28.  * Your password
  29.  */
  30. define(‘GR_PASSWORD’, ‘did you think I\’ll give you my pass? :) ’);
  31. ?>

In functions.php I define our 3 functions. requestContent looks as following:

  1. function requestContent($url, $postData = array(), $referer = ”, $headers = array())
  2. {
  3.         $ch = curl_init();//creating cUrl instance
  4.         curl_setopt($ch, CURLOPT_URL, $url);//setting our URL
  5.         curl_setopt($ch, CURLOPT_HEADER, 0);
  6.         if(GR_USER_AGENT) curl_setopt($ch, CURLOPT_USERAGENT, GR_USER_AGENT);//setting user agent
  7.         if($postData)//if we have post data
  8.         {
  9.                 curl_setopt($ch, CURLOPT_POST, 1);//tell cUrl we’ll be using POST
  10.                 curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);//set post data
  11.         }
  12.         curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  13.         curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);//force cUrl to follow any redirects
  14.         if($referer) curl_setopt($ch, CURLOPT_REFERER, $referer);//if there is referrer specified, set it
  15.         if(GR_COOKIE_FILE)//if there is cookie file
  16.         {
  17.                 //set next 2 options needed in order to uuse cookies correctly everywhere
  18.                 curl_setopt($ch, CURLOPT_COOKIEFILE, GR_COOKIE_FILE);
  19.                 curl_setopt($ch, CURLOPT_COOKIEJAR, GR_COOKIE_FILE);
  20.         }
  21.         if ($headers)//if there are additional headers
  22.         {
  23.                 curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);//set them
  24.         }
  25.         $result = curl_exec($ch);//execute cUrl query
  26.         curl_close($ch);//close cUrl
  27.         return $result;//return cUrl result
  28. }

Basically, here we init cURL, and get the handler. It is similar to the fopen(). Then we set all options we need and finally execute the HTTP query and return result.
Authentication is based on the previous function, that carries our all the low-level job. We only have to examine our login form and determine, what variables to send. So, open the login page of the service you want to access, find the <form> tag and copy the login URL. You’ll have to put it into the config.php file. Then see what fields are there. Note, that there my be hidden fields also, note their variables and create an associative array for them (see the following code). If you’re not sure what variables are actually sent, you may use Live Http Headers plugin for the Firefox.
Usage is simple. Open the login page. Fill it in. Then start the plugin (Tools -> Live HTTP headers). Submit the form and see the number of queries are shown. You actually need only the first one – this is your request. See the screenshot:

Http headers

Http headers


And now the code:

  1. function authenticate()
  2. {
  3.         $post_info = array(
  4.                            ‘wpName’ => GR_USERNAME,
  5.                            ‘wpPassword’ => GR_PASSWORD,
  6.                            ‘wpLoginAttempt’  => ‘Log+in’);
  7.         $postData = ”;
  8.         foreach($post_info as $name => $value)
  9.         {
  10.                 $postData .= $name.‘=’.$value.‘&’;
  11.         }
  12.         $postData = trim(substr($postData, 0, -1));
  13.         $result = requestContent(GR_LOGIN_URL, $postData, GR_LOGIN_URL);
  14.         //here you may check if you were logged in successfully
  15. }

After getting result, you may parse it and see if you got a successful login. After you performed this query and got response, you’ve also got the session ID stored in cookie. You may open the cookie file and take a look yourself. After this you may perform any requests to the servic and it will respond as if you are logged user. The trick is that cUrl will send him cookies from the cookie file. And we have the session ID there or anything else that site uses for user identification.
So my getPage() function is a simple wrapper over the requestContent():

  1. function getPage()
  2. {
  3.         return requestContent(‘http://en.wikipedia.org/’);
  4. }

Basically, that’s all you need in order to get into the protected area. Now you’re free to do anything you want. Using regular expressions for parsing is the most common way to get the data from the page, so maybe you’ll need the tool for testing your regexps. I really recommend “The Regext Coach”, it greatly simplifies debugging :)

HTTP authentication

Another approach to the authentication, that is sometimes used in different control panels is HTTP authentication. You should have seen that – you’re presented with a standard screen, prompting for username and password. This username and password are sent with each request. You are not prompted fro them every time just because browser remembers it. To access such area, you don’t need any authenticate() function. You just pass login and pass with each request.
Here are modifications you should do in the above code:

  1. function requestContentHttpAuth($url, $postData = array(), $referer = ”, $headers = array())
  2. {
  3.         $ch = curl_init();//creating cUrl instance
  4.         curl_setopt($ch, CURLOPT_URL, $url);//setting our URL
  5.         curl_setopt($ch, CURLOPT_HEADER, 0);
  6.         curl_setopt($ch, CURLOPT_USERPWD, GR_USERNAME.‘:’.GR_PASSWORD);//<—– here is the change – you pass yout auth data each time
  7.         if(GR_USER_AGENT) curl_setopt($ch, CURLOPT_USERAGENT, GR_USER_AGENT);//setting user agent
  8.         if($postData)//if we have post data
  9.         {
  10.                 curl_setopt($ch, CURLOPT_POST, 1);//tell cUrl we’ll be using POST
  11.                 curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);//set post data
  12.         }
  13.         curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  14.         curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);//force cUrl to follow any redirects
  15.         if($referer) curl_setopt($ch, CURLOPT_REFERER, $referer);//if there is referrer specified, set it
  16.         if(GR_COOKIE_FILE)//if there is cookie file
  17.         {
  18.                 //set next 2 options needed in order to uuse cookies correctly everywhere
  19.                 curl_setopt($ch, CURLOPT_COOKIEFILE, GR_COOKIE_FILE);
  20.                 curl_setopt($ch, CURLOPT_COOKIEJAR, GR_COOKIE_FILE);
  21.         }
  22.         if ($headers)//if there are additional headers
  23.         {
  24.                 curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);//set them
  25.         }
  26.         $result = curl_exec($ch);//execute cUrl query
  27.         curl_close($ch);//close cUrl
  28.         return $result;//return cUrl result
  29. }

That’s all I wanted to share with you today. Hope you fins it helpful! If you have any questions/comments, feel free to add them!

Quick Note on OOP

I didn’t create the Curl class or present solution as a class because I don’t think it makes my explanations more clear or the code easier to use. This is not full solution, this is just a technique and I tried to present the code in such a way, that you could use them with minimal changes. You can easily put that functions to the class :)

Further reading

Download code for the post

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/05/24/grabbing-password-protected-content-with-curl/feed/ 14 OOP in jQuery http://programmersnotes.info/2009/04/30/oop-in-jquery/ http://programmersnotes.info/2009/04/30/oop-in-jquery/#comments Thu, 30 Apr 2009 20:19:51 +0000 Konstantin Mirin http://konstantin.takeforce.net/?p=12 Yet Another Related Posts Plugin.]]> Intro

OOP is a great concept, it simplifies the programmer’s life, especially in the field of interface development. Nearly all modern web-interfaces use JS in some way – for field highlighting, validation, help etc. It’s OK when there are such simple things.
And now imagine, that we have a set of dialogue windows (HTML+CSS) for adding a comment, rating, sending to the friend. Each of these boxes has a number of functions – check if user is logged in(if not – show login form), send AJAX request, process response, show error/success messages, position the box on the page etc.

The problem

First idea I had about them is to use simple inheritance, create Box class, derive CommentBox, RatingBox and other classes, but it turned out, that jQuery (I used it there, because system is built using Yii framework, which uses jQuery for client-side coding) doesn’t support extending classes like, for example, ExtJS or YUI. Yes, it has $.extend() function, but it just copies the object, not actually extends class. Why it is not the same? Let’s look here:

  1. //namespace
  2. var Popups = {};
  3. //class constructor
  4. Popups.Box = function()
  5. {
  6.         //initialization here
  7. }
  8. Popups.Box.prototype = {
  9.         container:$(‘<div class="container"></div>’),
  10.         content:null,
  11.         create:function()
  12.         {
  13.                 //dom elements generation here
  14.         },
  15.         showHide:function()
  16.         {
  17.                 //some logic here
  18.         }
  19.         //other methods and properties go here
  20. }

Normally (in ExtJS, for example) when writing:

  1. Popups.CommentBox = $.extend(true,{}, Popups.Box, {
  2.         create:function()
  3.         {
  4.                 alert(‘overriden!’);
  5.         }
  6. })

I expect, that if I do:

  1. var c = new CommentBox();
  2. c.create();

I should get an alert window. However, this is not true for jQuery. It just copies everything and we get some messed things. So I gave this up and followed another pattern.

The solution

Box object became container for all boxes. I took care of messaging, determining if user is logged, and all other routines. It also serves as a factory, because all clicking icons go to the showHide method and it has to determine, which box to show and create this inner object.
And this inner object takes care of the box-specific logic. If it needs some common info, for example, the ID of the product it is called for, it calls the Box method (see LowCoupling GRASP principle here :) ). So in the Box I have:

  1.         getBoxType:function()
  2.         {
  3.                 //determine the box we’re dealing with by the link class
  4.                 var tg = $(this.event.target);
  5.                 if (tg.hasClass(‘f’))
  6.                         return ‘favourite’;
  7.                 else if (tg.hasClass(‘t’))
  8.                         return ‘friend’;
  9.                 else if (tg.hasClass(‘ra’))
  10.                         return ‘rate’;
  11.                 else if (tg.hasClass(‘c’))
  12.                         return ‘comment’;
  13.                 else if (tg.hasClass(‘r’))
  14.                         return ‘rss’;
  15.                 else if (tg.hasClass(‘add’))
  16.                         return ‘addTag’;
  17.                 else if (tg.hasClass(‘b’))
  18.                         return ‘bookmark’;
  19.                 return ”;
  20.         },
  21.         getInnerObj:function()
  22.         {
  23.                 if (!us.logged)//if user is not logged, we show him the login box instead of anything else
  24.                 {
  25.                         return new Popups.LoginObject(this);
  26.                 }
  27.                 var cls;
  28.                 this.type = this.getBoxType();
  29.                 switch (this.type)
  30.                 {
  31.                         case ‘favourite’:
  32.                                         cls = this.objects[this.type] = new Popups.FavouriteObject(this);
  33.                                 break;
  34.                         case ‘friend’:
  35.                                         cls = this.objects[this.type] = new Popups.FriendObject(this);
  36.                                 break;
  37.                         case ‘rate’:
  38.                                         cls = this.objects[this.type] = new Popups.RateObject(this);
  39.                                 break;
  40.                         case ‘comment’:
  41.                                         cls = this.objects[this.type] = new Popups.CommentObject(this);
  42.                                 break;
  43.                         case ‘rss’:
  44.                                         cls = this.objects[this.type] = new Popups.RssObject(this);
  45.                                 break;
  46.                         case ‘bookmark’:
  47.                                         cls = this.objects[this.type] = new Popups.BookmarkObject(this);
  48.                                 break;
  49.                         default:
  50.                                 cls = this.objects[this.type] = null;
  51.                 }
  52.                 return cls;
  53.         },
  54.         show:function()
  55.         {
  56.                 //some stuff
  57.                 var obj = this.getInnerObj();
  58.                 if (typeof obj == ‘object’ && obj != null)
  59.                 {
  60.                         this.content.append(obj.getContents());
  61.                 }
  62.         },

So instead of inheritance I implemented aggregation (Box aggregates CommmentObject, Favourite object etc).

Everything goes well until you want to to attach some method of current object as a handler to some event. For example:

  1. Popups.Box.prototype = {
  2.         //properties here
  3.         productID:0,
  4.  
  5.         create:function()
  6.         {
  7.                 this.form.submit(this.onSubmit);
  8.                 productID = this.parentBox.getItemID();
  9.         },
  10.         onSubmit:function()
  11.         {
  12.                 alert(this.productID);
  13.         }
  14. //other methods follow
  15. }

Right? Wrong! Because when submit event occurs, jQuery will call the onSubmit method, but this will refer to the form, not to the Box object! However, it is logical to put processing of the form into the same object. In ExtJS and YUI you can pass the scope parameter for every callback function, so you can control the “this” variable in the callback call. jQuery doesn’t allow this. I used the following workaround:

  1. Popups.Box.prototype = {
  2.         //properties here
  3.         productID:0,
  4.  
  5.         create:function()
  6.         {
  7.                 var ctx = this;
  8.                 this.form.submit(function(event){
  9.                         ctx.onSubmit(event);
  10.                 });
  11.                 productID = this.parentBox.getItemID();
  12.         },
  13.         onSubmit:function(event)
  14.         {
  15.                 alert(this.productID);
  16.         }
  17. //other methods follow
  18. }

If you need the object, that triggered an event, just pass “this” there:

  1. create:function()
  2. {
  3.         var ctx = this;
  4.         this.form.submit(function(event){
  5.                 ctx.onSubmit(this, event);
  6.         });
  7.         productID = this.parentBox.getItemID();
  8. },
  9. onSubmit:function(form, event)
  10.  
  11.         alert(this.productID);
  12. }

Having these 2 tricks you can use OOP approach to designing reusable interface components in a handy modern way.

Back to inheritance

Since I discovered the problem with inheritance, I want to develop a plugin, that fixes this, that will provide nice class inheritance model to take jQuery a step ahead. I was going to digg the ExtJS’s Ext.extend() code and find out how this is done correctly, but while preparing an article, I found this material, which is actually a guide for creation of such plugin: http://phrogz.net/JS/Classes/OOPinJS2.html. Unfortunately, I don’t have time for this now, but I plan to do this in the nearest future. If someone does this earlier, please let me know :)

Links

  • ExtJS – a component, interface-oriented JS framework
  • YUI – Yahoo library for the user interface. ExtJS was derived from it long ago. They are similar in extending and event processing (the “scope” I mentioned initially appeared in YUI)
  • jQuery – a lightweight VERY popular JS framework with an easily extensible structure, but lacks OOP techniques!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/04/30/oop-in-jquery/feed/ 2 Trick with php switch() http://programmersnotes.info/2009/03/06/trick-with-php-switch/ http://programmersnotes.info/2009/03/06/trick-with-php-switch/#comments Fri, 06 Mar 2009 10:41:39 +0000 Konstantin Mirin http://programmersnotes.info/?p=151 Yet Another Related Posts Plugin.]]> I bet you didn’t know, that switch can be used to answer the question: “Which of the N following statements is true?” Here is how this can be done:

  1. <?php
  2. $a = 1;
  3. $b = 15;
  4. $c = 20;
  5. switch (true)
  6. {
  7.         case $a > $b:
  8.                 {
  9.                         echo ‘$a > $b’;
  10.                         break;
  11.                 }
  12.         case ($a > $b) && ($c > $a):
  13.                 {
  14.                         echo ‘($a > $b) && ($c > $a); $c is the biggest, $b is the lowest’;
  15.                         break;
  16.                 }
  17.         case ($c > $b) && ($c > $a):
  18.                 {
  19.                         echo ‘$c is the biggest again!’;
  20.                         break;
  21.                 }
  22.         default:
  23.                 {
  24.                         echo ‘some default value’;
  25.                         break;
  26.                 }
  27. }
  28. ?>

And little description – you can write true or false in switch(), this will cycle through the case (statement) and execute the commands, that goes below it. In our case, only third statement is true – it prints out, that $c is the biggest.
You shouldn’t use it instead of if or if…elseif…else every time, but if you know this trick, you’ll see when this can be applied.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/03/06/trick-with-php-switch/feed/ 4 Return several values from PHP function http://programmersnotes.info/2009/03/06/return-several-values-from-php-function/ http://programmersnotes.info/2009/03/06/return-several-values-from-php-function/#comments Fri, 06 Mar 2009 07:38:40 +0000 Konstantin Mirin http://programmersnotes.info/?p=148 Yet Another Related Posts Plugin.]]> As you know, PHP can return only one value from the function. However, there is a simple workaround – we return an indexed array from function and immediately split it into variables using list():

  1.  
  2. function ret()
  3. {
  4. //something useful here
  5. return array($varA, $varB);
  6. }
  7. list($a, $b) = ret();//we’ll have $varA in $a and $varB in $b after this
  8.  

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/03/06/return-several-values-from-php-function/feed/ 1 Speeding up Yii or why should you use DB sessions http://programmersnotes.info/2009/03/05/speeding-up-yii-or-why-should-you-use-db-sessions/ http://programmersnotes.info/2009/03/05/speeding-up-yii-or-why-should-you-use-db-sessions/#comments Thu, 05 Mar 2009 04:02:35 +0000 Konstantin Mirin http://programmersnotes.info/?p=135 Yet Another Related Posts Plugin.]]> Yesterday I’ve read a post on The Developer’s Day blog. Then tested this thing in Yii. There is a jQuery in my application, so I created test controller and test action, that only outputted “Hello, world” and called it from the firebug console like this (I have url rewrite set up):

  1. $.ajax({url:‘/test/test.html’});
  2. $.ajax({url:‘/test/test.html’});
  3. $.ajax({url:‘/test/test.html’});
  4. $.ajax({url:‘/test/test.html’});
  5. $.ajax({url:‘/test/test.html’});
  6. $.ajax({url:‘/test/test.html’});
  7.  

Yes, really it matters. Each call starts after the previous one ends and it takes long to finish them all. But each web application handles the concurrent requests by nature. So I started a forum thread to clarify this. And sebas recommended me to use CDbHttpSession instead of standard one. Yes, I knew, that storing sessions in DB speeds up the application, but I thought that is because of more effective search. And it turns out, that besides more effective search it gives one more DB benefit – concurrent access to the session storage. That’s actually the reason #1 why we usually use databases instead of files :)
So here is a quick trick, that solves the problem:

  1.  
  2. // application components
  3.         ‘components’=>array(
  4. ……….
  5.                 ‘session’ => array(
  6.                         ‘class’ => ‘system.web.CDbHttpSession’,
  7.                         ‘connectionID’ => ‘db’,
  8.                 ),
  9.         ),
  10.  

It creates a new table YiiSession and uses it for session data. This time the JS test above runs much faster. Try it yourself!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/03/05/speeding-up-yii-or-why-should-you-use-db-sessions/feed/ 12 Handling Application Parameters in Yii – Using the Database http://programmersnotes.info/2009/03/04/handling-application-parameters-in-yii-using-the-database/ http://programmersnotes.info/2009/03/04/handling-application-parameters-in-yii-using-the-database/#comments Wed, 04 Mar 2009 09:48:22 +0000 Konstantin Mirin http://programmersnotes.info/?p=129 Yet Another Related Posts Plugin.]]> The problem

Nearly every large application has some configuration parameters – site admin email, cache time for different blocks, number of latest news, number of items per search or catalogue page etc.

Standard Way

Yii comes with the built-in mechanism for this. You can create a file with an associative array of all your variables, like this:

  1. <?php
  2. return array(
  3.                 ‘brandNew’ => 7, // number of days to consider new
  4.                 ‘onSale’ => 7, // number of days to consider on sale
  5.                 ‘favourites’ => array(
  6.                                 ‘max’ => 9//max number of last favourites to show
  7.                                 ),
  8.                 ‘search’ => array(
  9.                                 ‘itemsPerPage’ => 20,
  10.                                 ‘tagsInFilter’ => 30,
  11.                                 ),
  12.                 ‘product’ => array(
  13.                                 ‘maxAlts’ => 5, //max alternatives
  14.                                 ),
  15.                 ‘cacheTime’ => array(
  16.                                 ‘assets’ => 1800,
  17.                                 ),
  18.         );
  19. ?>

Then put it somewhere, for example in /protected/config/params.php and configure your application to use it:

  1. return array(
  2.         // application components
  3.         ‘components’=>array(
  4. //components go here
  5.         ),
  6.  
  7.         // application-level parameters that can be accessed
  8.         // using Yii::app()->params['paramName']
  9.         ‘params’=>include(dirname(__FILE__).‘/params.php’),//<– here is our file
  10. );

Usage is simple, just refer to it as to array:

  1. echo Yii::app()->params[‘siteAdmin’];

This is quite convenient when you just want to save some constants. End-user will not be able to change this from admin in any way. Sure, you can create the interface for editing this config and update the file, but much more flexible way is to store the configuration in DB.

The Database Way

To make the life easier, I’ve created an extension, that can be used as an application component. Extension can be downloaded from the Yii extension page. Here I’ll give an overview of it’s work.

First thing to note, it implements IApplicationComponent, so it should implement init() method where initialization is performed. In this method we check if there are any parameters to load. If so – we try to load them. If there is an exception, it means, that no table is present, so we create it and initialize the parameters that should be preloaded with null values

  1. /**
  2.  * Initializes component, creates table if it doesn’t exist and populates preloaded attributes
  3.  * @throws CException Throws exception if table does not exist and auto creation is set to false
  4.  */
  5. public function init()
  6. {
  7.         $this->_init = true;
  8.         $db = $this->getDbConnection();
  9.         $this->preload = $this->preprocessParams($this->preload);
  10.         if (!empty($this->preload) || $this->autoLoad)
  11.         {
  12.                 $sql = ‘SELECT name, value FROM ‘.$this->paramsTableName;
  13.                 if (sizeof($this->preload))
  14.                         $sql .= ‘ WHERE name IN (\’‘.implode(‘\’,\’‘, $this->preload).‘\’)’;
  15.                 $cmd = $db->createCommand($sql);
  16.                 try
  17.                 {
  18.                         $reader = $cmd->query();
  19.                         foreach ($reader as $row)
  20.                         {
  21.                                 $this->add($row[‘name’], $row[‘value’]);
  22.                         }
  23.                 }
  24.                 catch (CException $e)
  25.                 {
  26.                         //table is not present
  27.                         $createTable = true;
  28.                         //if there is no table, then attributes are empty.
  29.                         for ($i = 0, $s = sizeof($this->preload); $i < $s; $i++)
  30.                         {
  31.                                 $this->add($this->preload[$i], null);
  32.                         }
  33.                 }
  34.         }
  35.         else
  36.         {
  37.                 //check if table exist
  38.                 if ($db->createCommand(‘SHOW TABLES LIKE \’‘.$this->paramsTableName.‘\’‘)->query()->rowCount <= 0)
  39.                         $createTable = true;
  40.         }
  41.         if ($createTable === true)
  42.         {
  43.                 if($this->autoCreateParamsTable)
  44.                         $this->createParamsTable($db,$this->paramsTableName);
  45.                 else
  46.                         throw new CException(Yii::t(‘xparam’,‘Params table "{tableName}" does not exist.’,
  47.                                 array(‘{tableName}’=>$this->paramsTableName)));
  48.         }
  49. }

If parameter is requested, in the __get() method we reuse the parent’s (CAttributesCollection) contains() method, which checks if we have this parameter in the collection already. If so – we simply return it. If not – we call the loadParam() method, it loads if from DB and adds to the component collection. If there is no attribute, it throws an exception. We catch it in the __get() and call the parent::__get(). Here is the code:

  1. /**
  2.  * Returns a property value or an event handler list by property or event name.
  3.  * This method overrides the parent implementation by returning
  4.  * a parameter value if the it exist in the collection or loading it from DB
  5.  * if not.
  6.  *
  7.  * @param string the property name or the event name
  8.  * @return mixed the property value or the event handler list
  9.  * @throws CException if the property/event is not defined.
  10.  */
  11. public function __get($name)
  12. {
  13.         if($this->contains($name))
  14.                 return $this->itemAt($name);
  15.         else
  16.         {
  17.                 try
  18.                 {
  19.                         return $this->loadParam($name);
  20.                 }
  21.                 catch (CException $e)
  22.                 {
  23.                         return parent::__get($name);
  24.                 }
  25.         }
  26. }
  27.  
  28. /**
  29.  * Returns parameter from DB or cache if it was requested earlier
  30.  *
  31.  * @param string $name
  32.  */
  33. protected function loadParam($name)
  34. {
  35.         if ($this->caseSensitive)
  36.         $name = strtolower($name);
  37.         $db = $this->getDbConnection();
  38.         $res = $db->createCommand(‘SELECT value FROM ‘.$this->paramsTableName.‘ WHERE name=\’‘.$name.‘\’‘)->query();
  39.         if ($res->rowCount == 1)
  40.         {
  41.                 $row = $res->read();
  42.                 $this->add($name,$row[‘value’]);
  43.                 return $row[‘value’];
  44.         }
  45.         else
  46.         {
  47.                 throw new CException(Yii::t(‘xparam’,‘XDbParam->{name} does not exist!’,
  48.                         array(‘{name}’=>$name)));
  49.         }
  50. }

Setter method simply updates the collection. We check if the parameter is already present in DB. If so – we update it. If not – we create it. And, if everything was OK, we add the parameter to the internal collection ($this->add($name,$value)):

  1. /**
  2.  * Sets value of a component property.
  3.  * This method overrides the parent implementation by adding a new param
  4.  * value to the collection and updating the DB.
  5.  * @param string the property name or event name
  6.  * @param mixed the property value or event handler
  7.  * @throws CException If the property is not defined or read-only.
  8.  */
  9. public function __set($name,$value)
  10. {
  11.         if ($this->caseSensitive)
  12.                 $name = strtolower($name);
  13.         $db = $this->getDbConnection();
  14.         if ($db->createCommand(‘SELECT name FROM ‘.$this->paramsTableName.‘ WHERE name=\’‘.$name.‘\’‘)->query()->rowCount >= 1)
  15.         {
  16.                 $sql = ‘UPDATE `’.$this->paramsTableName.‘` SET `value`=:value WHERE `name`=:name’;
  17.                 $cmd = $db->createCommand($sql);
  18.                 $cmd->bindValue(‘:value’, $value, PDO::PARAM_LOB);
  19.                 $cmd->bindValue(‘:name’, $name, PDO::PARAM_STR);
  20.         }
  21.         else
  22.         {
  23.                 $sql = ‘INSERT INTO `’.$this->paramsTableName.‘`(name, value) VALUES(:name, :value)’;
  24.                 $cmd = $db->createCommand($sql);
  25.                 $cmd->bindValue(‘:name’, $name, PDO::PARAM_STR);
  26.                 $cmd->bindValue(‘:value’, $value, PDO::PARAM_LOB);
  27.         }
  28.         $cmd->execute();
  29.         $this->add($name,$value);
  30. }

When configuring the extension or using load() or purge() method, we can specify parameters either as array or as comma-separated string. This is achieved by the preprocessing method, which also converts parameter names to lowercase and they are case insensitive (if this is required by the caseSensitive property, it must be set to true). Here is it:

  1. /**
  2.  * Sets value of a component property.
  3.  * This method overrides the parent implementation by adding a new param
  4.  * value to the collection and updating the DB.
  5.  * @param string the property name or event name
  6.  * @param mixed the property value or event handler
  7.  * @throws CException If the property is not defined or read-only.
  8.  */
  9. public function __set($name,$value)
  10. {
  11.         if ($this->caseSensitive)
  12.                 $name = strtolower($name);
  13.         $db = $this->getDbConnection();
  14.         if ($db->createCommand(‘SELECT name FROM ‘.$this->paramsTableName.‘ WHERE name=\’‘.$name.‘\’‘)->query()->rowCount >= 1)
  15.         {
  16.                 $sql = ‘UPDATE `’.$this->paramsTableName.‘` SET `value`=:value WHERE `name`=:name’;
  17.                 $cmd = $db->createCommand($sql);
  18.                 $cmd->bindValue(‘:value’, $value, PDO::PARAM_LOB);
  19.                 $cmd->bindValue(‘:name’, $name, PDO::PARAM_STR);
  20.         }
  21.         else
  22.         {
  23.                 $sql = ‘INSERT INTO `’.$this->paramsTableName.‘`(name, value) VALUES(:name, :value)’;
  24.                 $cmd = $db->createCommand($sql);
  25.                 $cmd->bindValue(‘:name’, $name, PDO::PARAM_STR);
  26.                 $cmd->bindValue(‘:value’, $value, PDO::PARAM_LOB);
  27.         }
  28.         $cmd->execute();
  29.         $this->add($name,$value);
  30. }

And, finally, load and purge methods:

  1.  
  2. /**
  3.  * Loads several params from DB at once. If nothing specified, all parameters
  4.  * are loaded. This saves queries if you plan to use all that params in the
  5.  * next lines.
  6.  *
  7.  * @param array|string $params
  8.  * @throws CException
  9.  */
  10. public function load($params = array())
  11. {
  12.         $params = $this->preprocessParams($params);
  13.         $db = $this->getDbConnection();
  14.         $sql = ‘SELECT name, value FROM ‘.$this->paramsTableName;
  15.         if (sizeof($params) > 0)
  16.         {
  17.                 $sql .= ‘WHERE name IN (\’‘.implode(‘\’,\’‘, $params).‘\’)’;
  18.         }
  19.         $cmd = $db->createCommand($sql);
  20.         $reader = $cmd->query();
  21.         foreach ($reader as $row)
  22.         {
  23.                 $this->add($row[‘name’], $row[‘value’]);
  24.                 $loaded++;
  25.         }
  26.         if ($loaded < sizeof($params))//this will not be thrown if loading all attributes
  27.         {
  28.                 throw new CException(Yii::t(‘xparam’,‘Some of the requested params do not exist!’));
  29.         }
  30. }
  31.  
  32. /**
  33.  * Deletes specified params (or all params if none specified) from the parameters table
  34.  *
  35.  * @param array|string $params Comma-separated list of params or array of param names
  36.  */
  37. public function purge($params = array())
  38. {
  39.         $sql = ‘DELETE FROM ‘.$this->paramsTableName;
  40.         if (sizeof($params) > 0)
  41.         {
  42.                 $params = $this->preprocessParams($params);
  43.                 $sql .= ‘ WHERE name IN (\’‘.implode(‘\’,\’‘, $params).‘\’)’;
  44.         }
  45.         $db = $this->getDbConnection();
  46.         $db->createCommand($sql)->execute();
  47.         if (sizeof($params) > 0)
  48.         {
  49.                 for ($i = 0, $s = sizeof($params); $i < $s; $i++)
  50.                 {
  51.                         $this->remove($params[$i]);
  52.                 }
  53.         }
  54.         else
  55.                 $this->clear();
  56. }

Usage Instructions

Using the extension is simple. Like any extension, you should unzip it to the /protected/extensions/ folder and then set it up as an application component. To do this, you should add it with any name to the components array in the application configuration:

  1. ‘par’=>array(
  2.     ‘class’ => ‘application.extensions.dbparam.XDbParam’,
  3.     ‘connectionID’ => ‘db’,//id of the connection component, just the same as with CDbCache
  4. //  ’preload’ => ‘test,test2′, //comma-separated string or array of params to be loaded anyway. Other params are loaded only when requested.
  5. //  ’autoLoad’ => true, //setting to true enables loading of all attributes present in DB in the beginning
  6. //  ’caseSensitive’ => true, //setting to true makes all parameters case sensitive
  7.     ),

‘par’ is the ID of the component, you will refer to it in any place of the application by it’s ID:

  1.  
  2. Yii::app()->par->test = ’1234′;//set parameter. If it is not present, it will be created
  3. echo Yii::app()->par->test;//output parameter. If it is not present, exception is thrown
  4.  
  5. //load several parameters in one query. Useful if you;re going to use them in the next lines
  6. Yii::app()->par->load(array(‘test’, ‘test2′));
  7. //OR
  8. Yii::app()->par->load(‘test,test2′);
  9. //OR
  10. Yii::app()->par->load();//load all parameters
  11.  
  12. //delete the specified parameters or all of them if none specified
  13. Yii::app()->par->purge(‘test,test2′);//delete test and test 2
  14. //OR
  15. Yii::app()->par->purge(array(‘test’, ‘test3′));//delete test and test 3
  16. //OR
  17. Yii::app()->par->purge();//delete ALL parameters!

That’s it! Enjoy!

UPDATE: Updated on March, 5 2009 according to the Jonah’s comments an this discussion. Version 3 is planned with great functionality :)

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/03/04/handling-application-parameters-in-yii-using-the-database/feed/ 13 Registering group of JS files in Yii http://programmersnotes.info/2009/03/03/registering-group-of-js-files-in-yii/ http://programmersnotes.info/2009/03/03/registering-group-of-js-files-in-yii/#comments Tue, 03 Mar 2009 13:27:32 +0000 Konstantin Mirin http://programmersnotes.info/?p=123 Yet Another Related Posts Plugin.]]> The problem…

In Yii you can have all your JS and CSS files together with your PHP. For example, you have some view in /views/user/create.php and you can have your css & js in /views/user/assets/css/ and /views/user/assets/js/. That’s quite convenient when you reuse your views/controllers/components from application to application. But these directories are not accessible from the web, so there is publishing mechanism there. You should do:

  1. $files = array(‘Box’, ‘CommentObject’, ‘FavouriteStore’, ‘FavouriteObject’, ‘FriendObject’, ‘RateObject’, ‘init’, ‘rateComment’, ‘TagObject’);
  2. foreach ($files as $file)
  3. {
  4.         $path = CHtml::asset(dirname(__FILE__).‘/product/assets/’.$file.‘.js’);
  5.         $cs->registerScriptFile($path);
  6. }

Yes, it’s simple, but why should I do these routines all the time!

…and solution

Luckily, in Yii you can change everything. When we write

  1. Yii::app()->clientScript

we access the clientScript application component. This is default component, it is created from the class CClientScript. So we can extend this CClientScript and replace it in the application configuration file with our class. OK, here is /protected/components/AdvancedClientScript.php file:

  1. <?php
  2. class AdvancedClientScript extends CClientScript
  3. {
  4.         /**
  5.          * Registeres all files within the directory as JS
  6.          *
  7.          * @param string $path Path to the directory with files. With trailing ‘/’
  8.          * @param bool $makeAssets Wether or not publish file as asset. If the directory is under /protected/, you should set true (default)
  9.          * @param int $position Position where to put the file. @see CClientScript::registerScriptFile
  10.          */
  11.         public function registerScriptDirectory($path, $makeAssets = true, $position = self::POS_HEAD)
  12.         {
  13.                 $id = md5(serialize($path.$makeAssets.$position));
  14.                 $data = Yii::app()->cache->get($id);
  15.                 if($data === false)
  16.                 {
  17.                         $di = new DirectoryIterator($path);
  18.                         foreach ($di as $file)
  19.                         {
  20.                                 if (!$file->isDot() && !$file->isDir())
  21.                                 {
  22.                                         $data[] = $makeAssets ? CHtml::asset($path.$file->getFileName()) : $path.$file->getFileName();
  23.                                 }
  24.                         }
  25.                         Yii::app()->cache->set($id, $data, 1800);
  26.                 }
  27.                 foreach ($data as $file)
  28.                 {
  29.                         $this->registerScriptFile($file, $position);
  30.                 }
  31.         }
  32.  
  33.         /**
  34.          * Registeres all files within the directory as CSS
  35.          *
  36.          * @param string $path Path to the directory with files. With trailing ‘/’
  37.          * @param bool $makeAssets Wether or not publish file as asset. If the directory is under /protected/, you should set true (default)
  38.          * @param int $position Position where to put the file. @see CClientScript::registerCssFile
  39.          */
  40.         public function registerCSSDirectory($path, $makeAssets = true, $position = self::POS_HEAD)
  41.         {
  42.                 $id = md5(serialize($path.$makeAssets.$position));
  43.                 $data = Yii::app()->cache->get($id);
  44.                 if($data === false)
  45.                 {
  46.                         $di = new DirectoryIterator($path);
  47.                         foreach ($di as $file)
  48.                         {
  49.                                 if (!$file->isDot() && !$file->isDir() && (stristr($file->getFileName(), ‘.css’) !== false))
  50.                                 {
  51.                                         $data[] = $makeAssets ? CHtml::asset($path.$file->getFileName()) : $path.$file->getFileName();
  52.                                 }
  53.                         }
  54.                         Yii::app()->cache->set($id, $data, 1800);
  55.                 }
  56.                 foreach ($data as $file)
  57.                 {
  58.                         $this->registerCssFile($file, $position);
  59.                 }
  60.         }
  61.  
  62.         /**
  63.          * Registeres all files inthe passed array as JS
  64.          *
  65.          * @param array $files Array of file names
  66.          * @param string $basePath Path to the container directory
  67.          * @param bool $makeAssets Wether or not publish file as asset. If the directory is under /protected/, you should set true (default)
  68.          * @param int $position Position where to put the file. @see CClientScript::registerScriptFile
  69.          */
  70.         public function registerScriptFiles($files = array(), $basePath = -1, $makeAssets = true, $position = self::POS_HEAD)
  71.         {
  72.                 if (sizeof($files) <= 0) return;
  73.                 if ($basePath === -1) $basePath = dirname(__FILE__);
  74.                 $id = md5(serialize(implode($files).$basePath.$makeAssets.$position));
  75.                 $data = Yii::app()->cache->get($id);
  76.                 if($data === false)
  77.                 {
  78.                         for ($i = 0, $s = sizeof($files); $i < $s; $i++)
  79.                         {
  80.                                 $data[] = $makeAssets ? CHtml::asset($path.$files[$i]) : $path.$files[$i];
  81.                         }
  82.                         Yii::app()->cache->set($id, $data, 1800);
  83.                 }
  84.                 foreach ($data as $file)
  85.                 {
  86.                         $this->registerScriptFile($file, $position);
  87.                 }
  88.         }
  89.  
  90.         /**
  91.          * Registeres all files inthe passed array as CSS
  92.          *
  93.          * @param array $files Array of file names
  94.          * @param string $basePath Path to the container directory
  95.          * @param bool $makeAssets Wether or not publish file as asset. If the directory is under /protected/, you should set true (default)
  96.          * @param int $position Position where to put the file. @see CClientScript::registerCssFile
  97.          */
  98.         public function registerCssFiles($files = array(), $basePath = -1, $makeAssets = true, $position = self::POS_HEAD)
  99.         {
  100.                 if (sizeof($files) <= 0) return;
  101.                 if ($basePath === -1) $basePath = dirname(__FILE__);
  102.                 $id = md5(serialize(implode($files).$basePath.$makeAssets.$position));
  103.                 $data = Yii::app()->cache->get($id);
  104.                 if($data === false)
  105.                 {
  106.                         for ($i = 0, $s = sizeof($files); $i < $s; $i++)
  107.                         {
  108.                                 $data[] = $makeAssets ? CHtml::asset($path.$files[$i]) : $path.$files[$i];
  109.                         }
  110.                         Yii::app()->cache->set($id, $data, 1800);
  111.                 }
  112.                 foreach ($data as $file)
  113.                 {
  114.                         $this->registerScriptFile($file, $position);
  115.                 }
  116.         }
  117. }
  118. ?>

I’ve added 4 methods there:

  • registerScriptDirectory – registers all files in the specified directory as JS files (no check is performed)
  • registerCssDirectory – registers all files in the specified directory as CSS files
  • registerScriptFiles – same as first, but deals with an array of files
  • registerCssFiles – same as second, but deals with an array of files

And no we’ll replace default component with ours. My config is in the /protected/config/main.php:

  1. // application components
  2. ‘components’=>array(
  3.         ‘clientScript’ => array(
  4.                 ‘class’ => ‘application.components.AdvancedClientScript’ //here we specified the path to our component
  5.         ),
  6.         //other components follow
  7.         ……
  8. ),

In order to make it work, you should enable caching. This is quite simple. I’ll show, how to use caching in DB:

  1. You should have your DB connection configured. See details here.
  2. Add new component to the application config:
    1. …
    2. ‘cache’ => array(
    3.         ‘class’ => ‘system.caching.CDbCache’,
    4.         ‘connectionID’ => ‘db’//this is id of the DB module. If you didn’t change anything, it should have name "id"
    5. ),
    6. …

We’re done! Now we can publish the files simply calling:

  1. Yii::app()->clientScript->registerScriptDirectory(dirname(__FILE__).‘/user/assets/ui/’);

Note, that the code I wrote uses caching and when I call

  1. Yii::app()->cache->set($id, $data, 1800);

I cache this for 30 mins (1800 seconds). However, it is much more flexible to have either constant or file with parameters.

UPDATE: Post updated after discussion in the Yii forum according to the Maxximus advice.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/03/03/registering-group-of-js-files-in-yii/feed/ 3 SEO Titles in wordpress http://programmersnotes.info/2009/02/26/seo-titles-in-wordpress/ http://programmersnotes.info/2009/02/26/seo-titles-in-wordpress/#comments Thu, 26 Feb 2009 16:02:46 +0000 Konstantin Mirin http://programmersnotes.info/?p=28 Yet Another Related Posts Plugin.]]> One of the ways to make your blog more search engines friendly is to take cate of your <title> tag. That’s what’s showin in the browser’s window title. By default, wordpress shows Blog name on the homepage and [BolgName] > Blog archive > [PostTitle] on the posts page (the most important to us). Best variant is to show only post title or ctegory title in the <title> tag, without any blog name. Best variant to do this is to install All in one SEO Pack and configure everything there. However, if you can’t/don’t want do this, you can do one simple change.

  1. Go to Design->Theme Editor if you have version less, than 2.7 or Appearance -> Editor if you have 2.7.
  2. Choose file header.php
  3. Find line <title>…..
  4. Replace this line with these two:
    1.  
    2. <?php if (($t = wp_title(" ", false)) == ”){$t = get_bloginfo(‘name’);} ?>
    3. <title><?php echo $t; ?></title>
    4.                
  5. You’re done!

Further reading – wordpress reference

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

]]> http://programmersnotes.info/2009/02/26/seo-titles-in-wordpress/feed/ 6