Using Redis in PHP for handling background Jobs

Introduction On Redis

Redis is an open source, BSD licensed advanced key-value store. It is often referred to as a data structure server since the keys can contain strings, hashes, lists, sets and sorted sets.
Redis is written in C.

Refer to the below link which would help us to understand how to install Redis. Also, it gives the basic understanding of how to use Redis.how to install and use Redis

Before you start using Redis in your PHP programs, you need to make sure that you have Redis PHP driver and PHP set up on the machine. You can check PHP tutorial for PHP installation on your machine.

Installation

Now, let us check how to set up Redis PHP driver.
You need to download the phpredis from github repository phpredis.
Once you’ve downloaded it, extract the files to phpredis directory.

On Ubuntu, install the following extension.

cd phpredis 
sudo phpize 
sudo ./configure 
sudo make 
sudo make install

Now, copy and paste the content of “modules” folder to the PHP extension directory and add the following lines in php.ini.

extension = redis.so

Now, your Redis PHP installation is complete

PHP Resque Worker (and Enqueue)

Resque is a Redis-backed library for creating background jobs, placing those jobs on one or more queues, and processing them later.
For more information: Php-resque

Jobs

Queueing Jobs

Jobs are queued as follows:





$job_type      = 'comment'; // Specify the type to job in my case its for adding a comment
$queue_type = 'notification';


$args = array(                                                                                                                                                              'user_id' => $login_user_id, // Adding Commenter's User-Id                        'post_id' => $post_id, // Specify for which Post the comment is made                        'Comment_text' => $comment, // Adding the commented text                        'push_notify' => true, // Should be notified or not                        'type' => $job_type                                                                                                                         ); 


$redis = new redisQueue(); $addQueue = $redis->addToQueue($queue_type, $job_type, $args);

NOTE: $addQueue returns the Job IDs and store this ID's for future reference

Defining Jobs

Each job should be in its own class and include a perform method.

class notification { 
     public function perform() { 
         // Work work work 
         echo $this->args['user_id']; 
         echo $this->args['post_id']; 
         echo $this->args['comment_text']; 
         echo $this->args['push_notify']; 
         echo $this->args['type']; 
    } 
}

Dequeueing Jobs

This method can be used to conveniently remove a job from a queue.

// Removes job class 'My_Job' of queue 'default' Resque::dequeue('notification','comment');


//Removes job class 'My_Job' with Job ID
'087df5819a790ac666c9608e2234b21e' of queue 'default'

Resuque::dequeue('default',
['My_Job' => '087df5819a790ac666c9608e2234b21e']);

//Removes all jobs of queue 'nano-notification'Resque::dequeue('nano-notification');

Handling the Failed Jobs

To track the status of a job, pass true as the fourth argument to Resque:: enqueue. A token used for tracking the job status will be returned:

$token = Resque:: enqueue('notification', 'comment', $args, true);
echo $token;

To fetch the status of a job:

$status = new Resque_Job_Status($token);
echo $status->get(); // Outputs the status

Job statuses are defined as constants in the Resque_Job_Status class.

Valid statuses include:

Resque_Job_Status::STATUS_WAITING - Job is still queued
Resque_Job_Status::STATUS_RUNNING - Job is currently running
Resque_Job_Status::STATUS_FAILED - Job has failed
Resque_Job_Status::STATUS_COMPLETE - Job is complete

false - Failed to fetch the status - is the token valid?

Managing Workers

Before understanding the workers. We need to understand some terminologies used.


QUEUE: The name of the queues to poll

INTERVAL: The polling interval (waiting time in seconds between each polling). Default to 5 seconds.

COUNT: Number of workers to create. All workers will have the same properties. Will always create one worker by default.

REDIS_BACKEND: Address of your Redis server, formatted as hostname: port.

e.g: 127.0.0.1:6379, or localhost:6379. Default to localhost:6379 

VERBOSE: 1 to enable verbose, will print basic debugging information

VVERBOSE: 1 to enable advanced verbose, will print detailed debugging informations.

Starting a worker

Workers can be created by running the file resque.php, located inside the php-resque folder.
In your terminal, navigate to your php-resque folder
cd /app/redis

Then run:
INTERVAL=10 QUEUE=nano-notification VERBOSE=1 php resque.php

The terminal will output:
*** Starting worker nano:3771:nano-notification
*** Registered signals
*** Pruning dead worker: nano:9225:nano-notification
*** Pruning dead worker: nano:9263:nano-notification
*** Pruning dead worker: nano:9203:nano-notification
*** Checking nano-notification
*** Sleeping for 10
*** Checking nano-notification
*** Sleeping for 10

... etc ...

The worker will automatically be named nano: 84499: notification.

It follows the conventions hostname:process-id:queue-names.

Daemon-ify the workers

It’ll also be nice to thrown in a nohup, enabling the command to keep running even if the user logged out.
Let’s pipe the output to a file before daemonizing:
nohup sudo -u www-data QUEUE=notification php resque.php >> /var/logs/working_logs.log 2>&1 &With this, all the standard and error output will be written to working_logs.log. To monitor the file content:
tail -F /var/logs/working_logs.log Ensure workers were created successfully
Since we are piping the output. A way to verify that is to monitor your log file.
You can also check your system process. It’ll list all your active workers:
ps u | grep resque.php

Output looks like:
your-user 86681 0.0 0.1 2470332 4712 s002 S 2:03AM 0:00.01 php ./resque.php
your-user 86680 0.0 0.1 2470332 4712 s002 S 2:03AM 0:00.01 php ./resque.php
Second column is the worker PID.
It’s a good way to check if some of your workers died at some time.

Stopping a worker

sudo kill - 9 86681sudo kill - 9 86681

Configuration or Issues

a) If your Redis is located at a different address:
Go to nano/app/config/redis.config.php file make the following changes


'port_config' => array(

    'redis_backend' => 'XX.XX.XX.XXX:6379', // External Ip : Port
),

b) If there are any changes on the files under app/redis folder

i) Stopping the worker

ii) Daemon-ify the workers
iii) Ensure worker is created again

c) open redis port for remote connections

i) set the bind option to allow remote access to the Redis server

Change from bind 127.0.0.1 to bind 0.0.0.0

ii) First I'd check to verify it is listening on the IPs you expect it to be:
netstat -nlpt | grep 6379

iii) After establishing it is listening where you expect it to, from a remote node which should have access try
redis-cli -h REMOTE.HOST ping // You must get PONG in response.

karthik Chakravarthi

Web Development Everywhere