By default all created jobs are queued to run asynchronously (on New Laravel applications). Laravel job classes created using artisan command has ShouldQueue interface.
Things to remembers
- Jobs are created to run on background with a queue service.
- handle() method of the job class is executed when running a job.
- Jobs are used to handle time consuming tasks to improve performance.
Table of Content
- Creating a job
- onQueue
- onConnection
- Tries (max retry attempts)
- Retry Job Until timeout – for failed jobs
- Stop after max exception count – for failed jobs
- Set Job Process duration Timeout
Basics of creating and running a queue
You can use artisan command to create a queueable job easily. You need to have a background service to manage queue service. If not queued jobs may not run. For production you can use systemctl to run queue service on the background. In this article I will show how to run queue service in the local development environment.
how to create a job class?
php artisan make:job MyJobClassNamehow to run queue service in local development?
php artisan queue:listenExample: Create a simple queueable job and dispatch it.
Below shows step by step of using jobs and queues. Before checking the code try to implement this example and check it with the given code. For every example make sure run web server and queue service
- Run Laravel web application using your preferred method (Apache server, XAMPP, artisan Serve, etc.).
- Then run a queue service using artisan queue:listen command.
First create a sample job class using below command.
php artisan make:job ProcessImageDataNext open created job file and add below code in the handle() method to check whether job is properly executing.
<?php
namespace App\Jobs;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;
class ProcessImageData implements ShouldQueue
{
use Queueable;
/**
* Execute the job.
*/
public function handle(): void
{
Log::info("Running Process Image Data Job on the queue");
}
}Now dispatch created job using the routes file web.php.
<?php
use App\Jobs\ProcessImageData;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Route;
Route::get('/', function () {
Log::info('Received a request on the root domain');
ProcessImageData::dispatch();
});Check laravel logs file after hitting root domain of your Laravel web application. Make sure to run queue service.
[2024-12-08 06:39:39] local.INFO: Received a request on the root domain
[2024-12-08 06:39:53] local.INFO: Running Process Image Data Job on the queue onQueue
Queue Connection (redis, database, sqs) and queue name (myqueue2, secondaryqueue, etc) are two different things. OnQueue is used to set queue name. You can use any preferred name as the queue name.
Example: Change the queue of a Laravel job
Here we are using the Laravel job class created above. Here we can:
- set name of the queue before dispatching
- Or set the name of the queue within the job class.
Option 1: using onQueue() method before dispatching
Change web.php file like below. Add onQueue() method on the ProcessImageData job class the change the queue.
<?php
use App\Jobs\ProcessImageData;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Route;
Route::get('/', function () {
//Log request
Log::info("Received a get request on root route");
ProcessImageData::dispatch()->onQueue('myqueue1');
});Option 2: Using constructor method of the job.
....
class ProcessImageData implements ShouldQueue
{
use Queueable;
/**
* Create a new job instance.
*/
public function __construct()
{
$this->onQueue('myqueue1');
}
....
}
Change the name of the queue using option 1 or option 2. Then run two queue workers as “myqueue1” and “myqueue2” using below artisan command. Use two terminal windows to run below command.
php artisan queue:work --queue=myqueue1php artisan queue:work --queue=myqueue2Run this Laravel web application using the artisan command and check Terminal window one and two. You will see above job uses Terminal window one which has the myqueue1 queue to execute the job.
PS D:\laravel> php artisan queue:work --queue=myqueue1
11:02:19 App\Jobs\ProcessImageData ...................... RUNNING
11:02:19 App\Jobs\ProcessImageData ...................... 12.25ms DONEPS D:\laravel> php artisan queue:work --queue=myqueue2onConnection
onConnection method is used to select the queue connection if your Laravel application has multiple queue connections like Database, sqs, redis, etc. The name mentioned here should match with the name on the queue config file.
Example: Using onConnection()
Option 1: Using onConnection() method on dispatch() method
Use onConnection() method on the dispatch() method to change the queue connection.
Route::get('/', function () {
//Log request
Log::info("Received a get request on root route");
ProcessImageData::dispatch()->onConnection('database');
});Option 2: Using onConnection on job constructor method
...
class ProcessImageData implements ShouldQueue
{
use Queueable;
/**
* Create a new job instance.
*/
public function __construct()
{
$this->onConnection('database');
}
...
}
Tries
Tries attributes or max attempts of a job decides how many times that job should be tried if it failed.
Set max attempts to a queue worker
- You can set max attempts to a queue worker.
- This will set maximum attempts for all the jobs executed in that queue worker.
- You can override this value using the tries variable or tries() method within the job class,
Example: Using tries attribute on queue command.
Open ProcessImagaData job class and throw exception on the handle() method to retry that job again and again.
<?php
namespace App\Jobs;
use Exception;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;
class ProcessImageData implements ShouldQueue
{
use Queueable;
public function __construct(){}
/**
* Execute the job.
*/
public function handle(): void
{
Log::info('Running Process Image Data Job');
throw new Exception('job failed');
}
}
Now run this web application. Then run a queue worker with 4 retry attempts.
php artisan queue:work --tries=4Check Laravel logs. You will that this failed job has retried 4 times only. Because we use the value 4 on the tries attribute of the queue worker command.
[2024-12-29 11:22:03] local.INFO: Received a get request on root route
[2024-12-29 11:22:04] local.INFO: Running Process Image Data Job
[2024-12-29 11:22:04] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:22:04] local.INFO: Running Process Image Data Job
[2024-12-29 11:22:04] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:22:04] local.INFO: Running Process Image Data Job
[2024-12-29 11:22:04] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:22:04] local.INFO: Running Process Image Data Job
[2024-12-29 11:22:04] local.ERROR: job failed {"exception":"[object] "Set Max attempts to a Laravel Job Class
- Override public $tries variable within the job class. For below examples our queue worker will be below artisan command.
- Using tries() method that return integer.
php artisan queue:work --tries=4Example: set max attempts lower than the attempts of the queue worker
...
class ProcessImageData implements ShouldQueue
{
use Queueable;
public $tries=2;
....
}Logs:
Executed only two times respecting the attempts value defined within the job class.
[2024-12-29 11:28:04] local.INFO: Received a get request on root route
[2024-12-29 11:28:06] local.INFO: Running Process Image Data Job
[2024-12-29 11:28:06] local.ERROR: job failed {"exception":"[object]"
[2024-12-29 11:28:06] local.INFO: Running Process Image Data Job
[2024-12-29 11:28:06] local.ERROR: job failed {"exception":"[object]" Example: set max attempts higher than the attempts of the queue worker
....
class ProcessImageData implements ShouldQueue
{
use Queueable;
public $tries=5;
...
}Logs:
You can override max attempt mentioned within the queue worker by defining the number of retries within the job class. Here job executed 5 times which is more than the defined attempts by the queue worker.
[2024-12-29 11:31:08] local.INFO: Received a get request on root route
[2024-12-29 11:31:11] local.INFO: Running Process Image Data Job
[2024-12-29 11:31:11] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:31:11] local.INFO: Running Process Image Data Job
[2024-12-29 11:31:11] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:31:11] local.INFO: Running Process Image Data Job
[2024-12-29 11:31:11] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:31:11] local.INFO: Running Process Image Data Job
[2024-12-29 11:31:11] local.ERROR: job failed {"exception":"[object] "
[2024-12-29 11:31:11] local.INFO: Running Process Image Data Job
[2024-12-29 11:31:11] local.ERROR: job failed {"exception":"[object] "Retry job until Time Out
If a job failed, that job will be tried again. Instead of defining number of tries you can set how long that failed job should be re-tried. Here you can use RetryUntil() method.
Example: Retry failing job for 30 seconds only
Open ProcessImageData job class and add below code.
- Add 5 seconds sleep to job class to simulate time consuming task.
- Define retryUntil() method and return 30 seconds as the attempt time out value.
namespace App\Jobs;
use DateTime;
use Exception;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;
class ProcessImageData implements ShouldQueue
{
use Queueable;
public function __construct(){}
public function handle(): void
{
Log::info('Running Process Image Data Job');
sleep(5);
throw new Exception('job failed');
}
public function retryUntil():DateTime
{
return now()->addSeconds(30);
}
}Run your Laravel application and a queue worker.
php artisan queue:workLogs:
Job start to process at 07:19:17 and end retrying that job at 07:19:47 which is 30 seconds after the job start to process.
[2024-12-30 07:19:14] local.INFO: Received a get request on root route
[2024-12-30 07:19:17] local.INFO: Running Process Image Data Job
[2024-12-30 07:19:22] local.ERROR: job failed {"exception":"[object]"
[2024-12-30 07:19:22] local.INFO: Running Process Image Data Job
[2024-12-30 07:19:27] local.ERROR: job failed {"exception":"[object]"
[2024-12-30 07:19:27] local.INFO: Running Process Image Data Job
[2024-12-30 07:19:32] local.ERROR: job failed {"exception":"[object]"
[2024-12-30 07:19:32] local.INFO: Running Process Image Data Job
[2024-12-30 07:19:37] local.ERROR: job failed {"exception":"[object]"
[2024-12-30 07:19:37] local.INFO: Running Process Image Data Job
[2024-12-30 07:19:42] local.ERROR: job failed {"exception":"[object] "
[2024-12-30 07:19:42] local.INFO: Running Process Image Data Job
[2024-12-30 07:19:47] local.ERROR: job failed {"exception":"[object] "Stop re-trying a job when it reach max exception count
In order to test this you need to have a Job that failed by throwing an exception and set max attempts count on job class or queue worker.
Open ProcessImageData job class and define tries count and allowed number of maximum unhandled exceptions. Tries count is required in order to retry failed job to count exceptions. Without retry attempt count failed job does not retry again.
namespace App\Jobs;
use DateTime;
use Exception;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;
class ProcessImageData implements ShouldQueue
{
use Queueable;
public $tries = 10;
public $maxExceptions = 3;
....
public function handle(): void
{
Log::info('Running Process Image Data Job');
throw new Exception('job failed');
}
}Run your Laravel application and a queue worker.
Logs:
Job re-trying stopped after it detect 3 unhandled exceptions.
[2024-12-30 09:05:46] local.INFO: Received a get request on root route
[2024-12-30 09:05:47] local.INFO: Running Process Image Data Job
[2024-12-30 09:05:47] local.ERROR: job failed {"exception":"[object]"
[2024-12-30 09:05:47] local.INFO: Running Process Image Data Job
[2024-12-30 09:05:47] local.ERROR: job failed {"exception":"[object] "
[2024-12-30 09:05:47] local.INFO: Running Process Image Data Job
[2024-12-30 09:05:47] local.ERROR: job failed {"exception":"[object] "Set Job Process Duration Timeout
Default job execution timeout is 60 seconds. Any job that takes longer than 60 seconds will cause queue worker to exit with an error. You can override timeout value by adding timeout atrribute on the artisan command or using $timeout variable on the job class.
To prevent queue worker being exiting you can use $failOnTimeout. Setting $failOnTimeout to true will marked the job as failed job after timeout instead of stopping the queue worker with an error.
....
class ProcessImageData implements ShouldQueue
{
use Queueable;
public $timeout = 5;
public $failOnTimeout = true;
....
}