Throttling Laravel Job Exceptions

ThrottlesExceptions middleware of Laravel is used to throttle the number of exceptions thrown by a job. It mean you can set a time to delay execution of a certain job after throwing certain number of exceptions.

Exception throwing jobs are immediately processed again until it reaches the retry limit. If the exceptions occur due to a third-party service retrying immediately increases the chance or receiving a another failed job. It is a best practice to throttle jobs that throw exceptions to increase it success rate.

ThrottlesExceptions middleware:

  1. Can set a delay to retry after given number of job execution with exceptions. Example: after 4 failed attempts wait 10 minutes. Then try another 4 times. then wait 10 minutes. It will continue till it reach maximum retry limit or retryUntil time.
  2. Throwing exceptions are not recorded or not shown on Laravel logs by default. Use reports() method on the middleware to get details about the exceptions.

Setup basic Laravel project to test throttlesException

Lets setup our Laravel project to test below given examples. Setup fresh Laravel project. Then create Laravel job class using below artisan command.

php artisan make:job ActivateSubscription

Open ActivateSubscription job class and modify.

  1. Create a variable to hold Date time object.
  2. Add 30 seconds to that variable. So that we can send exceptions for 30 seconds.
  3. Throw exceptions in the handle() method till it passes 30 seconds.
  4. Make sure to log information.
<?php

namespace App\Jobs;

use Exception;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;

class ActivateSubscription implements ShouldQueue
{
    use Queueable;
    private $workingAfter;

    /**
     * Create a new job instance.
     */
    public function __construct()
    {
        //After 30 seconds stop throwing exceptions
        $this->workingAfter = now()->addSeconds(30);
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Log::info("Processing.... subscription activation");
        
        if(now()->lessThan($this->workingAfter)){
            throw new Exception("Exception thrown here");
        }        
        Log::info("Successfully executed the subscription activation");
    }
    
}

Testing the above created Laravel Job

Open the web.php file and add below code to dispatch ActivateSubscription job on root route.

<?php

use App\Jobs\ActivateSubscription;
use App\Jobs\DeactivateSubscription;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Route;

Route::get('/', function () {
    Log::info("Activate subscription request received");
    ActivateSubscription::dispatch();
});

Then run Laravel web application using below command.

php artisan serve

Run a queue worker

php artisan queue:work

Logs:

[2024-12-19 12:38:16] local.INFO: Activate subscription request received  
[2024-12-19 12:38:17] local.INFO: Processing.... subscription activation  
[2024-12-19 12:38:17] local.ERROR: Exception thrown here {"exception" ...
...
...

You will see long list of exceptions are thrown and maximum retry count timeout within 5 or 6 seconds leaving no time to have a successful attempt. What to do if the third-party service experience high load and able to handle your request after 1 minute. We need to slow down job execution speed for jobs that throw exceptions.

Example 1: ThrottlesExceptions

Task: Create a Laravel web application that throws exception for first 30 seconds of initialization (to simulate a job that throws exception and goes back to retry). Use throttlesException middleware to throttle those job exceptions. Throttle exceptions after 4 consecutive attempts. Max retries should be 20 and can retry for 5 minutes.

  1. First argument of ThrottlesExceptions – Number of exceptions allowed before throttling.
  2. Second argument of ThrottlesExceptions – Rest time after reaching the limit mentioned in the first argument.
  3. You should use retryUntil() method with ThrottlesExceptions.
  4. How long a job must retry is defined in retryUntil() method.
<?php

namespace App\Jobs;

use DateTime;
use Exception;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\Middleware\ThrottlesExceptions;
use Illuminate\Support\Facades\Log;

class ActivateSubscription implements ShouldQueue
{
    use Queueable;
    private $workingAfter;

    /**
     * Create a new job instance.
     */
    public function __construct()
    {
        //After 30 seconds stop throwing exceptions
        $this->workingAfter = now()->addSeconds(30);
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Log::info("Processing.... subscription activation");
        
        if(now()->lessThan($this->workingAfter)){
            throw new Exception("Exception thrown here");
        }        
        Log::info("Successfully executed the subscription activation");
    }
    
    public function middleware():array
    {
        return [new ThrottlesExceptions(4,20)];
    }

    public function retryUntil():DateTime
    {
        return now()->addMinutes(5);
    }
}

Check Laravel log files. You will see immediate execution of this job 4 times and rest for 20 seconds before beginning the next execution.

Example 2: ThrottlesExceptions with backoff

According example 1, before throttling the job it ran immediately after throwing the exceptions. You can set a time gap to execute the job that throws the exception before reaching the throttling limit. Currently backoff() support only minutes not seconds.

  1. Change exception simulation time to 70 seconds since backoff time support only minutes. Change value in the constructor.
  2. Add backoff method to initialized ThrottlesExceptions middleware class.
<?php

namespace App\Jobs;

use DateTime;
use Exception;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\Middleware\ThrottlesExceptions;
use Illuminate\Support\Facades\Log;

class ActivateSubscription implements ShouldQueue
{
    use Queueable;
    private $workingAfter;

    /**
     * Create a new job instance.
     */
    public function __construct()
    {
        //After 65 seconds stop throwing exceptions
        $this->workingAfter = now()->addSeconds(70);
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Log::info("Processing.... subscription activation");
        
        if(now()->lessThan($this->workingAfter)){
            throw new Exception("Exception thrown here");
        }        
        Log::info("Successfully executed the subscription activation");
    }
    
    public function middleware():array
    {
        return [
            (new ThrottlesExceptions(4,20))->backoff(1)
        ];
    }

    public function retryUntil():DateTime
    {
        return now()->addMinutes(5);
    }
}

Check Laravel logs files. There you will see the job that throws exceptions does not execute immediately after it failed. It wait the time given in the backoff method which is one minute. If the job throws exceptions continuously it will stop the execution when it reach the time mentioned in the retryUntil method or reached max retry count.

Example 3: Do not hide exceptions thrown by jobs when using ThrottleExceptions

There is no report about the exceptions thrown by Laravel jobs due to failed attempts on Laravel logs. If you want to reports those exceptions you have to use report() method on the Laravel middleware. You can set report for certain type of exceptions by passing a closure to report() method.

Report only selected exceptions

.....
 public function middleware():array
    {
        return [
            (new ThrottlesExceptions(2,15))->report(
                function (Throwable $throwable){
                    return $throwable instanceof HttpClientException;
                }
            )
        ];
    }
.....

report all throttled exceptions by ThrottleExceptions middleware.

.....
 public function middleware():array
    {
        return [
            (new ThrottlesExceptions(2,15))->report()
        ];
    }
.....