Limit Laravel jobs per given time with a filter

Learn how to run preferred number of Laravel jobs per given time with a key (grouping jobs to have different time limits). Here we are using RateLimiter feature of the Laravel in Laravel Jobs.

Table of Content

1. Setup and create job class
2. Dispatch the created job
3. Limit job requests
4. Testing the application
5. Prevent throwing exceptions

Real World Example

Task: Imagine you have a Laravel web application for a company. That company has several departments. Most expensive and time consuming task of the application is report generation. Due to limited resources you had to limit report generation. You should allow 2 report generations per hour per department. (In order to see the results quickly I am going to limit 2 reports per 15 seconds.)

Let’s see how to create a Laravel application to cope with above conditions. Here I am not going to code report creation. Main task of this example is to demonstrate how to run a specific job 2 time per hour per given key? You can change time duration, allowed number of jobs and the key based on your preferences.

Setup & create a Laravel job

First setup a fresh Laravel application. Then use below artisan command to create a job class.

php artisan make:job CreateReports

Open created job class which is CreateReports. Modify it to accept department id. Make sure to log department id to Laravel log file.

<?php

namespace App\Jobs;

use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;

class CreateReports implements ShouldQueue
{
    use Queueable;

    /**
     * Create a new job instance.
     */
    public function __construct(public string $departmentId) {}

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Log::info("Report generated for the department: $this->departmentId");
    }

}

Dispatch the created Job

Here we are using web.php file to dispatch above created job. Lets create two routes for 2 departments.

1. http://127.0.0.1:8000/ – for department 201
2. http://127.0.0.1:8000/2 – for department 202

In our example CreateReports job class accept only department id. Lets pass department id and dispatch CreateReports job on each route.

use App\Jobs\CreateReports;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Route;

Route::get('/', function () {
   $departmentId = 201;
    Log::info("Received get request $departmentId");
    CreateReports::dispatch($departmentId);
   
});

Route::get('/2', function () {
    $departmentId = 202;
    Log::info("Received get request $departmentId");
    CreateReports::dispatch($departmentId);
});

Run web application using below command

php artisan serve

Run queue worker

php artisan queue:work

Send get requests

Open your browser and send get requests to http://127.0.0.1:8000/ and http://127.0.0.1:8000/2. You can send any number of requests to both URLs to create reports without any restrictions.

Check Laravel Logs file. You will see nor matter the number of requests jobs are executed to create reports. Let’s limit report creation process.

Limit number of job requests

Here we are using RateLimiter middleware. No need to create your own middleware class to limit Laravel Jobs. Write your rate limiter inside the boot() method.

namespace App\Providers;

use App\Listeners\UserEventSubscriber;
use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\Event;
use Illuminate\Support\Facades\RateLimiter;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{

    public function register(): void{      //    }

    /**
     * Bootstrap any application services.
     */
    public function boot(): void
    {
        RateLimiter::for('create-reports', function(Object $job){
            return Limit::perSecond(2,15)->by($job->departmentId);
        });
    }
}

1. Import RateLimiter from facades not from cache. (line 6)
2. Use for() method of the RateLimiter to limit job attempts.
3. First argument of the for() method is the name given for your rate limiter.
   • You can use any name to name your rate limiter.
   • Make sure to pass the name used here when binding this middleware to job class using middleware() method in the job class.
4. Second argument of the for() method
   • Closure that accept object which is the job class.
5. Return type of the for() method
   • Make sure to return Limit (Cache\RateLimiting)
   • You can use perSeconds, preMinute, perMinutes, perHour method on Limit class.
6. perSecond
   • First argument is – maximum number of attempts. If 2 is mentioned. You can execute 2 times (job class) before given time expired. Extra requests within the expiration time will throw an exception.
   • Second argument is – time in seconds. Limit lifted time or expiration time to remove limit.
   • You can use perHour to limit jobs per one hour. For testing purposes I am going to use preSecond option.
7. Argument passed to by() method
   • You can use any name.
   • It is a unique id or key used to identify or separate jobs passed by the same* job class.
   • Ex 1: ReportCreation job class can send jobs initiated by users like arun and dave. You can use unique id on the by() method to distinguish those two jobs. It will set limiter independently* for each user.
   • Ex 2: You can use department id to limit report creation jobs for department.
   • Ex 3: You can use user id to limit report creation jobs based on the user.
   • You use job class properties to set unique name (ex: $job->department_id)

More about for() and by() method used here. Here you can name your rate limiter using the for() method. You can apply your rate limiting condition to any given Laravel job by using the name given at for() method. Think your Laravel job access by many different parties and limitations need to be added independently for each party. If one party hit the limiter it will not limit for all other parties. They will have access to Laravel job until they hit the limiter. Simple and easy. For that you can use by() method. by() method of Limit class is used to impose above condition.

Let’s bind above created middleware (rate limiter) to the CreateReports Laravel job.

namespace App\Jobs;

use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\Middleware\RateLimited;
use Illuminate\Support\Facades\Log;

class CreateReports implements ShouldQueue
{
    use Queueable;

    /**
     * Create a new job instance.
     */
    public function __construct(public string $departmentId) {}

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Log::info("Report generated for the department: $this->departmentId");
    }

    public function middleware():array
    { 
        return [new RateLimited('create-reports')];
    }
}

Test Laravel job class with the rate limiter

1. First send 3 get requests to http://127.0.0.1:8000/. Three requests are used to pass allowed maximum attempts which is 2.
2. Next send one request to http://127.0.0.1:8000/2. This request is used to test limit hit for one department does not affect other departements.

Now check logs files.

[2024-12-16 09:02:55] local.INFO: Received get request 201  
[2024-12-16 09:02:57] local.INFO: Report generated for the department: 201  
[2024-12-16 09:02:58] local.INFO: Received get request 201  
[2024-12-16 09:03:00] local.INFO: Report generated for the department: 201  
[2024-12-16 09:03:02] local.INFO: Received get request 201  
[2024-12-16 09:03:06] local.INFO: Received get request 202  
[2024-12-16 09:03:07] local.INFO: Report generated for the department: 202  
[2024-12-16 09:03:16] local.ERROR: App\Jobs\CreateReports has been attempted too many times.

According to above log information, you will see department 201 received 3 requests to generate reports within 2 minutes. Only first two requests are processed. Because third request cannot be proceed due to the rate limiter. You can only proceed another report generation after 15 seconds. That’s why line 8 has a error with exception. Once it hit the limiter it will throw an exception. Only department 201 hit the limiter not the department 202. So department 202 can send another request to generate report until it hit the limiter.

Prevent throwing exceptions for extra job requests

Laravel throws an exception when it received an job requests after hitting the limiter. It is because extra job requests are added to Laravel queue. Job in the queue executed and send failed response on the terminal and you will see exception thrown on logs file.

Since rate limited jobs does not need to retry. Avoid adding extra job requests received after reaching the limit. Therefore make sure to add dontRelease() method.

About dontRelease() method

• It prevent unnecessary retries after hitting the limiter.
• Prevent unwanted resource usage since extra jobs requests are not added to queues.
• Do not use if you do not want to discard extra job requests send when it hit the limiter.

<?php

namespace App\Jobs;

use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Queue\Middleware\RateLimited;
use Illuminate\Support\Facades\Log;

class CreateReports implements ShouldQueue
{
    use Queueable;

    /**
     * Create a new job instance.
     */
    public function __construct(public string $departmentId) {}

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        Log::info("Report generated for the department: $this->departmentId");
    }

    public function middleware():array
    { 
        return [(new RateLimited('create-reports'))->dontRelease()];
    }
}

If you carryout the above tests you will see there is not failed jobs in the queue as well as no exceptions thrown.