Should I Turn My Lightweight API Scheduler into an Open-Source Project?

September 19th 2024 50 views • 2 comments

Hey everyone,

I recently came across a scenario where I needed a lightweight solution to schedule delayed and recurring callbacks for my API. Specifically, I wanted my API to interact with a service that could trigger a designated endpoint, for example, after 2 hours. Additionally, this service should also be capable of making daily calls to a specific endpoint. I'd also like to run this service in a Docker container as part of my application setup.

In my search for a solution, I explored several options:

  • RabbitMQ (no support for recurring tasks)

  • Celery and Redis Queue (too heavy for my needs)

  • AWS EventBridge and Step Functions (not self-hostable)

  • Temporal and Apache Airflow (both heavy and not self-hostable)

Eventually, I stumbled upon APScheduler. I ended up building a simple Flask app around it and bundled everything into a Docker container. It’s been working pretty well so far.

I'm surprised there isn't a go-to solution for such a simple use case (or maybe I just haven't found it yet). Are these kinds of requirements unusual? If not, do you think it would be worthwhile to refine what I’ve put together and turn it into an open-source project?

Your use case is quite common, especially in scenarios where lightweight scheduling and delayed task execution are needed without the overhead of more complex solutions. APScheduler, combined with Flask and Docker, is indeed a smart choice for self-hosted and relatively simple task scheduling.

Turning your solution into an open-source project could be very beneficial to others who have similar requirements but are looking for something more straightforward than RabbitMQ, Celery, or Airflow. If you decide to open-source it, consider adding features like:

  1. A simple web UI for managing tasks.

  2. REST API endpoints for creating, updating, and deleting jobs.

  3. Persistent storage options (e.g., using SQLite or Redis) for scheduled jobs to survive restarts.

  4. Docker orchestration and examples for easy deployment.

This could turn your project into a go-to lightweight scheduling service! 🔥

Thank you for responding and the feature ideas :)

Interesting to hear your view. That confirmes the feeling I have.