Skip to main content

Gathering Sensor Data

  • Status: accepted
  • Deciders: Piotr Dankowski
  • Date: 2023-05-10

Technical Story: https://orikami.atlassian.net/browse/KRN-476

Context and Problem Statement​

We have to build engine which can produce events to the queue at the certain point in time. Scheduling should be done:

  • periodically
  • one-ff
  • conditionally

Decision Drivers​

  • Scalability. We should be able to produce hundreds of thousands of events in parallel
  • Precision. The event should occur on the queue as close to it schedule time as possible
  • Open to optimization in the future

Considered Options​

  • [AgendaJS]

Agenda is a light-weight job scheduling library for Node.js Which allows to define jobs schedule, persist them, retrieve and persist results

  • [Custom implementation]

We would need a mongo collection optimized for reads which will contains the occurrences of next schedule of each item and process which will periodically browse that collection, pick which are in time and put them to the queue

Results of the PoC for agenda​

PoC was run on development machin:

  • AMD Ryzen 7 5800H
  • 32GB RAM

Bursting 100k events at one time results with 2-3 minutes difference between first and last in batch Which is not a brilliant outcome. Moreover, i was noticing some nondeterministic behavior when the job collection went big like 1kk

Despite the experiment

  • In my understanding of the documentation, the idea behind agenda is: to keep and maintain a relatively small pool of periodic jobs. Those jobs can be heavy in logic, store outputs and so on. Agenda is also rather about serial execution and ensuring the job is executed once a time. This is contrary to what we want to achieve. We want to spawn thousands of events in parallel to the queue. We don't have any logic in that “jobs”
  • Agenda is not horizontally scalable: The documentation is focusing on how to lock tasks they does not interfere witch each other,
    There is no way how to divide collection between agenda instances
  • agenda does not support conditional schedules - this need to be implemented by us anyway

Decision outcome​

I will implement custom solution to avoid situation when we are bending the tool to purpose it was not designed for.