Currently, when a user requests an action by tag, and that tag is associated with a large number of workflows (e.g., one million), the tag engine must send a message to each workflow before acknowledging the request. This process can lead to several issues:

• It takes an excessive amount of time (over an hour for a million workflows)
• It can cause Pulsar acknowledging timeouts
• It may result in tag engine timeouts
• It can lead to memory issues

Possible Solutions:

• Pagination with Dedicated Workflow: Implement pagination for the results. If the first page is full, start a dedicated workflow to handle the request. This workflow engine would be embedded within the tag engine

• Self-messaging for Pagination: Implement pagination for the results. Have the tag engine send a message to itself to handle the next page of results