Pub/Sub message ordering with asynchronous processing
Hey everyone,
I am looking for the best approach to maintain message ordering in Cloud Pub/Sub when dealing with mixed processing times.
Currently, I use Pub/Sub with message ordering enabled, but I face a challenge when a message requiring heavy background processing (via Cloud Tasks and Cloud Functions) is sent immediately before a message that requires none.
In my current setup, I only publish to Pub/Sub after the background processing completes, which causes the second "fast" message to be consumed before the first "slow" one, breaking the intended sequence. To solve this, I’m considering publishing all messages instantly, using a "placeholder" for the slow messages and having my push subscription endpoint check a database flag to see if the background task is finished. If not, the endpoint would NACK the message to trigger a retry.
While this "NACK-until-ready" approach preserves the order (since subsequent messages in that ordering key will wait), it introduces latency and overhead from retries, so I’m wondering if there is a more efficient way to handle this dependency without relying on frequent NACKs.
Would love to hear what you think!