Distributed Locks Are a Code Smell
Distributed Locks Are a Code Smell The Lock That Lied A single angry support ticket is usually an anomaly. Three identically angry support tickets arriving within 60 seconds about the exact same mi...

Source: DEV Community
Distributed Locks Are a Code Smell The Lock That Lied A single angry support ticket is usually an anomaly. Three identically angry support tickets arriving within 60 seconds about the exact same missing money? That is a pattern. Last quarter, our supposedly bulletproof payment pipeline successfully charged a single customer three times for one order. The investigation dragged on for hours, but the root cause took only four minutes to explain. Here's what actually happened. Service A acquired a Redis lock with a 10-second TTL to process a payment. Right in the middle of executing, the JVM triggered a stop-the-world garbage collection. The entire process froze for 12 seconds. It didn't crash. It didn't log anything. It just... stopped. While Service A was completely frozen, the lock expired in Redis. Service B picked up the very same lock, processed the exact same payment, and committed the charge. Seconds later, Service A woke up from its GC pause. It had absolutely no idea the lock was