The Go Scheduler vs Rust Ownership: Two Different Ways to Control Chaos
We were staring at a service that sat in the money path and ate thousands of requests per second. Latency spikes meant refunds. A crash meant angry calls from sales. We had to pick a language and live with it. We went with Go because we cared about shipping fast, and then we lived with a quiet fear that one hidden race would take us down. Why we chose Go when money was on fire We had one job: take load fast, return answers fast, and not block the rest of the system. We did not have time to write heavy plumbing around memory, threading, and retries. We needed boring concurrency that we could ship this week, not next quarter. Go gave us that story. We could spin workers, batch calls, and see something running in hours. Rust, at that point, felt like a promise for our future selves, not our current deadline. // trafficHandler faned out work to N workers func trafficHandler(reqs <-chan Request, out chan<- Result) {
for r := range reqs {
go func(r Request) {
data := callDownstream(r) // network call
out <- buildResult(r, data) // push response
}(r)
}
} This style let us absorb burst traffic without begging ops for bigger boxes. We could flood the service with parallel calls and still keep median latencies low. The effect was real: during launch week our p95 response time stayed under 220 ms even when traffic jumped, and nobody from finance called us in panic. If you are in that same moment, Go helps you move without ceremony. It lets a small team act rude and fast while the business is yelling for numbers, not elegance. What Go’s scheduler did under real traffic The Go scheduler was the reason we slept the first few nights. We did not have to juggle OS threads by hand. We could just throw work into goroutines and let the runtime multiplex them on a small pool of real threads. That kept CPU stable even when downstream calls stalled. The runtime parked waiting work instead of wasting a whole core per request. +--------- incoming requests ---------+ | req A req B req C req D req E | +-------------------------------------+
| | | | |
v v v v v
+-------------------------------+
| goroutines (cheap, thousands) |
+-------------------------------+
|
v
+-------------------------------+
| OS threads (few, reused) |
+-------------------------------+
Because goroutines parked while waiting on IO, the service stopped burning threads on slow partners. That alone prevented a spiral where stuck calls blocked fresh calls, which blocked more calls, until the queue blew up. Under load tests, CPU never spiked above 62 percent and we did not see queue depth explode. That kept us inside our SLOs without begging for more machines. If your bottleneck is network, not CPU, this model buys time. It lets you survive before deeper fixes, like smarter caching or query shaping, are even ready. Where Go made us sweat later Then the fear started. Go made it very easy to share state across goroutines. Too easy. We had places where two goroutines touched the same map in a hot path. That is the kind of bug that sleeps for weeks, then ruins your weekend. We tried to fix it fast with a lock, because that felt normal and safe. The fix looked fine in review. It still scared us. type Store struct {
mu sync.Mutex data map[string]CachedItem
}
func (s *Store) Put(k string, v CachedItem) {
s.mu.Lock() defer s.mu.Unlock() s.data[k] = v
} func (s *Store) Get(k string) (CachedItem, bool) {
s.mu.Lock() defer s.mu.Unlock() v, ok := s.data[k] return v, ok
} This code is clean and readable. It also hides risk. One missed lock in a helper path and the whole promise dies. You do not notice until memory turns to garbage under traffic, and by then you are already bleeding error rate. That was the first night we asked ourselves if we had picked the safe tool or just the fast one. If you use Go like this, write down every shared structure and decide who is allowed to mutate it. Do not trust that everyone remembers. Write it down like policy. How Rust changed our risk math We pulled Rust into a smaller service to learn its shape. The first feeling was pain. The borrow checker rejected patterns we used every day in Go. It felt slow, strict, and at times annoying. Then we realized it was doing our review job for us. It refused to let two parts of the code hold mutable access to the same data at the same time. That cut off an entire class of bugs before they ever ran. struct Store {
data: HashMap<String, CachedItem>
}
impl Store {
fn put(&mut self, k: String, v: CachedItem) {
self.data.insert(k, v);
}
fn get(&self, k: &str) -> Option<&CachedItem> {
self.data.get(k)
}
} Rust forced us to be explicit about who owned data and who only borrowed it. That made our mental model tighter. We could point at a line and say who was allowed to mutate state. That level of certainty felt expensive at first and then felt like relief. In production tests, the Rust service never showed corrupted cache entries under stress replay. The Go service had, twice. If your fear is silent data damage, not crash loops, Rust gives you guard rails you cannot fake with code reviews or comments. It gives you policy in code, not in memory. The fear that stayed after launch The real cost was not language. It was the constant question we carried for months: are we one weird race away from public failure. We built a mental map of what could explode. It looked like this.
traffic spike
|
v
+-------------------+
| goroutine burst |
+-------------------+
|
v
+-------------------+
| shared state hit |
| without lock |
+-------------------+
|
v
+-------------------+
| dirty cache item |
+-------------------+
|
v
user gets wrong data
That map sat in our heads every release. We were fast, yes. We were proud, yes. We were also nervous every time volume jumped after a promo. This was the moment we stopped seeing Go vs Rust as speed vs control. It became speed now vs fear later. We understood that both had a price, and the real job was choosing which price we could actually carry with our team size. If you are in that point, be honest with yourself about the type of failure you can survive. Wrong data hits trust. A 500 hits retries. Those are not the same wound. Lessons We shipped Go first because we had to prove revenue, not academic safety, and Go let us do that without drowning in thread math. We added Rust where silent corruption would have killed trust in the product, and that let us sleep. That was the balance: Go to move, Rust to protect the parts that could not lie. You do not always get to be elegant. You do get to decide where you are allowed to be scared. That choice is design. That choice is leadership. If this saved you a rollback, say which change did it and pass it on.
Read the full article here: https://medium.com/@kp9810113/the-go-scheduler-vs-rust-ownership-two-different-ways-to-control-chaos-968fca555d45