Go’s concurrency features are powerful and easy to use, but\nthat same ease can sometimes lead even seasoned developers to make\nmistakes.\nFortunately, the Go ecosystem comes equipped with useful tools for\ndebugging, e.g., the race detector,\nbut even existing tools may miss some concurrency bugs,\nsuch as the topic of this article, the goroutine leak.
\nGoroutines synchronize or exchange information\nvia shared concurrency primitives, e.g., channels, locks, and wait groups.\nWhile communicating, goroutines often block on these primitives,\nas in, wait until some condition is met;\nubiquitous examples include waiting to acquire a held mutex,\nor receive a message over a channel.\nGoroutines can also block on operating system operations, like reading from a network socket or a file.
\nWe may consider a goroutine leaked if it is blocked,\nbut the conditions needed to unblock it can never be met.\nOver time, an accumulation of leaked goroutines degrades\nperformance through excessive memory usage (by the leaked\ngoroutines themselves or the memory they reference), as well as\nCPU usage from the garbage collector, especially\nif GOMEMLIMIT is in use.
Goroutine leaks can be notoriously difficult to detect.\nIn unit testing, the most significant breakthroughs include\nthe open-source library goleak,\nwhich can instrument individual tests to signal any\nun-terminated goroutines after the test wraps up as suspicious.\nSimilarly, Go 1.25 introduced the synctest package to\nthe standard library; it can significantly improve\nthe quality of unit tests in concurrent code by giving\nGo developers more control over the ordering of concurrent events\nin order to reliably test hard-to-reproduce scenarios.
Unfortunately, neither approach can check for goroutine leaks in production systems, especially at larger scales, which may behave in ways unaccounted for by tests. Goroutine profiles are a rudimentary way to check for operations that block too many goroutines, or analyze growth trends. However, goroutine profiles cannot distinguish between goroutines which are leaked, and those which are temporarily blocked in high numbers by design, e.g., as caused by increased traffic in a microservice. Likewise, leaks which are low in number may slip by undetected for many years.
\nGo 1.27 introduces the goroutine leak profiler,\na flexible and lightweight mechanism for finding\ngoroutine leaks in running Go programs, including production systems.\nUnlike previous approaches, which require human analysis,\nthis mechanism is precise and generates little-to-no false positives.\nThe trade-off is that it is limited to a subset of goroutine leaks:\ngoroutines permanently blocked on channels or primitives\nin the sync package.\nLuckily for us, this already covers a very large subset of goroutine leaks,\nas we’ll see in our examples.
In the following sections, we showcase how to use the feature, followed by some additional examples of detectable leaks, and a description of the underlying implementation and trade-offs.
\nConsider a function that processes work items concurrently:
\ntype result struct {\n res workResult\n err error\n}\nfunc processWorkItems(ws []workItem) ([]workResult, error) {\n // Process work items in parallel, aggregating results in ch.\n ch := make(chan result)\n for _, w := range ws {\n go func() {\n res, err := processWorkItem(w)\n ch <- result{res, err}\n }()\n }\n // Collect the results from ch, or return an error if one is found.\n var results []workResult\n for range len(ws) {\n r := <-ch\n if r.err != nil {\n // This early return may cause goroutine leaks.\n return nil, r.err\n }\n results = append(results, r.res)\n }\n return results, nil\n}Because ch is an unbuffered channel, each worker goroutine blocks when sending\nits result until the main goroutine receives from the channel.\nIf processWorkItems returns early due to an error, the receiving loop terminates,\nand all remaining sender goroutines block forever.
This example is emblematic of a common mistake discovered in real Go programs, including Uber production services. Let’s see how we can find these leaks by using the new goroutine leak profiler.
\nThe profile is available through the\nruntime/pprof package, as the\ngoroutineleak profile type, or by installing the profile handlers defined\nby the net/http/pprof package.\nIf you already have net/http/pprof set up in your service,\nthen you don’t need to do anything else! The profile will be\nautomatically made available for collection at the /debug/pprof/goroutineleak\nendpoint on whatever host and port the handlers are installed.
Let’s put our concurrency bug in context and set up the net/http/pprof package.\nThis way, you can try it yourself!
package main\nimport (\n "errors"\n "log"\n "net/http"\n _ "net/http/pprof"\n "time"\n)\ntype workItem int\ntype workResult int\nfunc processWorkItem(w workItem) (workResult, error) {\n time.Sleep(10 * time.Millisecond)\n if w == 5 {\n return 0, errors.New("simulated error")\n }\n return workResult(w * 2), nil\n}\ntype result struct {\n res workResult\n err error\n}\nfunc processWorkItems(ws []workItem) ([]workResult, error) {\n ch := make(chan result)\n for _, w := range ws {\n go func() {\n res, err := processWorkItem(w)\n ch <- result{res, err}\n }()\n }\n var results []workResult\n for range len(ws) {\n r := <-ch\n if r.err != nil {\n return nil, r.err\n }\n results = append(results, r.res)\n }\n return results, nil\n}\nfunc main() {\n // Start pprof server\n go func() {\n log.Println(http.ListenAndServe("localhost:6060", nil))\n }()\n // Repeatedly trigger the leak\n for {\n items := []workItem{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}\n _, err := processWorkItems(items)\n if err != nil {\n log.Printf("Error processing items: %v", err)\n }\n time.Sleep(time.Second)\n }\n}Build the program above, then run it:
\n$ go build -o leaky\n$ ./leakyIt won’t take long for the program to start accumulating leaks, which you can then view by using the web UI at http://localhost:6060/debug/pprof.
\nAlternatively, you can collect the goroutine\nleak profile using curl, and then examine it with go tool pprof:
$ curl http://localhost:6060/debug/pprof/goroutineleak > leak.prof\n$ go tool pprof leak.prof\nType: goroutineleak\nTime: 2026-03-01 13:19:49 UTC\nEntering interactive mode (type "help" for commands, "o" for options)\n(pprof) list processWorkItems\nTotal: 116\nROUTINE ======================== main.processWorkItems.func1 in .../main.go\n 0 116 (flat, cum) 100% of Total\n . . 31: go func() {\n . . 32: res, err := processWorkItem(w)\n . 116 33: ch <- result{res, err}\n . . 34: }()The profile reveals the goroutines leaked at\nch <- result{res, err} (line 33), pinpointing the culprit operation.\nNotably, the longer the program is running, the larger the number of leaked\ngoroutines.
This leak can be simply fixed by giving ch a buffer:
ch := make(chan result, len(ws))This allows all the work item goroutines to send a message without blocking\nin the event of a premature return of processWorkItems.
We list more real-world examples in this section.
\nThis section is for those interested how leak detection works under the hood of the goroutine leak profiler. For details strictly pertaining to performance overhead and limitations, skip ahead to this section.
\nLet’s start with an initial observation: if a goroutine\nis blocked over some concurrency primitive that no other goroutine has access to\n(in this case, via a reference in memory), then it is obviously leaked.\nThis is already a strong lead, we can generalize it further into a definition\nfor when a goroutine is not leaked, a property we term as liveness.\nWe formally define liveness, an inductive property\nas follows:
\nA goroutine is live if:
\nIn the trivial case, goroutines which are not blocked are obviously not leaked. In the inductive case, the underlying assumption is that any goroutine which is not leaked may eventually use concurrency primitives it references to unblock any other goroutines blocked by those primitives.
\nTo find all live goroutines, we start from the obviously live unblocked goroutines and trace any references they hold, i.e., through their local variables, to find the concurrency primitives they have access to. We then incrementally include any goroutines blocked over those primitives as live, and repeat the process until no additional live goroutines are discovered.
\nFortunately for us, the Go runtime already computes memory reachability through the garbage collector (GC), so the next step is to adapt the GC to suit our purposes. You can quickly compare the two GCs with the following diagrams:
\nA complete overhaul of the GC is not necessary. The Go runtime uses a concurrent tri-color mark-and-sweep garbage collector, (now with the Green Tea variant!), so its MO already neatly aligns with our goals. Only a few key changes are needed:
\nIn the initial phases, the regular GC marks all goroutines (and global variables)
as reachable, such that they would never be considered garbage,\ni.e., they aremark roots .\nWe change it to insteadonly include unblocked goroutines,\nsince these are guaranteed to be live.
This is followed by the marking phase, where the GC traces objects referenced
(transitively) by the mark roots, and marks them as usable memory.\nEven though we do not modify this phase directly, the changes in step 1.\nimplicitly ensure that the GC only marks memory referenced by live goroutines.
Once the GC cycle is complete, the goroutine leak profiler picks up like in a regular goroutine profile, and filters for strictly leaked goroutines.
\nThe examples above demonstrate the usefulness of goroutine leak profiles. Nevertheless, the garbage collector has some limitations that may lead it to miss leaks:
\nMemory overreach : if a concurrency primitive is\nconsistently reachable throughglobal variables orrunnable goroutines ,\nthen goroutines blocking on it are never reported as leaked, even if\nthat concurrency primitive is never used in the future.This can be alleviated by better regimenting access to concurrency primitive references, and more clearly delineating their lifecycle.
\nNon-standard blocking :\nFor the sake of correctness, goroutine leak detection is strictly limited\nto Go first-class concurrency primitives, which includes:\nchannel send and receive operations (including overnil channels),\nblockingselect statements, i.e., with nodefault case, up to, and includingselect statements with no cases, and members of thesync package, specificallyMutex ,RWMutex ,WaitGroup andCond .Goroutines blocked for any other reason, e.g., file and network IO, or direct system calls are never considered as leaked. This likewise applies for custom, user-defined concurrency, e.g., spin locks, unless they rely on the primitives outlined above for their underlying implementation.
Non-determinism : leaks can be detected only after\nthey have occurred, but cannot be otherwise predicted,\nso reproducing and diagnosing leaks in flaky programs\ncontinues to be a challenge.\nFor the best results, we encourage mixing approaches, by using\ngoroutine leak profiles at various layers, up to, and including production,\nas well as comprehensive test suites instrumented withgoleak andsynctest .
Goroutine leak detection is carefully designed to minimize performance impact, but there are, nevertheless, some costs.
\nWhile memory overhead is negligible, only limited to small additions required for bookkeeping, goroutine leak detection can be slower than the regular GC. This is best illustrated through a pathological case we call the “daisy-chain”: In this leak-free example, runnable goroutine G₀ has a reference to primitive P₁ which blocks G₁, and so on.
\nThis implies that proving liveness for some Pᵢ₊₁, requires proving liveness for Pᵢ, which introduces two costs:
\nWhile the second point can eventually be optimized for, the first point is an intrinsic limitation of leak detection that cannot be circumvented.
\nRegardless, we remind the reader that, unless configured otherwise via runtime flags, the GC still operates concurrently with user code. Furthermore, if a goroutine leak can be observed at some point in time, then it can also be observed at any future point during the same execution. Periodic profiling infrastructures can therefore tune profiling frequency, e.g., every 4 hours, to minimize overhead at virtually no cost in leak detection capabilities.
\nGoroutine leak detection is the result of a research collaboration between Aarhus University, Washington University in St. Louis, and Uber, as presented in “Dynamic Partial Deadlock Detection and Recovery via Garbage Collection” (Saioc et al., ASPLOS 2025).
\nThe transition from academic prototype to actual Go feature was made possible with the guidance of Michael Knyszek and Michael Pratt on the Go team at Google, and PJ Malloy (@thepudds).
\nThe following are coding patterns that lead to leaks, as observed in industrial-scale codebases and open source projects, in ascending order of complexity.
\nYou can quickly test drive the goroutine leak detector on them in the Go playground, as well as experiment with your own leaks.
\nSome of the simplest leaks occur when more messages\nare sent over a channel than expected.\nBelow, a goroutine is expected to send one message to the main goroutine\nover an unbuffered channel.\nHowever, the return statement is missing after the send operation\nin the error case.\nFor every error, the sender will, therefore, attempt to send two messages,\nwhich causes a leak.
func DoubleSend() {\n ch := make(chan any)\n go func(err error) {\n if err != nil {\n // In case of an error, send nil.\n ch <- nil\n // Return statement is missing.\n }\n // Otherwise, continue with normal behaviour.\n // This send is still executed, which causes a leak in the error case.\n ch <- struct{}{}\n }(fmt.Errorf("error"))\n // Receive only one message.\n <-ch\n}While the profile does not explicitly highlight the missing return\nas the cause, it at least directs you to the faulty function, by\nhighlighting the leaking send operation.
(pprof) list DoubleSend\nTotal: 1\nROUTINE ======================== main.DoubleSend.func1 in .../main.go\n 0 1 (flat, cum) 100% of Total\n . . 118: go func(err error) {\n . . 119: if err != nil {\n . . 121: ch <- nil\n . . 123: }\n . 1 126: ch <- struct{}{}\n . . 127: }(fmt.Errorf("error"))\n . . 129: <-chThis leak can be addressed simply by adding a return statement after the\nsend operation in the error case.
The inverse situation is just as common, where the receiver omits communication on some control flow paths, in what is effectively a simplified version of the introductory example.
\n// Incoming error simulates an error produced internally.\nfunc EarlyReturn(err error) {\n ch := make(chan any)\n // Create a worker goroutine.\n go func() {\n // Send something to the channel.\n // Leaks if the parent goroutine terminates early.\n ch <- struct{}{}\n }()\n if err != nil {\n // The parent goroutine quits too early in case of an error.\n // Sender leaks.\n return\n }\n // Receive is only executed if there is no error.\n <-ch\n}The goroutine leak is exposed by the profile:
\nROUTINE ======================== main.EarlyReturn.func1 in .../main.go\n 0 1 (flat, cum) 100% of Total\n . . 140: go func() {\n . 1 143: ch <- struct{}{}\n . . 144: }()\n . . 145:\n . . 146: if err != nil {The leak can be addressed by giving ch a buffer of size 1.
A variation of the Early return pattern above involves contexts\nand non-deterministic choice (select statements):
func Timeout(ctx context.Context) {\n // An unbuffered channel is used to coordinate\n // a worker and parent thread\n ch := make(chan any)\n // Create worker goroutine\n go func() {\n // Perform some work then signal to the parent thread.\n ch <- struct{}{}\n }()\n // Wait for message from worker or context\n // to be cancelled or timed out.\n select {\n case <-ch: // Receive message from worker\n case <-ctx.Done():\n // Sender leaks because there is no\n // future rendezvous over the channel.\n }\n}If the context is cancelled before the sender synchronizes with the parent, the sender will leak:
\n(pprof) list Timeout\nTotal: 10\nROUTINE ======================== main.Timeout.func1.1 in .../main.go\n 0 10 (flat, cum) 100% of Total\n . . 198: go func() {\n . 10 201: ch <- struct{}{}\n . . 202: }()As in the previous example, the fix is to give the channel buffer of size 1.
\nIterating over channels by using range\nallows you to repeatedly receive values from a channel in a loop.\nOnce the channel is closed and all values that have been enqueued\nin the channel’s buffer have been received, the loop exits.
Importantly, if the channel is never closed, a range loop will block\nthe executing goroutine forever.\nOmitting the close operation is a common mistake, as below:
// Incoming list of items and the number of workers.\nfunc noCloseRange(list []any, workers int) {\n // Create a channel that distributes work items.\n ch := make(chan any)\n // Create the worker goroutines.\n for i := 0; i < workers; i++ {\n go func() {\n // Each worker pulls items from the channel\n // and then processes it.\n for item := range ch {\n // Process each item\n _ = item\n }\n }()\n }\n // Queue items to the workers by using the channel.\n for _, item := range list {\n // The parent leaks by sending an item if workers == 0\n // or if all the workers panic, but the panic is recovered.\n ch <- item\n }\n // Otherwise, the channel is never closed, so workers\n // leak once there are no more items left to process.\n}\n...\ngo noCloseRange([]any{1, 2, 3}, 3) // Leaks all 3 workersA goroutine leak profile for such a program would include the following:
\nType: goroutineleak\n(pprof) list noCloseRange.func1\nTotal: 4\nROUTINE ======================== main.noCloseRange.func1 in .../main.go\n 0 3 (flat, cum) 75.00% of Total\n . . 82: go func() {\n . 3 84: for item := range ch {\n . . 86: _ = item\n . . 87: }\n . . 88: }()We see the 3 workers blocked at the range ch operation, which\ngives an ample hint as to the cause of the leak. The leak can be\naddressed by simply closing the channel once all messages have been sent:
for _, item := range list {\n ch <- item\n }\n // All items have been sent. It is now safe to close.\n close(ch)Bonus! Eagle-eyed readers may have spotted another potential\nleak in this example, if the number of workers is mistakenly set to zero,\nwhich will lead the parent sender to leak:
\ngo noCloseRange([]any{1, 2, 3}, 0) // Sender leaks with 0 workersThis is also captured by the profile:
\n(pprof) list noCloseRange$\nTotal: 4\nROUTINE ======================== main.noCloseRange in .../main.go\n 0 1 (flat, cum) 25.00% of Total\n . . 76:func noCloseRange(list []any, workers int) {\n...\n . . 92: for _, item := range list {\n . 1 95: ch <- item\n . . 96: }While workers > 0 can be assumed to hold in realistic production systems,\ngoroutine leak profiles can nevertheless be used to implicitly monitor for off-chance\nviolations without conservative workers <= 0 checks.
The patterns seen so far have been relatively constrained in their lexical scope. However, as functionality is spread out across functions, methods and packages, and implementations are obfuscated by interfaces, the difficulty of manually detecting leaks drastically increases.
\nSuch a case is exemplified in this section, with the custom worker type that embeds two channel\nfields, ch and done and creates a looping goroutine with its Start method that\nreads from both channels with a select statement.\nSaid goroutine can only be terminated by receiving a message through the done channel,\nwhich is closed by the Stop method.
The Start method can be invoked any number of times, but if it is invoked\nat least once, Stop should eventually be called.
As a result, Start and Stop form an implicit contract that dictates the order\nin which the methods should be invoked.\nBreaking that contract can lead to undesirable behavior,\nin this case, goroutine leaks:
func MethodContractViolation() {\n items := make([]any, 10)\n // Create a new worker\n w := NewWorker()\n // Start worker\n w.Start()\n // Operate on worker\n for _, item := range items {\n w.AddToQueue(item)\n }\n // Exits without calling ’Stop’.\n}\ntype worker struct {\n ch chan any\n done chan any\n}\ntype Worker interface {\n Start()\n Stop()\n AddToQueue(item any)\n}\nfunc NewWorker() Worker {\n return &worker{\n ch: make(chan any),\n done: make(chan any),\n }\n}\n// Start spawns a background goroutine that extracts items pushed to the queue.\nfunc (w *worker) Start() {\n go func() {\n for {\n select {\n case <-w.ch: // Normal workflow\n case <-w.done:\n return // Shut down\n }\n }\n }()\n}\nfunc (w *worker) Stop() {\n // Allows goroutine created by Start to terminate\n close(w.done)\n}\nfunc (w *worker) AddToQueue(item any) {\n w.ch <- item\n}This issue is further exacerbated in practice, where such custom types are only\nexported as interfaces, in this case, through the non-descript\nWorker type.\nClients may not even be aware of the underlying implementation and,\nconsequently, violate the implicit contract without realizing.
Fortunately, soliciting a goroutine leak profile can reveal the defect:
\n(pprof) list Start\nTotal: 1\nROUTINE ======================== main.(*worker).Start.func1 in .../main.go\n 0 1 (flat, cum) 100% of Total\n . . 266: go func() {\n . . 267: for {\n . 1 268: select {\n . . 269: case <-w.ch:\n . . 270: case <-w.done:\n . . 271: returnNaturally, the fix involves following the trail to the Start call\nand adding an invocation of Stop.
The following example\nis taken from CockroachDB.\nIt involves acquiring and releasing a lock in a loop,\nbut forgetting to unlock it\nbefore executing a break statement:
type Gossip struct {\n mu sync.Mutex\n closed bool\n}\nfunc (g *Gossip) bootstrap() {\n for {\n g.mu.Lock()\n if g.closed {\n // Missing g.mu.Unlock\n break\n }\n g.mu.Unlock()\n }\n}\nfunc Cockroach584() {\n g := &Gossip{\n closed: true,\n }\n // ...\n g.bootstrap()\n g.bootstrap() // Causes a leak\n}In such a case, the goroutine will leak when failing to acquire the lock.
\n(pprof) list Gossip\nTotal: 1\nROUTINE ======================== main.(*Gossip).bootstrap in .../main.go\n 0 1 (flat, cum) 100% of Total\n . . 165:func (g *Gossip) bootstrap() {\n . . 166: for {\n . 1 167: g.mu.Lock()\n . . 168: if g.closed {\n . . 170: break\n . . 171: }\n . . 172: g.mu.Unlock()Adding a call to Unlock before the break addresses the issue.
This example, found in etcd, shows how an unexpected ordering between channel operations can lead to a goroutine leak:
\ntype node struct {\n status chan chan struct{}\n stop chan struct{}\n done chan struct{}\n}\nfunc (n *node) Status() struct{} {\n c := make(chan struct{})\n n.status <- c\n return <-c\n}\nfunc (n *node) run() {\n for {\n select {\n case c := <-n.status:\n c <- struct{}{}\n case <-n.stop:\n close(n.done)\n return\n }\n }\n}\nfunc (n *node) Stop() {\n select {\n case n.stop <- struct{}{}:\n case <-n.done:\n return\n }\n <-n.done\n}\nfunc Etcd6857() {\n n := &node{\n status: make(chan chan struct{}),\n stop: make(chan struct{}),\n done: make(chan struct{}),\n }\n go n.run()\n go n.Status()\n go n.Stop()\n}The run method fires a loop which expects to\nrepeatedly receive messages over the status channel\n(sent by invoking the Status method).\nAt the same time, it can also receive one message over the\nstop channel (sent via the Stop method),\nat which point it closes the done channel and exits.\nThe Stop method itself then waits to receive message\nover done, which is unblocked once done is closed.
A leak may occur if the run, Status, and Stop methods\nrun concurrently.\nThe Stop and run goroutines can synchronize\nand exit without receiving the message issued\nby Status, causing it to block forever.
(pprof) list Status\nTotal: 8\nROUTINE ======================== main.(*node).Status in .../main.go\n 0 8 (flat, cum) 100% of Total\n . . 16:func (n *node) Status() struct{} {\n . . 17: c := make(chan struct{})\n . 8 18: n.status <- c\n . . 19: return <-c\n . . 20:}Wrapping the send to status in a select statement\nwhere the other case branch tries to receive a message\nover done allows the goroutine running to Status\nto gracefully exit if it lost the race with a Stop\ncall.
This example occurs in Kubernetes, as a result of mixing channels and locks:
\ntype Connection struct {\n closeChan chan bool\n}\ntype idleAwareFramer struct {\n resetChan chan bool\n writeLock sync.Mutex\n conn *Connection\n}\nfunc (i *idleAwareFramer) monitor() {\n var resetChan = i.resetChan\n for range i.conn.closeChan {\n i.writeLock.Lock()\n close(resetChan)\n i.resetChan = nil\n i.writeLock.Unlock()\n break\n }\n}\nfunc (i *idleAwareFramer) WriteFrame() {\n i.writeLock.Lock()\n defer i.writeLock.Unlock()\n if i.resetChan == nil {\n return\n }\n i.resetChan <- true\n}\nfunc NewIdleAwareFramer() *idleAwareFramer {\n return &idleAwareFramer{\n resetChan: make(chan bool),\n conn: &Connection{\n closeChan: make(chan bool),\n },\n }\n}\nfunc Kubernetes6632() {\n i := NewIdleAwareFramer()\n go func() {\n i.conn.closeChan <- true\n }()\n go i.monitor()\n go i.WriteFrame()\n}The goroutine running WriteFrame may acquire the\nidle-aware framer lock, followed by sending a message over the\nresetChan channel, while the monitor goroutine\nwaits to receive a message over the closeChan channel.\nOnce a message has been dispatched, the monitor goroutine\nwill attempt to acquire the same lock.\nHowever, since there isn’t any traffic over resetChan, the send operation\nblocks forever, preventing the monitor goroutine from releasing\nthe lock.\nThis, in turn, causes both goroutines to leak.
(pprof) list AwareFramer\nTotal: 200\nROUTINE ======================== main.(*idleAwareFramer).WriteFrame in .../main.go\n 0 100 (flat, cum) 50.00% of Total\n . . 32:func (i *idleAwareFramer) WriteFrame() {\n . . 33: i.writeLock.Lock()\n . . 34: defer i.writeLock.Unlock()\n . . 35: if i.resetChan == nil {\n . . 36: return\n . . 37: }\n . 100 38: i.resetChan <- true\n . . 39:}\nROUTINE ======================== main.(*idleAwareFramer).monitor in .../main.go\n 0 100 (flat, cum) 50.00% of Total\n . . 21:func (i *idleAwareFramer) monitor() {\n . . 22: var resetChan = i.resetChan\n . . 23: for range i.conn.closeChan {\n . 100 24: i.writeLock.Lock()\n . . 25: close(resetChan)The fix is to set up a separate goroutine after a message is received\nover closeChan in the monitor goroutine that drains the resetChan\nbefore attempting to acquire the lock.
sync.WaitGroupThe following example in Moby showcases how wait groups may cause leaks:
\ntype Manager struct {\n plugins []int\n}\nfunc (pm *Manager) init() {\n var group sync.WaitGroup\n group.Add(len(pm.plugins))\n for _, p := range pm.plugins {\n go func(p int) {\n defer group.Done()\n }(p)\n group.Wait() // Block here\n }\n}\nfunc Moby25384() {\n pm := &Manager{\n plugins: []int{1, 2},\n }\n go pm.init()\n}The group wait group increments its counter\ndepending on the number of plugins held by the\nplugin manager pm, then iterates over each plugin\nand spawns a goroutine.\nEach goroutine decrements the counter once it finishes\nits task with the Done method.\nHowever, group erroneously invokes Wait inside\nthe loop body, instead of after it!\nThis will cause any goroutine running the init method\nwhen the manager has more than one plugin to leak.
(pprof) list init\nTotal: 1\nROUTINE ======================== main.(*Manager).init in .../main.go\n 0 1 (flat, cum) 100% of Total\n . . 17: group.Add(len(pm.plugins))\n . . 18: for _, p := range pm.plugins {\n . . 19: go func(p int) {\n . . 20: defer group.Done()\n . . 21: }(p)\n . 1 22: group.Wait() // Block here\n . . 23: }This can be easily addressed by moving the Wait outside\nthe loop.
Another example in Moby showcases a mixed channel-lock leak:
\ntype (\n State struct {\n Health *Health\n }\n Container struct {\n sync.Mutex\n State *State\n }\n Store struct {\n ctr *Container\n }\n Daemon struct {\n containers Store\n }\n Health struct {\n stop chan struct{}\n }\n)\nfunc (d *Daemon) StateChanged() {\n c := d.containers.ctr\n c.Lock()\n d.updateHealthMonitorElseBranch(c)\n defer c.Unlock()\n}\nfunc (d *Daemon) updateHealthMonitorElseBranch(c *Container) {\n c.State.Health.CloseMonitorChannel()\n}\nfunc (s *Health) CloseMonitorChannel() {\n if s.stop != nil {\n s.stop <- struct{}{}\n }\n}\nfunc monitor(c *Container, stop chan struct{}) {\n for {\n select {\n case <-stop:\n return\n default:\n handleProbeResult(c)\n }\n }\n}\nfunc handleProbeResult(c *Container) {\n c.Lock()\n defer c.Unlock()\n // Additional work...\n}\nfunc NewDaemonAndContainer() (*Daemon, *Container) {\n c := &Container{\n State: &State{&Health{\n stop: make(chan struct{}),\n }},\n }\n d := &Daemon{Store{c}}\n return d, c\n}\nfunc Moby28462() {\n d, c := NewDaemonAndContainer()\n go monitor(c, c.State.Health.stop)\n go d.StateChanged()\n}The goroutine invoking StateChanged may acquire the lock\nof the container stored by the daemon, then invoke\nthe updateHealthMonitorElseBranch method on\nthe daemon, which attempts to send a message over\nthe stop channel of the container.\nHowever, the goroutine running monitor\nmay fail to receive a message over stop, if the message\nis not already in-flight, and instead unblock by picking\nthe default case of the select statement.\nThis will lead it to try to acquire the same container\nlock that is already held by the StateChanged\ngoroutine, leading both goroutines to leak.
(pprof) list .CloseMonitorChannel\nTotal: 2\nROUTINE ======================== main.(*Health).CloseMonitorChannel in .../main.go\n 0 1 (flat, cum) 50.00% of Total\n . . 66:func (s *Health) CloseMonitorChannel() {\n . . 67: if s.stop != nil {\n . 1 68: s.stop <- struct{}{}\n . . 69: }\n . . 70:}\n(pprof) list main.handleProbeResult\nTotal: 2\nROUTINE ======================== main.handleProbeResult in .../main.go\n 0 1 (flat, cum) 50.00% of Total\n . . 83:func handleProbeResult(c *Container) {\n . 1 84: c.Lock()\n . . 85: // Additional work...\n . . 86: defer c.Unlock()\n . . 87:}The fix is to close the stop channel instead\nof sending a message over it.\nSince closing a channel is not a blocking operation,\nthe StateChanged goroutine is then able to release\nthe lock.\nIn turn, this unblocks the monitor goroutine,\nwhich may now terminate by picking unblocked\n<-stop case branch in the select statement\non the next loop iteration.
**Next article:** Size-Specialized Memory Allocation\n\n \n \n \n \n **Previous article:** Generic Methods\n\n \n \n **Blog Index**