This resume is also available in English.

1. Go foundations3 / 4 · All columns

Columns › 1. Go foundations › Part 3

Goroutines, Channels and Concurrency Patterns in Go

Goroutine、Channel 與並發模式

Starting from launching a goroutine, this article walks through WaitGroup, data races and mutexes, channel semantics, select, worker pools and pipelines, then bounded concurrency, error collection and goroutine leaks.

  • Go
  • Concurrency
  • Goroutine
  • Channel
In this article
  1. What this article solves
  2. Step 1: Start a goroutine and wait for it to finish
  3. Step 2: Shared memory, data races and Mutex
  4. Step 3: Channels: passing data between goroutines
  5. Step 4: select: waiting for several things at once
  6. Step 5: Worker pools and pipelines
  7. Step 6: Limit concurrency and collect errors
  8. Step 7: Avoid goroutine leaks
  9. Common pitfalls
  10. Key takeaways
  11. Next

In the previous article, “Error Handling in Go: Wrapping, Sentinels and Custom Types”, we gave the order service consistent error handling: add context at each layer, inspect with sentinels and custom types, and translate uniformly at the boundary. But back then all the code ran one step after another; this article lets the order service "do several things at once" and safely collect the results and errors produced by concurrent work.

What this article solves

When an order service creates an order, it often needs to look up several things: the price of each item, whether there is enough stock, and the user's eligibility for discounts. If these lookups run one after another, the total time is the sum of all of them; if they are independent and sent out at the same time, the total time is close to that of the slowest one. The server itself is the same: each incoming HTTP request is handled in its own goroutine, with thousands of requests in flight at once.

Go makes concurrency part of the language: start a goroutine with the go keyword, pass data between goroutines with channels, and wait for several things at once with select. There is very little syntax, but getting it right is not simple. Common problems include:

  • Starting a goroutine but not waiting for it to finish, so the program exits first.
  • Several goroutines reading and writing the same variable at once, giving results that are sometimes right and sometimes wrong (a data race).
  • Nobody receiving from a channel, so the sending goroutine is stuck forever, slowly building up into a memory leak.
  • No upper bound on concurrency, firing off tens of thousands of downstream requests at once and blowing up the other side's or your own connection pool.
  • One goroutine failing, but its error never comes back, or the other goroutines keep doing pointless work.

This article builds the tools to deal with these problems step by step, ending up with a few patterns you can apply directly: fan-out with WaitGroup, the worker pool, the pipeline, and bounded concurrency with error collection.

Step 1: Start a goroutine and wait for it to finish

A goroutine is a lightweight thread scheduled by the Go runtime. Its initial stack is only a few KB and grows as needed, and it is far cheaper to create and switch than an operating-system thread, so having tens of thousands of goroutines at once is perfectly normal. Go's scheduler distributes these goroutines across a number of OS threads roughly equal to the number of CPU cores (GOMAXPROCS), and switches automatically on blocking system calls or network I/O so that other goroutines can keep running.

To start a goroutine, just put go in front of a function call. But go itself doesn't wait: as soon as the main function returns, the whole program exits, no matter how many goroutines haven't finished. So every time you start a goroutine, decide first: who is responsible for waiting for it to finish?

The most common tool is sync.WaitGroup. Since Go 1.25, WaitGroup has a Go method that wraps "increment the counter, start the goroutine, decrement the counter when it ends" into a single call, which avoids forgetting Add or putting Done in the wrong place:

package main

import (
	"fmt"
	"math/rand/v2"
	"sync"
	"time"
)

type Item struct {
	SKU string
	Qty int
}

// fetchPrice simulates calling the pricing service; each call takes a different amount of time.
func fetchPrice(sku string) int64 {
	time.Sleep(time.Duration(50+rand.IntN(100)) * time.Millisecond)
	return int64(1000 + len(sku)*100) // in minor units
}

func main() {
	items := []Item{{"book", 2}, {"pen", 10}, {"lamp", 1}}
	prices := make([]int64, len(items)) // each goroutine only writes its own index, so they don't interfere

	start := time.Now()
	var wg sync.WaitGroup
	for i, it := range items { // since Go 1.22 every iteration has a new i and it, safe to use in closures
		wg.Go(func() {
			prices[i] = fetchPrice(it.SKU)
		})
	}
	wg.Wait() // wait for all price lookups to finish

	var total int64
	for i, it := range items {
		total += prices[i] * int64(it.Qty)
	}
	fmt.Printf("total=%d minor units, took about %v (close to the slowest lookup, not the sum of all three)\n",
		total, time.Since(start).Round(10*time.Millisecond))
}

Three things in this program are worth noting:

  1. Each goroutine writes a different slice element. Different slice indices are different memory locations, so writing them at the same time doesn't conflict and no lock is needed. This is the simplest and safest way to collect results.
  2. Loop variables can be used directly in closures. Before Go 1.22, a for loop's variable existed only once for the whole loop, so a closure would capture the value from the last iteration, which is why old code often has i := i. Once go.mod declares 1.22 or later, every iteration gets a new variable and this trap disappears (this is the example of the go directive changing semantics mentioned in “Go Project Layout and Go Modules”).
  3. wg.Wait() always comes after every wg.Go. If you use the old-style wg.Add(1) with defer wg.Done(), Add must be called before the goroutine starts, not inside it; otherwise Wait may return before Add happens.
Sequential: total time = the three segments added together book price pen lamp price main Concurrent: total time ≈ the slowest segment goroutine 1 book goroutine 2 pen goroutine 3 lamp main wg.Wait() wg.Go ×3 all finished time

Figure: run sequentially, the total time is the three segments added together; run concurrently, the main goroutine only waits until the slowest one finishes.

Step 2: Shared memory, data races and Mutex

The previous step deliberately had each goroutine write to a different location. What if they all add to the same total variable instead? total += x is really three actions, "read, add, write back", and when two goroutines interleave, one result may overwrite the other. This situation, "several goroutines accessing the same memory at once, at least one of them writing, with no synchronisation", is called a data race. What makes it frightening is that most of the time the result looks correct, and it only goes wrong under heavy load or with specific timing. What's more, Go's memory model says the behaviour of a program with a data race is undefined: not just "the number comes out wrong", but anything up to reading a half-written string or corrupting a map's internal structure.

Go has a built-in race detector. Add -race when testing and running, and it prints both stacks when a race happens:

go test -race ./...
go run -race ./cmd/order

It has a runtime cost (memory and CPU usage both go up several times), so it is usually used in the CI test stage rather than in production. The detector can only find races that "actually execute", so test coverage of concurrent paths matters; “Testing in Go: Table-Driven Tests, httptest and Integration Tests” comes back to this.

There are several tools for fixing data races:

ToolSuitsNote
sync.MutexProtecting a group of related state (such as a map plus a count)Keep the locked section small; don't do I/O while holding the lock
sync.RWMutexReads far outnumber writes, and the read critical section isn't shortWhen the critical section is very short it isn't necessarily faster than Mutex; measure it
sync/atomic typesA single counter or flag (atomic.Int64, atomic.Bool)Only guarantees atomicity of a single variable, not consistency across several
sync.Once / sync.OnceValueInitialisation that happens only onceA failed initialisation is not retried either
channelHanding ownership of data to another goroutineCovered in detail in the next step

The convention for using a Mutex is to put the lock in the same struct as the data it protects, and lock inside methods rather than leaving it to callers:

// Snippet: a stock cache protected by a Mutex
type stockCache struct {
	mu    sync.Mutex
	stock map[string]int // protected by mu
}

func (c *stockCache) Reserve(sku string, qty int) bool {
	c.mu.Lock()
	defer c.mu.Unlock()
	if c.stock[sku] < qty {
		return false
	}
	c.stock[sku] -= qty // "check" and "decrement" are under the same lock, so nothing can cut in between
	return true
}

Note that the "check" and the "decrement" must happen within the same lock acquisition. If you lock to read the stock, unlock, and then lock again to decrement, two goroutines may both pass the check, causing overselling. This kind of "check then act" problem gets trickier across services and databases; “Concurrency Pitfalls: From Data Races to Overselling” is devoted to it.

Tip: A sync.Mutex must not be copied. Passing a struct containing a Mutex by value gives you an independent new lock, and the protection is gone. The copylocks check in go vet catches this kind of mistake, so methods on such structs always use pointer receivers.

Step 3: Channels: passing data between goroutines

Go's motto is "Don't communicate by sharing memory; share memory by communicating." A channel is that means of communication: one goroutine sends a value in and another takes it out, and "ownership" of the value moves with it. Only one goroutine handles the value at any time, so naturally there is no race.

Unbuffered and buffered

make(chan T) creates an unbuffered channel: the sender blocks until a receiver takes the value, and the two "meet" at that moment. It conveys both data and a synchronisation signal: when the send completes, you know the other side has received it.

make(chan T, n) creates a buffered channel: while the buffer has free space, a send returns immediately; it only blocks when the buffer is full. A receive blocks when the buffer is empty. The buffer allows the sender and receiver to differ in speed briefly, but it does not make a slow consumer faster; it only postpones the moment of blocking. When choosing a buffer size, you should be able to give a reason for the number (such as "equal to the number of workers" or "allow a batch of at most 100 items") rather than filling in a big number to paper over a design problem.

Unbuffered make(chan T): both sides meet sender ch <- v hand-over at one moment; blocks with no receiver receiver <-ch Buffered make(chan T, 3): blocks only when full sender v1 v2 v3 receiver buffer (capacity 3) is full sending v4 blocks A buffer only absorbs brief speed differences; it can't make a slow receiver faster

Figure: with the unbuffered channel at the top, the send and receive must happen at the same time; with the capacity-3 channel at the bottom, the first three sends go straight into the buffer and only the fourth blocks.

Closing and range

The sender uses close(ch) to say "there will be no more values". The receiver can check with v, ok := <-ch: ok is false when the channel is closed and the buffer is empty; or simply use for v := range ch, which ends automatically once the channel is closed and drained. There is only one rule to remember about closing: the sender closes, and only once. A receiver closing the channel makes any goroutine still sending panic.

Operations in each channel state

This table is worth memorising; most channel-related bugs come from here:

Operationnil channelOpenClosed
Send ch <- vBlocks foreverSucceeds if there is space or a receiver, otherwise blockspanic
Receive <-chBlocks foreverGets a value if there is one, otherwise blocksReturns the zero value immediately, ok == false
Close close(ch)panicSucceedspanic

"A nil channel blocks forever" looks useless, but it is actually handy in select: setting a case's channel to nil effectively disables that case for the time being.

Step 4: select: waiting for several things at once

select lets one goroutine wait on several channel operations at once and runs whichever becomes ready first; if several are ready at once, it picks one at random so that no case is starved forever. The three most common uses:

// Snippet: three common uses of select

// 1. Timeout: wait at most 2 seconds
select {
case res := <-results:
	fmt.Println("got", res)
case <-time.After(2 * time.Second):
	fmt.Println("timeout")
}

// 2. Non-blocking send: if the queue is full, drop and record it rather than stalling the request
select {
case events <- ev:
default:
	dropped.Add(1)
}

// 3. A stoppable loop: ends when done is closed
for {
	select {
	case <-done:
		return
	case job := <-jobs:
		handle(job)
	}
}

In the third use, done is a channel used only to "broadcast stop": when it is closed, every goroutine waiting on <-done wakes up at the same time (because receiving from a closed channel returns immediately). This is Go's traditional way of telling several goroutines to stop, and the next article's context standardises exactly this pattern, adding timeouts and a cause. In practice, new code should use ctx.Done() instead of a home-made done channel.

Tip: Calling time.After repeatedly in a loop keeps creating new timers. Since Go 1.23, timers that haven't fired can be reclaimed promptly, instead of piling up until they fire as in older versions, but on hot paths it is still advisable to reuse one time.Timer and Reset it each round, which makes the semantics clearer.

Step 5: Worker pools and pipelines

Combining the tools so far gives you a few classic concurrency patterns.

Worker pool: a fixed number of goroutines working through jobs

When there are many jobs (for example, recalculating the amounts of ten thousand orders), starting one goroutine per job works, but it fires off ten thousand database queries at once. A worker pool starts a fixed N goroutines that take jobs from the same jobs channel and send their results to a results channel. N is the concurrency limit, usually set by how much the downstream can take (the database connection pool size, the other side's rate limit).

producer then close(jobs) jobs worker 1 worker 2 worker 3 results collect after wg.Wait() close(results) Number of workers = concurrency limit, set by what the downstream can take

Figure: jobs is closed by the producer; results is closed only after all workers have finished, so the collector's range can end normally.

Below is a complete worker pool. The key is "who closes which channel": jobs is closed by its only sender (the producer) once it has sent everything; results has several senders (all the workers), so no single worker can close it; instead a separate goroutine closes it after all the workers have finished.

package main

import (
	"fmt"
	"sync"
	"time"
)

type Order struct {
	ID     string
	UserID string
	Total  int64 // in minor units
	Status string
}

type result struct {
	OrderID string
	Total   int64
	Err     error
}

// recalc simulates recalculating the amount of one order.
func recalc(o Order) (int64, error) {
	time.Sleep(20 * time.Millisecond)
	if o.Status == "cancelled" {
		return 0, fmt.Errorf("recalc %s: order is cancelled", o.ID)
	}
	return o.Total * 105 / 100, nil
}

func main() {
	orders := make([]Order, 12)
	for i := range 12 { // since Go 1.22 you can range over an integer directly
		orders[i] = Order{ID: fmt.Sprintf("o-%02d", i), UserID: "u-1", Total: int64(1000 * (i + 1)), Status: "pending"}
	}
	orders[5].Status = "cancelled"

	const workers = 3
	jobs := make(chan Order)
	results := make(chan result)

	// Producer: the only sender, closes jobs once it has sent everything.
	go func() {
		defer close(jobs)
		for _, o := range orders {
			jobs <- o
		}
	}()

	// N workers: take jobs from jobs until it is closed.
	var wg sync.WaitGroup
	for range workers {
		wg.Go(func() {
			for o := range jobs {
				total, err := recalc(o)
				results <- result{OrderID: o.ID, Total: total, Err: err}
			}
		})
	}

	// results has several senders: close it after all workers have finished.
	go func() {
		wg.Wait()
		close(results)
	}()

	// Collector: range ends once results is closed.
	var ok, failed int
	for r := range results {
		if r.Err != nil {
			failed++
			fmt.Println("error:", r.Err)
			continue
		}
		ok++
	}
	fmt.Printf("done: %d ok, %d failed\n", ok, failed)
}

The order of the results differs from the input, which is normal for concurrent processing. If you need to keep the order, write into a preallocated slice by index as in Step 1, or carry the original index in the result and sort afterwards.

Pipeline: a series of stages connected by channels

A pipeline splits processing into several stages, each a group of goroutines that read from an upstream channel, process, and write to a downstream channel. For example "read orders → calculate amounts → write back to the database". The benefits are that each stage can have a different degree of concurrency (calculation is CPU-bound, so use fewer; writing back is I/O-bound, so use more), and that data flows through like water without waiting for the previous stage to finish completely.

Every stage of a pipeline follows the same rule: a stage owns its output channel and closes it once its input is drained. That way a close upstream propagates all the way downstream, and the collector's range ends naturally. Starting several goroutines within a stage to read the same input is called fan-out; merging several channels into one is called fan-in, and it works just like how results is closed above: one goroutine per source forwards values, and a WaitGroup waits for all of them before the merged channel is closed.

Step 6: Limit concurrency and collect errors

A worker pool suits "a large batch of uniform jobs". Another, more common situation is this: while handling a request, you need to call anything from a few to dozens of downstream services concurrently, limit how many run at once, and know if any of them fails.

Using a buffered channel as a semaphore

The capacity of a buffered channel can serve as a semaphore: send a value to take a slot before entering, block when the buffer is full, and take a value out to release the slot when done. It is lighter than a worker pool: there is no need to start goroutines in advance, and each job is still its own goroutine, but at most N enter the critical section at once.

Collecting all the errors

Error collection can be combined with errors.Join from the previous article: each goroutine writes its error into its own slot in a slice, and they are all merged at the end.

package main

import (
	"errors"
	"fmt"
	"sync"
	"time"
)

// reserveStock simulates calling the inventory service to reserve an item.
func reserveStock(sku string) error {
	time.Sleep(30 * time.Millisecond)
	if sku == "sold-out" {
		return fmt.Errorf("reserve %s: insufficient stock", sku)
	}
	return nil
}

// reserveAll runs at most limit reservations at once and returns every failure.
func reserveAll(skus []string, limit int) error {
	sem := make(chan struct{}, limit) // capacity is the concurrency limit
	errs := make([]error, len(skus))  // each writes its own slot, no lock needed

	var wg sync.WaitGroup
	for i, sku := range skus {
		wg.Go(func() {
			sem <- struct{}{}        // take a slot; wait if full
			defer func() { <-sem }() // release
			errs[i] = reserveStock(sku)
		})
	}
	wg.Wait()
	return errors.Join(errs...) // nil when everything succeeded
}

func main() {
	skus := []string{"book", "pen", "sold-out", "lamp", "mug"}
	start := time.Now()
	err := reserveAll(skus, 2)
	fmt.Printf("took about %v\n", time.Since(start).Round(10*time.Millisecond))
	if err != nil {
		fmt.Println("partial failure:", err)
	}
}

This version waits for every job to finish, even if one failed long ago. If you want "tell the other jobs to stop as soon as the first error happens", you need a cancellation signal that can be broadcast across goroutines, which is exactly the topic of the next article, context. The official extension package golang.org/x/sync/errgroup wraps "WaitGroup + first error + cancellation + concurrency limit (SetLimit)" into one type and is the most common approach in practice; once you understand the hand-made version in this step, you know clearly what it is doing for you.

Which pattern to choose

SituationRecommended pattern
A fixed few independent lookups whose results are combinedWaitGroup.Go + writing into a slice by index
Many uniform jobs that need a fixed degree of concurrencyworker pool
Multi-stage processing where stages run at different speedspipeline (fan-out / fan-in)
Concurrent downstream calls within a request, needing a limit and errorssemaphore + errors.Join, or errgroup with SetLimit
A shared counter or cacheatomic or Mutex, not a channel

Step 7: Avoid goroutine leaks

Goroutines are cheap, but not free: each one occupies a stack, and none of the objects it references can be garbage-collected. A goroutine that blocks forever is a leak. If every request leaks one, the service runs out of memory after a few hours or days, and it is hard to see the problem from any single request.

The most typical leak looks like this: start a goroutine to do a lookup, return the result over an unbuffered channel, and have the caller select on the result or a timeout. When the timeout fires, the caller leaves and nobody receives any more; the lookup goroutine finishes and gets stuck on ch <- res, never ending. There are two fixes: give the channel a capacity of 1 so the sender can always send and then finish; or make the lookup itself cancellable (the next article).

Principles for preventing leaks:

  1. Every goroutine must have a clear exit condition: its input channel is closed, it receives a cancellation signal, or its work is done. When you start a goroutine, you should be able to answer "when does it end?"
  2. The sender must consider that the receiver may have gone away. Use a buffer of sufficient capacity, or listen for the cancellation signal in the same select.
  3. When the service shuts down, wait for background goroutines to finish rather than exiting immediately, or in-progress work is interrupted. “Configuration and Graceful Shutdown in Go” implements this in full.

For checking, you can compare runtime.NumGoroutine() at the end of a test; for a live service, use the goroutine profile from net/http/pprof to see which line every current goroutine is stuck on (“Performance Tuning in Go: pprof and Benchmarks” introduces this). Since Go 1.25, the standard library's testing/synctest provides a "virtual time" test bubble that lets you test concurrent code with timeouts and timers without really waiting, and detects goroutines that are still blocked when the bubble ends.

Common pitfalls

  • Forgetting to wait for goroutines: main or the handler returns first, and the goroutine's work is interrupted or its result lost. Every go needs a matching way to wait.
  • Calling wg.Add inside the goroutine: Wait may return before Add. Use wg.Go, or call Add before starting the goroutine.
  • "Confirming there's no race" by anything other than -race: getting the right result a hundred times doesn't mean there is no data race. Always add -race in CI.
  • Concurrent reads and writes on a map: the built-in map is not safe for concurrent use; when the runtime detects it, it terminates the program with a fatal error that cannot be recovered. Protect it with a Mutex, or use sync.Map in specific read-heavy situations.
  • Closing a channel from the receiver, or closing it twice: both panic. Only the single sender closes; with several senders, use a WaitGroup to wait for them all and then have one goroutine close it.
  • The sender blocking forever after the receiver has gone: a classic goroutine leak. Use a buffer of capacity 1 or a cancellation signal.
  • Starting unlimited goroutines to hit a downstream service: this exhausts the connection pool or triggers the other side's rate limiting. Limit concurrency with a worker pool or a semaphore.
  • Doing I/O while holding a lock: all the other goroutines queue up behind one network call. Locks only protect in-memory state; move I/O outside the lock.
  • Assuming channels are always better than a Mutex: for a simple counter or cache, a Mutex or atomic is simpler and faster; channels suit handing over ownership and coordinating flow.

Key takeaways

  • Goroutines are lightweight, but every go needs an answer to "who waits for it, and when does it end"; sync.WaitGroup.Go (Go 1.25) makes waiting harder to get wrong.
  • Since Go 1.22, loop variables are new on every iteration and closures can use them directly; for i := range n iterates over an integer directly.
  • When several goroutines access the same memory and there are writes, you must synchronise; find data races with go test -race, fix them with a Mutex, atomic or channel, and keep "check then act" under the same lock.
  • An unbuffered channel makes both sides meet, while a buffered channel only absorbs brief differences in speed; only the sender closes, only once, and remember the behaviour table for nil and closed channels.
  • select is for timeouts, non-blocking operations and stop signals; in new code, stop signals should use context instead.
  • A worker pool fixes the degree of concurrency, and a pipeline chains several stages; with several senders, wait for them all with a WaitGroup before closing the output.
  • A buffered channel can act as a semaphore to limit concurrency, combined with errors.Join to collect all errors; in practice this is usually wrapped by errgroup.
  • Goroutine leaks come from sends or receives that block forever; every goroutine needs an exit condition, and you can check with the goroutine profile and testing/synctest.

Next

The concurrent code in this article is still missing a piece: when a request is cancelled by the client, or one of the downstream services has already failed, there is no standard way to tell the other goroutines "you can stop now", nor any way to set a time limit on the whole call chain. The next article, “Go context: Cancellation, Timeouts and Request-Scoped Values”, uses context to carry cancellation, timeouts and request-scoped values from the HTTP handler all the way to every goroutine and downstream call.