Concurrency vs parallelism in Go
These two words are often used interchangeably but mean different things. Concurrency is about structure — designing a program as a collection of independent pieces that can be reasoned about and scheduled separately. Parallelism is about execution — running multiple things at the literal same instant on multiple CPU cores.
Go's concurrency model is built for the former. A concurrent Go program can run correctly on a single CPU core by interleaving goroutines. When multiple cores are available, the Go runtime may run goroutines in parallel automatically. But the programming model — goroutines and channels — is about structuring concurrent logic, regardless of how many cores are present.
Rob Pike summarised this in a famous talk: "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once." You write concurrent Go programs. The runtime decides how much of it runs in parallel based on available hardware.
The key tools for writing concurrent Go programs are:
- Goroutines — lightweight concurrent functions
- Channels — typed communication pipes between goroutines
- select — wait on multiple channel operations simultaneously
- sync.WaitGroup — wait for a group of goroutines to complete
- sync.Mutex — protect shared state with a mutual exclusion lock
- context.Context — propagate cancellation and deadlines across goroutines
Goroutines
A goroutine is a function that runs concurrently with other goroutines in the same address space. You start one with the go keyword followed by a function call. The call returns immediately — the goroutine runs independently in the background:
package main
import (
"fmt"
"time"
)
func say(msg string) {
for i := 0; i < 3; i++ {
fmt.Println(msg, i)
time.Sleep(100 * time.Millisecond)
}
}
func main() {
go say("background") // starts goroutine, returns immediately
say("foreground") // runs in main goroutine
}
Goroutines are not OS threads. They start with a stack of around 2–8 KB (compared to 1–8 MB for a typical OS thread) and the stack grows dynamically as needed up to a configurable maximum (default 1 GB). The Go runtime multiplexes many goroutines onto a small pool of OS threads. You can start hundreds of thousands of goroutines in a single program — something that would be impossible with native threads:
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
// Start 100,000 goroutines with no problem
for i := 0; i < 100_000; i++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
// do work
_ = n * n
}(i)
}
wg.Wait()
fmt.Println("all done")
}
One important nuance: the main goroutine is special. When main() returns, the entire program exits — all other goroutines are killed immediately, regardless of what they were doing. This means you must always synchronise with goroutines you care about before main returns.
Anonymous goroutines and closure capture
func main() {
// Common mistake: loop variable capture
for i := 0; i < 5; i++ {
go func() {
fmt.Println(i) // captures i by reference — all goroutines may print 5
}()
}
// Fix: pass i as a parameter to create a copy
for i := 0; i < 5; i++ {
go func(n int) {
fmt.Println(n) // n is a copy of i at the time of the call
}(i)
}
// (need to wait for goroutines to finish — see WaitGroup section)
}
for loop creates a new variable. In Go 1.21 and earlier, always pass loop variables as function parameters to goroutines to avoid accidental sharing.The Go scheduler (GMP model)
The Go runtime implements its own cooperative/preemptive scheduler on top of OS threads. The model has three main components: G (goroutines), M (machine — OS thread), and P (processor — logical CPU).
Each P has a run queue of goroutines waiting to execute. P binds to an M, and the M runs whatever goroutine P selects from its queue. The number of Ps is set by GOMAXPROCS, which defaults to the number of available CPU cores. Increasing GOMAXPROCS allows more goroutines to run in true parallel:
import (
"fmt"
"runtime"
)
func main() {
// Check how many CPUs Go will use
fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))
// Limit to 1 CPU (useful for testing race conditions)
runtime.GOMAXPROCS(1)
// Print current goroutine count
fmt.Println("Goroutines:", runtime.NumGoroutine())
}
When a goroutine blocks on a system call (like file I/O or a network operation), the Go runtime detaches the M from its P and creates a new M to keep the P busy with other goroutines. This is why Go programs can handle thousands of concurrent I/O operations efficiently — blocked goroutines do not hold up an OS thread.
Channels: communicating between goroutines
Channels are typed conduits for communication between goroutines. They are the mechanism through which goroutines share data safely — instead of sharing memory and protecting it with locks, goroutines pass data through channels, transferring ownership.
func main() {
// Create an unbuffered channel of strings
ch := make(chan string)
// Start a goroutine that sends a value
go func() {
ch <- "hello from goroutine" // send — blocks until receiver is ready
}()
// Receive from the channel (blocks until sender sends)
msg := <-ch
fmt.Println(msg) // hello from goroutine
}
An unbuffered channel has no internal queue. A send (ch <-) blocks until another goroutine is ready to receive (<-ch), and vice versa. This creates a rendezvous — both goroutines must be at the channel simultaneously. This synchronisation guarantee is one of the most powerful properties of unbuffered channels.
Using channels for results
import "net/http"
func fetchURL(url string, ch chan<- string) {
resp, err := http.Get(url)
if err != nil {
ch <- fmt.Sprintf("error: %v", err)
return
}
defer resp.Body.Close()
ch <- fmt.Sprintf("%s -> %d", url, resp.StatusCode)
}
func main() {
urls := []string{
"https://example.com",
"https://httpbin.org/get",
"https://httpbin.org/status/404",
}
ch := make(chan string)
// Start all requests concurrently
for _, url := range urls {
go fetchURL(url, ch)
}
// Collect all results (one receive per goroutine)
for range urls {
fmt.Println(<-ch)
}
}
Ranging over a channel
If a goroutine will send a variable number of values, close the channel when done. The receiver can then range over the channel and will exit the loop automatically when the channel is closed:
func generate(nums ...int) <-chan int {
ch := make(chan int)
go func() {
for _, n := range nums {
ch <- n
}
close(ch) // signal that no more values will be sent
}()
return ch
}
func main() {
for n := range generate(2, 3, 5, 7, 11) {
fmt.Println(n)
}
// range exits automatically when the channel is closed
}
Buffered channels
A buffered channel has an internal queue. Sends do not block as long as there is space in the buffer; receives do not block as long as there are values in the buffer. Buffered channels decouple the timing of senders and receivers:
func main() {
// Buffered channel with capacity 3
ch := make(chan int, 3)
// These three sends do not block — buffer has space
ch <- 1
ch <- 2
ch <- 3
// ch <- 4 // would block — buffer is full
// Receive all three without needing a matching sender goroutine
fmt.Println(<-ch) // 1
fmt.Println(<-ch) // 2
fmt.Println(<-ch) // 3
// Check current length and capacity
fmt.Println(len(ch), cap(ch)) // 0 3
}
Buffered channels as semaphores
A popular use of buffered channels is as a semaphore — limiting how many goroutines can do a thing concurrently. This is the idiomatic Go way to implement a worker pool with concurrency limits:
import (
"fmt"
"sync"
)
func main() {
urls := make([]string, 50)
for i := range urls {
urls[i] = fmt.Sprintf("https://example.com/%d", i)
}
// Semaphore: at most 10 concurrent requests
sem := make(chan struct{}, 10)
var wg sync.WaitGroup
for _, url := range urls {
wg.Add(1)
go func(u string) {
defer wg.Done()
sem <- struct{}{} // acquire slot
defer func() { <-sem }() // release slot
// only 10 goroutines can be here at once
fmt.Println("fetching", u)
}(url)
}
wg.Wait()
}
Note the use of chan struct{} — an empty struct takes zero bytes of memory, making it the idiomatic type for channels used purely for signalling or counting, where the value itself carries no information.
Channel direction in function signatures
Functions can declare that they only send to or only receive from a channel. This is enforced by the compiler and communicates intent clearly:
// chan<- T: send-only channel (function can only send)
func producer(ch chan<- int) {
for i := 0; i < 5; i++ {
ch <- i
}
close(ch) // close is allowed on send-only channels
}
// <-chan T: receive-only channel (function can only receive)
func consumer(ch <-chan int) {
for n := range ch {
fmt.Println("received:", n)
}
}
func main() {
ch := make(chan int, 5)
go producer(ch) // bidirectional chan converts to send-only
consumer(ch) // bidirectional chan converts to receive-only
}
Directional channels are automatically narrowed when passed to functions — a bidirectional chan T can be passed where a chan<- T or <-chan T is expected. The narrowing cannot be reversed: a receive-only channel cannot be used as a send-only channel.
The select statement
select is Go's mechanism for waiting on multiple channel operations simultaneously. It is like a switch statement, but for channels — it blocks until one of its cases can proceed, then executes that case. If multiple cases are ready simultaneously, one is chosen at random:
import (
"fmt"
"time"
)
func main() {
ch1 := make(chan string)
ch2 := make(chan string)
go func() {
time.Sleep(2 * time.Second)
ch1 <- "one"
}()
go func() {
time.Sleep(1 * time.Second)
ch2 <- "two"
}()
// Wait for whichever channel receives first
for i := 0; i < 2; i++ {
select {
case msg := <-ch1:
fmt.Println("from ch1:", msg)
case msg := <-ch2:
fmt.Println("from ch2:", msg)
}
}
}
select with a default case (non-blocking)
A default case in a select runs immediately if no other case is ready. This makes the select non-blocking:
func tryReceive(ch <-chan int) (int, bool) {
select {
case v := <-ch:
return v, true
default:
return 0, false // channel empty — don't block
}
}
select with a timeout
func fetchWithTimeout(url string) (string, error) {
result := make(chan string, 1)
go func() {
// simulate network request
time.Sleep(3 * time.Second)
result <- "response body"
}()
select {
case body := <-result:
return body, nil
case <-time.After(2 * time.Second):
return "", fmt.Errorf("request to %s timed out", url)
}
}
time.After returns a <-chan time.Time that receives a value after the specified duration. Using it in a select case is the standard Go pattern for adding a timeout to any channel operation. For production code that may call this in a loop, prefer time.NewTimer to avoid a minor timer leak.The done channel pattern
A done channel is a channel used purely for signalling — it signals goroutines to stop. When the done channel is closed, all goroutines waiting on it unblock and can exit cleanly. Closing a channel broadcasts to all receivers simultaneously, making it the right tool for cancellation:
func worker(id int, jobs <-chan int, done <-chan struct{}) {
for {
select {
case job, ok := <-jobs:
if !ok {
fmt.Printf("worker %d: jobs channel closed\n", id)
return
}
fmt.Printf("worker %d: processing job %d\n", id, job)
case <-done:
fmt.Printf("worker %d: shutting down\n", id)
return
}
}
}
func main() {
jobs := make(chan int, 10)
done := make(chan struct{})
for i := 1; i <= 3; i++ {
go worker(i, jobs, done)
}
for i := 1; i <= 9; i++ {
jobs <- i
}
close(done) // broadcast shutdown to all workers at once
time.Sleep(100 * time.Millisecond)
}
In modern Go, context.Context has largely replaced bare done channels for cancellation — it provides the same mechanism with built-in deadline and value propagation (see the context section below).
WaitGroup for fan-out
sync.WaitGroup allows the main goroutine (or any coordinator) to wait for a collection of goroutines to finish. The pattern: call Add(1) before each goroutine, Done() (usually deferred) inside each goroutine, and Wait() to block until the counter reaches zero:
import (
"fmt"
"sync"
"time"
)
func processItem(id int, wg *sync.WaitGroup) {
defer wg.Done() // always call Done, even if the function panics
time.Sleep(50 * time.Millisecond)
fmt.Printf("item %d processed\n", id)
}
func main() {
var wg sync.WaitGroup
items := []int{1, 2, 3, 4, 5}
for _, id := range items {
wg.Add(1) // increment before launching goroutine
go processItem(id, &wg)
}
wg.Wait() // blocks until all goroutines call Done()
fmt.Println("all items processed")
}
wg.Add(n) before the goroutine starts — not inside the goroutine. If Add is called inside the goroutine, there is a race condition: Wait might be reached before Add is called, causing Wait to return prematurely.WaitGroup plus channel: collecting results
func main() {
inputs := []int{1, 2, 3, 4, 5}
results := make(chan int, len(inputs))
var wg sync.WaitGroup
// Fan out: process all inputs concurrently
for _, n := range inputs {
wg.Add(1)
go func(x int) {
defer wg.Done()
results <- x * x // square each number
}(n)
}
// Close results when all workers finish
go func() {
wg.Wait()
close(results)
}()
// Fan in: collect all results
var total int
for r := range results {
total += r
}
fmt.Println("sum of squares:", total) // 55
}
Mutex for shared state
Channels are ideal when goroutines pass data to each other. When goroutines need to read and update shared state in place — like a counter or a cache — a sync.Mutex is more appropriate. A Mutex provides mutual exclusion: only one goroutine can hold the lock at a time:
import "sync"
type SafeCounter struct {
mu sync.Mutex
value int
}
func (c *SafeCounter) Inc() {
c.mu.Lock()
defer c.mu.Unlock()
c.value++
}
func (c *SafeCounter) Get() int {
c.mu.Lock()
defer c.mu.Unlock()
return c.value
}
func main() {
counter := &SafeCounter{}
var wg sync.WaitGroup
for i := 0; i < 1000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
counter.Inc()
}()
}
wg.Wait()
fmt.Println(counter.Get()) // always 1000
}
For read-heavy workloads where writes are infrequent, sync.RWMutex allows multiple concurrent readers while ensuring exclusive access for writers:
type Cache struct {
mu sync.RWMutex
items map[string]string
}
func (c *Cache) Get(key string) (string, bool) {
c.mu.RLock() // shared read lock — multiple goroutines can hold simultaneously
defer c.mu.RUnlock()
v, ok := c.items[key]
return v, ok
}
func (c *Cache) Set(key, value string) {
c.mu.Lock() // exclusive write lock
defer c.mu.Unlock()
c.items[key] = value
}
Context cancellation
The context package is the standard way to propagate cancellation signals, deadlines, and request-scoped values across goroutine boundaries. Every function that starts a goroutine or makes a blocking call in production Go code should accept a context.Context as its first parameter:
import (
"context"
"fmt"
"time"
)
func doWork(ctx context.Context, id int) {
for {
select {
case <-ctx.Done():
fmt.Printf("worker %d cancelled: %v\n", id, ctx.Err())
return
default:
fmt.Printf("worker %d: doing work\n", id)
time.Sleep(500 * time.Millisecond)
}
}
}
func main() {
// Context with 2-second timeout — automatically cancelled after 2s
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel() // always call cancel to release resources, even on normal exit
var wg sync.WaitGroup
for i := 1; i <= 3; i++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
doWork(ctx, id)
}(i)
}
wg.Wait()
}
func main() {
// WithCancel gives you a cancel function to call explicitly
ctx, cancel := context.WithCancel(context.Background())
go func() {
time.Sleep(1 * time.Second)
cancel() // cancel from another goroutine
}()
<-ctx.Done() // wait for cancellation
fmt.Println("context cancelled:", ctx.Err())
// context cancelled: context canceled
}
Goroutine leaks — causes and prevention
A goroutine leak is a goroutine that starts but never exits. Leaked goroutines accumulate for the lifetime of the process, consuming memory and CPU. The three most common causes:
1. Blocked forever on a channel send or receive
// LEAK: goroutine blocks on ch forever if main returns before sending
func leaky() {
ch := make(chan int)
go func() {
val := <-ch // blocks forever if nobody sends
fmt.Println(val)
}()
// oops — nobody ever sends to ch
}
// FIX: use a context for a clean exit condition
func fixed(ctx context.Context) {
ch := make(chan int, 1)
go func() {
select {
case val := <-ch:
fmt.Println(val)
case <-ctx.Done():
return // clean exit when context is cancelled
}
}()
}
2. Infinite loop with no exit condition
// LEAK: runs forever
go func() {
for {
process()
}
}()
// FIX: always check for cancellation
go func() {
for {
select {
case <-ctx.Done():
return
default:
process()
}
}
}()
3. Ranging over an unclosed channel
// LEAK: range never exits if sender doesn't close the channel
go func() {
for v := range ch { // blocks forever if ch is never closed
fmt.Println(v)
}
}()
// FIX: always close the channel when the sender is done,
// or use select with a done channel:
go func() {
for {
select {
case v, ok := <-ch:
if !ok {
return
}
fmt.Println(v)
case <-ctx.Done():
return
}
}
}()
To detect leaks in tests, the goleak library checks that no unexpected goroutines are running at the end of a test:
import "go.uber.org/goleak"
func TestMyFunction(t *testing.T) {
defer goleak.VerifyNone(t) // fail if any goroutines leak
// ... run test code ...
}
Production concurrency patterns
Pipeline
func gen(ctx context.Context, nums ...int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for _, n := range nums {
select {
case out <- n:
case <-ctx.Done():
return
}
}
}()
return out
}
func square(ctx context.Context, in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
select {
case out <- n * n:
case <-ctx.Done():
return
}
}
}()
return out
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Wire the pipeline: gen -> square -> square -> print
for n := range square(ctx, square(ctx, gen(ctx, 2, 3, 4))) {
fmt.Println(n) // 16, 81, 256
}
}
Worker pool
func workerPool(numWorkers int, jobs <-chan int, results chan<- int) {
var wg sync.WaitGroup
for i := 0; i < numWorkers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
results <- job * job // process job
}
}()
}
go func() {
wg.Wait()
close(results)
}()
}
func main() {
jobs := make(chan int, 100)
results := make(chan int, 100)
workerPool(5, jobs, results) // 5 concurrent workers
// Send 20 jobs
for i := 1; i <= 20; i++ {
jobs <- i
}
close(jobs)
// Collect results
var total int
for r := range results {
total += r
}
fmt.Println("total:", total)
}
iter and slices packages with map/filter/reduce helpers. For concurrency-specific utilities — fan-in, merge, retry with backoff — the golang.org/x/sync package provides errgroup and semaphore which are production-proven implementations of common patterns.