Columns › 3. Splitting into microservices › Part 11
Building an API Gateway in Go: Routing, Authentication and Rate Limiting
API Gateway:路由、認證、限流
Build a small API gateway from scratch with Go's standard httputil.ReverseProxy, adding request IDs, API key and JWT checks and token-bucket rate limiting, then learn when an off-the-shelf gateway is the better choice.
In this article
- What this article solves
- Step 1: Routing with ReverseProxy
- Step 2: Adding a request ID to every request
- Step 3: Authentication: API keys and JWT
- Step 4: Rate limiting with a token bucket
- Step 5: Assembling the complete gateway
- Step 6: Multiple gateway instances and distributed rate limiting
- Step 7: When to use an off-the-shelf gateway
- Common pitfalls
- Key takeaways
- Next
In the previous article, “gRPC and Protocol Buffers in Go”, we gave order, payment and inventory type-safe internal communication. This article deals with the system's "front door": where do browsers, mobile apps and external partners come in? Who checks who they are and limits how fast they can call? The answer is an api-gateway placed in front of every service.
What this article solves
Once the system is split into several services, letting external clients call each service directly causes a few problems:
- Clients have to know every service's address, and any internal change forces every client to change.
- Every service has to implement authentication, rate limiting, CORS and request logging itself, which is repetitive and easily inconsistent.
- Internal services are exposed directly to the network, enlarging the attack surface.
An API gateway is a reverse proxy that stands in front of all the services and handles the work that "has nothing to do with the business, but every request needs":
| Responsibility | Description |
|---|---|
| Routing | Forward requests to the matching internal service by path |
| Authentication | Establish who the caller is (API key, JWT and so on) |
| Rate limiting | Limit each caller's request rate to protect the backends |
| Request ID | Generate a unique ID for each request to tie logs together across services |
| Other | TLS termination, CORS, compression, protocol translation (e.g. HTTP/JSON to gRPC) |
The strangler routing layer in “When to Split: Service Boundaries and Decomposition Strategies” is really the most bare-bones gateway. This article extends it into a usable version, using only the standard library, and at the end discusses when to switch to an off-the-shelf product.
Tip: the gateway only does "cross-cutting" work and holds no business logic. As soon as the gateway starts knowing that "orders above a certain amount need an extra check", it becomes a bottleneck that every team has to change and nobody dares to touch.
Step 1: Routing with ReverseProxy
The Go standard library's net/http/httputil.ReverseProxy already handles the most tedious parts of reverse proxying: forwarding request and response bodies, removing hop-by-hop headers that must not be forwarded (such as Connection), and handling streaming responses and connection reuse.
Since Go 1.20, the recommended way to rewrite requests is the Rewrite field rather than the older Director. The difference is that Rewrite has access to both the original request (pr.In) and the outgoing request (pr.Out), and by the time Rewrite is called the hop-by-hop headers have already been removed, so a client cannot use the Connection header to sneakily make the gateway delete headers you added.
The routing table is described with a simple slice:
| Public path | Internal service |
|---|---|
/api/orders/ | order |
/api/payments/ | payment |
/api/inventory/ | inventory |
Each route gets its own ReverseProxy, registered on an http.ServeMux. ServeMux matches paths by "longest prefix", so /api/orders/123 lands on /api/orders/. Inside Rewrite we do three things: pr.SetURL(target) changes the destination to the internal service, pr.SetXForwarded() sets X-Forwarded-For and related headers so the backend knows who the original client was, and we handle the identity header (explained in Step 3).
You must also set an ErrorHandler. When a backend is unreachable or times out, the default behaviour just returns an empty 502; with a custom handler you can log which request and which service had the problem, and give responses a uniform format.
In the example, internal service addresses are written as local ports for now so it runs on a development machine; in deployment they become service names, such as http://order:8080 in Docker Compose or Kubernetes, which is the subject of “Service Discovery and Load Balancing for Go Microservices”.
Step 2: Adding a request ID to every request
A single user action may pass through four services, the gateway, order, inventory and payment, each writing its own logs. When something goes wrong, you need a common key to tie those logs together, and that is the request ID.
The middleware is written the same way as in “net/http and Middleware in Go”: wrap the next handler and do work before and after it. The rules are:
- If the request already carries an
X-Request-ID(for example from an outer load balancer), reuse it; but limit its length so nobody can stuff an extremely long string in to pollute the logs. - Otherwise, generate one with
crypto/rand. - Write it back into the request headers (to pass downstream) and the response headers (so clients can include it when reporting a problem).
- When the request finishes, write one structured log line with
log/slog: ID, method, path and duration.
When a downstream Go service receives it, it puts it into the context and, when making gRPC calls, into the metadata, which is what the interceptor example in the previous article did. Later, in “Observability in Go: Logs, Metrics and Tracing”, this role is taken over by the OpenTelemetry trace ID, but the principle of "generate at the entrance, pass it all the way down" stays the same.
Step 3: Authentication: API keys and JWT
One of the gateway's most important jobs is to establish "who you are" before a request reaches an internal service. The example supports two common methods:
API key: suited to server-to-server integration, such as a partner's system. The client sends a long string in the X-API-Key header, and the gateway looks it up to find the corresponding user. Compare with crypto/subtle.ConstantTimeCompare so attackers cannot guess the key from differences in comparison time. In practice, the database should store only a hash of the key, not the plain text.
JWT (JSON Web Token): suited to requests after a user logs in. A JWT consists of three base64url strings: header, payload (claims) and signature. Once the gateway has verified that the signature is correct and the token has not expired, it trusts the sub (user ID) inside. The example implements HS256 (HMAC-SHA256) verification; the key points are:
- Accept only the expected algorithm. The
algin the header is controlled by the attacker; historically, many vulnerabilities came from verifiers that "verify with whatever algorithm the header says", which were bypassed by changing it tononeor to another algorithm. The example rejects any value other thanHS256outright. - Compare signatures with
hmac.Equal, which is a constant-time comparison. - Check
expand reject expired tokens. Production systems usually also checkiss(issuer),aud(audience) andnbf.
Tip: to stick to the standard library, the example writes its own HS256 verification. For real projects, use a mature JWT library and prefer asymmetric algorithms such as RS256 or ES256: only the authentication service holds the private key, the gateway needs only the public key, and even if the gateway is compromised it cannot issue new tokens.
After successful verification, the gateway puts the user ID into the context, and Rewrite writes it to the backend as an X-User-ID header. There is a crucial security detail here: first delete any X-User-ID the client sent, then write the verified value. Otherwise an attacker only needs to add the line X-User-ID: admin to the request and the backend will believe them. Likewise, backend services should accept traffic only from the gateway (enforced with network policies or mTLS); otherwise someone could bypass the gateway and call the backend directly, and the header could still be forged.
Authentication failures return 401 Unauthorized with a WWW-Authenticate header. Note that "authentication" (who you are) and "authorisation" (what you may do) are two different things: the gateway is a good place for authentication and coarse-grained authorisation (for example, a given API key may only call read-only endpoints), while fine-grained decisions such as "is this order yours" belong in the order service's own business logic.
Step 4: Rate limiting with a token bucket
Rate limiting protects the backends from being overwhelmed by bursts from a single caller, and it is the basis of fair use for a public API. The most common algorithm is the token bucket:
- Each caller has a bucket with capacity
burst. - The bucket is refilled with tokens at a fixed rate
rate, up to full. - Each request has to take one token; if it gets one it is allowed through, otherwise it receives
429 Too Many Requests.
Figure: the bucket is refilled with tokens at a fixed rate and each request takes one; when the bucket is empty the gateway returns 429 and uses Retry-After to tell the client how long to wait before trying again.
The token bucket's advantage is that it controls two things at once: the long-term average rate (rate) and the tolerated instantaneous burst (burst). For example, with 5 per second and a capacity of 10 (the numbers are only illustrative), a client can send 10 requests at once and can then continue only at 5 per second.
In the implementation you don't need a timer that actually drops tokens into the bucket every second. You only need to remember the time of the last update; whenever a request arrives, top the bucket up in one go by "elapsed seconds × rate", then check whether there are enough. This "lazy refill" approach needs just one mutex and a few lines of arithmetic. When no token is available, you can also calculate how long until the next token arrives and put it in the Retry-After header.
Two more details:
- What to rate-limit by: the example limits per authenticated user, so it must come after authentication. Unauthenticated endpoints are often limited per source IP, but bear in mind that many users behind NAT may share the same IP.
- Memory reclamation: with one bucket per user, the map keeps growing. The example uses a background goroutine to periodically delete buckets that have been idle for too long; if a deleted bucket's user reappears, they simply get a fresh full bucket, which has no effect on correctness.
Other common algorithms are the fixed window (the simplest to implement, but it can let through twice the traffic around a window boundary), the sliding window (more accurate, more expensive) and the leaky bucket (fixed output rate, suited to traffic shaping). For API rate limiting, the token bucket is usually the most balanced choice. If you don't mind one more dependency, golang.org/x/time/rate provides a ready-made, well-tested token bucket implementation.
Step 5: Assembling the complete gateway
Putting the previous four steps together, the middleware order is: request ID → authentication → rate limiting → routing. The order has its reasons: the request ID is outermost so that even rejected requests have an ID to look up; authentication comes before rate limiting so we know whose bucket to charge; and blocked requests end at the gateway without wasting backend resources.
Figure: a request passes through request ID, authentication, rate limiting and routing in turn; requests that fail authentication or exceed the rate are stopped at the gateway and never reach the backends.
The complete program follows. It uses only the standard library and can be run directly with go run:
// A small API gateway using only the standard library:
// request ID → authentication (API key or HS256 JWT) → rate limiting (one token bucket per user) → reverse proxy.
package main
import (
"context"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"log/slog"
"math"
"net/http"
"net/http/httputil"
"net/url"
"os"
"strconv"
"strings"
"sync"
"time"
)
// ---------- 1. Routing table ----------
type route struct {
prefix string // public path prefix, e.g. /api/orders/
upstream string // internal service address
}
var routes = []route{
{"/api/orders/", "http://127.0.0.1:9001"},
{"/api/payments/", "http://127.0.0.1:9002"},
{"/api/inventory/", "http://127.0.0.1:9003"},
}
func newProxy(target *url.URL) *httputil.ReverseProxy {
return &httputil.ReverseProxy{
Rewrite: func(pr *httputil.ProxyRequest) {
pr.SetURL(target)
pr.SetXForwarded()
// The Out that Rewrite receives has hop-by-hop headers removed; here we replace the identity with the value the gateway verified.
pr.Out.Header.Del("X-User-ID")
if uid, ok := pr.In.Context().Value(userKey{}).(string); ok {
pr.Out.Header.Set("X-User-ID", uid)
}
},
ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
slog.Error("upstream error", "path", r.URL.Path, "err", err,
"request_id", r.Header.Get("X-Request-ID"))
http.Error(w, "bad gateway", http.StatusBadGateway)
},
}
}
// ---------- 2. Request ID ----------
func withRequestID(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
id := r.Header.Get("X-Request-ID")
if id == "" || len(id) > 64 {
b := make([]byte, 8)
rand.Read(b)
id = hex.EncodeToString(b)
}
r.Header.Set("X-Request-ID", id) // pass downstream
w.Header().Set("X-Request-ID", id)
start := time.Now()
next.ServeHTTP(w, r)
slog.Info("request", "id", id, "method", r.Method, "path", r.URL.Path,
"dur", time.Since(start))
})
}
// ---------- 3. Authentication ----------
type userKey struct{}
var (
jwtSecret = []byte(envOr("JWT_SECRET", "dev-only-secret"))
apiKeys = map[string]string{ // key → user; in practice kept in a database or secrets service
envOr("DEMO_API_KEY", "demo-key-123"): "partner-a",
}
errUnauthorized = errors.New("unauthorized")
)
func authenticate(r *http.Request) (string, error) {
if key := r.Header.Get("X-API-Key"); key != "" {
for k, user := range apiKeys {
if subtle.ConstantTimeCompare([]byte(k), []byte(key)) == 1 {
return user, nil
}
}
return "", errUnauthorized
}
token, ok := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer ")
if !ok {
return "", errUnauthorized
}
return verifyHS256(token, time.Now())
}
// verifyHS256 accepts only alg=HS256 and checks the signature and exp.
func verifyHS256(token string, now time.Time) (string, error) {
parts := strings.Split(token, ".")
if len(parts) != 3 {
return "", errUnauthorized
}
var header struct {
Alg string `json:"alg"`
}
if err := decodeSegment(parts[0], &header); err != nil || header.Alg != "HS256" {
return "", errUnauthorized
}
mac := hmac.New(sha256.New, jwtSecret)
mac.Write([]byte(parts[0] + "." + parts[1]))
sig, err := base64.RawURLEncoding.DecodeString(parts[2])
if err != nil || !hmac.Equal(sig, mac.Sum(nil)) {
return "", errUnauthorized
}
var claims struct {
Sub string `json:"sub"`
Exp int64 `json:"exp"`
}
if err := decodeSegment(parts[1], &claims); err != nil {
return "", errUnauthorized
}
if claims.Sub == "" || now.Unix() >= claims.Exp {
return "", errUnauthorized
}
return claims.Sub, nil
}
func decodeSegment(seg string, v any) error {
b, err := base64.RawURLEncoding.DecodeString(seg)
if err != nil {
return err
}
return json.Unmarshal(b, v)
}
// ---------- 4. Rate limiting: token bucket ----------
type bucket struct {
tokens float64
last time.Time
}
type limiter struct {
mu sync.Mutex
rate float64 // tokens added per second
burst float64 // bucket capacity
buckets map[string]*bucket
}
func newLimiter(rate, burst float64) *limiter {
l := &limiter{rate: rate, burst: burst, buckets: map[string]*bucket{}}
go l.cleanup(time.Minute)
return l
}
// allow reports whether to let the request through and, if not, how long to wait.
func (l *limiter) allow(key string, now time.Time) (bool, time.Duration) {
l.mu.Lock()
defer l.mu.Unlock()
b, ok := l.buckets[key]
if !ok {
b = &bucket{tokens: l.burst, last: now}
l.buckets[key] = b
}
// Refill tokens by elapsed time, up to burst
b.tokens = math.Min(l.burst, b.tokens+now.Sub(b.last).Seconds()*l.rate)
b.last = now
if b.tokens >= 1 {
b.tokens--
return true, 0
}
wait := time.Duration((1 - b.tokens) / l.rate * float64(time.Second))
return false, wait
}
// cleanup removes buckets that have been idle too long so the map doesn't grow without bound.
func (l *limiter) cleanup(every time.Duration) {
for range time.Tick(every) {
l.mu.Lock()
for k, b := range l.buckets {
if time.Since(b.last) > every {
delete(l.buckets, k)
}
}
l.mu.Unlock()
}
}
// ---------- 5. Putting it together ----------
func gateway(lim *limiter) http.Handler {
mux := http.NewServeMux()
for _, rt := range routes {
u, err := url.Parse(rt.upstream)
if err != nil {
panic(err)
}
mux.Handle(rt.prefix, newProxy(u))
}
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) {
w.Write([]byte("ok"))
})
protected := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/healthz" {
mux.ServeHTTP(w, r)
return
}
user, err := authenticate(r)
if err != nil {
w.Header().Set("WWW-Authenticate", `Bearer realm="api"`)
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
if ok, wait := lim.allow(user, time.Now()); !ok {
w.Header().Set("Retry-After", strconv.Itoa(int(math.Ceil(wait.Seconds()))))
http.Error(w, "too many requests", http.StatusTooManyRequests)
return
}
r = r.WithContext(context.WithValue(r.Context(), userKey{}, user))
mux.ServeHTTP(w, r)
})
return withRequestID(protected)
}
func envOr(key, def string) string {
if v := os.Getenv(key); v != "" {
return v
}
return def
}
func main() {
lim := newLimiter(5, 10) // 5 per second, bursts of up to 10 (illustrative numbers)
srv := &http.Server{
Addr: envOr("ADDR", ":8080"),
Handler: gateway(lim),
ReadHeaderTimeout: 5 * time.Second,
IdleTimeout: 60 * time.Second,
}
slog.Info("gateway listening", "addr", srv.Addr)
if err := srv.ListenAndServe(); err != nil {
slog.Error("server stopped", "err", err)
os.Exit(1)
}
}The server itself also needs timeouts: ReadHeaderTimeout stops anyone from tying up a connection by sending headers slowly (a Slowloris attack), and IdleTimeout reclaims idle keep-alive connections. /healthz needs no authentication; it is for health checks by the load balancer or Kubernetes.
To try it locally, first start any HTTP service that responds on port 9001 to stand in for order, then start the gateway and check the various scenarios with curl:
go run . # the gateway listens on :8080
# No credentials: 401
curl -i localhost:8080/api/orders/1
# API key: forwarded to order; the backend receives X-User-ID: partner-a and X-Request-ID
curl -i localhost:8080/api/orders/1 -H "X-API-Key: demo-key-123"
# Forging the identity header doesn't work: the gateway rewrites it to the verified user
curl -i localhost:8080/api/orders/1 -H "X-API-Key: demo-key-123" -H "X-User-ID: admin"
# Call 12 times in a row: the first 10 return 200, then 429 with Retry-After
for i in $(seq 1 12); do
curl -s -o /dev/null -w "%{http_code} " localhost:8080/api/orders/1 -H "X-API-Key: demo-key-123"
done; echo
# A route whose backend isn't running: 502, and the gateway's log records the request_id
curl -i localhost:8080/api/payments/1 -H "X-API-Key: demo-key-123"Every piece of this gateway can be tested with the methods from “Testing in Go: Table-Driven Tests, httptest and Integration Tests”: start a fake backend with httptest.NewServer, point routes at it, then use a table to list the combinations of "credentials or not, expired or not, over the rate or not" and check the status codes and the headers the backend receives. limiter.allow takes a now parameter, so tests can pass in a fake time instead of actually waiting.
Step 6: Multiple gateway instances and distributed rate limiting
The gateway is the entrance for all traffic, so it can't run as a single instance. But with several instances, the in-memory rate limiting above has a problem: each instance keeps its own count, so three instances effectively triple the limit.
Common approaches:
| Approach | Description | Trade-off |
|---|---|---|
| Split the limit across instances | Divide the total limit by the number of instances | Simplest; but inaccurate when the instance count changes or traffic is uneven |
| Sticky routing | Make the same user always land on the same instance | No shared state needed; users are redistributed when instances are added or removed |
| Centralised counting | Keep the count in shared storage such as Redis, deducting tokens atomically or via a script | Accurate; one extra network round trip, and Redis itself becomes a dependency |
| Two-tier rate limiting | A rough local check first, then precise centralised control | Balances performance and accuracy; more complex to implement |
In most cases rate limiting doesn't need to be exact; its purpose is to protect the system, not to bill. Start with "split across instances" and introduce centralised counting only when you really need precision. If you do switch to centralised counting, you also have to decide whether to "allow" or "deny" when Redis is down; usually you allow and raise an alert, so that the rate-limiting component doesn't itself become a single point of failure for the whole system.
The same thinking applies to other state: JWT verification needs no shared state (which is one reason it is popular), but if you need to support "revoke a token immediately", you need a shared denylist, or shorter lifetimes combined with a refresh mechanism.
Step 7: When to use an off-the-shelf gateway
The value of writing your own gateway lies in understanding what it does, and in staying lightweight when the requirements are simple. But production often needs more features:
- Automatic TLS certificate renewal, HTTP/2 and HTTP/3
- Translation between HTTP/JSON and gRPC (for example grpc-gateway or Envoy's transcoding)
- Dynamic configuration: adding routes or adjusting limits without redeploying
- Circuit breaking, retries, canary releases and traffic mirroring
- Complete metrics, tracing and access logs
- An admin interface, a developer portal and an API key application flow
You can write all of these yourself, but each one needs long-term maintenance. Common off-the-shelf choices are Envoy, NGINX, Kong, Traefik and the managed API gateways of the cloud platforms. In Kubernetes, you can describe routes with declarative configuration through the Gateway API, and a controller that implements it (such as Envoy Gateway or Istio) does the actual forwarding:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: shop-api
namespace: shop
spec:
parentRefs:
- name: public-gateway # a Gateway created by the cluster administrator
rules:
- matches:
- path: { type: PathPrefix, value: /api/orders }
backendRefs:
- name: order # Kubernetes Service name
port: 8080
- matches:
- path: { type: PathPrefix, value: /api/payments }
backendRefs:
- name: payment
port: 8080The decision can be quite practical:
| Situation | Recommendation |
|---|---|
| A few services, simple rules, a team that knows Go | A small home-grown gateway is enough and the easiest to customise |
| Need protocol translation, dynamic configuration, advanced traffic control | An off-the-shelf gateway product |
| Already on Kubernetes | The Gateway API or an Ingress controller; keep business-specific checks inside the services |
| Opening an API to many third parties | An API management platform with a developer portal and key management |
You can also mix them: an off-the-shelf gateway handles TLS, routing and coarse-grained rate limiting, while a small Go service handles special authentication logic, connected through an external authorisation mechanism.
Common pitfalls
- Trusting identity headers sent by the client: if you don't first delete the
X-User-IDthe client sent, an attacker can impersonate anyone. Backends must also accept traffic only from the gateway. - Verifying a JWT with the header's
alg: always hard-code the allowed algorithm and rejectnoneand other unexpected values. - Verifying keys with ordinary string comparison: use
subtle.ConstantTimeCompareorhmac.Equalto avoid timing attacks. - Putting business logic in the gateway: the gateway becomes every team's bottleneck, and deployment risk concentrates in one place.
- No timeouts on the gateway: without
ReadHeaderTimeoutit is easy for slow attacks to exhaust connections; backend calls also need timeouts so one slow service can't drag down the whole entrance. - A rate-limiting map that grows without bound: every new user or new IP adds a bucket, and without cleanup you have a memory leak.
- Keeping single-instance rate limiting with multiple instances and thinking it is the total limit: the amount actually allowed is the limit multiplied by the number of instances.
- Running only one gateway instance: it is the entrance for all traffic, so a single point of failure means the whole system fails.
- Returning error details to the client: backend error messages, stack traces or internal addresses must not leak out through the gateway; return a short, uniform error together with the request ID.
Key takeaways
- An API gateway is a reverse proxy standing in front of all services, centralising cross-cutting work such as routing, authentication, rate limiting and request IDs, with no business logic.
httputil.ReverseProxywithRewriteis enough for routing, plus a customErrorHandlerto handle backend failures.- The request ID is generated or reused at the entrance, length-limited, written into request and response headers, and passed all the way downstream.
- Authentication supports API keys (constant-time comparison) and JWT (fixed algorithm, signature verification,
expcheck); after verification, overwrite the identity header and never trust the value the client sent. - The token bucket is implemented with lazy refill, controls both the average rate and the burst size, and responds with 429 and
Retry-Afterwhen rejecting; remember to clean up idle buckets. - Middleware order: request ID → authentication → rate limiting → routing.
- With multiple instances, rethink rate limiting; when requirements grow complex, switch to an off-the-shelf solution such as Envoy, Kong or the Kubernetes Gateway API.
Next
The gateway's routing table currently hard-codes every service's address, but services scale out to multiple instances, get redeployed and change IPs. The next article, “Service Discovery and Load Balancing for Go Microservices”, explains how callers can always find healthy instances and spread requests evenly across them.