TT Lab
Get started
Learn Learning paths Courses

Istio Service Mesh

What an extension point buys you and what it costs

Continue in TT Lab

In one line

EnvoyFilter is not an Istio abstraction but an escape hatch that directly touches Envoy's internal structure, so it is weak against version upgrades, and WasmPlugin lets you write the same demand in a form less tied to versions, but it does not substitute for every case.

Why an escape hatch was needed

Istio's traffic API is a well-chosen abstraction. Where to send, how long to wait, how many times to retry — most demands are expressed in this language. But if you use a mesh for a long time, demands that are not in this language are sure to come. Things like "attach an internal tracing header to every response," "put a rate limit only in front of this workload," and "add one more business field to the access log."

The proxy itself (Envoy) can do all of these. What is lacking is Istio's language. EnvoyFilter is the door left open to fill that gap — you directly write where and what to paint over in the Envoy configuration that Istio generated.

How it works

One EnvoyFilter is a list of patches, and each patch decides four things.

Place What it decides Example values
applyTo Which kind of Envoy configuration to touch HTTP_FILTER, NETWORK_FILTER, CLUSTER, LISTENER
match.context Which direction's configuration SIDECAR_INBOUND, SIDECAR_OUTBOUND, GATEWAY, ANY
patch.operation How to paint over MERGE, ADD, REMOVE, INSERT_BEFORE, INSERT_FIRST
priority The order in which several patches are applied An integer. The smaller, the earlier

All four values are enumerations, so the API server enforces them. If you put in a wrong value, the object is not created and the list of allowed values comes straight back — this is good news. The bad news is that even if all the values are right, there is no guarantee the configuration takes effect. match is a condition that finds the target inside the Envoy configuration, and if it cannot find it, nothing happens at all. No error, no event. If a version upgrade changes a filter name or the chain structure, this is exactly the state you get.

There is one more layer here. An operation that uses another patch as its reference, like INSERT_BEFORE, is meaningful only when the order is fixed, and if you do not write priority, the order is not guaranteed. istioctl analyze catches this case as IST0151. The same tool also points out the case where only a filter name is written without typed_config — writing only the name is an old notation and a place that will change in the future.

Scope matters too. An EnvoyFilter applies only to the namespace it is placed in, but if you put it in the root namespace (by default istio-system), it applies to the whole mesh. If you attach a workloadSelector there, it narrows to the workloads whose labels match. The size of an incident is decided by this one choice.

WasmPlugin writes the same demand differently. Instead of Envoy's internal structure, it says only what to insert at which stage — phase is only four values, AUTHN, AUTHZ, STATS, and UNSPECIFIED_PHASE, and the code is deployed separately as an OCI image. It does not reference the internal structure, so it is much more resilient to version upgrades. In exchange, there are things it cannot do — changing the cluster configuration or the listener itself is still the place of EnvoyFilter.

What it looks like in the field

The most common incident is "the EnvoyFilter we put in last year stopped working at some point." Nobody deleted it and the object is still there. It is just that at some point in a version upgrade the match condition began to miss. There are only two ways to catch this in advance — have a checklist that scans the list before the version upgrade, or have a procedure that fetches and checks the real proxy configuration after the version upgrade. If you have neither, it is discovered at the very bottom of the list of suspected causes at the next outage.

The second is an incident caused by setting the scope too wide. There has been a case where an EnvoyFilter a team made to attach a header to their own service went into the root namespace and applied to every proxy in the mesh. The reason it does not stand out in review is that the manifest is short and looks fine. One line of namespace is itself the blast radius.

Limits of this lab environment

The lab Pod has neither istiod nor a real sidecar. So you cannot see how an EnvoyFilter is reflected in the actual Envoy configuration — comparing before and after a patch with istioctl proxy-config cannot be done in this environment. Instead, there is a real API server, so you can actually confirm schema enforcement, and istioctl analyze also reads the cluster's objects and gives the same verdict. "Whether the declaration is accepted" and "whether the declaration actually takes effect" are different questions, and this lab covers the former and the risk check that comes before it.

What you will do in the next lab

You get three enumerations wrong at once and see what the API server returns, then fix them and actually put it up. You build side by side a place that applies to the whole mesh and a place that applies to one workload and compare the scope, and see how a relative-position patch without a priority gets caught by the analyzer. You write the same demand as a WasmPlugin, and finally build a check script that scans all the EnvoyFilters in the cluster and prints the version-upgrade risks as code.