kubernetes - Adding custom response headers using Istio's (1.6.0) envoy lua filter

Question

I am running Istio 1.6.0. I wanted to add some custom headers to all the outbound responses originating from my service. So I was trying to use lua envoyfilter to achieve that. However, I don't see my proxy getting properly configured.

The envoy filter config that I'm trying to use is

kind: EnvoyFilter
metadata:
  name: lua-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.http_connection_manager"
            subFilter:
              name: "envoy.router"
    patch:
      operation: INSERT_BEFORE
      value:
       name: envoy.lua
       typed_config:
         "@type": "type.googleapis.com/envoy.config.filter.http.lua.v2.Lua"
         inlineCode: |
            function envoy_on_response(response_handle)
                response_handle:logInfo(" ========= XXXXX ========== ")
                response_handle:headers():add("X-User-Header", "worked")
            end

I do have my ingress-gateway pods running in the istio-system namespace

? kgp -l istio=ingressgateway -n istio-system
NAME                              READY   STATUS    RESTARTS   AGE
ingress-gateway-b4b5cffc9-wz75r   1/1     Running   0          3d12h
ingress-gateway-b4b5cffc9-znx9b   1/1     Running   0          28h

I was hoping that I would see X-User-Header when I curl for my service. Unfortunately, I'm not seeing any custom headers.

I tried checking the proxy-configs of the ingress-gateway pod in the istio-system, and I don't see the envoy.lua configured at all. I'm not sure whether I'm debugging it correctly.

 istioctl proxy-config listener ingress-gateway-b4b5cffc9-wz75r.istio-system  -n istio-system --port 443 -o json | grep "name"
        "name": "0.0.0.0_443",
                        "name": "istio.stats",
                        "name": "envoy.tcp_proxy",
                        "name": "istio.stats",
                        "name": "envoy.tcp_proxy",
                "name": "envoy.listener.tls_inspector",

Please let me know what is that I'm missing or incorrectly configured. Any advice on how to debug further also would be really helpful.

Thank you so much.

Answer 1

As far as I checked on my istio cluster with version 1.6.3 and 1.6.4 your example works just fine. Take a look at below code from my cluster.

I checked it with curl

$ curl -s -I -X HEAD x.x.x.x/
HTTP/1.1 200 OK
server: istio-envoy
date: Mon, 06 Jul 2020 08:35:37 GMT
content-type: text/html
content-length: 13
last-modified: Thu, 02 Jul 2020 12:11:16 GMT
etag: "5efdcee4-d"
accept-ranges: bytes
x-envoy-upstream-service-time: 2
x-user-header: worked

AND

I checked it with config_dump in istio ingress-gateway pod.

I exec there with

kubectl exec -ti istio-ingressgateway-78db9f457d-xfhl7  -n istio-system -- /bin/bash 

Results from config_dump

curl 0:15000/config_dump | grep X-User-Header
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  128k    0  128k    0     0  9162k      0 --:--:-- --:--:-- --:--:-- 9162k
               "inline_code": "function envoy_on_response(response_handle)
    response_handle:logInfo(" ========= XXXXX ========== ")
    response_handle:headers():add("X-User-Header", "worked")
end
"

So as you can see it works, header is added to request and function is active in istio ingress gateway.

---

Could you try to check it again with above curl, check istio ingress-gateway tcp_dump and let me know if it works for you?