ip - Why unable to access a service if setting externalTrafficPolicy to Local in a kubernetes cluster

Question

I'm following this guide to apply the source ip feature to my kubernetes cluster.

Firstly, I created a pod by running:

$ kubectl run source-ip-app --image=gcr.io/google_containers/echoserver:1.4

Then expose it as a NodePort service:

kubectl expose deployment source-ip-app --name=nodeport --port=80 --target-port=8080 --type=NodePort

At this point, I'm able to access the service from outside of the cluster and get correct client_address:

$ curl 10.74.68.49:16860 | grep client
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 296 0 296 0 0 43167 0 --:--:-- --:--:-- --:--:-- 49333
client_address=10.168.193.130

But if applying the source ip feature:

kubectl patch svc nodeport -p '{"spec":{"externalTrafficPolicy":"Local"}}'

I'll get timeout:

$ curl 10.74.68.49:16860 | grep client
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:01:14 --:--:-- 0curl: (7) Failed to connect to 10.74.68.49 port 16860: Operation timed out

I'm wondering what's the reason behind this and how to resolve it.

My env info:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.3", GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1", GitTreeState:"clean", BuildDate:"2017-08-03T07:00:21Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.3", GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1", GitTreeState:"clean", BuildDate:"2017-08-03T06:43:48Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Update:

My cluster has 2 nodes, I get the timeout issue no matter which node ip is accessed.

Answer 1

Create kube-proxy.yaml

kubectl get ds -n kube-system kube-proxy -o yaml > kube-proxy.yaml

Edit kube-proxy.yaml to include the HOST_IP argument

# ...
spec:
  containers:
  - command:
    - ./hyperkube
    - proxy
    - --cluster-cidr=10.2.0.0/16
    - --hostname-override=$(HOST_IP)
    - --kubeconfig=/etc/kubernetes/kubeconfig
    - --proxy-mode=iptables
    env:
    - name: HOST_IP
      valueFrom:
          fieldRef:
              apiVersion: v1
              fieldPath: status.hostIP
    #...

Update the pods:

kubectl apply -f kube-proxy.yaml

This will apply the fix mentioned in https://github.com/kubernetes/kubernetes/issues/48437, resolving the dropped packets issue.