Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Post an Article
Welcome To Ask or Share your Answers For Others

Categories

routing - how to terminate ssl at ingress-gateway in istio?

0 votes
380 views
in Technique[技术] by (71.8m points)

routing - how to terminate ssl at ingress-gateway in istio?

I am trying to experiment ssl connection in istio ingress gateway.

From here istio ssl gateway without termination, i assume that istio ingress gateway by default should terminate ssl.

I have installed istio with demo profile, via istioctl. I have also installed my service svc1.

Apart from these, below are what my resources are with routng logic:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: vs-gateway
  namespace: myns
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vs
  namespace: myns
spec:
  hosts:
  - "*"
  gateways:
  - vs-gateway
  http:
  - match:
    - uri:
        prefix: "/svc1/"
    rewrite:
      uri: "/"

    route:
    - destination:
        host: svc1
        port:
          number: 80

I found the Gateway url via this. For experiment i had also enabled http on ingress gateway. That way curl http://172.17.0.2:<http_node_port> works with 200 response. Later i removed http from ingress gateway and kept only https (as https is my primary goal to be recieved by ingress gateway)

And then trying curl with https on gateway url. I get 503.

$ curl -ivk https://172.17.0.2:<https_node_port>/svc1/user
*   Trying 172.17.0.2...
* TCP_NODELAY set
* Connected to 172.17.0.2 (172.17.0.2) port 30278 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=CA; O=Abc; CN=example.com
*  start date: Dec 31 08:22:32 2019 GMT
*  expire date: Jan 30 08:22:32 2020 GMT
*  issuer: C=US; ST=CA; O=Abc; CN=example.com
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x55c961626580)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET /svc1/user HTTP/2
> Host: 172.17.0.2:30278
> User-Agent: curl/7.58.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/2 503 
HTTP/2 503 
< content-length: 95
content-length: 95
< content-type: text/plain
content-type: text/plain
< date: Thu, 02 Jan 2020 08:13:49 GMT
date: Thu, 02 Jan 2020 08:13:49 GMT
< server: istio-envoy
server: istio-envoy

< 
* Connection #0 to host 172.17.0.2 left intact
upstream connect error or disconnect/reset before headers. reset reason: connection termination

I also enabled sidecar istio proxy debuggin on svc pod sidecar. I got below logs

[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/extensions/filters/listener/original_dst/original_dst.cc:18] original_dst: New connection accepted
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/extensions/filters/listener/tls_inspector/tls_inspector.cc:72] tls inspector: new connection accepted
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:30] Called tcp filter: Filter
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:40] Called tcp filter: initializeReadFilterCallbacks
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/common/tcp_proxy/tcp_proxy.cc:204] [C88] new tcp proxy session
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:133] [C88] Called tcp filter onNewConnection: remote 10.244.0.5:34148, local 10.244.0.16:3000
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/common/tcp_proxy/tcp_proxy.cc:347] [C88] Creating connection to cluster inbound|80|serviceport|svc1.myns.svc.cluster.local
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:83] creating a new connection
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:364] [C89] connecting
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:711] [C89] connecting to 127.0.0.1:3000
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:720] [C89] connection in progress
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:109] queueing request due to no available connections
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:333] [C88] new connection
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:559] [C89] connected
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:285] [C89] assigning connection
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][filter] [external/envoy/source/common/tcp_proxy/tcp_proxy.cc:541] TCP:onUpstreamEvent(), requestedServerName: 
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:143] Called tcp filter completeCheck: OK
[Envoy (Epoch 0)] [2020-01-02 06:53:19.395][28][debug][filter] [src/istio/control/client_context_base.cc:139] Report attributes: attributes {
  key: "connection.event"
  value {
    string_value: "open"
  }
}
attributes {
  key: "connection.id"
  value {
    string_value: "38a9b348-1730-4e0b-9664-fbbaeedd9215-88"
  }
[Envoy (Epoch 0)] [2020-01-02 06:53:19.395][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:100] [C88] Called tcp filter onRead bytes: 664
[Envoy (Epoch 0)] [2020-01-02 06:53:19.396][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:123] [C88] Called tcp filter onWrite bytes: 28
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:123] [C88] Called tcp filter onWrite bytes: 0
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:100] [C88] Called tcp filter onRead bytes: 34
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:527] [C88] remote close
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:193] [C88] closing socket: 0
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:174] [C88] Called tcp filter onEvent: 0 upstream 127.0.0.1:3000
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/istio/control/client_context_base.cc:139] Report attributes: attributes {
  key: "connection.duration"
  value {
    duration_value {
      nanos: 6151000
    }
  }
}
attributes {
  key: "connection.event"
  value {
    string_value: "close"
  }
}
at
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:104] [C89] closing data_to_write=34 type=0
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:88] [C88] adding to cleanup list
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:35] Called tcp filter : ~Filter
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:589] [C89] write flush complete
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:193] [C89] closing socket: 1
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:124] [C89] client disconnected
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:238] [C89] connection destroyed

With the logs, it seems like ingress gateway is forwarding ssl request to svc. (please correct here if i am wrong)

So in the end, can anyone please help me out getting ssl terminated at ingress gateway, and forward plain http request to svc?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Based on this github issue and this istio documentation

Named service ports: Service ports must be named. The port name key/value pairs must have the following syntax: name: [-]. See Protocol Selection for more details.

Based on mockserver service 

ports:
    - name: serviceport

I would recommend to change it to http/https like mentioned there and confirmed by community member who had same issue on github.

Manual protocol selection

Protocols can be specified manually by naming the Service port name: [-]. The following protocols are supported:

  • grpc
  • grpc
  • web
  • http
  • http2
  • https
  • mongo
  • mysql*
  • redis*
  • tcp
  • tls
  • udp

*These protocols are disabled by default to avoid accidentally enabling experimental features. To enable them, configure the corresponding Pilot environment variables.


与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界，一个免费开放的程序员编程交流平台！开放，进步，分享！让技术改变生活，让极客改变未来！ Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

Just Browsing Browsing

1.4m articles

1.4m replys

5 comments

57.0k users

Most popular tags

Snow Theme by Q2A Market
Powered by Question2Answer
...