Exposing Kubernetes apps with Gateway API in GKE
How to make applications running in GKE publicly available through the Gateway API, with managed wildcard TLS certificates and Cloud Armor WAF protection.
Overview
If you don't know what the Gateway API is, here is what the Special Interest Group (SIG) dedicated to Networking in Kubernetes says about it:
Gateway API is an official Kubernetes project focused on L4 and L7 routing in Kubernetes. This project represents the next generation of Kubernetes Ingress, Load Balancing, and Service Mesh APIs. From the outset, it has been designed to be generic, expressive, and role-oriented.
In other words, the Gateway API is a specification that aims to improve and standardize service networking in Kubernetes. This is the future for exposing Kubernetes applications. For more, have a look at Gateway API Introduction.
Next, we will use the Gateway API implementation in GKE to publicly expose some of our applications running in GKE.
Creating the Gateway
Gateway API feature activation
Before creating the Gateway resource, we have to make sure the Gateway API feature is enabled on the GKE cluster. Have a look at Activate the Gateway API on GKE clusters or Verify the Gateway API activation on GKE clusters for that.
For those using the standard GKE cluster, here is the command we use to enable the Gateway API on an existing cluster:
gcloud container clusters update <cluster_name> --gateway-api=standard --location=<cluster_region>
Once activated, we should see Gateway API related CRDs:
$ kubectl api-resources | grep gateway
gatewayclasses gc gateway.networking.k8s.io/v1beta1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1beta1 true Gateway
httproutes gateway.networking.k8s.io/v1beta1 true HTTPRoute
referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant
gcpgatewaypolicies networking.gke.io/v1
GatewayClass selection
We also need to choose a GatewayClass for the Gateway resource we will create. The choosen GatewayClass will determine the available capabilities/features for the Gateway. Have a look at Available Gateway capabilities/features per GatewayClass for details.
Currently, the GatewayClass with the most features is the gke-l7-global-external-managed. Thats the one we are going to use to create the Gateway. Here is a list of the currently available GatewayClass resources inside the GKE cluster:
$ kubectl get gatewayclass
NAME CONTROLLER ACCEPTED AGE
gke-l7-global-external-managed networking.gke.io/gateway True 54m
gke-l7-gxlb networking.gke.io/gateway True 54m
gke-l7-regional-external-managed networking.gke.io/gateway True 54m
gke-l7-rilb networking.gke.io/gateway True 54m
Here is the result of a describe on the gke-l7-global-external-managed GatewyaClass resource:
$ kubectl describe gatewayclass/gke-l7-global-external-managed
Name: gke-l7-global-external-managed
Namespace:
Labels: <none>
Annotations: <none>
API Version: gateway.networking.k8s.io/v1beta1
Kind: GatewayClass
Metadata:
Creation Timestamp: ***
Generation: 1
Resource Version: 27032
UID: 6062d521-6d0f-46f7-bc0d-19d3c968d4f2
Spec:
Controller Name: networking.gke.io/gateway
Description: Preview class for new L7 External Load Balancers.
Status:
Conditions:
Last Transition Time: ***
Message:
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 55m sc-gateway-controller gke-l7-global-external-managed
# Result on older versions
$ kubectl describe gatewayclass/gke-l7-global-external-managed
Name: gke-l7-global-external-managed
Namespace:
Labels: <none>
Annotations: <none>
API Version: gateway.networking.k8s.io/v1beta1
Kind: GatewayClass
Metadata:
Creation Timestamp: *****
Generation: 1
Managed Fields:
API Version: gateway.networking.k8s.io/v1beta1
Fields Type: FieldsV1
fieldsV1:
f:spec:
.:
f:controllerName:
f:description:
Manager: GoogleGKEGatewayController
Operation: Update
Time: *****
API Version: gateway.networking.k8s.io/v1beta1
Fields Type: FieldsV1
fieldsV1:
f:status:
f:conditions:
k:{"type":"Accepted"}:
f:lastTransitionTime:
f:message:
f:observedGeneration:
f:reason:
f:status:
Manager: GoogleGKEGatewayController
Operation: Update
Subresource: status
Time: *****
Resource Version: 585627605
UID: f67bc0b6-5da4-441d-bad6-cdf9a4195f56
Spec:
Controller Name: networking.gke.io/gateway
Description: Preview class for new L7 External Load Balancers.
Status:
Conditions:
Last Transition Time: *****
Message:
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
Events: <none>
The Gateway resource
Now let's create the Gateway. Here is the gateway.yml manifest:
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: testgwapi-global-external-managed
namespace: default
spec:
gatewayClassName: gke-l7-global-external-managed
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All
For a list of all the supported Gateway API fields for the GKE Gateway resource, have a look at Gateway API fields supported by the GKE Gateway resource per GatewayClass.
Lets apply the manifest and see what happens:
$ kubectl apply -f gateway.yml
gateway.gateway.networking.k8s.io/testgwapi-global-external-managed created
$ kubectl get gateway -w
NAME CLASS ADDRESS PROGRAMMED AGE
testgwapi-global-external-managed gke-l7-global-external-managed Unknown 17s
testgwapi-global-external-managed gke-l7-global-external-managed Unknown 82s
testgwapi-global-external-managed gke-l7-global-external-managed 34.111.146.21 True 82s
testgwapi-global-external-managed gke-l7-global-external-managed 34.111.146.21 True 114s
$ kubectl get events
LAST SEEN TYPE REASON OBJECT MESSAGE
2m23s Normal ADD gateway/testgwapi-global-external-managed default/testgwapi-global-external-managed
61s Normal UPDATE gateway/testgwapi-global-external-managed default/testgwapi-global-external-managed
29s Normal SYNC gateway/testgwapi-global-external-managed SYNC on default/testgwapi-global-external-managed was a success
After creating the Gateway, a Google Cloud HTTP/HTTPS Load Balancer has been automatically provisioned by the GKE Gateway Controller. That Load Balancer will be able to route external traffic to the GKE clusters pods according to our Gateway API routing configurations. Note that a random public IP has also been automatically assigned to the Load Balancer. We see that IP in the ADDRESS column of the previous kubectl get gateway command output.
Using static IP on the Gateway
Whithout specifying an already reserved public static IP address to use on the Gateway, one will automatically be reserved and assigned to the Gateway L7 Load Balancer.
The automatically reserved IP address will also be automatically released as soon as the Gateway is deleted. So, if for some reason we want to delete and recreate the Gateway, its IP address may change.
To avoid that, we can use our own reserved public static IP address on the Gateway. The public static IP address can be reserved from the GCP console public IP addresses reservation page or by using the gcloud CLI as follows:
gcloud compute addresses create ADDRESS_NAME --project=GCP_PROJECT_ID --global
To see the IP address, use:
gcloud compute addresses list --filter="name=( 'ADDRESS_NAME' )" --project=GCP_PROJECT_ID
Once the IP reserved, we need to tell the Gateway to use that IP address during or after its creation. For that, we need to specify the gateway.spec.addresses in the Gateway manifest as follows:
kind: Gateway
...
spec:
(...)
addresses:
- type: NamedAddress
value: ADDRESS_NAME
Configuring TLS on the Gateway
Overview
In this section, we will configure TLS on the Gateway for secure communications with clients reaching our applications. We will use the Google Certificate Manager to generate TLS certificates (including wildcard certificates) that will be automatically renewed.
The HTTPS listener
We need a new listener on the Gateway for the HTTPS protocol. Here is the manifest for adding that listener:
kind: Gateway
(...)
spec:
gatewayClassName: gke-l7-global-external-managed
listeners:
(...)
- name: https
protocol: HTTPS
port: 443
allowedRoutes:
namespaces:
from: All
Creating DNS authorization
Before creating the Google-managed TLS certificate with Google Certificate Manager, we need to create a DNS authorization and add the required DNS entry inside the corresponding DNS zone, in order to prove that we own the domain name for which we want to generate a TLS certificate.
Let's create a DNS authorization called testgatewayapi and show its content:
$ gcloud certificate-manager dns-authorizations create testgatewayapi --domain="testgatewayapi.hackerstack.org"
Create request issued for: [testgatewayapi]
Waiting for operation [projects/(...)] to complete...done.
Created dnsAuthorization [testgatewayapi].
$ gcloud certificate-manager dns-authorizations describe testgatewayapi
createTime: '***'
dnsResourceRecord:
data: 8bf4bd32-c3c7-4af1-ac25-7b38e07807f6.10.authorize.certificatemanager.goog.
name: _acme-challenge.testgatewayapi.hackerstack.org.
type: CNAME
domain: testgatewayapi.hackerstack.org
name: projects/***/locations/global/dnsAuthorizations/testgatewayapi
type: FIXED_RECORD
updateTime: '***'
This DNS authorization tells us to create a DNS CNAME record called _acme-challenge.testgatewayapi.hackerstack.org. resolving to 8bf4bd32-c3c7-4af1-ac25-7b38e07807f6.10.authorize.certificatemanager.goog..
After creating the DNS record, let's verify by making a DNS request:
$ dig +short -t CNAME _acme-challenge.testgatewayapi.hackerstack.org
8bf4bd32-c3c7-4af1-ac25-7b38e07807f6.10.authorize.certificatemanager.goog.
Looks good. We are ready to generate the TLS certificate for testgatewayapi.hackerstack.org and *.testgatewayapi.hackerstack.org with the Google Certificate Manager.
We use the DNS autorization for domains ownership verification because it allows the creation of TLS certificates independently of the Gateway resource and supports wildcard certificates. For more about the different domain authorization methods, have a look at Domain authorization or Domain authorization for Google-managed certificates.
Creating the Google-managed TLS certificate
We use the Google Certificate Manager for that.
Let's generate a TLS certificate for testgatewayapi.hackerstack.org and *.testgatewayapi.hackerstack.org domains, using the previously created DNS authorization called testgatewayapi:
$ gcloud certificate-manager certificates create testgatewayapi --domains="*.testgatewayapi.hackerstack.org,testgatewayapi.hackerstack.org" --dns-authorizations=testgatewayapi
Create request issued for: [testgatewayapi]
Waiting for operation [projects/***/locations/global/operations/operation-***] to complete...done.
Created certificate [testgatewayapi].
$ gcloud certificate-manager certificates list
NAME SUBJECT_ALTERNATIVE_NAMES DESCRIPTION SCOPE EXPIRE_TIME CREATE_TIME UPDATE_TIME
testgatewayapi *.testgatewayapi.hackerstack.org *** ***
testgatewayapi.hackerstack.org
To see the state of the certificate generation, we use the following command:
$ gcloud certificate-manager certificates describe testgatewayapi
createTime: '***'
managed:
authorizationAttemptInfo:
- domain: '*.testgatewayapi.hackerstack.org'
state: AUTHORIZING
- domain: testgatewayapi.hackerstack.org
state: AUTHORIZING
dnsAuthorizations:
- projects/***/locations/global/dnsAuthorizations/testgatewayapi
domains:
- '*.testgatewayapi.hackerstack.org'
- testgatewayapi.hackerstack.org
state: PROVISIONING
name: projects/***/locations/global/certificates/testgatewayapi
sanDnsnames:
- '*.testgatewayapi.hackerstack.org'
- testgatewayapi.hackerstack.org
updateTime: '***'
Once the certificate generated (took ~ 6 minutes in this case), we should see something like this:
createTime: '***'
expireTime: '***'
managed:
authorizationAttemptInfo:
- domain: '*.testgatewayapi.hackerstack.org'
state: AUTHORIZED
- domain: testgatewayapi.hackerstack.org
state: AUTHORIZED
dnsAuthorizations:
- projects/***/locations/global/dnsAuthorizations/testgatewayapi
domains:
- '*.testgatewayapi.hackerstack.org'
- testgatewayapi.hackerstack.org
state: ACTIVE
name: projects/***/locations/global/certificates/testgatewayapi
pemCertificate: |
-----BEGIN CERTIFICATE-----
MIIFpDCCBIygAwIBAgIQQP/U4oP/gZASSvJLrgPhTDANBgkqhkiG9w0BAQsFADBG
MQswCQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZpY2VzIExM
(...)
IkoroZytqlFE5vKpJfN0tvxsS877ZnXhEtcAjvlqzRoeq2CjmecBl85009LAQd0f
az2mOclaqYlrxP2APwJIvJLoTJ8Y5TsGiPcHcGFcSQEEyaoAx9vUBpd/Dwvxn5Bp
VYomKUfHZyw=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFjDCCA3SgAwIBAgINAgCOsgIzNmWLZM3bmzANBgkqhkiG9w0BAQsFADBHMQsw
CQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZpY2VzIExMQzEU
(...)
JDwRjW/656r0KVB02xHRKvm2ZKI03TglLIpmVCK3kBKkKNpBNkFt8rhafcCKOb9J
x/9tpNFlQTl7B39rJlJWkR17QnZqVptFePFORoZmFzM=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIFYjCCBEqgAwIBAgIQd70NbNs2+RrqIQ/E8FjTDTANBgkqhkiG9w0BAQsFADBX
MQswCQYDVQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEQMA4GA1UE
(...)
+qduBmpvvYuR7hZL6Dupszfnw0Skfths18dG9ZKb59UhvmaSGZRVbNQpsg3BZlvi
d0lIKO2d1xozclOzgjXPYovJJIultzkMu34qQb9Sz/yilrbCgj8=
-----END CERTIFICATE-----
sanDnsnames:
- '*.testgatewayapi.hackerstack.org'
- testgatewayapi.hackerstack.org
updateTime: '***'
If the certificate generation fails because of an issue with the DNS authorization, we should see something like this:
createTime: '***'
managed:
authorizationAttemptInfo:
- domain: testgatewayapi.hackerstack.org
failureReason: CONFIG
state: FAILED
dnsAuthorizations:
- projects/***/locations/global/dnsAuthorizations/testgatewayapi
domains:
- testgatewayapi.hackerstack.org
provisioningIssue:
reason: AUTHORIZATION_ISSUE
state: PROVISIONING
name: projects/***/locations/global/certificates/testgatewayapi
sanDnsnames:
- testgatewayapi.hackerstack.org
updateTime: '***'
Have a look at Managing Google Certificate Managers certificates for more.
Adding the TLS certificate inside a certificate map
We need a certificate map to store one or many of our certificates, Google-managed or not, in order to have a unique place from where the Gateway will retrieve certificates for our applications using the HTTPS protocol.
Let's create and show a certificate map called testgatewayapi:
$ gcloud certificate-manager maps create testgatewayapi
Waiting for 'operation-***' to complete...done.
Created certificate map [testgatewayapi].
$ gcloud certificate-manager maps list
NAME ENDPOINTS DESCRIPTION CREATE_TIME
testgatewayapi - ***
$ gcloud certificate-manager maps describe testgatewayapi
createTime: '***'
name: projects/***/locations/global/certificateMaps/testgatewayapi
updateTime: '***'
To add certificates into the certificate map for specific domain names, we need to create certificate map entries as follows:
$ gcloud certificate-manager maps entries create testgatewayapi1 --map="testgatewayapi" --certificates="testgatewayapi" --hostname="testgatewayapi.hackerstack.org"
Waiting for 'operation-***' to complete...done.
Created certificate map entry [testgatewayapi1].
$ gcloud certificate-manager maps entries create testgatewayapi2 --map="testgatewayapi" --certificates="testgatewayapi" --hostname="*.testgatewayapi.hackerstack.org"
Waiting for 'operation-***' to complete...done.
Created certificate map entry [testgatewayapi2].
We have created two certificate map entries called testgatewayapi1 and testgatewayapi2 referencing TLS certificates respectively for the testgatewayapi.hackerstack.org and *.testgatewayapi.hackerstack.org domains.
$ gcloud certificate-manager maps entries list --map="testgatewayapi"
NAME DESCRIPTION HOSTNAME MATCHER CERTIFICATES STATE CREATE_TIME
testgatewayapi1 testgatewayapi.hackerstack.org testgatewayapi ACTIVE ***
testgatewayapi2 *.testgatewayapi.hackerstack.org testgatewayapi ACTIVE ***
Next, we will tell the Gateway to use the testgatewayapi certificate map as its TLS certificates store. Have a look at Managing Certificate Map and Managing Certificate Map Entries for more.
Using the TLS certificate on the Gateway
To tell the Gateway which certificate map to use for finding certificates for our applications domain names that are reachable through the HTTPS listener, we use the networking.gke.io/certmap key inside the Gateway resources metadata.annotations field as follows:
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
(...)
annotations:
networking.gke.io/certmap: testgatewayapi # Name of the certificate map
spec:
(...)
After that, when the Gateway receives HTTPS requests for one of the domain names present inside the certificate map, it will look at the certificate map entry for that domain to get the name of the certificate to use for setting up the TLS sessions.
Creating HTTPRoutes
The HTTPRoute resource can be used to route HTTP or HTTPS traffic to our Kubernetes applications workloads through the Gateway.
On the Gateway we have created previously, we have used the gateway.spec.allowedRoutes field to authorize HTTPRoute resources from any namespace of the cluster, to use the Gateway to route traffic to backend applications.
Here is the manifest for an example application running inside the cluster we are going to expose through the Gateway:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: default
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
labels:
app: nginx
spec:
ports:
- port: 80
targetPort: 80
name: http
selector:
app: nginx
type: ClusterIP
Here is the manifest we use for creating the HTTPRoute resource for exposing the application through the Gateway:
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: nginx-https
namespace: default
spec:
parentRefs:
- kind: Gateway
name: testgwapi-global-external-managed # Name of the Gateway to use
namespace: default
sectionName: https # Which Gateway listener to use
hostnames:
- testgatewayapi.hackerstack.org
rules:
- backendRefs:
- name: nginx # Name of the backend service on which traffic will be routed
kind: Service
namespace: default # Namespace where the backend service resides
port: 80
weight: 100
For infos about HTTPRoute resources fields we can use and detailed explanation about each of them, have a look at Gateway API fields supported by the GKE HTTPRoutes resource per gatewayclass.
Let's have a look at the status of the HTTPRoute resource:
$ kubectl get httproute
NAME HOSTNAMES AGE
nginx-https ["testgatewayapi.hackerstack.org"] 35m
$ kubectl describe httproute/nginx-https
Name: nginx-https
Namespace: default
Labels: <none>
Annotations: <none>
API Version: gateway.networking.k8s.io/v1beta1
Kind: HTTPRoute
Metadata:
Creation Timestamp: ***
Generation: 2
Resource Version: 93649
UID: ef84cf65-37d7-4b32-94ec-e4926596c0ee
Spec:
Hostnames:
testgatewayapi.hackerstack.org
Parent Refs:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: testgwapi-global-external-managed
Namespace: default
Section Name: https
Rules:
Backend Refs:
Group:
Kind: Service
Name: nginx
Namespace: default
Port: 80
Weight: 100
Matches:
Path:
Type: PathPrefix
Value: /
Status:
Parents:
Conditions:
Last Transition Time: ***
Message:
Observed Generation: 2
Reason: Accepted
Status: True
Type: Accepted
Last Transition Time: ***
Message:
Observed Generation: 2
Reason: ReconciliationSucceeded
Status: True
Type: Reconciled
Controller Name: networking.gke.io/gateway
Parent Ref:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: testgwapi-global-external-managed
Namespace: default
Section Name: https
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 17m sc-gateway-controller default/nginx-https
Normal UPDATE 2m1s sc-gateway-controller default/nginx-https
Normal SYNC 9s sc-gateway-controller Bind of HTTPRoute "default/nginx-https" to ParentRef {Group: "gateway.networking.k8s.io",
Kind: "Gateway",
Namespace: "default",
Name: "testgwapi-global-external-managed",
SectionName: "https",
Port: nil} was a success
Normal SYNC 9s sc-gateway-controller Reconciliation of HTTPRoute "default/nginx-https" bound to ParentRef {Group: "gateway.networking.k8s.io",
Kind: "Gateway",
Namespace: "default",
Name: "testgwapi-global-external-managed",
SectionName: "https",
Port: nil} was a success
The nginx-https HTTPRoute resource has been successfully bound to the Gateway. Let's try to reach the nginx application through the Gateway by making an HTTPS request:
$ curl -v https://testgatewayapi.hackerstack.org
* Trying 34.111.146.21:443...
* TCP_NODELAY set
* Connected to testgatewayapi.hackerstack.org (34.111.146.21) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.testgatewayapi.hackerstack.org
* start date: ***
* expire date: ***
* subjectAltName: host "testgatewayapi.hackerstack.org" matched cert's "testgatewayapi.hackerstack.org"
* issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1D4
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x559aaa0680e0)
> GET / HTTP/2
> Host: testgatewayapi.hackerstack.org
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< server: nginx/1.25.4
< date: ***
< content-type: text/html
< content-length: 615
< last-modified: ***
< etag: "65cce434-267"
< accept-ranges: bytes
< via: 1.1 google
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
<
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
* Connection #0 to host testgatewayapi.hackerstack.org left intact
We have successfully reached our nginx application through the Gateway by using the HTTPS protocol.
Protecting the Gateway backends with Cloud Armor
Overview
The Google Cloud Armor Web Application Firewall (WAF) can be used to protect applications exposed through the Gateway API. Have a look at Additional Google Cloud services for compatibility with the Gateway depending on the GatewayClass used.
To use the WAF to protect the Gateway backends applications we need to:
- create one or more security policies containing security policies rules
- tell the Gateway to use a specific security policy to protect a specific backend service by creating a GCPBackendPolicy resource inside the GKE cluster
Next we will expose the OWASP Juiceshop vulnerable web application through the Gateway we have previously created, perform a successful SQL injection attack on the login page and then protect that app against SQL injection attacks by using one of the Google Cloud Armor preconfigured WAF rules. We will then create a custom Cloud Armor policy rule to block specific requests.
Using the OWASP Juiceshop vulnerable app as backend
Here are the manifests we use to deploy the OWASP Juiceshop vulnerable web application:
apiVersion: apps/v1
kind: Deployment
metadata:
name: juiceshop
namespace: default
labels:
app: juiceshop
spec:
replicas: 1
selector:
matchLabels:
app: juiceshop
template:
metadata:
labels:
app: juiceshop
spec:
containers:
- name: juiceshop
image: bkimminich/juice-shop
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: juiceshop
namespace: default
labels:
app: juiceshop
spec:
ports:
- port: 80
targetPort: 3000
name: http
selector:
app: juiceshop
type: ClusterIP
After that, we make the application available through the Gateway by creating the following HTTPRoute resource:
kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
name: juiceshop-https
namespace: default
spec:
parentRefs:
- kind: Gateway
name: testgwapi-global-external-managed # Name of the Gateway to use
namespace: default
sectionName: https # Which Gateway listener to use
hostnames:
- "testgatewayapi.hackerstack.org"
rules:
- backendRefs:
- name: juiceshop # Name of the backend service on which traffic will be routed
kind: Service
namespace: default # Namespace where the backend service resides
port: 80
weight: 100
Verification:
$ curl https://testgatewayapi.hackerstack.org
<!--
~ Copyright (c) 2014-2023 Bjoern Kimminich & the OWASP Juice Shop contributors.
~ SPDX-License-Identifier: MIT
--><!DOCTYPE html><html lang="en"><head>
<meta charset="utf-8">
<title>OWASP Juice Shop</title>
<meta name="description" content="Probably the most modern and sophisticated insecure web application">
(...)
</script><script src="vendor.js" type="module"></script><script src="main.js" type="module"></script></body></html>
Protecting the Juiceshop app against SQL injection attacks
The login page of the OWASP Juiceshop application is vulnerable to some SQL injection attacks. Using ' OR TRUE-- in the login/email field with any characters in the password field make us login as the admin user.
Let's block that attack with the WAF.
First, we create a security policy and a security policy rule to block SQL injection attacks. We use the sqli-v33-stable predefined rule for that. Predefined rules are sourced from the OWASP ModSecurity Core Rule Set. Here are the commands:
# Create the Cloud Armor security policy
$ gcloud compute security-policies create protect
# Create a rule inside the security policy to block sql injections
$ gcloud compute security-policies rules create 1000 \
--security-policy protect \
--expression "evaluatePreconfiguredExpr('sqli-v33-stable')" \
--action deny-403
For info about the permissions required for configuring security policies, have a look at Security policies IAM perms and Security policies custom IAM perms.
The priority of the rule is 1000. Security policies rules with the lowest priorities are evaluated first. Have a look at Security policies rules evaluation order for details. For more about creating, updating and removing Cloud Armor policies and policies rules, have a look at Manage Cloud Armor Security Policies, Manage Cloud Armor Security Policies rules and Example policies.
Then, we need to tell the Gateway to use the protect security policy for the juiceshop backend service by creating a GCPBackendPolicy resource:
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
name: protect-juiceshop
namespace: default
spec:
default:
securityPolicy: protect
logging:
enabled: true
sampleRate: 500000
targetRef:
group: ""
kind: Service
name: juiceshop
Whithout a spec.default.logging section that enables logging, we will not see Cloud Armor logs for blocked requests in Cloud Logging. For more about logging configuration parameters, have a look at Gateway HTTP access logging.
$ kubectl get gcpbackendpolicies
NAME AGE
protect-juiceshop 47m
$ kubectl describe gcpbackendpolicies/protect-juiceshop
Name: protect-juiceshop
Namespace: default
Labels: <none>
Annotations: <none>
API Version: networking.gke.io/v1
Kind: GCPBackendPolicy
Metadata:
Creation Timestamp: ***
Generation: 1
Resource Version: 58233
UID: 9fcacbb7-d874-4b7e-bf40-b5ca93c6681e
Spec:
Default:
Security Policy: protect
Target Ref:
Group:
Kind: Service
Name: juiceshop
Status:
Conditions:
Last Transition Time: ***
Message:
Reason: Attached
Status: True
Type: Attached
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ADD 47m sc-gateway-controller default/protect-juiceshop
Normal SYNC 49s (x14 over 46m) sc-gateway-controller Application of GCPGatewayPolicy "default/protect-juiceshop" was a success
Note that only one GCPBackendPolicy resource can be associated to a Gateway backend service. Have a look at Troubleshooting Gateway GCPBackendPolicy for details.
Let's test the protection by making the SQL injection attack again. The attack is successfully blocked by the WAF (Web Application Firewall):
Using custom Cloud Armor WAF rules
Now, we want to block requests based on specific attributes like source IPs, request headers... For that, we need to create a custom security policy rule. Here is the rules language reference we use for that.
Let's create a rule to block requests containing user-agent: robot in the header:
$ gcloud compute security-policies rules create 1001 \
--security-policy protect \
--expression "has(request.headers['user-agent']) && request.headers['user-agent'].contains('robot')" \
--action deny-403
The custom rule is set using the --expression flag. After the rule has been successfully applied, let's verify:
$ curl -H 'user-agent: robot' https://testgatewayapi.hackerstack.org
<!doctype html><meta charset="utf-8"><meta name=viewport content="width=device-width, initial-scale=1"><title>403</title>403 Forbidden
To visualize and customize security policies logs outputs for Google Cloud Logging, have a look at Security policies logs entries and Security policies verbose logging.
More Gateway features
GCPGatewayPolicy and GCPBackendPolicy resources for the Gateway are the equivalent of the FrontendConfig and BackendConfig resources for the GCP ingress-gce ingress controller.
They can be used to configure additional features like:
For a complete list of those features and links describing how to enable them, have look at Frontend security, Backend services properties and Additional Google Cloud services.
For additional features related to routing and traffic management available through the HTTPRoute resource, see Routing and traffic management.