Multi-cluster Allocation

In order to allow allocation from multiple clusters, Agones provides a mechanism to set redirect rules for allocation requests to the right cluster.

There may be different types of clusters, such as on-premise, and Google Kubernetes Engine (GKE), used by a game to help with the cost-saving and availability. For this purpose, Agones provides a mechanism to define priorities on the clusters. Priorities are defined on GameServerAllocationPolicy agones CRD. A matchmaker can enable the multi-cluster rules on a request and target agones-allocator endpoint in any of the clusters and get resources allocated on the cluster with the highest priority. If the cluster with the highest priority is overloaded, the allocation request is redirected to the cluster with the next highest priority.

The remainder of this article describes how to enable multi-cluster allocation.

Define Cluster Priority

GameServerAllocationPolicy is the CRD defined by Agones for setting multi-cluster allocation rules. In addition to cluster priority, it describes the connection information for the target cluster, including the game server namespace, agones-allocator endpoint and client K8s secrets name for redirecting the allocation request. Here is an example of setting the priority for a cluster and it’s connection rules. One such resource should be defined per cluster. For clusters with the same priority, the cluster is chosen with a probability relative to its weight.

In the following example the policy is defined for cluster B in cluster A.

cat <<EOF | kubectl apply -f -
kind: GameServerAllocationPolicy
  name: allocator-cluster-B
  namespace: cluster-A-ns
    clusterName: "clusterB"
    namespace: cluster-B-ns
    secretName: allocator-client-to-cluster-B
    serverCa: c2VydmVyQ0E=
  priority: 1
  weight: 100

To define the local cluster priority, similarly, an allocation rule should be defined, while leaving allocationEndpoints unset. If the local cluster priority is not defined, the allocation from the local cluster happens only if allocation from other clusters with the existing allocation rules is unsuccessful.

Establish trust

To accept allocation requests from other clusters, agones-allocator for cluster B should be configured to accept the client’s certificate from cluster A and the cluster A’s client should be configured to accept the server TLS certificate, if it is not signed by a public Certificate Authority (CA).

Follow the steps to configure the agones allocator gRPC service. The client certificate pair in the mentioned document is stored as a K8s secret. Here are the secrets to set:

1.Client certificate to talk to other clusters:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
  name: allocator-client-to-cluster-B
  namespace: cluster-A-ns
type: Opaque
  tls.crt: <REDACTED>
  tls.key: <REDACTED>
  ca.crt: <REDACTED>

The certificates are base 64 string of the certificate file e.g. cat ${CERT_FILE} | base64 -w 0

Agones recommends using solution for generating client certificates.

2.Add client CA to the list of authorized client certificates by agones-allocator in the targeted cluster.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
  name: allocator-client-ca
  namespace: agones-system
type: Opaque
  client1.crt: <REDACTED>
  client2.crt: <REDACTED>
  clientN.crt: <REDACTED>

Allocate multi-cluster

To enable multi-cluster allocation, set multiClusterSetting.enabled to true in allocation.proto and send allocation requests. For more information visit agones-allocator. In the following, using allocator-client sample , a multi-cluster allocation request is sent to the agones-allocator service.

EXTERNAL_IP=`kubectl get services agones-allocator -n agones-system -o jsonpath='{.status.loadBalancer.ingress[0].ip}'`

NAMESPACE=default # replace with any namespace

go run examples/allocator-client/main.go --ip ${EXTERNAL_IP} \
    --namespace ${NAMESPACE} \
    --key ${KEY_FILE} \
    --cert ${CERT_FILE} \
    --cacert ${TLS_CA_FILE} \
    --multicluster true