To allow stable endpoints in an environment of ever changing starting and stopping Pods (and therefore constantly changing IP addresses), Kubernetes introduces (and OpenShift uses) the concept of services. Services are stable IP addresses (taken per default from the 172.30.0.0/16 subnet) that remain the same as long as the service exists.
Connection requests to a service are forwarded to a pod which matches the service’s selector. A selector is a predicate that describes which pods to target based on labels (key/value pairs) applied to a pod. While the discussions of labels and selectors is outside of this paper, the mechanism how the forwarding takes place is not. Here, a component called “kube-proxy” comes into play. Just like the SkyDNS service on the masters, it listens on the cluster configuration database for changes (e.g. pods matching the selector starting or stopping) and adjusts the forwarding rules.
Again there are two methods how this forwarding can work in detail, and it is a cluster-wide configuration which one is used:
- User-space mode: Here IPTables rules are used to forward packages destined to the service IP address to the kube-proxy, who will in turn initiate connections to the actual destination IP and proxy between the two endpoints. Key advantage of user-space mode is that it is able to detect non-responding pods and retry connection to other pods.
- IPTables mode: Here the kube-proxy continuously updates the hosts’ IPTables rules to forward packets directly to one of the target pod’s IP address. Key advantage of this mode is the increased throughput.
Both methods default to a round-robin distribution when more than 1 pod is available, but allow for session affinity based on the client’s IP address. Of course, this works the same no matter if the target pod is on the same node or a different node.
OpenShift services add the following flows (using a single node example for simplicity):
- From a Pod to another Pod (on the same node) via Service / Usermode : PodA eth0 → vethXXXX → (ovs) br0 → tun0 → IPTables NAT → kube-proxy → tun0 → (ovs) br0 → vethYYYY → PodB eth0
- From a Pod to another Pod (on the same node) via Service / IPTables : PodA eth0 → vethXXXX → (ovs) br0 → tun0 → IPTables NAT → tun0 → (ovs) br0 → vethYYYY → PodB eth0
Additional Benefit: Resolving service names via DNS
Also, the OpenShift DNS configuration resolves the service name into its IP address for all callers from the same namespace. This means that from a pod, you can simply access “http://foo” to connect to the “foo” service in the same namespace. This works because Kubernetes launches the pods so that the DNS configuration will point to the master nodes with a default search domain “.<pod_namespace>.cluster.local”. A SkyDNS service on the masters listens on configuration changes in the OpenShift database and updates its resolution tables whenever the OpenShift service configuration changes.