February 2, 2017

To allow stable endpoints in an environment of ever changing starting and stopping Pods (and therefore constantly changing IP addresses), Kubernetes introduces (and OpenShift uses) the concept of services. Services are stable IP addresses (taken per default from the subnet) that remain the same as long as the service exists.

Connection requests to a service are forwarded to a pod which matches the service’s selector. A selector is a predicate that describes which pods to target based on labels (key/value pairs) applied to a pod. While the discussions of labels and selectors is outside of this paper, the mechanism how the forwarding takes place is not. Here, a component called “kube-proxy” comes into play. Just like the SkyDNS service on the masters, it listens on the cluster configuration database for changes (e.g. pods matching the selector starting or stopping) and adjusts the forwarding rules.

Again there are two methods how this forwarding can work in detail, and it is a cluster-wide configuration which one is used:

  • User-space mode: Here IPTables rules are used to forward packages destined to the service IP address to the kube-proxy, who will in turn initiate connections to the actual destination IP and proxy between the two endpoints. Key advantage of user-space mode is that it is able to detect non-responding pods and retry connection to other pods.
  • IPTables mode: Here the kube-proxy continuously updates the hosts’ IPTables rules to forward packets directly to one of the target pod’s IP address. Key advantage of this mode is the increased throughput.

Both methods default to a round-robin distribution when more than 1 pod is available, but allow for session affinity based on the client’s IP address. Of course, this works the same no matter if the target pod is on the same node or a different node.

OpenShift services add the following flows (using a single node example for simplicity):

  • From a Pod to another Pod (on the same node) via Service / Usermode : PodA eth0 → vethXXXX → (ovs) br0 → tun0 → IPTables NAT → kube-proxy → tun0 → (ovs) br0 → vethYYYY → PodB eth0
  • From a Pod to another Pod (on the same node) via Service / IPTables : PodA eth0 → vethXXXX → (ovs) br0 → tun0 → IPTables NAT → tun0 → (ovs) br0 → vethYYYY → PodB eth0

Additional Benefit: Resolving service names via DNS

Also, the OpenShift DNS configuration resolves the service name into its IP address for all callers from the same namespace. This means that from a pod, you can simply access “http://foo” to connect to the “foo” service in the same namespace. This works because Kubernetes launches the pods so that the DNS configuration will point to the master nodes with a default search domain “.<pod_namespace>.cluster.local”. A SkyDNS service on the masters listens on configuration changes in the OpenShift database and updates its resolution tables whenever the OpenShift service configuration changes.

Leave a Reply


Subscribe to our newsletter.

Please select all the ways you would like to hear from Open Sourcerers:

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit our website.

We use Mailchimp as our newsletter platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.