An Introduction into OpenShift v3

Red Hat has released OpenShift v3 last week at the summit. Time for me to go out and explore the new features of OpenShift.

I thought the beta training content would be a good starting point. Soon I had to learn, that some of the example files were not adjusted for latest kubernetes API changes.

This blog post explains what steps are necessary to a working OpenShift v3 Installation. This is not one of these Zero to PaaS type of Screencasts, which just show how fast you can get something simple running, but rather a step-by-step instruction on how to get to a real-life Installation inside your datacenter that you can use to evaluate OpenShift. The final environment will contain:

  • 1 master which also acts as a node
  • 2 more dedicated nodes


I am using Amazon ec2 for my testing, but you can use whatever you would like to use. I have created a little helper repository to get up and running quickly.

Please note that I am using the Enterprise version and not the Origin Version. So make sure you have the necessary Entitlements handy. This installation assumes that you have the machines on which you would like to install OpenShift Enterprise v3 on installed with the latest RHEL 7.1

  • points to the ip of the desired master
  • points to the ip of the desired node1
  • points to the ip of the desired node2
  • * which points to the ip address of the desired master (This should point to the node that will host the OpenShift Router, which is the master in our case).

Another requirement is a working DNS. My setup cosists of three m3.large machines and I have reserved a dedicated domain ( that I have configured the following way.
Let us start by cloning the helper repository to our local workstation.

cd ~
mkdir openshift
cd openshift
git clone
cd openshift-kickstarter

This repository contains a couple of files, which I already explaind in the README. The next thing to do is to go ahead and copy the env.example to env and fill in the correct values. When that is done, source the file so the variables are set in your environment.

cp env.example env
vi env
source env

As I am using ansible to do all the prereqs on the hosts, you have to fill in the ansible_inventory file with your environment. Remember how I set my hostnames? I explained that above. You will find the same names and roles as described inside of the file. Please adjust it to match your environment but keep the sections [openshift_nodes] and [openshift_master] in tact.

The next file to edit is the example_hosts file. This file is required by the oo-install-ose tool which you will call later in the process. Don’t worry too much about it now. Just make sure you keep the sections intact and the names match your environment, as above.

The last file to change are the lines labeled with nodes: and masters: inside the file installer.cfg.yml. I am just listing the lines below, so it is clear which I mean:

masters: []
nodes: [,,]

This should cover all the pre-requisites for the kickstarter.


After you have prepared the files, we can go ahead and run ansible to handle all the prerequisites. on the systems. Here is what the kickstarter will do for you (on all of your hosts).

  1. Register your systems against RHN and attach to the correct pool
  2. Disable All Repositories and only subscribe the ones needed by OpenShift Enterprise
  3. Install deltaRPM
  4. Remove NetworkManager
  5. Install all required packages by the OpenShift Installer
  6. Run yum update on the systems
  7. Install docker
  8. Alter the docker registry to allow for insecure registries
  9. Pull the docker images from
  10. Dsitribute the openshift_aws SSH key to the systems

To start the ansible based preparation of the systems run the following command:

ansible-playbook -v -i ./ansible_inventory ./openshift_prereqs.yml

On the master, it will also prepare the installer, so you only have to do minimal configuration changes and can run the installer immediately. The ansible playbook might take several minutes to complete. It is highly dependant on your internet connection speed.
After it finished you can go ahead and log into your master as root and run the following command:


This command installs OpenShift Enterprise into your infrastructure. The installation process might also take several minutes to complete. But stay strong, we are almost done.

First Steps

Now that the installation is finished, there is not very much left for us to do. What we need is:

  1. Check whether every node is schedulable
  2. Check whether the labels have been applied correctly
  3. Deploy a docker registry to hold all the images, and
  4. Deploy a router who will serve as an endpoint to our external requests.

First let us check the node configuration. Run the command oc get nodes which should return the following output:

[root@ip-172-31-18-252 ~]# oc get nodes
NAME                                             LABELS                                                                                            STATUS,region=primary,zone=east    Ready,region=infra,zone=default   Ready,region=primary,zone=west    Ready

If you need to make a node schedulable run the command

oadm manage-node --schedulable=true

To assign labels to nodes, you can run the following commands:

oc label node region=infra zone=default
oc label node region=primary zone=east
oc label node region=primary zone=west

You will need the selectors, for the commands in the training manual to run successfully. The Managing Nodes article in the openshift documentation is a great resource to review what we are doing.
To deploy a registry run the following command. Take special note on the–selector attribute which makes sure to place the registry on the host who is labelled with region=infra

oadm registry --credentials=/etc/openshift/master/openshift-registry.kubeconfig --images='${component}:${version}' --selector="region=infra"

Since I have not yet been able to pass the mount-host option successfully to the oadm registry command, I am happy to take any comments.
Now the only thing left is to deploy a router. A router is able to present different SSL Certificates to the clients, depending on what resource they are requesting. In the training material we reference the domain * I am using the wildcard domain * To make sure that the router is able to present a SSL Certificate to the client we have to create that first. We are using the built-in CA for that.

oadm create-server-cert --signer-cert=$CA/ca.crt --signer-key=$CA/ca.key --signer-serial=$CA/ca.serial.txt --hostnames='*' --cert=cloudapps.crt --key=cloudapps.key

The --cert and --key just represent the later filenames so they can really be anything. Let us merge the two separate files into one.

cat cloudapps.crt cloudapps.key $CA/ca.crt > cloudapps.router.pem

And now we can deploy the router (see the selector again?)

oadm router --default-cert=cloudapps.router.pem --credentials=/etc/openshift/master/openshift-router.kubeconfig --selector='region=infra' --images='${component}:${version}’

It can take a bit for the router to deploy. Once it is deployed, you will see something like

[root@ip-172-31-18-252 ~]# oc get pods
NAME                      READY     REASON    RESTARTS   AGE
docker-registry-1-edt0z   1/1       Running   0          5
drouter-1-wje21            1/1       Running   1          5d

Now it is time to check the status page of the HA Proxy Router. The status page is still secured by iptables, so let us open the port:

iptables -I OS_FIREWALL_ALLOW -p tcp -m tcp --dport 1936 -j ACCEPT

Now you are all set and most of the training material should work. I have opened a pull request for the Training material which contains my modified files


This will hopefully get you started quickly with OpenShift version 3. It would be great if you could leave some comments.

5 comments on “An Introduction into OpenShift v3
  1. Great blog entry! Thank you Buddy.

    I am building something similar running in vagrant and this is really useful.

  2. Hi,

    I believe the mount-host option won’t work without adding a service account and adding it to the privileged SSC. I went with NFS shared storage by editing the deploymentConfig after creating the registry – this will force the registry pods to be redeployed with your changes :-

    # oc edit dc/docker-registry – replace emptyDir

    – mountPath: /registry
    name: registry-storage

    – name: registry-storage
    path: /exports/registry
    server: NNN.NNN.NNN.NNN

Leave a Reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: