Getting Started with OCI Container Engine for Kubernetes (OKE): Produc

Deploy containerized applications on Oracle Kubernetes Engine with node pools, load balancers, persistent storage, and CI/CD integration.

Overview

In the dynamic landscape of cloud-native applications, Kubernetes has emerged as the de facto standard for orchestrating containerized workloads. Oracle Cloud Infrastructure (OCI) offers a robust, fully managed Kubernetes service known as OCI Container Engine for Kubernetes (OKE). OKE simplifies the deployment, scaling, and management of Kubernetes clusters, allowing developers and operators to focus on building applications rather than managing complex infrastructure.

This guide, penned for TechNews Venture readers, delves into the intricacies of setting up an OKE cluster specifically tailored for production deployments. We'll move beyond basic setups to cover the critical aspects that ensure your applications are highly available, secure, scalable, and cost-efficient. From initial OCI tenancy configuration to advanced security practices and monitoring, we aim to provide a comprehensive roadmap for leveraging OKE's full potential in a production environment.

Leveraging OKE means benefiting from Oracle's enterprise-grade infrastructure, including a high-performance network, resilient storage, and seamless integration with other OCI services like OCI Container Registry (OCIR), OCI Vault, and OCI Monitoring. For organizations seeking to modernize their application delivery and embrace cloud-native principles, OKE presents a compelling solution that combines the power of Kubernetes with the reliability and security of OCI.

Prerequisites

Before embarking on your OKE production deployment journey, ensure you have the following prerequisites in place:

OCI Account and Tenancy

An active Oracle Cloud Infrastructure tenancy with administrative privileges. You'll need access to the OCI Console and the ability to create and manage resources within your compartments.

IAM Policies

Appropriate IAM policies are crucial for granting necessary permissions to create and manage OKE clusters and related resources. At a minimum, a user or group needs policies similar to these (replace with your actual group):


# Allow group to manage all resources in the tenancy (broad, for simplicity in dev/test)
Allow group  to manage all resources in tenancy

# More granular policies for production:
Allow group  to manage instance-family in compartment 
Allow group  to manage vnics in compartment 
Allow group  to manage volume-family in compartment 
Allow group  to manage cluster-family in compartment 
Allow group  to manage oke-virtual-node-pools in compartment 
Allow group  to manage virtual-node-pool-family in compartment 
Allow group  to use public-ips in compartment 
Allow group  to manage load-balancers in compartment 
Allow group  to manage metrics in compartment 
Allow group  to manage repos in tenancy # For OCI Container Registry
Allow group  to read users in tenancy # For Kubeconfig generation
Allow group  to inspect compartments in tenancy

OCI CLI

The Oracle Cloud Infrastructure Command Line Interface (CLI) is essential for automating tasks and interacting with OCI services. Install and configure it on your local machine or a bastion host. Follow the official OCI documentation for installation.
```
# Example installation (Linux/macOS)
bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)"

# Configure the CLI
oci setup config
        
```
Ensure your configuration file (~/.oci/config) and API key are correctly set up.

Kubectl

The Kubernetes command-line tool, kubectl, is used to run commands against Kubernetes clusters. Install it according to the Kubernetes official documentation.


# Example installation (Linux)
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

# Verify installation
kubectl version --client

Helm (Optional but Recommended)

Helm is the package manager for Kubernetes and simplifies the deployment and management of complex applications. It's highly recommended for production environments.


# Example installation (Linux/macOS)
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

# Verify installation
helm version

Virtual Cloud Network (VCN) Setup

For a production OKE cluster, a well-designed VCN is paramount. You'll need at least three subnets:
- Kubernetes API Endpoint Subnet: A regional, private subnet for the Kubernetes API endpoint. This enhances security by keeping the control plane internal.
- Worker Node Subnet: A regional, private subnet for your worker nodes. These nodes will host your application pods.
- Load Balancer Subnet: A regional, public subnet for OCI Load Balancers that expose your applications to the internet.
Ensure your VCN has the following gateways configured:
- Internet Gateway (IGW): For public subnet internet access (e.g., Load Balancer).
- NAT Gateway: For private subnet instances to initiate connections to the internet (e.g., pulling Docker images, OS updates).
- Service Gateway (SGW): For private subnet instances to access OCI public services (like OCI Object Storage, OCI Container Registry) without traversing the internet.

Detailed Steps for Production Deployment

1. Create a Production-Ready VCN

If you don't have a VCN configured as described in the prerequisites, let's create one. For production, we prioritize private subnets for security.


# Define variables
COMPARTMENT_OCID="ocid1.compartment.oc1..aaaaaaaanbxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Replace with your compartment OCID
VCN_NAME="oke-prod-vcn"
VCN_CIDR="10.0.0.0/16"
REGION="us-ashburn-1" # Adjust to your region

# Create VCN
echo "Creating VCN: ${VCN_NAME}..."
VCN_OCID=$(oci network vcn create --compartment-id ${COMPARTMENT_OCID} --display-name ${VCN_NAME} --cidr-block ${VCN_CIDR} --query 'data.id' --raw-output)
echo "VCN OCID: ${VCN_OCID}"

# Create Internet Gateway
echo "Creating Internet Gateway..."
IGW_OCID=$(oci network internet-gateway create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name "${VCN_NAME}-igw" --query 'data.id' --raw-output)
echo "IGW OCID: ${IGW_OCID}"

# Create NAT Gateway
echo "Creating NAT Gateway..."
NAT_GW_OCID=$(oci network nat-gateway create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name "${VCN_NAME}-nat-gw" --query 'data.id' --raw-output)
echo "NAT GW OCID: ${NAT_GW_OCID}"

# Create Service Gateway
echo "Creating Service Gateway..."
SGW_OCID=$(oci network service-gateway create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --service-id $(oci network service list --service-name 'OCI PHX Object Storage' --query 'data[0].id' --raw-output) --display-name "${VCN_NAME}-sgw" --query 'data.id' --raw-output)
echo "SGW OCID: ${SGW_OCID}"

# Get Default Route Table OCID (for public subnets)
DEFAULT_RT_OCID=$(oci network route-table list --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name "Default Route Table for ${VCN_NAME}" --query 'data[0].id' --raw-output)
echo "Default RT OCID: ${DEFAULT_RT_OCID}"

# Update Default Route Table for Internet Gateway
echo "Updating Default Route Table with IGW route..."
oci network route-table update --rt-id ${DEFAULT_RT_OCID} --route-rules '[{"cidrBlock":"0.0.0.0/0", "networkEntityId":"'${IGW_OCID}'"}]'

# Create Private Route Table (for worker nodes and API endpoint)
PRIVATE_RT_NAME="${VCN_NAME}-private-rt"
echo "Creating Private Route Table: ${PRIVATE_RT_NAME}..."
PRIVATE_RT_OCID=$(oci network route-table create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name ${PRIVATE_RT_NAME} --route-rules '[{"cidrBlock":"0.0.0.0/0", "networkEntityId":"'${NAT_GW_OCID}'"}, {"serviceCidrBlock":"all-phx-services-in-oracle-services-network", "networkEntityId":"'${SGW_OCID}'"}]' --query 'data.id' --raw-output)
echo "Private RT OCID: ${PRIVATE_RT_OCID}"

# Create Subnets
# Public Subnet for Load Balancers
LB_SUBNET_CIDR="10.0.10.0/24"
LB_SUBNET_OCID=$(oci network subnet create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name "${VCN_NAME}-lb-subnet" --cidr-block ${LB_SUBNET_CIDR} --prohibit-public-ip-on-vnic false --route-table-id ${DEFAULT_RT_OCID} --query 'data.id' --raw-output)
echo "LB Subnet OCID: ${LB_SUBNET_OCID}"

# Private Subnet for Worker Nodes
WORKER_SUBNET_CIDR="10.0.20.0/24"
WORKER_SUBNET_OCID=$(oci network subnet create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name "${VCN_NAME}-worker-subnet" --cidr-block ${WORKER_SUBNET_CIDR} --prohibit-public-ip-on-vnic true --route-table-id ${PRIVATE_RT_OCID} --query 'data.id' --raw-output)
echo "Worker Subnet OCID: ${WORKER_SUBNET_OCID}"

# Private Subnet for Kubernetes API Endpoint
API_SUBNET_CIDR="10.0.30.0/24"
API_SUBNET_OCID=$(oci network subnet create --compartment-id ${COMPARTMENT_OCID} --vcn-id ${VCN_OCID} --display-name "${VCN_NAME}-api-subnet" --cidr-block ${API_SUBNET_CIDR} --prohibit-public-ip-on-vnic true --route-table-id ${PRIVATE_RT_OCID} --query 'data.id' --raw-output)
echo "API Subnet OCID: ${API_SUBNET_OCID}"

2. Create an OKE Cluster (Custom Create for Production)

For production, always use the "Custom Create" option in OCI Console or specify all parameters via CLI. This gives you granular control over networking, node pools, and Kubernetes versions.


# Define cluster variables
CLUSTER_NAME="techventure-prod-cluster"
K8S_VERSION="v1.28.2" # Always use a stable, supported version
NODE_POOL_NAME="prod-worker-pool"
NODE_COUNT=3 # Start with at least 3 for HA
NODE_SHAPE="VM.Standard.E4.Flex" # Choose appropriate shape for your workload
BOOT_VOLUME_SIZE_GB=100 # Adjust based on application needs
SSH_PUBLIC_KEY_PATH="~/.ssh/id_rsa.pub" # Path to your SSH public key

# Retrieve Image OCID for worker nodes (e.g., Oracle-Linux-8-OKE-GPU)
# For production, use the latest OKE-optimized image for your chosen K8s version
IMAGE_OCID=$(oci compute image list --compartment-id ${COMPARTMENT_OCID} --operating-system "Oracle Linux" --operating-system-version "8" --shape "${NODE_SHAPE}" --query "data[?contains(\"display-name\",'OKE')].id" --raw-output | tr -d '[]"')

# Create the OKE Cluster
echo "Creating OKE Cluster: ${CLUSTER_NAME}..."
oci ce cluster create \
    --compartment-id ${COMPARTMENT_OCID} \
    --name ${CLUSTER_NAME} \
    --kubernetes-version ${K8S_VERSION} \
    --vcn-id ${VCN_OCID} \
    --endpoint-config '{"isPublicIpEnabled": false, "subnetId": "'${API_SUBNET_OCID}'"}' \
    --options '{
        "serviceLbSubnetIds": ["'${LB_SUBNET_OCID}'"],
        "kubernetesNetworkConfig": {
            "podsCidr": "10.244.0.0/16",
            "servicesCidr": "10.96.0.0/16"
        }
    }' \
    --node-pools '[
        {
            "name": "'${NODE_POOL_NAME}'",
            "compartmentId": "'${COMPARTMENT_OCID}'",
            "kubernetesVersion": "'${K8S_VERSION}'",
            "nodeShape": "'${NODE_SHAPE}'",
            "nodeShapeConfig": {
                "ocpus": 2,
                "memoryInGBs": 16
            },
            "size": '${NODE_COUNT}',
            "subnetIds": ["'${WORKER_SUBNET_OCID}'"],
            "nodeSourceDetails": {
                "sourceType": "OCID",
                "imageId": "'${IMAGE_OCID}'",
                "bootVolumeSizeInGBs": '${BOOT_VOLUME_SIZE_GB}'
            },
            "sshPublicKey": "$(cat ${SSH_PUBLIC_KEY_PATH})",
            "initialNodeLabels": [
                {"key": "node.kubernetes.io/lifecycle", "value": "on-demand"}
            ],
            "nodeConfigDetails": {
                "isPvEncryptionInTransitEnabled": true,
                "placementConfigs": [
                    {
                        "availabilityDomain": "Uocm:US-ASHBURN-1-AD-1", # Adjust ADs based on your region
                        "faultDomain": ["FAULT-DOMAIN-1", "FAULT-DOMAIN-2", "FAULT-DOMAIN-3"]
                    },
                    {
                        "availabilityDomain": "Uocm:US-ASHBURN-1-AD-2",
                        "faultDomain": ["FAULT-DOMAIN-1", "FAULT-DOMAIN-2", "FAULT-DOMAIN-3"]
                    },
                    {
                        "availabilityDomain": "Uocm:US-ASHBURN-1-AD-3",
                        "faultDomain": ["FAULT-DOMAIN-1", "FAULT-DOMAIN-2", "FAULT-DOMAIN-3"]
                    }
                ]
            }
        }
    ]'
echo "Cluster creation initiated. This may take several minutes."

Note: The endpoint-config is set to isPublicIpEnabled: false and points to API_SUBNET_OCID, ensuring your Kubernetes API endpoint is private. The nodeConfigDetails with placementConfigs across multiple Fault Domains (and Availability Domains if your region supports it) ensures high availability for your worker nodes.

3. Configure Kubectl Access

Once the cluster is in an ACTIVE state, configure kubectl to interact with it. This involves generating a kubeconfig file.


# Get the Cluster OCID (replace with actual OCID if you don't have it from previous step)
CLUSTER_OCID=$(oci ce cluster list --compartment-id ${COMPARTMENT_OCID} --name ${CLUSTER_NAME} --query 'data[0].id' --raw-output)

# Create a directory for kubeconfig
mkdir -p ~/.kube

# Generate kubeconfig
# The --file option specifies where to save the kubeconfig.
# The --overwrite option ensures previous configs are replaced if they exist.
# The --region is crucial for authentication.
echo "Generating kubeconfig for cluster ${CLUSTER_OCID}..."
oci ce cluster create-kubeconfig \
    --cluster-id ${CLUSTER_OCID} \
    --file ~/.kube/config \
    --region ${REGION} \
    --token-version 2.0 \
    --overwrite

# Set KUBECONFIG environment variable
export KUBECONFIG=~/.kube/config

# Verify kubectl access
echo "Verifying kubectl access..."
kubectl get nodes

You should see your worker nodes listed. If you encounter issues, check your IAM policies and network connectivity from where you are running kubectl to the OKE API endpoint (if private, you might need a bastion host or VPN).

4. Deploy a Sample Application

Let's deploy a simple Nginx application to test the cluster and expose it via an OCI Load Balancer.


# Create a deployment YAML (nginx-deployment.yaml)
cat < nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.23.3 # Use a specific, stable image version for production
        ports:
        - containerPort: 80
EOF

# Create a service YAML (nginx-service.yaml)
cat < nginx-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  annotations:
    service.beta.kubernetes.io/oci-load-balancer-shape: "flexible" # Use flexible shape for production
    service.beta.kubernetes.io/oci-load-balancer-shape-details-min: "10" # Min bandwidth Mbps
    service.beta.kubernetes.io/oci-load-balancer-shape-details-max: "100" # Max bandwidth Mbps
    service.beta.kubernetes.io/oci-load-balancer-internal: "false" # Public Load Balancer
    # service.beta.kubernetes.io/oci-load-balancer-subnet1: "" # Optional if only one public subnet
spec:
  selector:
    app: nginx
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  externalTrafficPolicy: Local # Preserves client source IP
EOF

# Apply the deployment and service
kubectl apply -f nginx-deployment.yaml
kubectl apply -f nginx-service.yaml

# Monitor the service creation and get the Load Balancer IP
echo "Waiting for Load Balancer IP..."
kubectl get svc nginx-service --watch

Once the `EXTERNAL-IP` for `nginx-service` appears (it will be an OCI Load Balancer IP), you can access your Nginx application through that IP in a web browser.

5. Integrate with OCI Container Registry (OCIR)

For production, you'll likely host your Docker images in a private registry like OCIR. Here's how to integrate.


# 1. Generate an Auth Token for your OCI user
# Go to OCI Console -> Identity & Security -> Users -> Your User -> Auth Tokens -> Generate Token.
# Copy the generated token. This token acts as your password.

# 2. Log in to OCIR using Docker CLI
# The username format is / or /oracleidentitycloudservice/
# You can find your tenancy namespace in the OCI Console (Profile menu -> Tenancy: ).
TENANCY_NAMESPACE="your_tenancy_namespace" # e.g., axxxxxxxxxxxxxxxxxxx
OCIR_REGION_KEY="iad" # e.g., iad for us-ashburn-1, phx for us-phoenix-1
OCIR_REPO="container_repo_name" # e.g., my-app/web-frontend
IMAGE_NAME="${OCIR_REGION_KEY}.ocir.io/${TENANCY_NAMESPACE}/${OCIR_REPO}:v1.0.0"

echo "Logging in to OCIR..."
docker login ${OCIR_REGION_KEY}.ocir.io --username ${TENANCY_NAMESPACE}/oracleidentitycloudservice/your.email@example.com # Use your OCI username format
# Enter the generated auth token when prompted for password.

# 3. Tag and Push an image to OCIR
echo "Building and pushing image to OCIR..."
docker build -t ${IMAGE_NAME} . # Build your application image
docker push ${IMAGE_NAME}

# 4. Create a Kubernetes Secret for Image Pull
# This secret allows your OKE cluster to pull images from your private OCIR repository.
kubectl create secret docker-registry ocir-secret \
  --docker-server=${OCIR_REGION_KEY}.ocir.io \
  --docker-username=${TENANCY_NAMESPACE}/oracleidentitycloudservice/your.email@example.com \
  --docker-password='' \
  --docker-email='your.email@example.com'

# 5. Update your deployment to use the OCIR image and the imagePullSecrets
# Modify nginx-deployment.yaml to reflect this:
# ...
# spec:
#   containers:
#   - name: nginx
#     image: ${IMAGE_NAME} # Your OCIR image
#     ports:
#     - containerPort: 80
#   imagePullSecrets:
#   - name: ocir-secret
# ...

# Apply the updated deployment
# kubectl apply -f updated-nginx-deployment.yaml

Security in OKE Production Deployments

Security is paramount for production workloads. OCI provides a robust security framework that integrates seamlessly with OKE.

IAM Policies for Granular Control

Always adhere to the principle of least privilege. Instead of granting broad "manage all resources" access, create specific IAM policies for different roles (e.g., developers, operators, security auditors) and compartments.

Example: A developer group might only need to manage pods, deployments, services in a specific namespace, while an operator group needs to manage cluster-family and manage node-pool-family.

Network Security Groups (NSGs)

NSGs offer a more granular way to control traffic at the VNIC level compared to Security Lists (which operate at the subnet level). For OKE, NSGs are critical:

Control Plane NSG: OKE automatically creates NSGs for the control plane. Do not modify these.

Worker Node NSG: Create custom NSGs for your worker nodes to define ingress/egress rules for your applications.


# Example NSG for worker nodes allowing ingress from LB subnet and egress to internet/OCIR
# Rule for ingress from Load Balancer subnet
oci network nsg rule add --nsg-id  --direction INGRESS --protocol TCP --destination-port-range min=30000,max=32767 --source-type CIDR_BLOCK --source ${LB_SUBNET_CIDR} --description "Allow NodePort from LB"
# Rule for egress to NAT Gateway for internet access
oci network nsg rule add --nsg-id  --direction EGRESS --protocol ALL --destination-type CIDR_BLOCK --destination 0.0.0.0/0 --description "Allow egress to internet via NAT GW"

Load Balancer NSG: Ensure your LB NSG allows ingress from the internet (0.0.0.0/0) on required ports (e.g., 80, 443) and egress to the worker node NSG on NodePort ranges.

Kubernetes RBAC (Role-Based Access Control)

Beyond OCI IAM, Kubernetes RBAC controls who can do what inside the cluster. Define Roles/ClusterRoles and bind them to ServiceAccounts, Users, or Groups using RoleBindings/ClusterRoleBindings. Always configure RBAC to limit user and application access to only the resources they need.


# Example:

Getting Started with OCI Container Engine for Kubernetes (OKE): Production Deployment Guide