Setting up Tanzu Community Edition (TCE) on MAC

VMware announced Tanzu Community Edition (TCE) this week. It’s an OpenSource and free to use version. However, from a capability perspective, it got all the capabilties that TKG comes with including Tanzu packages. TCE support different deployment model as shown in below pictures.

Standalone Cluster
Managed Cluster

In the first picture, you see the flow fo creating a standalone cluster where bootstrap cluster and then workload cluster. In other picture, you can see the general flow where bootstrap cluster creates a management cluster and then we can have 1 or more workload clusters.

In this blog post, I would like to provide you the step by step intructions on how to setup TCE standalone cluster on a MAC.

Pre-requirements

Note: Docker desktop setting for resources.

Installing TCE on Docker Destop running on MAC

  • Add a repository and install tanzu cli
    # Run the below commands
    $ brew tap vmware-tanzu/tanzu
    $ brew install tanzu-community-edition
    Updating Homebrew...
    ==> Auto-updated Homebrew!
    Updated 5 taps (cloudfoundry/tap, homebrew/core, homebrew/cask, homebrew/services and vmware-tanzu/carvel).
    ==> New Formulae
    aws-sso-util               clickhouse-odbc            cmake-docs                 texlive
    ca-certificates            cloudfoundry/tap/cf-cli@8  git-credential-libsecret
    ==> Updated Formulae
    Updated 348 formulae.
    ==> New Casks
    bespoke                              plistedplus                          remarkable
    fig                                  rancher                              thetimemachinemechanic
    ==> Updated Casks
    Updated 258 casks.
    ==> Deleted Casks
    google-backup-and-sync               google-photos-backup-and-sync        nosqlclient

    ==> Tapping vmware-tanzu/tanzu
    Cloning into '/usr/local/Homebrew/Library/Taps/vmware-tanzu/homebrew-tanzu'...
    remote: Enumerating objects: 53, done.
    remote: Counting objects: 100% (53/53), done.
    remote: Compressing objects: 100% (41/41), done.
    remote: Total 53 (delta 22), reused 23 (delta 9), pack-reused 0
    Receiving objects: 100% (53/53), 21.41 KiB | 5.35 MiB/s, done.
    Resolving deltas: 100% (22/22), done.
    Tapped 1 formula (19 files, 53.8KB).
    ==> Downloading https://github.com/vmware-tanzu/community-edition/releases/download/v0.9.1/tce-darwin-amd64-v
    ==> Downloading from https://github-releases.githubusercontent.com/303802332/f34557b0-acad-41d4-a12a-24474258
    ######################################################################## 100.0%
    ==> Installing tanzu-community-edition from vmware-tanzu/tanzu
    Error: The `brew link` step did not complete successfully
    The formula built, but is not symlinked into /usr/local
    Could not symlink bin/tanzu
    Target /usr/local/bin/tanzu
    already exists. You may want to remove it:
    rm '/usr/local/bin/tanzu'

    To force the link and overwrite all conflicting files:
    brew link --overwrite tanzu-community-edition

    To list all files that would be deleted:
    brew link --overwrite --dry-run tanzu-community-edition

    Possible conflicting files are:
    /usr/local/bin/tanzu
    ==> Thanks for installing Tanzu Community Edition!
    ==> The Tanzu CLI has been installed on your system
    ==> 

    ==> ******************************************************************************
    ==> * To initialize all plugins required by Tanzu Community Edition, an additional
    ==> * step is required. To complete the installation, please run the following
    ==> * shell script:
    ==> *
    ==> * /usr/local/Cellar/tanzu-community-edition/v0.9.1/libexec/configure-tce.sh
    ==> *
    ==> ******************************************************************************
    ==> 

    ==> * To cleanup and remove Tanzu Community Edition from your system, run the
    ==> * following script:
    ==> /usr/local/Cellar/tanzu-community-edition/v0.9.1/libexec/uninstall.sh
    ==> 

    ==> Summary
    /usr/local/Cellar/tanzu-community-edition/v0.9.1: 15 files, 642.6MB, built in 10 seconds

Note: If you are seeing the below warning, this may come in case you have another Tanzu cli running

  • Warning: vmware-tanzu/tanzu/tanzu-community-edition v0.9.1 is already installed, it's just not linked. Run the below command

  • To link this version

  • run:

$ brew link tanzu-community-edition
Linking /usr/local/Cellar/tanzu-community-edition/v0.9.1... 1 symlinks created.
  • Validate the Tanzu CLI by running the below command
$ tanzu 
Tanzu CLI

Usage:
  tanzu [command]

Available command groups:

  Run
    cluster                 Kubernetes cluster operations 
    kubernetes-release      Kubernetes release operations 
    management-cluster      Kubernetes management cluster operations 
    package                 Tanzu package management 
	
  System
    completion              Output shell completion code 
    config                  Configuration for the CLI 
    init                    Initialize the CLI 
    login                   Login to the platform 
    plugin                  Manage CLI plugins 
    update                  Update the CLI 
    version                 Version information 
	

Flags:
  -h, --help   help for tanzu

Use "tanzu [command] --help" for more information about a command. 

  • Validat the Tanzu CLI version.
tanzu version                                           
version: v0.2.1
buildDate: 2021-09-29
sha: ceaa474
  • Configure the Tanzu CLI by running the following command
# Note: You will find the below path including script name after executing very first command, Check the output and copy the command from above.
$ /usr/local/Cellar/tanzu-community-edition/v0.9.1/libexec/configure-tce.sh
MY_DIR: /usr/local/Cellar/tanzu-community-edition/v0.9.1/libexec
/Users/dinetrip/Library/Application Support
Removing old plugin cache from /Users/dinetrip/.cache/tanzu/catalog.yaml
Making a backup of your Kuberentes config files int /tmp
| initializing ✔  successfully initialized CLI 
Installation complete!

  • Now, Validate the tanzu cli and you will notice the differences. There are more plugins and commands avilable.
$ tanzu        
Tanzu CLI

Usage:
  tanzu [command]

Available command groups:

  Admin
    builder                 Build Tanzu components 
	
  Run
    cluster                 Kubernetes cluster operations 
    conformance             Run Sonobuoy conformance tests against clusters 
    diagnostics             Cluster diagnostics 
    kubernetes-release      Kubernetes release operations 
    management-cluster      Kubernetes management cluster operations 
    package                 Tanzu package management 
    standalone-cluster      Create clusters without a dedicated management cluster 
	
  System
    completion              Output shell completion code 
    config                  Configuration for the CLI 
    init                    Initialize the CLI 
    login                   Login to the platform 
    plugin                  Manage CLI plugins 
    update                  Update the CLI 
    version                 Version information 
	

Flags:
  -h, --help   help for tanzu

Use "tanzu [command] --help" for more information about a command. 
  • Let’s run a command to create a standalone cluster.

$ tanzu standalone-cluster create -i docker tkgondocker
Downloading TKG compatibility file from 'projects.registry.vmware.com/tkg/framework-zshippable/tkg-compatibility'
Downloading the TKG Bill of Materials (BOM) file from 'projects.registry.vmware.com/tkg/tkg-bom:v1.4.0'
Downloading the TKr Bill of Materials (BOM) file from 'projects.registry.vmware.com/tkg/tkr-bom:v1.21.2_vmware.1-tkg.1'

Validating the pre-requisites...
Identity Provider not configured. Some authentication features won't work.

Setting up standalone cluster...
Validating configuration...
Using infrastructure provider docker:v0.3.23
Generating cluster configuration...
Setting up bootstrapper...
Bootstrapper created. Kubeconfig: /Users/dinetrip/.kube-tkg/tmp/config_tImxi3O1
Installing providers on bootstrapper...
Start creating standalone cluster...
Saving standalone cluster kubeconfig into /Users/dinetrip/.kube/config
Waiting for bootstrap cluster to get ready for save ...
Waiting for addons installation...
Moving all Cluster API objects from bootstrap cluster to standalone cluster...
Context set for standalone cluster tkgondocker as 'tkgondocker-admin@tkgondocker'.
Cleaning up unneeded resources (for standalone clusters)...

Standalone cluster created!


You can now use Kubectl to access your cluster:

  kubectl get pods -A


You can also delete the cluster by running the following:

  tanzu standalone-cluster delete [name]


Some addons might be getting installed! Check their status by running the following:

  kubectl get apps -A


  • while above command is running, you can grab the kubeconfig file from below location and check the progress status.
# find the latest kubeconfig file from below directory
$ ls -ltra ~/.kube-tkg/tmp     
total 48
-rw-r--r--  1 dinetrip  staff  5660 Nov  6  2020 config_THfjUqmF
-rw-r--r--  1 dinetrip  staff  5664 Nov  9  2020 config_7kMvqdKt
-rw-r--r--  1 dinetrip  staff  5688 Oct  4 20:03 config_tImxi3O1
drwxr-xr-x  5 dinetrip  staff   160 Oct  4 20:03 .
drwxr-xr-x  4 dinetrip  staff   128 Oct  4 20:08 ..

# view the list of pods running on bootstrap node.
 (main) $ k get po -A --kubeconfig ~/.kube-tkg/tmp/config_tImxi3O1
NAMESPACE                           NAME                                                                  READY   STATUS    RESTARTS   AGE
capd-system                         capd-controller-manager-5885f59558-9tp8t                              2/2     Running   0          3m25s
capi-kubeadm-bootstrap-system       capi-kubeadm-bootstrap-controller-manager-6494884869-hs2tm            2/2     Running   0          3m31s
capi-kubeadm-control-plane-system   capi-kubeadm-control-plane-controller-manager-857d687b9d-lpgf6        2/2     Running   0          3m28s
capi-system                         capi-controller-manager-778bd4dfb9-d65g2                              2/2     Running   0          3m36s
capi-webhook-system                 capi-controller-manager-9995bdc94-jf7jp                               2/2     Running   0          3m40s
capi-webhook-system                 capi-kubeadm-bootstrap-controller-manager-68845b65f8-7w5ms            2/2     Running   0          3m34s
capi-webhook-system                 capi-kubeadm-control-plane-controller-manager-9847c6747-wmngn         2/2     Running   0          3m30s
cert-manager                        cert-manager-77f6fb8fd5-mrtps                                         1/1     Running   0          5m3s
cert-manager                        cert-manager-cainjector-6bd4cff7bb-vtsw9                              1/1     Running   0          5m3s
cert-manager                        cert-manager-webhook-fbfcb9d6c-csmwz                                  1/1     Running   0          5m2s
kube-system                         coredns-8dcb5c56b-cs4b7                                               1/1     Running   0          5m45s
kube-system                         coredns-8dcb5c56b-kchrv                                               1/1     Running   0          5m45s
kube-system                         etcd-tkg-kind-c5dh03sbcv49n0qus2kg-control-plane                      1/1     Running   0          5m54s
kube-system                         kindnet-bcrth                                                         1/1     Running   1          5m45s
kube-system                         kube-apiserver-tkg-kind-c5dh03sbcv49n0qus2kg-control-plane            1/1     Running   0          5m54s
kube-system                         kube-controller-manager-tkg-kind-c5dh03sbcv49n0qus2kg-control-plane   1/1     Running   0          5m59s
kube-system                         kube-proxy-w4smg                                                      1/1     Running   0          5m45s
kube-system                         kube-scheduler-tkg-kind-c5dh03sbcv49n0qus2kg-control-plane            1/1     Running   0          5m54s
local-path-storage                  local-path-provisioner-8b46957d4-vxzzb                                1/1     Running   0          5m45s
 (main) $ 

  • You can also see the kind cluster running
$ docker ps                                                                          
CONTAINER ID   IMAGE                                                         COMMAND                  CREATED         STATUS         PORTS                       NAMES
b25052dc2da9   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   4 minutes ago   Up 4 minutes   127.0.0.1:57265->6443/tcp   tkg-kind-c5dh03sbcv49n0qus2kg-control-plane
  • Once the kind cluster is ready, you will notice that the ha-proxy load balancer and control plane containers will start
$ docker ps
CONTAINER ID   IMAGE                                                         COMMAND                  CREATED         STATUS         PORTS                                  NAMES
225d61b6f9ba   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   2 minutes ago   Up 2 minutes                                          tkgondocker-md-0-7b9758c6d6-wqdlk
347bf31db466   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   4 minutes ago   Up 3 minutes   40819/tcp, 127.0.0.1:40819->6443/tcp   tkgondocker-control-plane-cmqpg
8167fc8ee5e4   kindest/haproxy:v20210715-a6da3463                            "haproxy -sf 7 -W -d…"   4 minutes ago   Up 4 minutes   41431/tcp, 0.0.0.0:41431->6443/tcp     tkgondocker-lb
b25052dc2da9   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   8 minutes ago   Up 8 minutes   127.0.0.1:57265->6443/tcp              tkg-kind-c5dh03sbcv49n0qus2kg-control-plane

  • After few mins, when the cluster is created, you will notice that the kind container is deleted.

docker ps
CONTAINER ID   IMAGE                                                         COMMAND                  CREATED         STATUS         PORTS                                  NAMES
225d61b6f9ba   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   3 minutes ago   Up 3 minutes                                          tkgondocker-md-0-7b9758c6d6-wqdlk
347bf31db466   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   5 minutes ago   Up 5 minutes   40819/tcp, 127.0.0.1:40819->6443/tcp   tkgondocker-control-plane-cmqpg
8167fc8ee5e4   kindest/haproxy:v20210715-a6da3463                            "haproxy -sf 7 -W -d…"   5 minutes ago   Up 5 minutes   41431/tcp, 0.0.0.0:41431->6443/tcp     tkgondocker-lb
  • Now, Set the right context
# Set the contexts
$ kubectl config use-context tkgondocker-admin@tkgondocker              
Switched to context "tkgondocker-admin@tkgondocker".

# View the post of pods from TCE Cluster
$ k get po -A                                                           
NAMESPACE     NAME                                                      READY   STATUS    RESTARTS   AGE
kube-system   antrea-agent-ddvx9                                        2/2     Running   0          5m55s
kube-system   antrea-agent-ztc7p                                        2/2     Running   1          5m55s
kube-system   antrea-controller-86f8988c5f-r2w8d                        1/1     Running   0          5m56s
kube-system   coredns-8dcb5c56b-btk8j                                   1/1     Running   0          6m34s
kube-system   coredns-8dcb5c56b-g97v6                                   1/1     Running   0          6m34s
kube-system   etcd-tkgondocker-control-plane-cmqpg                      1/1     Running   0          6m32s
kube-system   kube-apiserver-tkgondocker-control-plane-cmqpg            1/1     Running   0          6m32s
kube-system   kube-controller-manager-tkgondocker-control-plane-cmqpg   1/1     Running   1          6m32s
kube-system   kube-proxy-9wsjt                                          1/1     Running   0          5m56s
kube-system   kube-proxy-ckjqs                                          1/1     Running   0          6m35s
kube-system   kube-scheduler-tkgondocker-control-plane-cmqpg            1/1     Running   1          6m32s
tkg-system    kapp-controller-6499b8866-jxdrd                           1/1     Running   0          5m56s
tkg-system    tanzu-capabilities-controller-manager-6ff97656b8-45qf2    1/1     Running   0          6m13s
tkr-system    tkr-controller-manager-6bc455b5d4-2pq2c                   1/1     Running   0          6m15s
  • View the list of nodes

$ k get nodes                 
NAME                                STATUS   ROLES                  AGE     VERSION
tkgondocker-control-plane-cmqpg     Ready    control-plane,master   8m42s   v1.21.2+vmware.1-360497810732255795
tkgondocker-md-0-7b9758c6d6-wqdlk   Ready                     7m57s   v1.21.2+vmware.1-360497810732255795
  • Lets validate the newly created TCE cluster by creating an nginx pod
# create a pod
$ k run nginx --image nginx                                                            
pod/nginx created
 $ k get po                  
NAME    READY   STATUS              RESTARTS   AGE
nginx   0/1     ContainerCreating   0          2s

# Check the pod status
$ k get po -o wide   
NAME    READY   STATUS    RESTARTS   AGE   IP           NODE                                NOMINATED NODE   READINESS GATES
nginx   1/1     Running   0          62s   100.96.1.6   tkgondocker-md-0-7b9758c6d6-wqdlk              

Now, TCE cluster is up and running and it got 1 master node and 1 worker node. Both nodes are running as docker container.

Let’s deploy one of the package too.

Deploying Tanzu Package on a TCE Cluster

  • Add a TCE package repository
$ tanzu package repository add tce-repo --url projects.registry.vmware.com/tce/main:0.9.1 --namespace tanzu-package-repo-global
/ Adding package repository 'tce-repo'... 
 Added package repository 'tce-repo'



  $ tanzu package repository list --namespace tanzu-package-repo-global
/ Retrieving repositories... 
  NAME      REPOSITORY                                   STATUS       DETAILS  
  tce-repo  projects.registry.vmware.com/tce/main:0.9.1  Reconciling           
 $ tanzu package repository list --namespace tanzu-package-repo-global
/ Retrieving repositories... 
  NAME      REPOSITORY                                   STATUS               DETAILS  
  tce-repo  projects.registry.vmware.com/tce/main:0.9.1  Reconcile succeeded           
 $ 

  • View the list of available packages.
$ tanzu package available list
/ Retrieving available packages... 
  NAME                                           DISPLAY-NAME        SHORT-DESCRIPTION                                                                                                             
  cert-manager.community.tanzu.vmware.com        cert-manager        Certificate management                                                                                                        
  contour.community.tanzu.vmware.com             Contour             An ingress controller                                                                                                         
  external-dns.community.tanzu.vmware.com        external-dns        This package provides DNS synchronization functionality.                                                                      
  fluent-bit.community.tanzu.vmware.com          fluent-bit          Fluent Bit is a fast Log Processor and Forwarder                                                                              
  gatekeeper.community.tanzu.vmware.com          gatekeeper          policy management                                                                                                             
  grafana.community.tanzu.vmware.com             grafana             Visualization and analytics software                                                                                          
  harbor.community.tanzu.vmware.com              Harbor              OCI Registry                                                                                                                  
  knative-serving.community.tanzu.vmware.com     knative-serving     Knative Serving builds on Kubernetes to support deploying and serving of applications and functions as serverless containers  
  local-path-storage.community.tanzu.vmware.com  local-path-storage  This package provides local path node storage and primarily supports RWO AccessMode.                                          
  multus-cni.community.tanzu.vmware.com          multus-cni          This package provides the ability for enabling attaching multiple network interfaces to pods in Kubernetes                    
  prometheus.community.tanzu.vmware.com          prometheus          A time series database for your metrics                                                                                       
  velero.community.tanzu.vmware.com              velero              Disaster recovery capabilities                                                                                                
  • List one of the package detail that we are going to install.

$ tanzu package available list cert-manager.community.tanzu.vmware.com
/ Retrieving package versions for cert-manager.community.tanzu.vmware.com... 
  NAME                                     VERSION  RELEASED-AT           
  cert-manager.community.tanzu.vmware.com  1.3.3    2021-08-06T12:31:21Z  
  cert-manager.community.tanzu.vmware.com  1.4.4    2021-08-23T16:47:51Z  
  cert-manager.community.tanzu.vmware.com  1.5.3    2021-08-23T17:22:51Z  
  • Install the cert-manager package

$ tanzu package install cert-manager \
  --package-name cert-manager.community.tanzu.vmware.com \
  --version 1.4.4
/ Installing package 'cert-manager.community.tanzu.vmware.com' 
| Getting namespace 'default' 
| Getting package metadata for 'cert-manager.community.tanzu.vmware.com' 
| Creating service account 'cert-manager-default-sa' 
| Creating cluster admin role 'cert-manager-default-cluster-role' 
| Creating cluster role binding 'cert-manager-default-cluster-rolebinding' 
- Creating package resource 
- Package install status: Reconciling 


 Added installed package 'cert-manager' in namespace 'default'
  • Validate the package installation

$ k get ns                                                
NAME                        STATUS   AGE
cert-manager                Active   24s
default                     Active   16m
kube-node-lease             Active   16m
kube-public                 Active   16m
kube-system                 Active   16m
tanzu-package-repo-global   Active   16m
tkg-system                  Active   16m
tkg-system-public           Active   16m
tkr-system                  Active   16m
 (main) $ k get po -n cert-manager         
NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-547fbbfbfb-b9bw6              1/1     Running   0          28s
cert-manager-cainjector-64bf4696c7-rz2fz   1/1     Running   0          28s
cert-manager-webhook-7497dbb564-bfwfp      1/1     Running   0          28s




$ tanzu package installed list               
/ Retrieving installed packages... 
  NAME          PACKAGE-NAME                             PACKAGE-VERSION  STATUS               
  cert-manager  cert-manager.community.tanzu.vmware.com  1.4.4            Reconcile succeeded  

Deleting a TCE cluster

Run the below command to delete the TCE Cluster.


$ tanzu standalone-cluster delete tkgondocker
Loading bootstrap cluster config for standalone cluster at '/Users/dinetrip/.config/tanzu/tkg/clusterconfigs/tkgondocker.yaml'
no bootstrap cluster config found - using default config
Deleting standalone cluster 'tkgondocker'. Are you sure? [y/N]: y

loading cluster config file at 
Setting up cleanup cluster...
Installing providers to cleanup cluster...
Moving all Cluster API objects from bootstrap cluster to standalone cluster...
Waiting for the Cluster API objects to be ready after restore ...
Deleting standalone cluster...
Standalone cluster 'tkgondocker' deleted.
Deleting the standalone cluster context from the kubeconfig file '/Users/dinetrip/.kube/config'
warning: this removed your active context, use "kubectl config use-context" to select a different one

Standalone cluster deleted!
Removing temporary bootstrap cluster config for standalone cluster at '/Users/dinetrip/.config/tanzu/tkg/configs/tkgondocker_ClusterConfig'
no bootstrap cluster config found - skipping
Removing temporary UI bootstrap cluster config for standalone cluster at '/Users/dinetrip/.config/tanzu/clusterconfigs/tkgondocker.yaml'
no UI bootstrap cluster config found - skipping

  • You will notice that the another kind cluster created, notice the container.


$ docker ps
CONTAINER ID   IMAGE                                                         COMMAND                  CREATED          STATUS          PORTS                                  NAMES
**768442c470da   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   9 seconds ago    Up 2 seconds    127.0.0.1:59192->6443/tcp              tkg-kind-c5dhe4sbcv4a5gqv1dv0-control-plane**
225d61b6f9ba   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   21 minutes ago   Up 21 minutes                                          tkgondocker-md-0-7b9758c6d6-wqdlk
347bf31db466   projects.registry.vmware.com/tkg/kind/node:v1.21.2_vmware.1   "/usr/local/bin/entr…"   23 minutes ago   Up 23 minutes   40819/tcp, 127.0.0.1:40819->6443/tcp   tkgondocker-control-plane-cmqpg
8167fc8ee5e4   kindest/haproxy:v20210715-a6da3463                            "haproxy -sf 7 -W -d…"   23 minutes ago   Up 23 minutes   41431/tcp, 0.0.0.0:41431->6443/tcp     tkgondocker-lb
  • Finally, it is cleaned up

$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s