One of the most requested feature with previous versions of the VMware Container Service Extension (CSE) is to add a native UI to it. As of CSE 2.6 we have added a native UI to CSE, which is adding to the friendliness of CSE and will make it much more appealing to many of our cloud providers. At just few clicks, our customers can deploy a K8S clusters at our Cloud Providers with filling few easy to understand fields.
Kubeconfig file will be auto generated as well and can be handed out right away to the developer limiting the efforts required by the tenant operation team/cloud providers administrators. Here is a quick screenshot teaser of what CSE look like. You can find a nice demo of CSE 2.6.1 UI at my following blog post: Cloud Director Kubernetes as a Service with CSE 2.6.x Demo
For more info on what is new with CSE 2.6.x please check my following blog post: vCloud Director Container Service Extension 2.6.x is here
In this post, I am assuming you have an existing vCloud Director environment and AMQP already configured. To start the installation of CSE 2.6.1, you will need a supported OS. In my case, I have decided to go with CentOS 8.1. After installation CentOS 8.1, you will need to install the development tools and few libraries. Use the below few commands to achieve that.
[root@vtcse01 ~]# sudo yum -y groupinstall "Development Tools" [root@vtcse01 ~]# sudo yum -y install openssl-devel bzip2-devel libffi-devel [root@vtcse01 ~]# gcc --version gcc (GCC) 8.3.1 20190507 (Red Hat 8.3.1-4) Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [root@vtcse01 ~]# sudo yum -y install wget
************ Steps to Fix the SQLite Issue Start ******
The SQLite version including with CentOS is not adequate for the installation of CSE/vcd-cli, as it’s still using an older version of it. We will need to follow the below steps to upgrade it and to avoid hitting the issue I have documented in the following post: Running vCD Cli fail with the following error: ModuleNotFoundError: No module named ‘_sqlite3’ . Please note that particular post go into more details in explaining the below steps, so if you want further understanding of it, please check it out.
[root@vtcse01 Python-3.7.3]# wget https://www.sqlite.org/2020/sqlite-autoconf-3310100.tar.gz [root@vtcse01 Python-3.7.3]# tar xvf sqlite-autoconf-3310100.tar.gz [root@vtcse01 Python-3.7.3]# cd sqlite-autoconf-3310100/ [root@vtcse01 Python-3.7.3]# ./configure --prefix=/usr [root@vtcse01 Python-3.7.3]# make install Check your sqlite version is upgrade as expected [root@vtcse01 Python-3.7.3]# sqlite3 –version 3.31.1 2020-01-27 19:55:54 3bfa9cc97da10598521b342961df8f5f68c7388fa117345eeb516eaa837bb4d6
************ Steps to Fix the SQLite Issue End ******
CSE will require Python 3.7.3 or higher. In here, I am going to install Python 3.7.3. Here is the steps to follow and install it from source code.
[root@vtcse01 ~]# wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tgz [root@vtcse01 ~]# tar xvf Python-3.7.3.tgz [root@vtcse01 ~]# cd Python-3.7*/ [root@vtcse01 Python-3.7.3]# ./configure --enable-optimizations [root@vtcse01 Python-3.7.3]# make install [root@vtcse01 Python-3.7.3]# python3.7 --version; Python 3.7.3 [root@vtcse01 Python-3.7.3]# pip3.7 –version
Check below that Python3.7.3 is detecting your updated SQLite
[root@vtcse01 Python-3.7.3]# python3 Python 3.7.3 (default, May 4 2020, 15:36:31) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>> sqlite3.sqlite_version '3.31.1'
Now we have all our pre-requisites satisfied, let’s start with the CSE installation.
[root@vtcse01 Python-3.7.3]# pip3 install container-service-extension
Test the CSE and vcd-cli commands
[root@vtcse01 Python-3.7.3]# cse version CSE, Container Service Extension for VMware vCloud Director, version 2.6.1 [root@vtcse01 Python-3.7.3]# vcd cse version Error: No such command 'cse'. Note: The above error is OK, as we will need to add the CSE extension to vcd config files.
At first test that you can login to your vCD setup with the vCD CLI
[root@vtcse01 Python-3.7.3]# vcd login vcd.vt.com system administrator -i Password: administrator logged in, org: 'system', vdc: ''
Please note above I have used the -i flag to ignore my self-signed certificates. Not recommended for a production environment, but if you have a production environment, I assume you will have a valid certs on your vCD and you won’t need the -i flag.
Now we need to modify the ~/.vcd-cli/profiles.yaml file by adding the following lines at the end of the file
extensions: - container_service_extension.client.cse
Here is what the file look like in my setup:
active: default profiles: - api_version: '33.0' disable_warnings: false host: vcd.doomdns.org is_jwt_token: true log_body: true log_header: true log_request: true name: default org: system org_href: https://vcd.vt.org/api/org/a93c9db9-7471-3192-8d09-a8f7eeda85f9 org_in_use: system token: eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbmlzdHJhdG9yIiwiaXNzIjoiYTkzYzlkYjktNzQ3MS0zMTkyLThkMDktYThmN2VlZGE4NWY5QDU3ZDIyMTY5LTcwYWEtNDNjNS04YzNkLWMyNGU5YzI3ZDc2NSIsImV4cCI6MTU4ODk0MTk2NCwidmVyc2lvbiI6InZjbG91ZF8xLjAiLCJqdGkiOiI2MGI3ZTU1N2MzZjA0MTk3OTk1YmM5NDQ0MGJhMWRlMCJ9.NFpVjmh0KpIpA43HcAbv7epovb7NZ_gm0Oz29ihMACnmTAl4rlMRx2VmFHjf004l0_TBYYZwv1FkB9khHKAbVqnQ_gl4C2O7v_odYg8WSB8AH6CqC54eAfXaQdFbk8zo3qHVu34xu3OkugfVYaH362AolGJ8O7e01Arf2hUctX3tTF2m7nARvk3CMyR9alCOnZofeoj7CTHGgmM6yqL87fEhTSq9s3FlcFc5M9HzeXqB6HNVHKi93g8WSB8AH6CqC54eAfXaQdFbk8zo3qHVu34xu3OkugfV user: administrator vapp_href: '' vapp_in_use: '' vdc_href: '' vdc_in_use: '' verify: false extensions: - container_service_extension.client.cse
Now let’s try the vcd cse command as the above modification of the profiles.yaml files should get it ready for us to use.
[root@vtcse01 Python-3.7.3]# vcd cse version CSE, Container Service Extension for VMware vCloud Director, version 2.6.1
Create CSE Config file
Let’s start by creating a sample config.yaml file by running the following command:
[root@vtcse01 Python-3.7.3]# cse sample -o config.yaml
Here is a sample config file
amqp: exchange: cse-ext host: amqp.vmware.com password: guest port: 5672 prefix: vcd routing_key: cse ssl: false ssl_accept_all: false username: guest vhost: / vcd: api_version: ‘33.0’ host: vcd.vt.com log: true password: my_secret_password port: 443 username: administrator verify: true vcs: - name: vc1 password: my_secret_password username: email@example.com verify: true - name: vc2 password: my_secret_password username: firstname.lastname@example.org verify: true service: enforce_authorization: false listeners: 10 log_wire: false telemetry: enable: true broker: catalog: cse default_template_name: ubuntu-16.04_k8-1.17_weave-2.6.0 default_template_revision: 1 ip_allocation_mode: pool network: mynetwork org: myorg remote_template_cookbook_url: https://raw.githubusercontent.com/vmware/container-service-extension-templates/master/template.yaml storage_profile: ‘*’ vdc: myorgvdc # [Optional] Template rule section # Rules can be defined to override template definitions as defined by remote # template cookbook. # Any rule defined in this section can match exactly one template. # Template name and revision must be provided for the rule to be processed.
Please note a good documentation on how to configure the above configuration file can be found at: https://github.com/vmware/container-service-extension/blob/master/docs/CSE_CONFIG.md. Please note that you will need to fill the default template field correctly as shown in the above sample file, else you will end up with the error I have documented in the following blog post: CSE 2.6.1 Error: Default template my_template with revision 0 not found. Unable to start CSE server.
CSE requires a valid SSH key, so before you can proceed any further, you will need to generate one. Use the below command to do so:
[root@vtcse01 Python-3.7.3]# ssh-keygen -t rsa -b 4096 -C “email@example.com” Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:YoATaF+IR9l5/o0pPtvSr5522nWkybNqwg0oRC6oI “firstname.lastname@example.org” The key's randomart image is: +---[RSA 4096]----+ | .+.= . | |.o O + o | |. * + + . . | |.. o o + + o | |W . = V = T..| | . . * * o =.o.| | . = o 0o | | o = .o . | | . o. o. | +----[SHA256]-----+
Now let’s run the CSE installation with config.yaml file we have created. Please make sure that was updated to include your environment information
We need to encrypt the configuration file first in version 2.6 and higher, while there is options to skip that, it is highly recommended in production environment and help protect your configuration files
[root@vtcse01 Python-3.7.3]# cse encrypt config.yaml --output encrypted-config.yaml Required Python version: >= 3.7.3 Installed Python version: 3.7.3 (default, May 4 2020, 15:36:31) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] Password for config file encryption: Encryption successful [root@vtcse01 Python-3.7.3]# chmod 600 encrypted-config.yaml
Now, it is show time. We can now run the CSE install command to prepare the environment and download the desired Kubernetes templates and prepare them in your catalog.
[root@vtcse01 Python-3.7.3]# cse install -c encrypted-config.yaml --ssh-key /root/.ssh/id_rsa.pub Required Python version: >= 3.7.3 Installed Python version: 3.7.3 (default, May 4 2020, 15:36:31) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)] Password for config file decryption: Decrypting 'encrypted-config.yaml' Validating config file 'encrypted-config.yaml' Connected to AMQP server (VTAMQP01.vt.com:5672) InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. Connected to vCloud Director (vcd.doomdns.org:443) Connected to vCenter Server 'vtvc01' as 'email@example.com' (vtvc01.vt.com:None) Config file 'encrypted-config.yaml' is valid Installing CSE on vCloud Director using config file 'encrypted-config.yaml' ………..
Try to run use the following run command, and things should work unless if you have a mis-configuration issue:
[root@vtcse01 Python-3.7.3]# cse run --config encrypted-config.yaml
Download the CSE Plugin from the following source: https://github.com/vmware/container-service-extension/raw/master/cse_ui/1.0.1/container-ui-plugin.zip, then upload it into your vCloud Director under More ==> Customize Portal
Now you are ready to start using CSE, you can get to it from the “Kubernetes Container Clusters” under the More menu at the top.