Merge pull request #11 from Ragin-LundF/feature/selection_of_templates

Feature/selection of templates
Ragin-LundF · Jun 2, 2020 · 84832d0 · 84832d0
2 parents d4b91e5 + 3734a51
commit 84832d0
Show file tree

Hide file tree

Showing 12 changed files with 352 additions and 79 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,36 @@
+# 1.12.0
+Adding support for cloud template selection
+
+In this release it is possible to create the project configurations more dynamically by selecting different cloud templates.
+The file `jcasc_config.yaml` should now have a `##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##` placeholder:
+
+```yaml
+  clouds:
+    - kubernetes:
+        name: "jenkins-build-slaves"
+        serverUrl: ""
+        serverCertificate: ##KUBERNETES_SERVER_CERTIFICATE##
+        directConnection: false
+        skipTlsVerify: true
+        namespace: "##NAMESPACE##"
+        jenkinsUrl: "http://##JENKINS_MASTER_DEPLOYMENT_NAME##:8080"
+        maxRequestsPerHostStr: 64
+        retentionTimeout: 5
+        connectTimeout: 10
+        readTimeout: 20
+        templates:
+##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##
+```
+
+**It is important, that the placeholder is at the beginning of the line.**
+
+If a folder called `cloud-templates` is existing in the `templates` folder, then all files in this directory will be shown as possible cloud-templates.
+The user can then select which (none or multiple) sub-templates should be added to the main template.
+
+These sub-templates must also start on the beginning of the line.
+For an example have a look here: [templates/cloud-templates/node.yaml](./templates/cloud-templates/node.yaml)
+
+
 # 1.11.0
 Adding support to disable Jenkins ingress as default while installing ingress controller.
 
@@ -8,12 +41,12 @@ To create the needed project, there is a new menu point called `createDeployment
 The installation for such projects is looking for Jenkins Helm Chart values and if they are not available it skips the installation of Jenkins and sets an internal flag to prevent the installation of a Jenkins ingress routing.
 
 
-
 # 1.10.1
 Hotfix for MacOS/BSD
 
 MacOS/BSD does not support `find -printf`. Now it uses the default `ls` command for directory listening.
 
+
 # 1.10.0
 Shell script support and directory selection.
 
@@ -43,7 +76,6 @@ k8smanagement:
     additionalNamespaces: []
 ```
 
-
 # 1.8.0 #
 - Improved scripts
   - Missing dialog for apply secrets to a namespace added

diff --git a/README.md b/README.md
@@ -12,6 +12,8 @@
 * [How to use](#how-to-use)
   * [k8s-jcasc.sh arguments](#k8s-jcascsh-arguments)
   * [k8s-jcasc.sh commands](#k8s-jcascsh-commands)
+  * [Templates](#templates)
+    * [Sub-Templates (cloud-templates)](#sub-templates-cloud-templates)
 * [Execution of Scripts](#execution-of-scripts)
 * [IP Management](#ip-management)
 * [Additional tools](#additional-tools)
@@ -286,8 +288,55 @@ The following commands are supported:
 | `applysecrets` | Apply the secrets to the Kubernetes namespace (global secrets or project secrets, depending on configuration). |
 | `applySecretsToAll` | Apply the secrets to all known namespaces in Kubernetes (global secrets only). |
 | `createproject` | Create a new Jenkins project for the configuration and deployment values from the templates. It uses a wizard to ask for relevant data. |
+| `createdeploymentonlyproject` | Create a new project without Jenkins for deployments. This can be used to reserve/manage the IP inside the tool and to install a loadbalancer and ingress controller to this namespace. |
 | `createJenkinsUserPassword` | Create a new bcryted Jenkins user password with `htpasswd`. You can also use this online site to create a password: https://www.devglan.com/online-tools/bcrypt-hash-generator  |
 
+## Templates ##
+
+At the `templates` directory contains the following:
+- `cloud-templates` -> support for subsets for Jenkins Cloud templates
+- `jcasc_config.yaml` -> Jenkins Configuration as Code template
+- `jenkins_helm_values.yaml` -> Basic Jenkins configuration of the Helm Charts template
+- `nginx_ingress_helm_values.yaml` -> Nginx Ingress Controller Helm Chart values template
+- `pvc_claim.yaml` -> Template for Persistent Volume Claim
+- `secrets.sh` -> Example of secrets.sh script
+
+### Sub-Templates (cloud-templates) ###
+
+`k8s-jcasc-management` supports additional sub-templates to create projects with dynamic container configuration.
+These templates are located in the `./templates/cloud-templates/` directory.
+If this directory does not exist, the `create project` wizard will not ask for other sub-templates.
+
+All files stored there can be selected with the process/menu `create project` and will added to the `jcasc_config.yaml`. 
+
+The file `jcasc_config.yaml` should now have a `##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##` placeholder:
+
+```yaml
+  clouds:
+    - kubernetes:
+        name: "jenkins-build-slaves"
+        serverUrl: ""
+        serverCertificate: ##KUBERNETES_SERVER_CERTIFICATE##
+        directConnection: false
+        skipTlsVerify: true
+        namespace: "##NAMESPACE##"
+        jenkinsUrl: "http://##JENKINS_MASTER_DEPLOYMENT_NAME##:8080"
+        maxRequestsPerHostStr: 64
+        retentionTimeout: 5
+        connectTimeout: 10
+        readTimeout: 20
+        templates:
+##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##
+```
+
+**It is important, that the placeholder is at the beginning of the line.**
+
+If files are inside of this directory, the user can then select which (none or multiple) sub-templates should be added to the main template.
+
+These sub-templates must also start on the beginning of the line.
+For an example have a look here: [templates/cloud-templates/node.yaml](./templates/cloud-templates/node.yaml)
+
+
 # Execution of Scripts #
 
 It is also possible to create shell scripts for namespaces/directories. This can be helpful, if you want to install other tools besides Jenkins.

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-1.11.0
+1.12.0
diff --git a/k8s-jcasc.sh b/k8s-jcasc.sh
@@ -30,6 +30,8 @@ fi
 source ./scripts/arguments_utils.sh
 # shellcheck source=scripts/cleanup_k8s_utils.sh
 source ./scripts/cleanup_k8s_utils.sh
+# shellcheck source=scripts/cloud_templates.sh
+source ./scripts/cloud_templates.sh
 # spellcheck source=scripts/filedir_utils.sh
 source ./scripts/filedir_utils.sh
 # spellcheck source=scripts/ipconfig_utils.sh

diff --git a/scripts/cloud_templates.sh b/scripts/cloud_templates.sh
@@ -0,0 +1,39 @@
+#!/bin/bash
+
+##########
+# Find templates from "templates/cloud-templates" directory.
+# Returns empty value if nothing was found or an array of the files
+#
+# argument 1: variable in which the result should be written (return value)
+##########
+function findJenkinsCloudTemplates() {
+    # arguments
+    local ARG_RETVALUE_CLOUD_TEMPLATES=$1
+    local __INTERNAL_TEMPLATES_FILE_ARRAY
+    if [[ -d "${TEMPLATES_BASE_DIRECTORY}cloud-templates" ]]; then
+        __INTERNAL_TEMPLATES_FILE_ARRAY=($(ls "${TEMPLATES_BASE_DIRECTORY}cloud-templates" | grep yaml | sort))
+    fi
+    eval ${ARG_RETVALUE_CLOUD_TEMPLATES}="(\${__INTERNAL_TEMPLATES_FILE_ARRAY[@]})"
+}
+
+##########
+# Read selected templates from "templates/cloud-templates" directory.
+# This method will do an "echo" if the result is not empty.
+# Usage: RESULT_VAR=$(readSelectedCloudTemplates "${VAR[@]}")
+#
+# argument 1: array with selected files
+##########
+function readSelectedCloudTemplates() {
+    # arguments
+    local ARG_CLOUD_TEMPLATES=("$@")
+
+    for __CLOUD_TEMPLATE_NAME in "${ARG_CLOUD_TEMPLATES[@]}"
+    do
+        local __INTERNAL_CLOUD_TEMPLATE_CAT
+        __INTERNAL_CLOUD_TEMPLATE_CAT=$(cat "${TEMPLATES_BASE_DIRECTORY}cloud-templates/${__CLOUD_TEMPLATE_NAME//\"/}")
+        # if content was not empty, add it to return value
+        if [[ -n "${__INTERNAL_CLOUD_TEMPLATE_CAT}" ]]; then
+            echo "${__INTERNAL_CLOUD_TEMPLATE_CAT}"
+        fi
+    done
+}
diff --git a/scripts/dialogs.sh b/scripts/dialogs.sh
@@ -176,6 +176,52 @@ function dialogAskForJenkinsSystemMessage() {
     eval ${ARG_RETVALUE}="\${__INTERNAL_JENKINS_SYSMSG}"
 }
 
+##########
+# Ask for Jenkins cloud templates if they exist.
+# If user is selecting file(s), the method returns the content.
+#
+# argument 1: variable in which the result should be written (return value)
+##########
+function dialogAskForCloudTemplates() {
+    # arguments
+    local ARG_RETVALUE=$1
+    local __INTERNAL_CLOUD_TEMPLATES_SELECTION
+    local __INTERNAL_CLOUD_TEMPLATES_FOUND
+    findJenkinsCloudTemplates __INTERNAL_CLOUD_TEMPLATES_FOUND
+
+    # check if cloud templates were found
+    if [[ -n "${__INTERNAL_CLOUD_TEMPLATES_FOUND}" ]]; then
+        # prepare them for dialogs
+        for __INTERNAL_CLOUD_TMPLP_IDX in "${!__INTERNAL_CLOUD_TEMPLATES_FOUND[@]}"
+        do
+            echo "$((__INTERNAL_CLOUD_TMPLP_IDX+1)) - ${__INTERNAL_CLOUD_TEMPLATES_FOUND[__INTERNAL_CLOUD_TMPLP_IDX]}"
+        done
+
+        # ask user for templates
+        echo "Enter the number(s) of the cloud templates you want to use. To select multiple templates, separate them with comma (,). Leave empty if you do not want to add a template."
+        read -p "Templates: " __INTERNAL_CLOUD_TEMPLATES_USER_ENTRIES
+        if [[ -n "${__INTERNAL_CLOUD_TEMPLATES_USER_ENTRIES}" ]]; then
+            __INTERNAL_CLOUD_TEMPLATES_USER_ENTRIES_ARR=(${__INTERNAL_CLOUD_TEMPLATES_USER_ENTRIES//,/ })
+            __INTERNAL_CLOUD_TEMPLATES_SELECTION=()
+            # iterate over selection
+            for __INTERNAL_CLOUD_TEMPLATES_USER_SELECT_IDX in "${__INTERNAL_CLOUD_TEMPLATES_USER_ENTRIES_ARR[@]}"
+            do
+                # first, calc user entry -1 to be in sync with index and remove existing spaces
+                local __INTERNAL_CLOUD_TMPL_IDX_TRIM
+                __INTERNAL_CLOUD_TMPL_IDX_TRIM=$((__INTERNAL_CLOUD_TEMPLATES_USER_SELECT_IDX-1))
+                __INTERNAL_CLOUD_TMPL_IDX_TRIM=$(echo -e "${__INTERNAL_CLOUD_TMPL_IDX_TRIM}" | sed -e 's/[[:space:]]//')
+                # add filenames from original array to selection array
+                __INTERNAL_CLOUD_TEMPLATES_SELECTION=("${__INTERNAL_CLOUD_TEMPLATES_SELECTION[@]}" "${__INTERNAL_CLOUD_TEMPLATES_FOUND[${__INTERNAL_CLOUD_TMPL_IDX_TRIM}]}" )
+            done
+        fi
+
+        # read content of selected templates
+        __INTERNAL_JENKINS_CLOUD_CONTENT=$(readSelectedCloudTemplates "${__INTERNAL_CLOUD_TEMPLATES_SELECTION[@]}")
+    fi
+
+    eval ${ARG_RETVALUE}="\${__INTERNAL_JENKINS_CLOUD_CONTENT}"
+}
+
 ##########
 # Ask for existing persistence claim.
 #

diff --git a/scripts/dialogs_dialog.sh b/scripts/dialogs_dialog.sh
@@ -233,6 +233,46 @@ function dialogAskForJenkinsSystemMessage() {
     eval ${ARG_RETVALUE}="\${__INTERNAL_JENKINS_SYSMSG}"
 }
 
+##########
+# Ask for Jenkins cloud templates if they exist.
+# If user is selecting file(s), the method returns the content.
+#
+# argument 1: variable in which the result should be written (return value)
+##########
+function dialogAskForCloudTemplates() {
+    # arguments
+    local ARG_RETVALUE=$1
+    local __INTERNAL_CLOUD_TEMPLATES_SELECTION
+    local __INTERNAL_CLOUD_TEMPLATES_FOUND
+    findJenkinsCloudTemplates __INTERNAL_CLOUD_TEMPLATES_FOUND
+
+    # check if cloud templates were found
+    if [[ -n "${__INTERNAL_CLOUD_TEMPLATES_FOUND}" ]]; then
+        # prepare them for dialogs
+        local __INTERNAL_CLOUD_TEMPLATE_DIALOG
+        for __INTERNAL_CLOUD_TEMPLATE in "${__INTERNAL_CLOUD_TEMPLATES_FOUND[@]}"
+        do
+            __INTERNAL_CLOUD_TEMPLATE_DIALOG="${__INTERNAL_CLOUD_TEMPLATE_DIALOG} ${__INTERNAL_CLOUD_TEMPLATE} ___ OFF "
+        done
+
+        # ask user for templates
+        __INTERNAL_CLOUD_TEMPLATES_SELECTION=($(dialog \
+            --title "Select cloud templates" \
+            --backtitle "Select cloud templates" \
+            --clear \
+            --checklist \
+            "Choose the templates you want to use for this Jenkins installation" 0 0 10 \
+            ${__INTERNAL_CLOUD_TEMPLATE_DIALOG} \
+            3>&1 1>&2 2>&3
+        ))
+
+        # read content of selected templates
+        __INTERNAL_JENKINS_CLOUD_CONTENT=$(readSelectedCloudTemplates "${__INTERNAL_CLOUD_TEMPLATES_SELECTION[@]}")
+    fi
+
+    eval ${ARG_RETVALUE}="\${__INTERNAL_JENKINS_CLOUD_CONTENT}"
+}
+
 ##########
 # Ask for existing persistence claim.
 #

diff --git a/scripts/dialogs_whiptail.sh b/scripts/dialogs_whiptail.sh
@@ -227,6 +227,45 @@ function dialogAskForJenkinsSystemMessage() {
     eval ${ARG_RETVALUE}="\${__INTERNAL_JENKINS_SYSMSG}"
 }
 
+##########
+# Ask for Jenkins cloud templates if they exist.
+# If user is selecting file(s), the method returns the content.
+#
+# argument 1: variable in which the result should be written (return value)
+##########
+function dialogAskForCloudTemplates() {
+    # arguments
+    local ARG_RETVALUE=$1
+    local __INTERNAL_CLOUD_TEMPLATES_SELECTION
+    local __INTERNAL_CLOUD_TEMPLATES_FOUND
+    findJenkinsCloudTemplates __INTERNAL_CLOUD_TEMPLATES_FOUND
+
+    # check if cloud templates were found
+    if [[ -n "${__INTERNAL_CLOUD_TEMPLATES_FOUND}" ]]; then
+        # prepare them for dialogs
+        local __INTERNAL_CLOUD_TEMPLATE_WHIPTAIL
+        for __INTERNAL_CLOUD_TEMPLATE in "${__INTERNAL_CLOUD_TEMPLATES_FOUND[@]}"
+        do
+            __INTERNAL_CLOUD_TEMPLATE_WHIPTAIL="${__INTERNAL_CLOUD_TEMPLATE_WHIPTAIL} ${__INTERNAL_CLOUD_TEMPLATE} ___ OFF "
+        done
+
+        # ask user for templates
+        __INTERNAL_CLOUD_TEMPLATES_SELECTION=($(whiptail \
+            --title "Select cloud templates" \
+            --clear \
+            --checklist \
+            "Choose the templates you want to use for this Jenkins installation" 0 0 10 \
+            ${__INTERNAL_CLOUD_TEMPLATE_WHIPTAIL} \
+            3>&1 1>&2 2>&3
+        ))
+
+        # read content of selected templates
+        __INTERNAL_JENKINS_CLOUD_CONTENT=$(readSelectedCloudTemplates "${__INTERNAL_CLOUD_TEMPLATES_SELECTION[@]}")
+    fi
+
+    eval ${ARG_RETVALUE}="\${__INTERNAL_JENKINS_CLOUD_CONTENT}"
+}
+
 ##########
 # Ask for existing persistence claim.
 #

diff --git a/scripts/project_wizard_controller.sh b/scripts/project_wizard_controller.sh
@@ -250,6 +250,26 @@ function createProjectFromTemplate() {
     fi
 }
 
+##########
+# Process the cloud template configuration placeholder
+#
+# argument 1: directory of the project
+# argument 2: content of the selected templates
+##########
+function processTemplatesWithCloudTemplates() {
+    # arguments
+    local ARG_FULL_PROJECT_DIRECTORY=$1
+    local ARG_SELECTED_CLOUD_TEMPLATES=$2
+
+    if [[ -n "${ARG_SELECTED_CLOUD_TEMPLATES}" ]]; then
+        echo "${ARG_SELECTED_CLOUD_TEMPLATES}" | while IFS=$'\n' read __INTERNAL_CONTENT_AS_ARRAY; do
+            replaceStringInFile "##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##" "          ${__INTERNAL_CONTENT_AS_ARRAY}\n##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##" "${ARG_FULL_PROJECT_DIRECTORY}/jcasc_config.yaml"
+        done
+    fi
+    # replace left template variable with empty comment
+    replaceStringInFile "##K8S_MGMT_JENKINS_CLOUD_TEMPLATES##" "          #" "${ARG_FULL_PROJECT_DIRECTORY}/jcasc_config.yaml"
+}
+
 ##########
 # Project wizard delegation method to trigger dialogs and
 # execute the resulting actions
@@ -260,6 +280,7 @@ function projectWizard() {
     local VAR_PROJECT_DIRECTORY
     local VAR_NAMESPACE
     local VAR_IP_ADDRESS
+    local VAR_CLOUD_TEMPLATES
     local VAR_JENKINS_SYSTEM_MESSAGE
     local VAR_JENKINS_JOB_CONFIGURATION_REPOSITORY
     local VAR_EXISTING_PERSISTENCE_CLAIM
@@ -276,20 +297,24 @@ function projectWizard() {
     # collect all information from dialogs
     dialogAskForNamespace VAR_NAMESPACE
     dialogAskForIpAddress VAR_IP_ADDRESS "${VAR_NAMESPACE}"
+    dialogAskForCloudTemplates VAR_CLOUD_TEMPLATES
     dialogAskForJenkinsSystemMessage VAR_JENKINS_SYSTEM_MESSAGE
     dialogAskForJenkinsJobConfigurationRepository VAR_JENKINS_JOB_CONFIGURATION_REPOSITORY
     dialogAskForExistingPersistenceClaim VAR_EXISTING_PERSISTENCE_CLAIM
 
     # target directory
     local __INTERNAL_FULL_PROJECT_DIRECTORY="${PROJECTS_BASE_DIRECTORY}${VAR_PROJECT_DIRECTORY}"
 
-    # all data collected -> start create new project
-    createProjectFromTemplate "${__INTERNAL_FULL_PROJECT_DIRECTORY}" "${VAR_EXISTING_PERSISTENCE_CLAIM}"
-
     # everything looks fine, lets add the IP address and namespace name to the configuration
     addIpToIpConfiguration "${VAR_IP_ADDRESS}" "${VAR_NAMESPACE}"
 
+    # all data collected -> start create new project
+    createProjectFromTemplate "${__INTERNAL_FULL_PROJECT_DIRECTORY}" "${VAR_EXISTING_PERSISTENCE_CLAIM}"
+
     # start processing the templates
+    ## First process the cloud templates, because they contain things, that will be replaced later!
+    processTemplatesWithCloudTemplates "${__INTERNAL_FULL_PROJECT_DIRECTORY}" "${VAR_CLOUD_TEMPLATES}"
+
     ## Jenkins system message should be the first, because it overwrites the message, if a custom message was defined
     processTemplatesWithJenkinsSystemMessage "${__INTERNAL_FULL_PROJECT_DIRECTORY}" "${VAR_JENKINS_SYSTEM_MESSAGE}"
     processTemplatesWithJenkinsJobRepository "${__INTERNAL_FULL_PROJECT_DIRECTORY}" "${VAR_JENKINS_JOB_CONFIGURATION_REPOSITORY}"