researchAnsible: updated with setup of latest AWX version

The latest version of AWX requires a Kubernetes Cluster. This
commit holds the update of the page and shows both installation
methodes (directly with docker and the latest version with a
minikube).

Additionally, I added a new section setting up a virtual machine
for test purpose. With it another section for accessing such
machines over SSH.

Signed-off-by: Marc Mattmüller <marc.mattmueller@netmodule.com>

doc/src/nextlevel-ci/researchAnsible.rst

@@ -3,38 +3,97 @@
 *******************************
 Research about Ansible Playbook
 *******************************
+This chapter describes all the trials, researches and work done for the topic ansible playbook and as well AWX
+(sponsored by Red Hat) as the upstream project from which the automation controller component is ultimately derived. To
+have more flexibility and a bigger playground the environment needs another machine. Therefore a virtual machine is set
+up using Virtual Box.
+
+This work gives input for future enhancements at HAC and NetModule regarding automated bring-up of CI services.
+
+
+Setting up a Virtual Machine
+############################
+As already mentioned above, an additional machine helps to collect more information about the topic. I set up a virtual
+machine with Debian 11 without graphical UI and with its network bridged to the host to ease things up. There are two
+users available:
+
+*  root - theRootPw4thisVM!
+*  user - theUser!
+
+    -  added to the group sudo
+
+.. note::
+    You might have troubles to start your virtual machine when you are using secure boot on your host. I disabled
+    secure boot on my host machine to be able starting a virtual machine.
+
+
+
 Installation Ansible AWX
 ########################
-The following steps bring up an AWX server using docker.
+There are some important links to keep in mind :-D
+
+*  the link to the AWX `repository <https://github.com/ansible/awx>`_
+*  the `docker-compose readme <https://github.com/ansible/awx/blob/devel/tools/docker-compose/README.md#start-with-minikube>`_
+*  a `getting started guide <https://www.raptorswithhats.com/2022-05-getting-started-with-awx/>`_
+*  the `minikube doc <https://minikube.sigs.k8s.io/docs/start/>`_
+*  another `installation guide <https://www.linuxtechi.com/install-ansible-awx-kubernetes-minikube/>`_
+*  another explanation `video <https://asciinema.org/a/416946>`_
+*  set up a `first playbook on GUI <https://www.linuxtechi.com/run-schedule-ansible-playbook-awx-gui/>`_
+*  example `playbook with git <https://www.middlewareinventory.com/blog/ansible-git-example/>`_
+
+From AWX version 18.0.0 the AWX operator (installation recommendation) requires a Kubernetes Cluster, i.e. when
+installing a version before (= 17.1.0) you can run AWX directly in a docker container. Both ways are described in the
+belonging subsections.
+
+For a new installation I propose to use the latest version, hence finally I will setup AWX by using a kubernetes cluster
+like minikube.
+
+But first there is some preparation work, see in the next section.
+
+
+Preparation Steps
+*****************
+There are some prerequisites being able to work:
+
+#.  Install necessary tools:
+
+     .. note::
+         Assuming docker and docker-composed is already installed
+
+     .. code-block:: bash
+
+         sudo apt install nodejs npm -y
+         #npm install npm --global
+         sudo apt install python3-pip pwgen -y
+         sudo apt install ansible -y
+
+
+#.  Clone the AWX repository
+
+     .. code-block:: bash
+
+         cd ~
+         git clone https://github.com/ansible/awx.git
+         cd awx
+         git fetch -t
+
+     .. note::
+         The next subsection will use different tags, thus we do not yet checkout a specific tag here.
+
+
+Run AWX directly with Docker (Version 17.1.0)
+*********************************************
+Assuming you reside in the path ``~/awx`` (repository root path), the following steps bring up an AWX server using
+docker.
 
 *  Preparation steps
 
-    #.  Install prerequisits:
+    #.  Check out the version needed
-
-        .. note::
-            Assuming docker and docker-composed is already installed
-
-        .. code-block:: bash  
-
-            sudo apt install nodejs npm -y
-            #npm install npm --global
-            sudo apt install python3-pip pwgen -y
-            sudo apt install ansible -y
-
-    #.  Clone the repository and checkout a release (we use 17.1.0):
 
          .. code-block:: bash
 
-             cd ~
-             git clone https://github.com/ansible/awx.git
-             cd awx
-             git fetch -t
              git checkout 17.1.0
 
-         .. note::
-             Apparently AWX Operator (installation recommendation) requires Kubernetes Cluster. Thus, we use version
-             17.1.0 which supports docker.
-
     #.  Enter the installer directory and generate a secret key (used for the inventory):
 
          .. code-block:: bash
@@ -94,13 +153,249 @@ The following steps bring up an AWX server using docker.
         # start the container with:
         docker-compose up -d
 
+Wohooo, now we have an AWX instance up and running.
 
-Wohooo, now we have an AWX instance up and running. This instance helps you manage, scale and automatize your (IT-)
+
-infrastructure. So let's have a further look in the next section.
+Run AWX latest Version (Version 22.1.0)
+***************************************
+Assuming you reside in the path ``~/awx`` (repository root path) and the repository is somehow cleaned :-D
+As mentioned I propose to use the latest version for a new installation. Therefore we use minikube for our tests.
+
+#.  First create a minikube cluster for testing:
+
+     .. code-block:: bash
+
+         # Install the latest stable minikube:
+         # --------------------------------------
+         curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
+         sudo install minikube-linux-amd64 /usr/local/bin/minikube
+
+         # Creating s minikube cluster for testing:
+         # -------------------------------------------
+         minikube start --cpus=4 --memory=6g --addons=ingress
+         😄  minikube v1.30.1 on Ubuntu 22.04
+         ✨  Automatically selected the docker driver. Other choices: virtualbox, ssh
+         📌  Using Docker driver with root privileges
+         👍  Starting control plane node minikube in cluster minikube
+         🚜  Pulling base image ...
+         💾  Downloading Kubernetes v1.26.3 preload ...
+             > preloaded-images-k8s-v18-v1...:  397.02 MiB / 397.02 MiB  100.00% 10.90 M
+         🔥  Creating docker container (CPUs=4, Memory=6144MB) ...
+         🐳  Preparing Kubernetes v1.26.3 on Docker 23.0.2 ...
+             ▪ Generating certificates and keys ...
+             ▪ Booting up control plane ...
+             ▪ Configuring RBAC rules ...
+         🔗  Configuring bridge CNI (Container Networking Interface) ...
+             ▪ Using image registry.k8s.io/ingress-nginx/controller:v1.7.0
+             ▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
+             ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
+             ▪ Using image registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20230312-helm-chart-4.5.2-28-g66a760794
+         🔎  Verifying Kubernetes components...
+         🔎  Verifying ingress addon...
+         🌟  Enabled addons: storage-provisioner, default-storageclass, ingress
+         💡  kubectl not found. If you need it, try: 'minikube kubectl -- get pods -A'
+         🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
+
+
+         # Check if the node(s) and kube-apiserver commincation is working:
+         # -------------------------------------------------------------------
+         minikube kubectl -- get nodes
+         NAME       STATUS   ROLES           AGE   VERSION
+         minikube   Ready    control-plane   39m   v1.26.3
+
+         minikube kubectl -- get pods -A
+         NAMESPACE       NAME                                        READY   STATUS      RESTARTS      AGE
+         ingress-nginx   ingress-nginx-admission-create-xmg2s        0/1     Completed   0             38m
+         ingress-nginx   ingress-nginx-admission-patch-2nlzb         0/1     Completed   1             38m
+         ingress-nginx   ingress-nginx-controller-6cc5ccb977-5qzfd   1/1     Running     0             38m
+         kube-system     coredns-787d4945fb-bc55h                    1/1     Running     0             38m
+         kube-system     etcd-minikube                               1/1     Running     0             39m
+         kube-system     kube-apiserver-minikube                     1/1     Running     0             39m
+         kube-system     kube-controller-manager-minikube            1/1     Running     0             39m
+         kube-system     kube-proxy-6jgjq                            1/1     Running     0             38m
+         kube-system     kube-scheduler-minikube                     1/1     Running     0             39m
+         kube-system     storage-provisioner                         1/1     Running     1 (38m ago)   39m
+
+         # Create alias for easier usage:
+         # ---------------------------------
+         alias kubectl="minikube kubectl --"
+         # or add this alias to your ~/.bash_aliases
+
+         # testing the alias:
+         kubectl get nodes
+         NAME       STATUS   ROLES           AGE   VERSION
+         minikube   Ready    control-plane   39m   v1.26.3
+
+#.  Install *kustomize*
+
+     .. code-block:: bash
+
+         curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash
+
+#.  Let's install the AWX operator using kustomize:
+
+     *  Create a file ``kustomization.yaml`` with this content:
+
+         .. code-block:: bash
+
+             apiVersion: kustomize.config.k8s.io/v1beta1
+             kind: Kustomization
+             resources:
+               # Find the latest tag here: https://github.com/ansible/awx-operator/releases
+               - github.com/ansible/awx-operator/config/default?ref=2.0.1
+
+             # Set the image tags to match the git version from above
+             images:
+               - name: quay.io/ansible/awx-operator
+                 newTag: 2.0.1
+
+             # Specify a custom namespace in which to install AWX
+             namespace: awx
+
+     *  Install the created manifest accordingly:
+
+         .. code-block:: bash
+
+             ./kustomize build . | kubectl apply -f -
+
+     *  Check and wait until the AWX operator is running:
+
+         .. code-block:: bash
+
+             kubectl get pods -n awx
+             NAME                                               READY   STATUS              RESTARTS   AGE
+             awx-operator-controller-manager-67bcc886fb-5szx6   0/2     ContainerCreating   0          28s
+
+             kubectl get pods -n awx
+             NAME                                               READY   STATUS    RESTARTS   AGE
+             awx-operator-controller-manager-67bcc886fb-5szx6   2/2     Running   0          108s
+
+     *  Set the current namespace to AWX so that we do not need to repeat the ``-n awx``:
+
+         .. code-block:: bash
+
+             kubectl config set-context --current --namespace=awx
+
+#.  Now we create the AWX instance:
+
+     *  Create a file ``awx-nwl.yaml`` with this content:
+
+         .. code-block:: bash
+
+             ---
+             apiVersion: awx.ansible.com/v1beta1
+             kind: AWX
+             metadata:
+               name: awx-nwl
+             spec:
+               service_type: nodeport
+
+     *  Adapt the ``kustomization.yaml`` with the following extra line:
+
+         .. code-block:: bash
+
+             ...
+             resources:
+               # Find the latest tag here: https://github.com/ansible/awx-operator/releases
+               - github.com/ansible/awx-operator/config/default?ref=2.0.1
+               # AWX extra line:
+               - awx-nwl.yaml
+             ...
+
+     *  Run kustomize again to create the AWX in our cluster:
+
+         .. code-block:: bash
+
+             ./kustomize build . | kubectl apply -f -
+
+             # you can check the log with
+             kubectl logs -f deployments/awx-operator-controller-manager -c awx-manager
+
+             # Quit with Ctrl+C
+
+             # Check the new resources:
+             kubectl get pods -A
+             NAMESPACE              NAME                                               READY   STATUS      RESTARTS        AGE
+             awx                    awx-nwl-postgres-13-0                              1/1     Running     0               2m58s
+             awx                    awx-nwl-task-64775dfcf8-86xn9                      4/4     Running     0               2m17s
+             awx                    awx-nwl-web-659b9b7cb-bmx9g                        3/3     Running     0               45s
+             awx                    awx-operator-controller-manager-67bcc886fb-5szx6   2/2     Running     0               60m
+             ...
+
+             kubectl get svc -l "app.kubernetes.io/managed-by=awx-operator"
+             NAME                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
+             awx-nwl-postgres-13   ClusterIP   None             <none>        5432/TCP       4m15s
+             awx-nwl-service       NodePort    10.100.100.240   <none>        80:31831/TCP   3m36s
+
+     *  Get the AWX instance URL with
+
+         .. code-block:: bash
+
+             minikube service -n awx awx-nwl-service --url
+
+     *  Get the AWX instance admin password with
+
+         .. code-block:: bash
+
+             kubectl get secret awx-nwl-admin-password -o jsonpath="{.data.password}" | base64 --decode ; echo
+
+
+#.  Enter the URL you got before
+#.  Login as user=admin with the password you got before
+
+Wohooo, now we have an AWX instance up and running.
+
+Here is some useful information:
+
+*  You may want to have a dashboard for the minikube:
+
+    .. code-block:: bash
+
+        minikube dashboard
+
+        # Press Ctrl+C to close the dashboard
+
+*  Uninstall AWX:
+
+    -  To uninstall an AWX deployment instance, you basically need to remove the AWX kind related to that instance.
+       For example, to delete an AWX instance named awx-nwl, you would do:
+
+       .. code-block:: bash
+
+           kubectl delete awx awx-nwl
+
+
+This instance helps you manage, scale and automatize your (IT-) infrastructure. So let's have a further look in the next
+section.
 
 
 Basic Setup in AWX
 ##################
+Machine Access over SSH
+***********************
+For my first playbook test I need access to a virtual machine (see section `Setting up a Virtual Machine`_). Thus,
+another SSH keypair is needed of which we copy the public one to the VM (assuming the VM is running):
+
+.. code-block:: bash
+
+    # generating the new keypair:
+    ssh-keygen -t ed25519 -f awx-client -C "awxclient@testenv"
+
+    # copy the public key to the VM:
+    ssh-copy-id -i ~/.ssh/awx-client.pub user@<VM-IP-address>
+
+    # test the connection:
+    ssh user@<VM-IP-address> "hostname;exit"
+
+
+Creating an Organization
+************************
+In your browser enter your AWX URL and log in as admin (see above for details).
+
+*  Select the menu *Organizations* and click *Add*
+*  Enter *Testenv* as Name and a Description
+
+
 SCM (=Git) Access
 *****************
 First of all your instance needs git credentials to have access to your infrastructure repository. I recommend to create
@@ -114,42 +409,40 @@ a new keypair so that you have a better overview about the access. So let's gene
 .. warning:: Use a strong passphrase and store it in a password manager
 
 
-In your browser enter **http://your-server-ip** and log in to AWX with the user and password you have set. Let's add the
+In your browser enter your AWX URL and log in as admin. Let's add the git credentials:
-git credentials:
 
-*  Select the menu Resources >> Credentials and click *Add*:
+*  Select the menu *Resources* >> *Credentials* and click *Add*:
 
     |awxCredentials|
 
 *  Add the git credential parameters:
 
     -  Enter a name, a description and set the Credential Type to ``Source Control``
+    -  Select *Testenv* as Organization
     -  Leave the username and password blank
     -  Copy the private key into the SCM Private Key field and the passphrase accordingly to its field.
     -  Press Save
 
     |awxGitCredentials|
 
+    .. note:: in this screenshot the organization is not yet set.
+
 *  Go to your git server and add the public key (*awx.pub*) to a dedicated user OR (more securely) add it as a Deploy
    Key to each individual repository you want to use.
 
+
 Machine Access
 **************
 Similar to the SCM access we setup login credentials for machines. You can create a placeholder credential which prompts
 you for the password (use case for the password rotation):
 
 *  For the Credentials Type select ``Machine`` instead of ``Source Control``.
+*  Select *Testenv* as Organization
 *  For the password enable the checkbox "Prompt on launch"
 
 
 
 
-subsection
-**********
-
-
-
-
 section
 ########
 subsection