K8S notes for realworldapp
=====
1. Install MicroK8s on Linux
```bash
sudo snap install microk8s --classic
```
If you are not root
```bash
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
```
You will also need to re-enter the session for the group update to take place:
```bash
su - $USER
```
2. Check the status while Kubernetes starts
```bash
microk8s status --wait-ready
```
3. Turn on the services you want
```bash
microk8s enable dashboard dns registry istio ingress
```
Try microk8s enable --help for a list of available services and optional features. microk8s disable <name> turns off a service.
4. Start using Kubernetes
```bash
microk8s kubectl get all --all-namespaces
```
> If you mainly use MicroK8s you can make our kubectl the default one on your command-line with alias mkctl="microk8s kubectl". Since it is a standard upstream kubectl, you can also drive other Kubernetes clusters with it by pointing to the respective kubeconfig
file via the --kubeconfig argument.
5. Access the Kubernetes dashboard
```bash
microk8s dashboard-proxy
```
6. Start and stop Kubernetes to save battery
Kubernetes is a collection of system services that talk to each other all the time. If you don’t need them running in the background then you will save battery by stopping them. *microk8s start* and *microk8s stop* will do the work for you.
7. If you need to clean the installation (remove everything)
```bash
sudo snap remove microk8s --purge
```
8. Allo download from private registry
For exemple I had to create the file /etc/docker/daemon.json
with following content
```txt
{
"insecure-registries" : ["localhost:32000", "registry.gitlab.inria.fr"]
}
```
and then restart docker
```bash
sudo service docker restart
```
## Kubebernetes sur le projet
9. Mettre une config dans kubernetes (équivalent au consul)
```bash
microk8s kubectl create configmap mysql-config \
--from-literal=mysql.service.name=mysql \
--from-literal=mysql.db.name=tlc
```
10. Mettre un secret dans kubernetes (équivalent au vault)
```bash
microk8s kubectl create secret generic db-security \
--from-literal=db.user.name=foo \
--from-literal=db.user.password=bar
```
11. Support image download from private repository (not required in this lab)
```bash
sudo nano -w /var/snap/microk8s/current/args/containerd-template.toml
```
change the end of the file
```txt
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io", ]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:32000"]
endpoint = ["http://localhost:32000"]
```
to add the two last following lines
```bash
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io", ]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:32000"]
endpoint = ["http://localhost:32000"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.gitlab.inria.fr"]
endpoint = ["https://registry.gitlab.inria.fr"]
```
```bash
microk8s stop
microk8s start
```
and add you registry login password as a secret. Update the information with you login and your password
```bash
microk8s kubectl create secret docker-registry my-private-registry1 --docker-server=registry.gitlab.inria.fr --docker-username="logingitlab" --docker-password="xxxx" --docker-email=youranonymouslogin@inria.fr
```
```bash
microk8s kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "my-private-registry1"}]}'
```
12. Create a secret
```bash
microk8s kubectl create secret generic my-jwt-private-key --from-literal=my.jwt.privatekey=foo
```
```bash
microk8s kubectl create secret generic my-jwt-public-key --from-literal=rest.my_jwt_publickey=bar
```
13. Create the image for your services
**Image for the back**
In the node-express-realworld-example-app folder, create a Dockerfile file with the following content
```txt
FROM node:10
RUN mkdir /app
WORKDIR /app
COPY package.json .
RUN npm install --production
COPY . .
CMD ["npm", "start"]
```
Next in this folder build the image
```bash
docker build -t localhost:32000/realworldnode .
```
**Image for the front**
Prepare the front for production
Edit the file *src/environments/environment.prod.ts*
Replace \'https://conduit.productionready.io/api\’
with \‘/api\’
You can edit this file using nano
```txt
nano -w src/environments/environment.prod.ts
```
In the node-express-realworld-example-app folder, create a Dockerfile file with the following content
```txt
FROM node:10-alpine as build-step
WORKDIR /app
COPY package.json /app
RUN npm install
COPY . /app
RUN npm run build
FROM bunkerity/bunkerized-nginx as prod
COPY --from=build-step /app/dist/ /www
```
Next in this folder build the image
```bash
docker build -t localhost:32000/realworldangular .
```
14. push the images to the public registry or localhost:32000 registry
```bash
docker push localhost:32000/realworldnode
docker push localhost:32000/realworldangular
```
15. Create the manifest for the back, the front and mongodb
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
devweb.service: mongorealworld
name: mongorealworld
spec:
replicas: 1
selector:
matchLabels:
devweb.service: mongorealworld
strategy: {}
template:
metadata:
labels:
devweb.network/realworld: "true"
devweb.service: mongorealworld
spec:
containers:
# - env:
# - name: MONGO_INITDB_ROOT_USERNAME
# value: root
# - name: MONGO_INITDB_ROOT_PASSWORD
# value: root
- image: mongo:latest
name: mongorealworld
resources: {}
restartPolicy: Always
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
devweb.network/realworld: "true"
devweb.service: realworldnode
name: realworldnode
spec:
replicas: 1
selector:
matchLabels:
devweb.service: realworldnode
strategy: {}
template:
metadata:
labels:
devweb.network/realworld: "true"
devweb.service: realworldnode
spec:
containers:
- image: localhost:32000/realworldnode:latest
env:
- name: MONGODB_URI
value: mongodb://mongorealworld/conduit
- name: NODE_ENV
value: production
- name: SECRET
value: demo
name: realworldnode
ports:
- containerPort: 3000
resources: {}
# volumeMounts:
# - mountPath: /work/config
# name: orangeconfigfolder
# - mountPath: /logs
# name: logsvolume
# volumes:
# - name: logsvolume
# hostPath:
# path: /logs # Path of the directory on the Host (node)
# type: Directory # DirectoryOrCreate, FileOrCreate
# - name: orangeconfigfolder
# hostPath:
# path: /root/compose/captcha-orange-server-config # Path of the directory on the Host (node)
# type: Directory # DirectoryOrCreate, FileOrCreate
restartPolicy: Always
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
devweb.service: realworldangular
name: realworldangular
spec:
replicas: 1
selector:
matchLabels:
devweb.service: realworldangular
strategy: {}
template:
metadata:
labels:
devweb.network/realworld: "true"
devweb.service: realworldangular
spec:
containers:
- env:
- name: SERVE_FILES
value: 'yes'
- name: PROXY_REAL_IP
value: 'yes'
- name: AUTO_LETS_ENCRYPT
value: 'no'
- name: REDIRECT_HTTP_TO_HTTPS
value: 'no'
- name: HTTP2
value: 'no'
- name: USE_DNSBL
value: 'no'
- name: USE_WHITELIST_IP
value: 'no'
- name: USE_BLACKLIST_IP
value: 'no'
- name: USE_LIMIT_REQ
value: 'no'
- name: USE_FAIL2BAN
value: 'no'
- name: USE_CLAMAV_UPLOAD
value: 'no'
- name: USE_CLAMAV_SCAN
value: 'no'
- name: USE_ANTIBOT
value: 'no'
- name: USE_MODSECURITY
value: 'no'
- name: BLOCK_USER_AGENT
value: 'no'
# - name: MONGO_INITDB_ROOT_PASSWORD
# value: root
image: localhost:32000/realworldangular:latest
ports:
- containerPort: 8080
name: realworldangular
resources: {}
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: mongorealworld
namespace: default
spec:
selector:
devweb.service: mongorealworld
ports:
- port: 27017
---
apiVersion: v1
kind: Service
metadata:
labels:
devweb.service: realworldnode
name: realworldnode
spec:
ports:
- name: "3000"
port: 3000
targetPort: 3000
selector:
devweb.service: realworldnode
---
apiVersion: v1
kind: Service
metadata:
name: realworldangular
namespace: default
spec:
selector:
devweb.service: realworldangular
ports:
- name: "8080"
port: 8080
targetPort: 8080
---
```
16. Deploy the service
First you have to update the different volume path and image version you want to use in the yaml file.
next.
```bash
microk8s kubectl apply -f services.yaml
```
start the K8S proxy to monitor the deployment.
next create a simple ingress for this service. Create a simple *ingress.yml* file with the following content.
```bash
nano -w ingress.yml
```
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: realworld-ingress
namespace: default
spec:
rules:
- host: www.monservice.fr
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: realworldangular
port:
number: 8080
- path: /api
pathType: Prefix
backend:
service:
name: realworldnode
port:
number: 3000
```
Next, apply this ingress.
```bash
microk8s kubectl apply -f ingress.yml
```
## Remarks
k8s yaml file could be templated following this approach
https://learnk8s.io/templating-yaml-with-code
### Remove https to dashboard and external access
```bash
sudo microk8s.kubectl -n kube-system edit deploy kubernetes-dashboard -o yaml
```
When Kubernetes dashboard is installed using the recommended settings, both authentication and HTTPS are enabled. Sometimes, you may want to disable authentication or HTTPS. For example, if Kubernetes dashboard is served behind a proxy, then it's unnecessary to e
nable authentication when the proxy has its own authentication enabled. Kubernetes dashboard uses auto-generated certificates for HTTPS, which may cause problems for HTTP client to access. This post shows how to disable authentication and HTTPS in Kubernetes dash
board.
First, we need to modify deployment of Kubernetes dashboard to remove the argument --auto-generate-certificates and add following extra arguments:
--enable-skip-login
--disable-settings-authorizer
--enable-insecure-login
--insecure-bind-address=0.0.0.0
After this change, Kubernetes dashboard server now starts on port 9090 for HTTP. Then we need to modify livenessProbe to use HTTP as the scheme and 9090 as the port. Port 9090 also needs to be added as containerPort.
Finally, we need to modify service of Kubernetes dashboard to open port 80 and use 9090 as the target port.
```bash
sudo microk8s.kubectl -n kube-system edit svc kubernetes-dashboard -o yaml
```
next create a simple ingress for this service
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dashboard-ingress
# annotations:
# nginx.ingress.kubernetes.io/rewrite-target: /
namespace: kube-system
spec:
rules:
- host: realapp.foo.fr
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: kubernetes-dashboard
# namespace: kube-system
port:
number: 80
```
To get the login token
```bash
token=$(microk8s kubectl -n kube-system get secret | grep default-token | cut -d " " -f1)
microk8s kubectl -n kube-system describe secret $token
```
### Configure network policies
I just create a template of network policy (network.yaml).
https://kubernetes.io/docs/concepts/services-networking/network-policies/
### Add a full spring property file as a config
```bash
microk8s kubectl create configmap api --from-file= src/main/resources/application.yml
```