So, I’ve got a single node kubernetes cluster up, go me. Traditionally I’ve never really considered something done until it was backed up to a source code manager. At work anyway, at home I’ve been… remiss. No longer. We’ve been using gitlab-ci at work for awhile now, I’m really getting into it, so I’m going to set it up at home too. This also looked to be the most problematical thing to get up, so hardest thing first, eh?
This is going to be a journey, obviously since I don’t really know kubernetes at all yet. There will probably be some digressions, and I’ll set up other blog pages for them. Specifically I’m pretty sure I’m going to have issues with persistent storage, but we’ll cope with that when we come to it.
And so we begin. Here is the web page I’m going from gitlab charts It is, of course, a helm chart. I don’t have helm installed yet. so… snap install helm after doing helm init I see tiller isn’t coming up. That’s because no nodes are untainted with something tiller doesn’t like. Let’s see what the taints are.
denebeim@frankiemouse:~/kubernetes$ kubectl get node k8s-control -o json|jq .spec.taints
[
{
"effect": "NoSchedule",
"key": "node-role.kubernetes.io/master"
}
]
Ah, makes sense. It won’t schedule my helm tiller pod because the only node I have is saying it’s unschedulable. So, my choices are, create a new node, remove the taint, or make the deployment ignore the taint. This original node was made small on purpose. I figure it’s only for control. So, I’m thinking I need to make a new node that’s bigger anyway. Guess I’m off to do a different blog post. Kubernetes Worker Node on Deepthot. So, now the tiller pod is up and running. Yay. Let’s continue
The gitlab documentation is kinda shitty, just like all of the other gitlab documentation I’ve ever read. I’m sure there’s going to be issues, however this is, I think, the command I’m going to need to do:
helm install gitlab/gitlab \
--set global.hosts.domain=deepthot.aa \
--set global.hosts.externalIP=192.168.42.50 \
--set certmanager-issuer.email=denebeim@gmail.com \
--set gitlab.unicorn.minReplicas=1 \
--set gitlab.unicorn.maxReplicas=1 \
--set gitlab.sidekiq.minReplicas=1 \
--set gitlab.sidekiq.maxReplicas=1 \
--set gitlab.gitlab-shell.minReplicas=1 \
--set gitlab.gitlab-shell.maxReplicas=1 \
--set registry.hpa.minReplicas=1 \
--set registry.hpa.maxReplicas=1 \
--set certmanager.install=false \
--set global.ingress.configureCertmanager=false \
--set gitlab-runner.install=false
```
I kept getting `Error: no available release name found` when I do helm install gitlab/gitlab. Not sure if this is what I want to do, but googling around for that error message I found something that indicates a workaround is `kubectl create clusterrolebinding permissive-binding --clusterrole=cluster-admin --user=admin --user=kubelet --group=system:serviceaccounts;` no idea exactly what that's doing, I think it's a sledge hammer, but it seems to work. I'm gonna use it for the moment.
I'm going to set up dns by hand for the 3 addresses it needs, gitlab, registry, and minio to 192.168.42.50. There's a reference in the docs to [external-dns](https://github.com/kubernetes-incubator/external-dns) which looks very interesting and I think I'm going to want to use it eventually. However for now let's do it manually.
```bash
ssh k8s-control
~~ipa dnsrecord-add --a-create-reverse --a-ip-address=192.168.42.50 deepthot.aa gitlab
ipa dnsrecord-add --a-ip-address=192.168.42.50 deepthot.aa registry
ipa dnsrecord-add --a-ip-address=192.168.42.50 deepthot.aa minio~~
IMPORTANT this has changed. I’ve moved this to our per site nginx ingress. Look at the Nginx Ingress Controller blog for more information.
Install gitlab using just the docker container
OK. I don’t know enough about kubernetes yet to do this. Let’s switch to installing the docker container. docker
PoC just play
First thing I want to just get the docker up. I think the easiest way to do that is with docker-compose. This gives us three options, install docker on the laptop, create a new VM to run docker, or run it on one of the kubernetes servers I’ve already set up. I checked port 80 and 443 on the k8s-control, they’re open, so that’s probably the easiest to set up and clean.
Now, why did I choose the VM that was just big enough to run the control plane for kubernetes and not the monster worker??? Silly girl.
docker-compose wants to use letsencrypt. disable it for right now. Long term I think I want it working. Although I guess it depends on if we want gitlab exposed to the internet. Also, I’m really thinking that I’m going to have one nginx server exposed and that’s the only place we need to have a web ssl cert.
OK, god, tons of grief because I wasn’t thinking straight. I couldn’t figure out how to make ssl go away. Turns out it was just the external_url in the docker-compose (or whatever, it’s what ends up in the ruby configuration). If you use https then it requires and configures https. If it doesn’t it does not. Duh. So, this works for getting up my docker container
web:
image: 'gitlab/gitlab-ee:latest'
restart: always
hostname: 'k8s-worker-1.deepthot.org'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'http://k8s-worker-1.deepthot.org'
# Add any other gitlab.rb configuration here, each on its own line
#letsencrypt['enabled'] = false
#nginx['listen_https'] = false
#nginx['listen_port'] = 80
ports:
- '80:80'
- '443:443'
- '2222:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
sheesh Joy, my love sometimes your grasp of the obvious fails.
Install Gitlab on Kubernetes
To get this docker image running on kubernetes I’m going to need at least a couple of things. An ingress controller and a persistent volumes. Each of those are blog posts for themselves. So I’m going to refer you to
Ingress Controller and Persistent Volumes
Written with StackEdit.
No comments:
Post a Comment