Current Configuration
Note: Helm 2.16.0 is broken don’t use it. That was the snap as of 11/21/19 emphasized text
This is a total win. I’ve got the cert manager working great. Internal and external. This works with the helm chart as shipped to deploy/upgrade simply:
helm repo add gitlab https://charts.gitlab.io/
helm repo update
helm upgrade --install -f values.yaml --namespace gitlab gitlab gitlab/gitlab
Here’s the current configure:
# values-minikube.yaml
# This example intended as baseline to use Minikube for the deployment of GitLab
# - Services that are not compatible with how Minikube runs are disabled
# - Configured to use 192.168.99.100, and nip.io for the domain
# Minimal settings
global:
ingress:
configureCertmanager: false
class: "nginx"
tls:
secretName: {}
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer : "ca-issuer"
hosts:
domain: deepthot.aa
externalIP: 192.168.42.64
https: false
gitlab:
name: gitlab.deepthot.aa
https: true
registry:
name: registry.deepthot.aa
https: true
minio:
name: minio.deepthot.aa
https: true
# Don't use certmanager, we'll use our own
certmanager:
install: false
nginx-ingress:
enabled: false
gitlab-runner:
install: false
certsSecretName: deepthot-aa-crt
gitlab:
unicorn:
ingress:
tls:
secretName: gitlab-gitlab-tls
registry:
ingress:
tls:
secretName: gitlab-registry-tls
minio:
ingress:
tls:
secretName: gitlab-minio-tls
Old configuration
Well, yet again I was making things too hard, ish. Where I am right now is I’ve set up a values.yaml file
# values-minikube.yaml
# This example intended as baseline to use Minikube for the deployment of GitLab
# - Services that are not compatible with how Minikube runs are disabled
# - Configured to use 192.168.99.100, and nip.io for the domain
# Minimal settings
global:
ingress:
configureCertmanager: false
class: "nginx"
hosts:
domain: deepthot.aa
externalIP: 192.168.42.64
https: false
gitlab:
name: gitlab.deepthot.aa
https: false
registry:
name: registry.deepthot.aa
https: false
minio:
name: minio.deepthot.aa
https: false
hosts:
# Don't use certmanager, we'll self-sign
certmanager:
install: false
# Use the `ingress` addon, not our Ingress (can't map 22/80/443)
nginx-ingress:
enabled: true
# GitLab Runner isn't a big fan of self-signed certificates
gitlab-runner:
install: false
Then I type helm upgrade --install -f values-deepthot.yaml --namespace gitlab gitlab . However I had to muck with the helm chart to work with the kubernetes I’m running here 1.16.2. The thing is they deleted several deprecated things and all of the helm charts I’ve played with have not been updated. I’m going to downgrade kubernetes and then get back to here.
Right now it is configured for a self-signed cert. That makes gitlab runner not function. So I need to generate a cert and get the deepthot.aa cert into it. I’ll do that after kubernetes downgrading though.
Downgrading was really easy. Just follow kubeadm-upgrade/ I downgraded the worker first, dunno if it was necessary, but all went clean. Now to try the helmchart from the repo again.
I think it’s working as well as it ever did. Now though it needs to have a real cert.
Wildcard_certificates that’s very problematical. OK. New plan, move my lan onto a delegated subdomain of deepthot.org. That’s a big production. I was planning on doing this anyway, but for the time being I really don’t need all that many certs. So, let’s do them by hand.
#!/bin/bash
dn=$(hostname -d)
me=$(hostname -f)
ns="gitlab"
ip="192.168.42.64"
kubectl create namespace "${ns}"
for host in gitlab registry minio; do
ipa dnsrecord-add "${dn}" $host --a-rec "${ip}"
ipa host-add $host."${dn}"
ipa service-add HTTP/$host."${dn}"
ipa service-add-host --hosts="${me}" HTTP/$host."${dn}"
sudo ipa-getcert request -r -f $host."${dn}".crt -k $host."${dn}".key -N CN=$host."${dn}" -D $host."${dn}" -K HTTP/$host."${dn}" -w
sudo chown denebeim: $host."${dn}".crt $host."${dn}".key
kubectl create secret tls gitlab-$host-tls --cert=./$host."${dn}".crt --key=./$host."${dn}".key --namespace="${ns}"
cat /etc/ipa/ca.crt >>"${host}"."${dn}".crt
done
kubectl --namespace ${ns} create secret generic deepthot-aa-crt --from-file={,{gitlab,minio,registry}.}deepthot.aa.crt
helm upgrade --install -f values.yaml --namespace "${ns}" "${ns}" gitlab/gitlab
Next step is to get gitlab runner up. It is not liking the custom CA that we have here.
gitlab-runner/issues/3968 describes the problem. There were a few workarounds, I tried figuring out the root cause, then I tried the solutions. The documentation says I should be concatenating the chain of certs.
I did this, replaced all the certs, didn’t help. Finally I looked at the stored cert in kubernetes. It was encrypted, but you know, it looked like the length was about the length of the key before it was concatenated. Also, I noticed the logs that came out when it was sucking the key in was saying ‘Updating local copy of SSL certificate “gitlab/gitlab-gitlab-tls” with missing intermediate CA certs’ Interesting. So I reread the bug report and one of the comments mentioned working around the problem by creating a generic secret. As you can see above the secrets we were building were tls secrets. So, I tried making a generic secret from the gitlab one we used to make the tls secret earlier. That worked. This is what you do:
cat /etc/ipa/ca.crt >>gitlab.deepthot.aa.crt
kubectl \
--namespace gitlab \
create secret generic gitlab-deepthot-aa-crt \
--from-file=gitlab.deepthot.aa.crt
helm upgrade --install -f values.yaml --namespace gitlab gitlab gitlab/gitlab \
--set gitlab-runner.install=true \
--set gitlab-runner.certsSecretName=gitlab-deepthot-aa-crt
The first script has been modified to include concatenating /etc/ipa/ca.crt onto all of the certs that it is making. However
there’s still issues. Basically reading on the internet someone mentioned how problematical it is to use your own CA. Every container on the planet has it’s own copy of SSL stuff. You can’t put your CA into all of them. I ran into the same sort of trouble at work. There I just rebuilt everything and put our CA into the base container. The solution is to get us on a public DNS we control and get let’s encrypt working. But that’s a pretty big thing, and I guess all this stuff can wait until then.
I guess that means I’m done with gitlab for the moment. I’ll call this a win.
No comments:
Post a Comment