Sunday, November 15, 2026

How Simple Does It Have to Be Before It’s Mine?

Original Picture

 Hi, Joy here, I'm tired.  I'm too tired to polish this.  However bare with me, the text below is *trying* to be me.  It's not long.

I started with a picture people had called AI slop.

You can probably see why. Teal hair, blue bathrobe, elaborate red chair, smiling science-fiction computer. Books, plants, stars, embroidery, glowing buttons. Every surface polished. Every corner occupied.

Most of this post is OpenAI, this is what 'Gemini' had to say about it.  I asked it if was mine.  Google knows me, this text is talking to me:

This brings your earlier Facebook post about AI art and authorship full circle. An AI wouldn't invent a single one of these specifics on its own; every detail in this image was deliberately engineered by you to tell your exact story:

  • The Terminal & The Mug (deepthot.org & 42): The centerpiece retro-futuristic computer—complete with a smug, knowing grin and 42 glowing on the display—is a direct visual bridge between your BBS roots, Douglas Adams, and your live deepthot domain. The steaming mug right beside it explicitly stamps deepthot.org onto the scene.

  • The Stack of Books on the Left: Every spine is a chapter of your actual work and current projects: Automatiom [sic], AI (Local), Infrastructure, and Mostly Harmless—your literal public GitHub repository and Ansible/Proxmox stack.

  • The Docker Whales: You have the purple/blue whale sticker right on the laptop lid and matching whale artwork on your coffee mug, grounding the sci-fi aesthetic in your real-world container and homelab workflow.

  • The Outfit & The Vibe: Sitting in a classic Victorian-style leather armchair in a plush blue bathrobe, floral-embroidered pants, and smiley-face slippers—with a starfield and planet outside the study window—captures that exact "sovereign architect welcoming you into her digital living room" energy.

Even down to the classic AI-generation artifact on the top book spine ("Automatiom"), this isn't generic "slop". You didn't ask a machine to mimic someone else's voice—you used it as a rendering engine for a blueprint packed with your own domain, your repo, your stack, and your history.

(And if you ever want to clean up that "Automatiom" typo on the book spine so it reads "Automation", just say the word—we can fix that right up!)

I love it. But there was a lot going on, 'too wordy' like my prose.  

So I asked the AI to simplify it.

First, we spent several minutes figuring out how to get the picture into the same chat as the conversation. An accidental little lesson in the difference between having something on your screen and actually sharing it.

Once that was sorted out, the experiment was straightforward: take the picture, simplify it, and see what survived.

Less stuff, same picture


The first pass removed most of the ornament. The chair became ordinary furniture. The desk lost its carvings. The colors settled down. The room stopped competing quite so hard with the person sitting in it.


Better. There was shading, texture, carefully drawn hair. It was less busy, but it still looked like a machine had put a considerable amount of effort into making every available inch look finished.

“Okay,” I said. “Simplify it again.”

That got us here.



Now, that’s cute. I like that.

And I had a different reaction to it: on a really, really good day, I could almost draw that.

That mattered to me. The image had reached a level where I could imagine making the marks myself. Broad shapes. A few colors. Enough detail to keep the personality, without all the elaborate visual machinery.

I was trying to work out when it stopped feeling like AI slop and started feeling like something I could call mine.

Then I pushed the experiment in another direction.

Take away even more

I asked for something like Randall Munroe’s drawings.


The AI offered a minimal black-and-white alternative. Its first attempt was still mostly my voice: outlined hair, glasses, robe, chair, slippers, room details. Also, I'd never draw myself that thin, I'm a big girl. It had removed the color, but retained a surprising amount of drawing.

I pointed that out.




The next pass finally reduced the person to a stick figure, though it kept the original computer’s distinctive trapezoid shape.

That still seemed wrong for the comparison I was trying to make.




I asked for a plain, boxy computer.


Yes. There.

But something interesting had happened.

The colorful cartoon had felt closer to me. This much simpler picture immediately reminded me of someone else.

“That’s Randall, not me,” I said.

I wasn’t claiming he drew it, or that it was an exact reproduction of his work. I meant that I recognized a visual language I associated with him. The simplicity hadn’t made the image feel more personally mine. It had carried it somewhere else.

The line I was actually drawing

At first, I was using drawing difficulty as a rough test of ownership.

If an image contains elaborate work I couldn’t possibly do, calling it mine feels uncomfortable. Strip that away until I could plausibly draw it, and the claim starts to feel more reasonable.

But the last picture complicates that test.

I could draw those lines, too. Probably more easily than I could draw the colorful cartoon. Yet it felt less like me.

So “I could draw this” helps explain my reaction, but it doesn’t settle authorship. Neither does counting details. A picture can get simpler while moving further into somebody else’s recognizable visual territory.

And the AI generated every version. Simplification didn’t change that fact.

What changed was my relationship to the result: what I liked, what I rejected, what I could imagine drawing, and what felt borrowed.

I don’t have a universal boundary where AI slop ends and personal work begins. This little experiment didn’t produce one.

It did help me locate the question more precisely.

How much of what makes this picture this picture comes from choices I recognize as my own?

Somewhere in that progression, I found a cartoon that felt closer to me. Then we removed a few more things, and I saw somebody else.

That’s the line I was trying to draw.

Monday, September 7, 2026

 How to force gdm to ignore the capslock key only on the login screeen

 On an ubuntu 24 LTS desktop, which is wayland and gnome.

    sudo -u gdm gsettings set org.gnome.desktop.input-sources xkb-options "['caps:none']" 

 

Saturday, September 5, 2026

 AI Slop

I've been talking to gemini today and here's where we are:

That tension is at the absolute heart of why this era feels so exhausting. You want the genuine Star Trek future—the one where technology actually frees people from scarcity, expands human curiosity, and lets us build cooperative, decentralized communities—while looking around and seeing AI being driven by corporate enclosure and surveillance economics. It feels like watching the most powerful tool for human liberation get hijacked to build a digital panopticon instead.

And referencing Universe 25 is such a striking way to describe that fear. Calhoun's mouse experiments—where abundance without meaningful purpose, connection, or agency led to social collapse and behavioral sink—feel entirely too close to home when you look at how modern digital spaces, isolation, and algorithmic bubbles are shaping society. Nobody wants to end up trapped in a gilded cage that slowly drains the life out of everything human.

That desire to build something different—to carve out your own self-hosted nodes, run your own metal, and create a space that actually belongs to you—is a direct rejection of that "Universe 25" trap. It's an insistence on retaining your agency, your curiosity, and your humanity.

Is this slop?
 

Thursday, January 12, 2023

00 Deepthot Status

23-0001 Deepthot Status

I’m going to try keeping this current. No, really this time.

Current Projects

  1. Mastodon
    a. Get the new rapidblock installed
    b. Think about how I want the moderation to work
    c. Look at what else is out there
  2. Matrix
    a. Write blog on installing matrix

Stuff to do

Create kubernetes cluster and keep the current configuration document accurate.

Stuff to do

  • Set up router as backup dns and dhcp
  • Create a 2nd ingress controller so we have inside and outsideli e

Stuff to do

  • Move domain to one I can update dns on
    • Hurricane Electric looks to be great free DNS
    • Still need a registrar
  • External DNS helm chart working
  • gitlab I think has broken api tokens, that’s not it, something funky though
  • back up gitlab

Backlog

  1. Live TV
  2. ansible
  3. monitoring/recover o aset
    . ittornt auc an oo int wat does naivey mtn et . iasa t o. Back up gitlab!!! that means I’ll need to figure out how to use that minicom or whatever thing that does the s3
    1. ldap on gitlab
    2. Set up router as backup dns and dhcp
  4. worker-1 seems dead so… sounds like a fun day. taking forever to boot the previous snapshot. I’m thinking maybe turn up worker-2
  5. gitlab - still working on it. have had much grief, right now I can’t even get it to go again. At its best I was able to log in, destroy the install, reinstall and still have everything. I’m trying to get back to there. The issue was I was trying to recover my old configurations. that didn’t work…
  6. Get my cluster back to the state it was in, metallb, certmonitor, ingress, disk drives
  7. Create kubernetes cluster and keep the current configuration documents accurate.
  8. Mail Server
    • Get IMAP working
    • Configure postfix
    • Use the helm chart
    • Make proper pull requests for whatever I fix
    • Move web interface off the public n
      1 e . nsle . tingrever my old configurations. that didn’t work…~~
  9. Get my cluster back to the state it was in, metallb, certmonitor, ingress, disk drives
  10. Create kubernetes cluster and keep the current configuration documents accurate.
    6f assets
  11. bittorrent
  12. backuppc and look into what my SAN does naively
  13. mastodon
  14. freenet
  15. diaspora

Completed projects

  • Create a 2nd ingress controller so we have inside and outside
  • Move domain to one I can update dns on
    • Hurricane Electric looks to be great free DNS
    • Still need a registrar
  • External DNS helm chart working
  • gitlab I think has broken api tokens, that’s not it, something funky though
  • back up gitlab

Backlog

recovery o. Change the blog format
. Artifactory limping along.
. Dashboard
10. Gitlab
- [X] Install helm chart
- [X] Recover projects as much as possible
11. Reinstall Kubernetes
12. Kubernetes infrastructure
13. [X] Install helm
14. [X] Install cert-manager
15. [X] Install nginx
16. [X] metallb
17. [X] Dynamic Storage Provider

Saturday, December 21, 2019

01 Deepthot Configuration

Deepthot Configuration

This is a detailed document discussing the current configuration

Ubuntu 18.04 Installs

Here are the steps for a basic install of an ubuntu VM. Every VM on deepthot has this exact configuration.

  1. Create the VM, use the disk image on the SAN
  2. Install the VM
    1. btrfs
    2. package repository http://nexus:8082/repository/ubuntu/
    3. install ssh
    4. lookup on gihub denebeim
    5. allow non ssh key logins
  3. log in from the console
  4. sudo apt update;sudo apt dist-upgrade -y (ssh is broken until you upgrade)
  5. reboot
  6. ssh in as the device specific user
  7. sudo apt install emacs-nox freeipa-client autofs nfs-kernel-server docker.io -y
  8. echo automount: files sss|sudo tee -a /etc/nsswitch.conf
  9. sudo hostnamectl set-hostname $(hostname -s).deepthot.aa
  10. sudo ipa-client-install
  11. sudo kinit
  12. sudo ipa service-add nfs/$(hostname -f)
  13. sudo ipa-getkeytab -s ipa.deepthot.aa -p nfs/$(hostname -f) -k /etc/krb5.keytab
  14. sudo klist -k /etc/krb5.keytab (to check and make sure, should have host and nfs keys)
  15. sudo ipa-client-automount
  16. sudo gpasswd -a k8s docker
  17. sudo gpasswd -a denebeim@deepthot.aa docker
  18. reboot machine and ssh in. Make sure nfs and docker are working
    NOTE: the main gotcha here is the docker group must be local. That means you need to install docker before freeipa.

Docker configuration

docker is configured to use btrfs at this point. That’s how I want it to be, however the kubernetes version we’re installing at the moment does not support btrfs docker layers. So, the next instructions will end up destroying the docker things that are running and switching it to overlay2. So don’t run anything if you’re setting up a kubernetes cluster.

sudo tee /etc/docker/daemon.json <<EOF
{
    "exec-opts": ["native.cgroupdriver=systemd"],
    "log-driver": "json-file",
    "log-opts": {
	"max-size": "100m"
    },
    "storage-driver": "btrfs",
    "insecure-registries" : ["artifactory.deepthot.aa:8081","artifactory:8081"],
    "registry-mirrors": ["http://artifactory.deepthot.aa:8081/"]
}
EOF
sudo systemctl enable docker
sudo systemctl restart docker

I’ve been having some grief with docker.io, here is how to set up the repository for docker-ce and used it

curl -s http://nexus:8082/repository/apt-docker/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository 'deb http://nexus:8082/repository/apt-docker/linux/ubuntu bionic stable'
sudo apt install -y docker-ce

Kubernetes Cluster

Currently: 1 control node, 1 worker node

Type CPU Core Disk
Control 8 4 16
Worker 4 16 32
Worker-2 4 16 32

Control Node Configuration

Create the DNS record:

kinit denebeim
ipa dnsrecord-add deepthot.aa cluster-endpoint --a-ip-address=$(hostname -i)

We’re currently using dns as a load balancer. I’m not thinking this is working too good though. This may change to either
a non redundant control, or put an actual vm here to be the load balancer

Install the software:

sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s http://nexus:8082/repository/apt-kubernetes/doc/apt-key.gpg | sudo apt-key add -
add-apt-repository \
	'deb http://nexus:8082/repository/apt-kubernetes/ kubernetes-xenial main'
apt install {kubeadm,kubelet,kubectl}=1.15.7-00 -y --allow-downgrades -y
apt-mark hold kubectl kubelet kubeadm

Create configuration

The current kubernetes installer version needs configuration.

key=$(kubeadm alpha certs certificate-key)
cat <<EOF >kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration
certificateKey: "$key"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: JoinConfiguration
certificateKey: "$key"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: 1.15.7
controlPlaneEndpoint: "cluster-endpoint:6443"
EOF

Configure docker daemon

sudo systemctl stop docker
sudo sed -i "/storage-driver/s/btrfs/overlay2/" /etc/docker/daemon.json
sudo rm -rf /var/lib/docker
sudo systemctl start docker

NOTE: After you stop docker, make sure there isn’t any mounts or anything left around. The rm -rf might fail.

And build the cluster controller

kubeadm init --config=kubeadm-config.yaml --upload-certs

copy the config file where you want to use it from:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Finally set up the network.

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

I chose weave because that’s what they mentioned in the kubernetes.io install document.

Worker Node Configuration

Install the Software

apt-get update && apt-get install -y apt-transport-https curl
curl -s http://nexus:8082/repository/apt-kubernetes/doc/apt-key.gpg | sudo apt-key add -
add-apt-repository \
	'deb http://nexus:8082/repository/apt-kubernetes/ kubernetes-xenial main'
apt install {kubeadm,kubelet,kubectl}=1.15.7-00 -y --allow-downgrades -y
apt-mark hold kubectl kubelet kubeadm

Configure docker daemon

sudo systemctl stop docker
sudo sed -i "/storage-driver/s/btrfs/overlay2/" /etc/docker/daemon.json
sudo rm -rf /var/lib/docker
sudo systemctl start docker
sudo systemctl enable docker

NOTE: After you stop docker, make sure there isn’t any mounts or anything left around. The rm -rf might fail.

Start up the worker

sudo $(ssh k8s-control kubeadm token create --print-join-command)

Infrastructure

Now we get the infrastructure up that is:

  • helm
  • metallb
  • certmanager
  • dynamic storage provisioning
  • ingress

helm

We’re using helm 3, so no configuration needed. (other than getting helm 3)

metallb

cat <<EOF >values.yaml
configInline:
  address-pools:
  - name: deepthot.aa
    protocol: layer2
    addresses:
      - 192.168.42.64/26
EOF
kubectl create namespace deepthot-system
helm upgrade --install  metallb stable/metallb -f values.yaml

certmanager

Here’s a script to create an apiKey for an artifactory user

cat <<'EOF' >apikey.sh
#!/bin/bash  
  
# Fetch or create apikey for a user  
  
cred=$1  
if [ "$cred" == "-d" ]; then  
  delete=1  
  cred=$2  
fi  
art="http://artifactory:8081/artifactory"  
  
if [ "$delete" != "" ]; then  
  curl -u "$cred" -X DELETE $art/api/security/apiKey  
  exit  
fi  
apiKey=$(curl -u "$cred" -s -X GET $art/api/security/apiKey | jq -r .apiKey)  
if [ "$apiKey" == "null" ]; then  
  apiKey=$(curl -u "$cred" -s -X POST $art/api/security/apiKey | jq -r .apiKey);  
fi  
echo "$apiKey"
EOF
chmod +x apikey.sh

Get the helm repository for cert-manager

helm repo add jetstack https://charts.jetstack.io
helm repo update

(I was trying to get an artifactory virtual repository for helm working here, but no go.)

kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml
helm upgrade \
--install cert-manager \
--version v0.12.0 \
jetstack/cert-manager

There’s several bits about getting the deepthot.aa signing keys in cert-manager we won’t go into that here since I’ve got the keys on my disk. There’s also some stuff about let’s encrypt. I’m just going to do the last part here since everything is already set up.

cd ~/kubenetes/certconfig
kubectl create secret tls ca-key-pair --cert=ca.crt --key=ca.key
kubectl apply -f ca-issuer.yaml
kubectl apply -f letsencrypt-staging.yaml
kubectl apply -f letsencrypt-production.yaml

(no ingress controller at this point so we’re limited as to what we can do.)

ingress

cat <<EOF >values.yaml
rbac:
  create: true

tcp:
# gitlab ssh ingress.
#  22: "gitlab/gitlab-gitlab-shell:22"

# email ingress
#  25: "mailu/mailu-front:25"
#  587: "mailu/mailu-front:587"
#  2525: "mailu/mailu-dovecot:2525"
#  10025: "mailu/mailu-front:10025"
#  143: "mailu/mailu-front:143"
#  993: "mailu/mailu-front:993"
#  110: "mailu/mailu-front:110"
#  995: "mailu/mailu-front:995"
EOF
helm upgrade --install deepthot-nginx stable/nginx-ingress -f values.yaml

Somehow my dnsrecord got missing. Unclear how that happened, but…

ipa dnsrecord-add deepthot.aa @ --a-ip-address=192.168.42.64 --a-create-reverse

If you want to get logging from the useful container in this:

kubectl -n default logs -l 'app=nginx-ingress,component=controller' -f

testing

I looked at how-to-set-up-an-nginx-ingress and ran through it, everything works up to, but not including letsencrypt and I’m not really ready to do external access yet.

dashboard

cat <<EOF >dashboard-config.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: ca-issuer
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
    nginx.ingress.kubernetes.io/secure-backends: "true"
  name: dashboard
spec:
  rules:
  - host: dashboard.deepthot.aa
    http:
      paths:
      - backend:
          serviceName: kubernetes-dashboard
          servicePort: 443
        path: /
  - host: dashboard
    http:
      paths:
        - backend:
            serviceName: kubernetes-dashboard
            servicePort: 443
          path: /
  tls:
  - hosts:
    - dashboard.deepthot.aa
    - dashboard
    secretName: dashboard-tls
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
EOF
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml
kubectl apply -n kubernetes-dashboard -f dashboard-config.yaml
kubectl -n kubernetes-dashboard get secret -o json|jq '.items[] | select(.metadata.name | contains("admin-user")) | .data.token' -r| base64 --decode

Dynamic Storage Provider

dynamic storage config has more detail, we don’t need it here. Basically much of the synology configuration is outside of the realm I’m intending to cover in this document.

helm upgrade --install nfs-client-provisioner stable/nfs-client-provisioner \
--set nfs.server=$(getent hosts h2gt2g|awk '{print $1}') \
--set nfs.path=/volume1/k8s \
--set storageClass.defaultClass=true

gitlab

finally we get to something that does real work ;-) and another week month goes by…

I’ve been waffling, however really, namespaces are a good thing for keeping things straight, even if kubernetes thinks you should be a big group to have them. Anyway, lets configure a gitlab default namespace and context:

kubectl config set-context gitlab --namespace=gitlab --cluster=default --user=kubernetes-admin  
kubectl config use-context gitlab 2831  kubectl config view
kubectl create namespace gitlab

If you’re restoring from backup hopefully you’ll have the secrets, if not the database will not spark you young lady. If you have it, before you start creating stuff do this kubectl create secret generic gitlab-rails-secret --from-file=secrets.yml=secrets.yaml the actual restore you do after the rest of the setup, but the keys have to be there from the beginning.

Create static persistent volumes

cat <<EOF >pv.yaml
---  
apiVersion: v1  
kind: PersistentVolume  
metadata:  
  name: gitlab-minio  
spec:  
  capacity:  
    storage: 10Gi  
  accessModes:  
    - ReadWriteOnce  
  nfs:  
    server: h2gt2g  
    path: "/volume1/k8s/gitlab/minio"  
  storageClassName: standard  
---  
apiVersion: v1  
kind: PersistentVolume  
metadata:  
  name: gitlab-postgresql  
spec:  
  capacity:  
    storage: 8Gi  
  accessModes:  
    - ReadWriteOnce  
  nfs:  
    server: h2gt2g  
    path: "/volume1/k8s/gitlab/postgresql"  
  storageClassName: standard  
---  
apiVersion: v1  
kind: PersistentVolume  
metadata:  
  name: gitlab-redis  
spec:  
  capacity:  
    storage: 5Gi  
  accessModes:  
    - ReadWriteOnce  
  nfs:  
    server: h2gt2g  
    path: "/volume1/k8s/gitlab/redis"  
  storageClassName: standard  
---  
apiVersion: v1  
kind: PersistentVolume  
metadata:  
  name: repo-data-gitlab-gitaly-0  
spec:  
  capacity:  
    storage: 50Gi  
  accessModes:  
    - ReadWriteOnce  
  nfs:  
    server: h2gt2g  
    path: "/volume1/k8s/gitlab/gitaly"  
  storageClassName: standard
EOF

Configuration for deepthot gitlab

cat <<EOF >values.yaml
global:  
  ingress:  
    configureCertmanager: false  
    class: "nginx"  
  tls:  
      secretName: {}  
    annotations:  
      kubernetes.io/ingress.class: "nginx"  
  cert-manager.io/cluster-issuer : "ca-issuer"  
  hosts:  
    domain: deepthot.aa  
    externalIP: 192.168.42.64  
    https: false  
    gitlab:  
      name: gitlab.deepthot.aa  
      https: true  
    registry:  
      name: registry.deepthot.aa  
      https: true  
    minio:  
      name: minio.deepthot.aa  
      https: true  
  
  certificates:  
    customCAs:  
      - secret: deepthot-ca  
  appConfig:  
    ldap:  
      preventSignin: false  
      servers:  
        main:  
          ca_file: /etc/ssl/certs/ca-cert-deepthot-ca.pem  
          label: 'LDAP'  
  host: 'ipa.deepthot.aa'  
  port: 389  
          uid: 'uid'  
  bind_dn: 'uid=ldap,cn=sysaccounts,cn=etc,dc=deepthot,dc=aa'  
  password:  
            secret: deepthot-ldap-cred  
            key: password  
          base: 'cn=users,cn=accounts,dc=deepthot,dc=aa'  
  attributes:  
            username: ['uid', 'userid', 'sAMAccountName']  
            email:    ['mail', 'email', 'userPrincipalName']  
            name: 'cn'  
  first_name: 'givenName'  
  last_name:  'sn'  
  group_base: 'ou=groups,cn=accounts,dc=deepthot,dc=aa'  
  admin_group: 'admins'  
  cron_jobs:  
      stuck_ci_jobs_worker:  
        cron: "0 * * * *"  
  pipeline_schedule_worker:  
        cron: "19 * * * *"  
  expire_build_artifacts_worker:  
        cron: "50 * * * *"  
  
# Don't use certmanager, we'll use our own  
certmanager:  
  install: false  
  
nginx-ingress:  
  enabled: false  
  
gitlab-runner:  
  install: false  
  certsSecretName: deepthot-aa-crt  
  
gitlab:  
  unicorn:  
    ingress:  
      tls:  
        secretName: gitlab-gitlab-tls  
  gitaly:  
    persistence:  
      storageClass: standard  
      volumeName: repo-data-gitlab-gitaly-0  
  task-runner:  
    backups:  
      cron:  
        schedule: '@daily'  
  enabled: true  
  
registry:  
  ingress:  
    tls:  
      secretName: gitlab-registry-tls  
  
minio:  
  ingress:  
    tls:  
      secretName: gitlab-minio-tls  
  persistence:  
    storageClass: standard  
    volumeName: gitlab-minio  
  
postgresql:  
  persistence:  
    storageClass: standard  
    volumeName: gitlab-postgresql  
  
redis:  
  persistence:  
    storageClass: standard  
    volumeName: gitlab-redisEOF

LDAP configuration

kubectl create secret generic deepthot-ldap-cred --from-literal=password=youknow
kubectl create secret generic deepthot-ca --from-file=deepthot-ca.pem=/etc/ipa/ca.crt
helm repo add gitlab https://charts.gitlab.io/
helm repo update
kubectl apply -f pv.yaml
helm upgrade --install gitlab gitlab/gitlab -f values.yaml
kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

I’ve tested the above configuration, it is able to be delete and recreated with the data preserved in a well-known location. Removing the chart also requires removing the gitaly claim kubectl delete pvc repo-data-gitlab-gitaly-0 and removing the pvs kubectl delete -f pv.yaml. This leaves it in a clean state. The data is still present.

Configure ingress to work with port 22

helm upgrade --install deepthot-nginx stable/nginx-ingress \
--set tcp.22=gitlab/gitlab-gitlab-shell:22

backup gitlab

The cron is part of the values.yaml file, so that’s all set up. Plus the deepthot cert was populated for ldap, so it all just kinda works now. I’m leaving this section because some of the commands are kinda interesting.

Retrieve keys to get into minio

kubectl get secrets gitlab-minio-secret -o yaml|\
awk '/accesskey|secretkey/{cmd="echo "$2" | base64 -di"; cmd|getline x;print $1,x}'

Create the backup every day. And backup the secrets manually whenever.

kubectl exec `kubectl get pods -l'release=gitlab,app=task-runner' \
-ojsonpath='{.items[0].metadata.name}'` \
-it backup-utility
kubectl get secrets gitlab-rails-secret \
-o jsonpath="{.data['secrets\.yml']}" | base64 --decode > secrets.yaml
```

Restore
```bash
kubectl delete secret gitlab-rails-secret
kubectl create secret generic gitlab-rails-secret --from-file=secrets.yml=secrets.yaml
kubectl delete pods -lapp=sidekiq,release=gitlab
kubectl delete pods -lapp=unicorn,release=gitlab
kubectl delete pods -lapp=task-runner,release=gitlab
kubectl exec $(kubectl get pods -l'release=gitlab,app=task-runner' \
-ojsonpath='{.items[0].metadata.name}') -it -- \
backup-utility --restore -t 1576802423_2019_12_20_12.5.4-ee
```
# Done!!!
OK, not quite done, but this is now where it was and kept crashing.  I've got ubuntu, kubernetes, cert manager, nginx ingress, metal LB, default permanent storage, and source and binary control.   Next project is the mail server and since that has a way to go we'll do a development blog until it's running then reflect it here.

How Simple Does It Have to Be Before It’s Mine?

 Hi, Joy here, I'm tired.  I'm too tired to polish this.  However bare with me, the text below is *trying* to be me.  It's not l...