6
netikras
17d

## Learning k8s

Sooo yeah, 2 days have been wasted only because I did not reset my cluster correctly the first time. Prolly some iptables rules were left that prevented me from using DNS. Nothing worked...

2 fucking days..

2 FUCKING DAYS!!! F!!!

Comments
  • 0
    ########
    ## RESET
    ########

    ## RUN_IN_MASTER
    kubectl drain netikras-xps --delete-local-data --force --ignore-daemonsets
    kubectl delete node netikras-xps

    ## RUN_IN_NODE (as root)
    kubeadm reset
    iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
    systemctl stop kubelet
    systemctl stop docker
    rm -rf /var/lib/cni/
    rm -rf /var/lib/kubelet/*
    rm -rf /etc/cni/
    rm -rf $HOME/.kube/
    ifconfig cni0 down
    ifconfig flannel.1 down
    ifconfig docker0 down
    ip link delete cni0
    ip link delete flannel.1
  • 0
    ##############
    ## INIT MASTER
    ##############
    sudo systemctl start docker
    sudo systemctl start kubelet
    sudo kubeadm init --pod-network-cidr=10.244.0.0/1

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    sudo kubectl apply -f https://raw.githubusercontent.com/c...

    ############
    ## JOIN NODE
    ############
    sudo systemctl start docker
    sudo systemctl start kubelet

    sudo kubeadm join 192.168.1.6:6443 --token vhtfdq.54qajk75062js595 --discovery-token-ca-cert-hash sha256:a26ba8f0b078254c275ec344026c3383a851a6f975d2eabe5e06c31f380da88c
  • 1
    In case anyone else stumbled upon this post tortured by similar magical issues..
  • 1
    Wow :| that is some serious linuxing
  • 0
    @netikras
    The F?

    1. use 'sudo kubeadm reset'.
    2. set your hostname.
    3. Do not put your cluster tokens on the internet.
  • 1
    @magicMirror
    1. Far, FAR from enough
    2. Eh? Hostnames are set
    3. Private cluster. Couldn't care less
  • 1
    @magicMirror reset won't clear iptables, cnis, kubelet configs, vNICS.

    I was using reset too. Then spent 2 days digging through various bug reports trying to figure out why is flannelD not running on some nodes, why is coredns timeouting when propagating queries to host dns, why can't I access aby clusterip, why isn't coredns resolving anything but the pod that's querying it,.... I could go on :)
  • 0
    You using Kubernetes the Hard Way? It's pretty good for starting this monolith of a project
  • 0
    Imma have to do the same :) Got a lil project at work involving k8s but I still gotta wrap my head around it. Keep up the good work btw, seems like you're managing!
  • 1
    @netikras Makes sense. My reset includes kubeadm reset + firewall rules reset.
    But I use calico in my private clusters, so that might be a factor.

    Coredns.... annoying POS. can someone explain why does it need to contact the upstream dns server in order to resolve between namwspaces??
  • 0
    @magicMirror I think it's optional, in case you need are querying public domains. You can disable upstream.

    It could be iptables fuckup prevented new pods from being registered or triggered kube to deregister them from dns as it could not reach those pods. Thx to iptables... Hence the upstream query

    just a guess :)
Add Comment