Skip to content

Install Docker

Since I'm going to use Docker to build my OpenFaaS functions, I need to install Docker and have buildx in it. I have installed Docker before and could not run command:

docker buildx

So, if you have docker installed and can execute that, you can skip this part.

Note

I'm doing this only on one of my nodes, control01, so I can build arm64 native images and push them to registry, and also build OpenFaaS functions. Building OpenFaaS functions on the same node where the server is running is not recommended, and Alex Ellis told me personally that that’s not the way it should be done! You should build on your client and push to OpenFaaS gateway from there, and so on... However, I can confirm this works just fine, and it’s less hassle for me as I can also setup GitLab worker on this node and have my stuff build automatically there whenever I push to my local GitLab (GitLab is on a different server in my network, outside the scope of this guide.)

Clean slate first

Remove the installation you have now.

sudo apt-get remove docker docker-engine docker.io containerd runc

Install Docker

Install Docker on Ubuntu arm64 as follow. You will need all of these 🙂

Install new repo

sudo add-apt-repository \
   "deb [arch=arm64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

Install Docker

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

Edit Docker settings

Add /etc/docker/daemon.json configuration for docker daemon.

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "insecure-registries": ["registry.cube.local:5000"],
  "experimental": true,
  "log-driver": "json-file",
  "storage-driver": "overlay2",
  "log-opts": {
    "max-size": "100m"
  }
}

Note

"insecure-registries": ["registry.cube.local:5000"] This is for insecure private registry running on HTTP, if you use TLS, you don't need this line.

Enable at boot and start docker daemon.

sudo systemctl enable docker
sudo systemctl start docker

Adding support for multi-arch.

Some additional packages are needed.

sudo apt-get install binfmt-support qemu-user-static

Using buildx with private registry.

Note

This is mainly related to OpenFaaS, since it uses buildx with faas-cli

Buildx is running inside container.

oot@control01:/home/ubuntu# docker ps
CONTAINER ID   IMAGE                           COMMAND       CREATED      STATUS        PORTS     NAMES
97820140a740   moby/buildkit:buildx-stable-1   "buildkitd"   2 days ago   Up 24 hours             buildx_buildkit_multiarch

I'm a bit fuzzy on exactly how and when this is created, to be honest. If it’s not there, try to run docker buildx.

Here is the issue. The container does not know about your private registry, nor does it have a root certificate of it if you use TLS...

The solution for not knowing the private registry is to add it to containers /etc/hosts.

You can find it by inspecting the ID of the container:

root@control01:/home/ubuntu# docker inspect 97820140a740 | grep /hosts
        "HostsPath": "/var/lib/docker/containers/97820140a7402ac215702c0db751fa5992bf000aa6a9dfcb683aca796ae49090/hosts",

My private registry address is 192.168.0.232 registry registry.cube.local, so lets add it:

echo '192.168.0.232 registry registry.cube.local' | sudo tee -a /var/lib/docker/containers/97820140a7402ac215702c0db751fa5992bf000aa6a9dfcb683aca796ae49090/hosts >/dev/null

This should be enough if you use the HTTP registry only.

TLS Fix for buildx

To push to a private TLS protected registry, you need to make the container aware of the root certificate.

You get the certificate I'm talking about when setting up a private Docker registry. Docker-Registry TLS

BUILDER=$(sudo docker ps | grep buildkitd | cut -f1 -d' ')
sudo docker cp registry.crt $BUILDER:/usr/local/share/ca-certificates/
sudo docker exec $BUILDER update-ca-certificates
sudo docker restart $BUILDER

This solution is from: GitHub Issue 80

This fixed the issue for me, and faas-cli works as expected 🙂

Did it help you ?


Last update: May 26, 2021

Comments