How to get logs from kubernetes using Go?

Question

I'm looking for the solution of how to get logs from a pod in Kubernetes cluster using Go. I've looked at "https://github.com/kubernetes/client-go" and "https://godoc.org/sigs.k8s.io/controller-runtime/pkg/client", but couldn't understand how to use them for this purpose. I have no issues getting information of a pod or any other object in K8S except for logs.

For example, I'm using Get() from "https://godoc.org/sigs.k8s.io/controller-runtime/pkg/client#example-Client--Get" to get K8S job info:

found := &batchv1.Job{}
err = r.client.Get(context.TODO(), types.NamespacedName{Name: job.Name, Namespace: job.Namespace}, found)

Please share of how you get pod's logs nowadays. Any suggestions would be appreciated!

Update: The solution provided in Kubernetes go client api for log of a particular pod is out of date. It have some tips, but it is not up to date with current libraries.

Solution 1

Here is what we came up with eventually using client-go library:

func getPodLogs(pod corev1.Pod) string {
    podLogOpts := corev1.PodLogOptions{}
    config, err := rest.InClusterConfig()
    if err != nil {
        return "error in getting config"
    }
    // creates the clientset
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        return "error in getting access to K8S"
    }
    req := clientset.CoreV1().Pods(pod.Namespace).GetLogs(pod.Name, &podLogOpts)
    podLogs, err := req.Stream()
    if err != nil {
        return "error in opening stream"
    }
    defer podLogs.Close()

    buf := new(bytes.Buffer)
    _, err = io.Copy(buf, podLogs)
    if err != nil {
        return "error in copy information from podLogs to buf"
    }
    str := buf.String()

    return str
}

I hope it will help someone. Please share your thoughts or solutions of how you get logs from pods in Kubernetes.

Solution 2

And if you want read stream in client-go v11.0.0+, the code is like this, feel free for create clientset by yourself:

func GetPodLogs(namespace string, podName string, containerName string, follow bool) error {
    count := int64(100)
    podLogOptions := v1.PodLogOptions{
        Container: containerName,
        Follow:    follow,
        TailLines: &count,
    }

    podLogRequest := clientSet.CoreV1().
        Pods(namespace).
        GetLogs(podName, &podLogOptions)
    stream, err := podLogRequest.Stream(context.TODO())
    if err != nil {
        return err
    }
    defer stream.Close()

    for {
        buf := make([]byte, 2000)
        numBytes, err := stream.Read(buf)
        if numBytes == 0 {
            continue
        }
        if err == io.EOF {
            break
        }
        if err != nil {
            return err
        }
        message := string(buf[:numBytes])
        fmt.Print(message)
    }
    return nil
}

Solution 3

The controller-runtime client library does not yet support subresources other than /status, so you would have to use client-go as shown in the other question.

Solution 4

Combining some answers found elsewhere and here to stream (tailing) logs for all containers (init included):

func GetPodLogs(namespace string, podName string) {
    pod, err := clientSet.CoreV1().Pods(namespace).Get(ctx, podName, metav1.GetOptions{})
    if err != nil {
        return err
    }
    wg := &sync.WaitGroup{}
    functionList := []func(){}
    for _, container := range append(pod.Spec.InitContainers, pod.Spec.Containers...) {
        podLogOpts := v1.PodLogOptions{}
        podLogOpts.Follow = true
        podLogOpts.TailLines = &[]int64{int64(100)}[0]
        podLogOpts.Container = container.Name
        podLogs, err := clientSet.CoreV1().Pods(namespace).GetLogs(podName, &podLogOpts).Stream(ctx)
        if err != nil {
            return err
        }
        defer podLogs.Close()
        functionList = append(functionList, func() {
            defer wg.Done()
            reader := bufio.NewScanner(podLogs)
            for reader.Scan() {
                select {
                case <-ctx.Done():
                    return
                default:
                    line := reader.Text()
                    fmt.Println(worker+"/"+podLogOpts.Container, line)
                }
            }
            log.Printf("INFO log EOF " + reader.Err().Error() + ": " + worker + "/" + podLogOpts.Container)
        })
    }

    wg.Add(len(functionList))
    for _, f := range functionList {
        go f()
    }
    wg.Wait()
    return nil
}

View more solutions

Comments

[Rawkode]

Possible duplicate of Kubernetes go client api for log of a particular pod

[Emixam23]

How do you manage when you have replicas for the same pod? they have different IP but how can you know which one corresponds to which one? I want to get my name, how do I do?

[scniro]

Anyway to specify all containers via PodLogOptions? Seems to want a specified container name... would be interested to find a possible --all-containers equivalent

[Asalle]

either you comment is not up-to-date or you didn't word it clearly, either way it doesn't add much value to the existing answer in my opinion

[opricnik]

/logs is a subresource of /pods, and there is no support for that in the controller-runtime client library at this time.

[shuiqiang]

what is the and/or relationship in podLogOptions?

[Jingshao Chen]

Works as of today 2020-02-28

[Bilbo Baggins]

@scniro we solved this by getting logs for all of a pod's containers concurrently using Go routines. github.com/homedepot/go-clouddriver/blob/…

[tormodatt]

might be a good idea to add a sleep before continue. otherwise CPU will try to run the loop as fast as possible, resulting in a high CPU usage

[ruckc]

To use this, I needed to move the if numBytes == 0 conditional to after the if err == io.EOF as it just went infinite as there was no EOF checking

[Steve]

How do you handle logs timeout?