Streaming and live tailing log entries

This document describes how to view your log entries in real time by streaming and live tailing. Streaming logs lets you view log entries in real time and is available in the Logs Explorer. Live tailing lets you view log entries in real time and is available as the gcloud CLI command gcloud alpha logging tail and as the Cloud Logging API method entries.tail.

When you view and analyze your logs using the Logs Explorer, gcloud logging read, or the API method entries.list, you're viewing log entries that Cloud Logging has stored. When you stream or live tail log entries, you're viewing log entries as your applications write them to the Cloud Logging API.

Stream logs in the Logs Explorer

In the Logs Explorer, you can view your logs data in real time using Stream logs. When you use Stream logs, you can add a query to stream only those logs that match the query. To stream logs, do the following:

Use live tailing in the Google Cloud CLI

Live tailing lets you view your log entries in real time as Cloud Logging writes them, by using either the Google Cloud CLI or the Cloud Logging API.

For information on the API method for live tailing, see the entries.tail method.

Installing gcloud alpha logging tail

To use gcloud alpha logging tail, you need to have Python 3 and the grpcio Python package installed.

For instructions on how to install Python, see the Python page. For instructions on how to install the Python package manager, pip, that is needed to install the grpcio package, see The Python Package Installer page.

Buffering and ordering

Because Logging can receive log entries out of chronological order, live tailing provides a buffer-window setting so you can balance the tradeoff between viewing the log entries as they are being written and viewing them in ascending order. You can set the buffer window between 0 and 60 seconds.

When configuring the buffer window, you make a tradeoff between viewing logs as they are written and viewing the entries out of order.

Limits and quotas

Client limitations

For a Google Cloud project that writes lots of entries quickly, your client might be unable to consume them as quickly as they're being written. In this case, Logging limits the total number of entries sent, prioritizing the most recent entries. At the end of the tail session, Logging returns the number of entries that were not displayed due to the limits of the client.

Use live tailing with client libraries

Live tailing lets you view your log entries in real time as Cloud Logging writes them. For information on the API method for live tailing, see the entries.tail method.

Go

To learn how to install and use the client library for Logging, see Logging client libraries.

To authenticate to Logging, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

import (
	"context"
	"fmt"
	"io"

	logging "cloud.google.com/go/logging/apiv2"
	"cloud.google.com/go/logging/apiv2/loggingpb"
)

// tailLogs creates a channel to stream log entries that were recently ingested for a project
func tailLogs(projectID string) error {
	// projectID := "your_project_id"

	ctx := context.Background()
	client, err := logging.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("NewClient error: %w", err)
	}
	defer client.Close()

	stream, err := client.TailLogEntries(ctx)
	if err != nil {
		return fmt.Errorf("TailLogEntries error: %w", err)
	}
	defer stream.CloseSend()

	req := &loggingpb.TailLogEntriesRequest{
		ResourceNames: []string{
			"projects/" + projectID,
		},
	}
	if err := stream.Send(req); err != nil {
		return fmt.Errorf("stream.Send error: %w", err)
	}

	// read and print two or more streamed log entries
	for counter := 0; counter < 2; {
		resp, err := stream.Recv()
		if err == io.EOF {
			break
		}
		if err != nil {
			return fmt.Errorf("stream.Recv error: %w", err)
		}
		fmt.Printf("received:\n%v\n", resp)
		if resp.Entries != nil {
			counter += len(resp.Entries)
		}
	}
	return nil
}

Java

To learn how to install and use the client library for Logging, see Logging client libraries.

To authenticate to Logging, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

import com.google.cloud.logging.LogEntry;
import com.google.cloud.logging.LogEntryServerStream;
import com.google.cloud.logging.Logging;
import com.google.cloud.logging.Logging.TailOption;
import com.google.cloud.logging.LoggingOptions;

public class TailLogEntries {

  public static void main(String[] args) throws Exception {
    // TODO(developer): Optionally provide the logname as an argument.
    String logName = args.length > 0 ? args[0] : "";

    LoggingOptions options = LoggingOptions.getDefaultInstance();
    try (Logging logging = options.getService()) {

      // Optionally compose a filter to tail log entries only from specific log
      LogEntryServerStream stream;

      if (logName != "") {
        stream =
            logging.tailLogEntries(
                TailOption.filter(
                    "logName=projects/" + options.getProjectId() + "/logs/" + logName));
      } else {
        stream = logging.tailLogEntries();
      }
      System.out.println("start streaming..");
      for (LogEntry log : stream) {
        System.out.println(log);
        // cancel infinite streaming after receiving first entry
        stream.cancel();
      }
    }
  }
}

Node.js

To learn how to install and use the client library for Logging, see Logging client libraries.

To authenticate to Logging, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.

const {Logging} = require('@google-cloud/logging');
const logging = new Logging();

/**
 * TODO(developer): Replace logName with the name of your log.
 */
const log = logging.log(logName);
console.log('running tail log entries test');

const stream = log
  .tailEntries({
    filter: 'timestamp > "2021-01-01T23:00:00Z"',
  })
  .on('error', console.error)
  .on('data', resp => {
    console.log(resp.entries);
    console.log(resp.suppressionInfo);
    // If you anticipate many results, you can end a stream early to prevent
    // unnecessary processing and API requests.
    stream.end();
  })
  .on('end', () => {
    console.log('log entry stream has ended');
  });

// Note: to get all project logs, invoke logging.tailEntries

Streaming and live tailing log entries Stay organized with collections Save and categorize content based on your preferences.

Stream logs in the Logs Explorer

Use live tailing in the Google Cloud CLI

Installing gcloud alpha logging tail

Buffering and ordering

Limits and quotas

Client limitations

Use live tailing with client libraries

Go

Java

Node.js

Installing `gcloud alpha logging tail`