Skip to content

Document the storage.backlog.flush_on_shutdown Fluent Bit buffer option #344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
promptless-for-oss wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Document the storage.backlog.flush_on_shutdown Fluent Bit buffer option #344

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions content/docs/logging-infrastructure/fluentbit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,29 @@ spec:
For the detailed list of available parameters for this plugin, see {{% xref "/docs/configuration/crds/v1beta1/fluentbit_types.md#bufferstorage" %}}.
[More Info](https://docs.fluentbit.io/manual/v/1.3/configuration/buffering).

### Flush the filesystem buffer on shutdown

Setting `storage.backlog.flush_on_shutdown` to `"On"` makes Fluent Bit flush all filesystem-backlog chunks to their destination during a graceful shutdown. This way, you don't lose buffered records when a node is drained or a spot instance is reclaimed.

This option is opt-in and defaults to `"Off"`. The operator renders the option only if you set it.

Enable filesystem buffering first, or this option has no effect: set `storage.type: filesystem` on the tail input and set `storage.path`. This setting only affects filesystem-backlog chunks.

```yaml
apiVersion: logging.banzaicloud.io/v1beta1
kind: FluentbitAgent
metadata:
name: default-logging-simple
spec:
inputTail:
storage.type: filesystem
bufferStorage:
storage.path: /buffers
storage.backlog.flush_on_shutdown: "On"
```

Increase [`terminationGracePeriodSeconds`]({{< relref "/docs/configuration/crds/v1beta1/fluentbit_types.md#fluentbitspec-terminationgraceperiodseconds" >}}) so the pod has enough time to finish flushing before Kubernetes force-kills it.

### HostPath volumes for buffers and positions

```yaml
Expand Down