Skip to content

in_http: add_remote_addr #2130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aletourneau wants to merge 1 commit into
base: master
Choose a base branch
from
Open

in_http: add_remote_addr #2130

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 79 additions & 11 deletions pipeline/inputs/http.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,18 @@

## Configuration parameters

| Key | Description | Default |
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|-----------|
| `buffer_chunk_size` | This sets the chunk size for incoming JSON messages. These chunks are then stored and managed in the space available by `buffer_max_size`. | `512K` |
| `buffer_max_size` | Specify the maximum buffer size to receive a JSON message. | `4M` |
| `http2` | Enable HTTP/2 support. | `true` |
| `listen` | The address to listen on. | `0.0.0.0` |
| `port` | The port for Fluent Bit to listen on. | `9880` |
| `success_header` | Add an HTTP header key/value pair on success. Multiple headers can be set. For example, `X-Custom custom-answer`. | _none_ |
| `successful_response_code` | Allows setting successful response code. Supported values: `200`, `201`, and `204`. | `201` |
| `tag_key` | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key. | _none_ |
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
| Key | Description | Default |
|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
| `add_remote_addr` | Adds a `REMOTE_ADDR` field to the record. The value of `REMOTE_ADDR` is the client's address, which is extracted from the `X-Forwarded-For` header. | `false` |
| `buffer_chunk_size` | This sets the chunk size for incoming JSON messages. These chunks are then stored and managed in the space available by `buffer_max_size`. | `512K` |
| `buffer_max_size` | Specify the maximum buffer size to receive a JSON message. | `4M` |
| `http2` | Enable HTTP/2 support. | `true` |
| `listen` | The address to listen on. | `0.0.0.0` |
| `port` | The port for Fluent Bit to listen on. | `9880` |
| `success_header` | Add an HTTP header key/value pair on success. Multiple headers can be set. For example, `X-Custom custom-answer`. | _none_ |
| `successful_response_code` | Allows setting successful response code. Supported values: `200`, `201`, and `204`. | `201` |
| `tag_key` | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key. | _none_ |
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |

### TLS / SSL

Expand All @@ -41,6 +42,32 @@
curl -d '{"key1":"value1","key2":"value2"}' -XPOST -H "content-type: application/json" http://localhost:8888/app.log
```

### Add remote addr

Check warning on line 45 in pipeline/inputs/http.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Spelling] Spelling check: 'addr'?

Raw Output:
{"message": "[FluentBit.Spelling] Spelling check: 'addr'?", "location": {"path": "pipeline/inputs/http.md", "range": {"start": {"line": 45, "column": 16}}}, "severity": "INFO"}

The `add_remote_addr` configuration option, when activated, adds a `REMOTE_ADDR` field to the records. The value of `REMOTE_ADDR` is the client's address, which is extracted from the `X-Forwarded-For` header.

In most cases, only a single `X-Forwarded-For` header is in the request, so the following curl would add a `REMOTE_ADDR` field which would be set to `host1`:

```shell
curl -d '{"key1":"value1"}' -XPOST -H 'Content-Type: application/json' -H 'X-Forwarded-For: host1, host2' http://localhost:8888
```

However, if your system sets multiple `X-Forwarded-For` headers in the request, the one used (first, or last) depends on the value of the `http2` config. For example:

Check warning on line 55 in pipeline/inputs/http.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Spelling] Spelling check: 'config'?

Raw Output:
{"message": "[FluentBit.Spelling] Spelling check: 'config'?", "location": {"path": "pipeline/inputs/http.md", "range": {"start": {"line": 55, "column": 147}}}, "severity": "INFO"}

Assuming the following X-Forwarded-For headers are in the request:

```text
X-Forwarded-For: host1, host2
X-Forwarded-For: host3, host4
```

The value of REMOTE_ADDR will be:

| http2 config | value of REMOTE_ADDR |

Check warning on line 66 in pipeline/inputs/http.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Spelling] Spelling check: 'config'?

Raw Output:
{"message": "[FluentBit.Spelling] Spelling check: 'config'?", "location": {"path": "pipeline/inputs/http.md", "range": {"start": {"line": 66, "column": 9}}}, "severity": "INFO"}
|------------------|----------------------|
| `true` (default) | host3 |
| `false` | host1 |

### Configuration file

{% tabs %}
Expand Down Expand Up @@ -161,6 +188,47 @@
{% endtab %}
{% endtabs %}

#### Set `add_remote_addr`

The `add_remote_addr` configuration option lets you activate a feature that systematically adds the `REMOTE_ADDR` field to events, and set its value to the client's address. The address will be extracted from the `X-Forwarded-For` header of the request. The format is:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
- name: http
listen: 0.0.0.0
port: 8888
add_remote_addr: true
outputs:
- name: stdout
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[INPUT]
Name http
Listen 0.0.0.0
Port 8888
Add_remote_addr true

[OUTPUT]
Name stdout
```

{% endtab %}
{% endtabs %}

#### Example curl to test this feature

```shell
curl -d '{"key1":"value1"}' -XPOST -H 'Content-Type: application/json' -H 'X-Forwarded-For: host1, host2' http://localhost:8888
```

#### Set multiple custom HTTP headers on success

The `success_header` parameter lets you set multiple HTTP headers on success. The format is:
Expand Down
Loading