<!-- status autogenerated section -->
# Queue Batch Processor
| Status        |           |
| ------------- |-----------|
| Stability     | [development]: traces, metrics, logs, profiles   |
| Distributions | [] |
| Issues        | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector?query=is%3Aissue%20is%3Aopen%20label%3Aprocessor%2Fqueuebatch%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aopen+is%3Aissue+label%3Aprocessor%2Fqueuebatch) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector?query=is%3Aissue%20is%3Aclosed%20label%3Aprocessor%2Fqueuebatch%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector/issues?q=is%3Aclosed+is%3Aissue+label%3Aprocessor%2Fqueuebatch) |
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner)    | [@jmacd](https://www.github.com/jmacd), [@iblancasa](https://www.github.com/iblancasa) |

[development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development
<!-- end autogenerated section -->

The queue/batch processor accepts spans, metrics, logs, or profiles
and places them into batches, using the [exporterhelper sending queue
and batch](../../exporter/exporterhelper/README.md#sending-queue) logic
and configuration.

This component replaces the [batchprocessor](../batchprocessor); see
[docs/rfcs/batching-migration.md](../../docs/rfcs/batching-migration.md). This
component improves on the former batch processor while maintaining its
general behavior. This component uses default settings taken from the
`batchprocessor` mapped into `exporterhelper` settings:

```yaml
# The processor will produce and export one batch at a time.
num_consumers: 1

# The queue is measured in request units.
sizer: requests

# The queue admits 10 requests.
queue_size: 10

# Backpressure: The queue blocks the caller when full until
# canceled.
block_on_overflow: true

# Error propagation: The caller receives success immediately 
# after the request enters the queue. Enable to wait for success
# or failure.
wait_for_result: false

# These defaults match both batchprocessor and exporterhelper.
batch: 
  enabled: true
  sizer: items
  min_size: 8192
  flush_timeout: 200ms
```

Refer to the [exporterhelper
documentation](../../exporter/exporterhelper/README.md) for more
details on this configuration structure. Other than changing the
default configuration, the entire `queuebatch` functionality is
available through this processor, including partitioning and storage.

## Examples

The following examples show two capabilities that Collector users
frequently rely on: batching by client metadata and buffering the
queue on disk.

### Partitioning with `metadata_keys`

The `metadata_keys` setting enables forming distinct batch for each
combination of `client.Metadata` values, so that a multi-tenant
pipeline could batch each tenant's data separately. The queue/batch
processor offers the same behavior through the
`batch::partition::metadata_keys` setting.

```yaml
receivers:
  otlp:
    protocols:
      grpc:
        # Capture client metadata so it is available for partitioning.
        include_metadata: true

processors:
  queuebatch:
    batch:
      partition:
        # Produce one batch per distinct tenant_id value.
        metadata_keys:
        - tenant_id

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [queuebatch]
      exporters: [otlp]
```

Receivers must be configured with `include_metadata: true` so the
metadata keys are available to the processor. An empty value and unset
metadata are treated as distinct cases, and entries are
case-insensitive.

### Persisting the queue with a storage extension

By default the queue is held in memory, so any buffered data is lost
if the Collector stops. Set the top-level `storage` field to the ID of
a [storage extension][filestorage] to persist the queue on disk, so
that buffered data is recovered and exporting continues after a
restart.

```yaml
extensions:
  file_storage:
    directory: /var/lib/otelcol/queuebatch

processors:
  queuebatch:
    # Persist the queue using the file_storage extension.
    storage: file_storage

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [otlp]
      processors: [queuebatch]
      exporters: [otlp]
```

When `storage` is set there is no in-memory queue. Each request is
serialized as it enters the queue and read back from disk before
export. The `wait_for_result` setting is not supported together with
`storage`.

[filestorage]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/storage/filestorage
