# Scraper Controller Extensions ## Overview This RFC proposes introducing an extension interface for scraper controllers to enable event-driven scraping patterns. The current time-based scraper controller will be maintained with an option to disable the timer, and new configuration will allow specifying one or more scraper controller extensions that can trigger scrapes based on external events. ## Motivation The current scraper controller implementation uses a time-based ticker to periodically invoke scrapers. While this works well for many use cases, there are scenarios where event-driven scraping would be more appropriate, such as: 1. **Webhook-driven scrapes**: On-demand profiling or detailed metric collection triggered by alerts (e.g., CPU/memory spikes) 2. **Job-queue based scraping**: Horizontal scaling where multiple collector instances consume scrape jobs from a shared queue 3. **External scheduling**: Using external schedulers (e.g. cron, Kubernetes CronJobs) to avoid mostly-idle, long-lived processes ## Current State The scraper controller is implemented in `scraper/scraperhelper/internal/controller/controller.go`: - Uses `ControllerConfig` with `CollectionInterval`, `InitialDelay`, and `Timeout` fields - Creates a time-based ticker in `startScraping()` that periodically calls the scrape function - Scrapers are created during controller initialization and started/shutdown with the controller - The controller manages the lifecycle of multiple scrapers and coordinates their execution ## Proposed Changes We propose to introduce a new extension interface for scraper controllers, enabling flexibility over how scrapes are triggered. The timer-based scraping will remain the default behavior but can be disabled via configuration. ### 1. Extension Interface Create a new extension interface (e.g. in `extension/extensionscrapercontroller`): ```golang // ControllerExtension is an extension that provides a means of registering scrapers, // and giving the extension control over when registered scrapers are invoked. type ControllerExtension interface { extension.Extension // RegisterScraper registers a scraper with this controller extension. // The extension will invoke the provided scrape function according to its implementation. // Returns a registration handle that can be used to deregister the scraper. RegisterScraper(ctx context.Context, scraperID component.ID, scrapeFunc func(context.Context) error) (RegistrationHandle, error) } // RegistrationHandle provides a way to deregister a scraper from a controller extension type RegistrationHandle interface { // Deregister removes the scraper from the controller extension Deregister(ctx context.Context) error } ``` We will define a `DeregisterFunc` function type implementing `RegistrationHandle` for convenience: ```golang // DeregisterFunc implements RegistrationHandle using a simple function. // // If the function value is nil, the method call will be a no-op. type DeregisterFunc func(ctx context.Context) error // Deregister calls the underlying function to deregister the scraper. func (f DeregisterFunc) Deregister(ctx context.Context) error { if f == nil { return nil } return f(ctx) } ``` ### 2. Configuration Changes Modify `ControllerConfig` to support: - Optional timer configuration (can be disabled) - List of controller extension IDs ```golang type ControllerConfig struct { // CollectionInterval sets how frequently the scraper should be called. // If zero or negative, the timer-based scraping is disabled. // At least one controller extension must be configured if timer is disabled. CollectionInterval time.Duration `mapstructure:"collection_interval"` // InitialDelay sets the initial start delay for the scraper timer. InitialDelay time.Duration `mapstructure:"initial_delay"` // Timeout is an optional value used to set scraper's context deadline. Timeout time.Duration `mapstructure:"timeout"` // Controllers specifies one or more controller extensions that // will control when scrapes are triggered. Extensions must implement // scraperhelper.ControllerExtension. Controllers []component.ID `mapstructure:"controllers"` } ``` ### 3. Controller Implementation Changes Modify `controller.Controller` to: - Support disabling the timer when `CollectionInterval <= 0` - Register scrapers with configured controller extensions during `Start()` - Deregister scrapers during `Shutdown()` - Validate that at least one controller extension is configured if timer is disabled ### 4. Reference Implementation We will introduce a reference implementation of the controller extension interface that is triggered by HTTP requests. This will be included in the OpenTelemetry Collector Contrib repository. The exact component name and configuration are to be determined, but will be along the lines of the webhook-driven controller described below. ## Use Cases Below are use cases that motivate this change. Some of the use cases describe hypothetical scrapers, such scraper-based profiling, that do not currently exist in the OpenTelemetry Collector. They are included to illustrate how event-driven scraping could be beneficial. ### Use Case 1: Webhook-Driven Scraping A collector receives webhook notifications when CPU usage exceeds a threshold and triggers an on-demand profiling scrape of a pprof endpoint: ```yaml extensions: webhook_scraper_controller: endpoint: http://0.0.0.0:8080 path: / receivers: # Hypothetical pprof receiver that scrapes pprof HTTP endpoints pprof/cpu_10s: endpoint: https://localhost:6060/debug/pprof?seconds=10 controllers: - webhook_controller service: extensions: [webhook_scraper_controller] pipelines: profiles: receivers: [pprof/cpu_10s] exporters: [debug] ``` ### Use Case 2: Job Queue Based Scraping Multiple collector instances consume scrape jobs from a shared queue: ```yaml extensions: queue_scraper_controller: queue_type: redis queue_url: redis://queue:6379 queue_name: httpcheck-jobs receivers: httpcheck: collection_interval: 0 targets: - ... controllers: - queue_controller ``` ### Use Case 3: External Cron Scheduling (one-shot execution) A Kubernetes CronJob runs the collector as a one-shot job that scrapes once and exits, avoiding a mostly-idle long-lived process. A similar approach could be used with any external scheduler, such as AWS EventBridge or Google Cloud Scheduler. In any case, the collector is expected to run, perform the scrape, and then exit. Multiple implementation options exist for this use case, and the best approach may depend on the specific platform. For example, integrating with AWS EventBridge may may involve running the collector as an AWS Lambda function, handling the relevant invocation event. For plain old cron, or Kubernetes CronJobs, two possible approaches are outlined below. #### Option A: new "scrape" CLI command ```bash # Kubernetes CronJob runs a new command, specifying the pipeline(s) and scraper(s) # to run once and then exit. otelcol scrape --config=config.yaml metrics hostmetrics ``` The collector would support a new `scrape` CLI command that: - Loads the specified configuration - Injects a controller extension into the specified scraper(s) that the command can invoke to trigger a scrape - Triggers the scraper(s) once via the injected controller extension - Exits after the scrape completes #### Option B: self-contained, one-shot controller ```yaml extensions: oneshot_controller: # Extension that triggers scrape on registration and signals shutdown after completion receivers: hostmetrics: collection_interval: 0 controllers: - oneshot_controller ``` The challenge with Option B is that it becomes difficult if not impossible to invoke multiple scrapers, as they will start concurrently and independently. Consequently, there needs to be some additional coordination mechanism to wait for all scrapers to complete before exiting. ## Open Questions 1. Should there be a way to pass dynamic configuration from the extension to the scraper, e.g. `targets` for httpcheck?