Overview

Wind flagging algorithms identify operational states and conditions in turbine time series data. This document describes the standardized pipeline architecture used by the wind flagging implementations.

Workflow

  1. Algorithm receives a turbine collection and an ObjectTimeSeriesCollection
  2. Validation: Verify required sensor signals exist with data quality flags, required derived signals exist, and required signals have sufficient data
  3. Build DataFrame: Construct a working DataFrame from the time series collection, using signal categories as columns
  4. Filter: Apply the data quality mask and algorithm-specific filters, preserving the combined discard mask
  5. Extract Parameters: Get turbine configuration or estimates needed by the algorithm
  6. Compute: Run algorithm logic on filtered data to produce flag values
  7. Unfilter: Reindex to the original timestamps and fill discarded or missing points with the algorithm fill value, normally -1
  8. Return: Add or replace output time series, preserving temporal alignment

Design Philosophy

The base flagging algorithm separates data preparation from algorithm logic. This separation ensures that concrete algorithms focus on domain-specific logic while the framework handles data quality, validation, and consistency concerns.

Why Standardization Matters

  • Reliability: Consistent validation prevents silent failures and misleading results
  • Maintainability: Algorithm developers focus on flagging logic, not infrastructure
  • Composability: Flags from one algorithm can serve as inputs to another

Pipeline Stages

1. Input Validation

Purpose: Catch invalid inputs with clear errors before producing downstream results.

Why This Matters: - Catches configuration errors (missing signals, wrong object) at the earliest practical point - Validates data sufficiency (minimum data points) to prevent meaningless statistical computations - Ensures data quality infrastructure exists before attempting analysis on sensor inputs

What Gets Validated: - Required sensor and derived signal categories exist in the input collection - Each required sensor signal has an associated dq_flag column - Each required signal contains at least 10 non-NaN values

2. Filtering

Purpose: Isolate clean data for algorithm processing while preserving the ability to restore full time series alignment.

Why This Matters: - Algorithm logic operates on reliable data without quality concerns cluttering the code - Bad quality data does not contaminate statistical computations - Preserves the discard mask so outputs can be aligned to the original timestamps

What Gets Filtered: - Data quality flags: Points where any required sensor input has non-zero dq_flag - Custom filters: Algorithm-specific conditions, such as non-running periods, non-stationary periods, or invalid control-state periods

Nota Bene: Filtering is temporary for computation. Discarded points are restored in the output index and filled with the algorithm fill value, normally -1.

3. compute_flag

Purpose: Execute the core algorithm logic on validated, filtered data.

Why This Matters: - Algorithms work with a guarantee that input data has passed the framework filters - Domain logic remains uncluttered by quality handling or validation concerns - Turbine-specific parameters can be injected cleanly

Implementation Contract: - Receives a filtered DataFrame containing required signals - Receives turbine parameters extracted from configuration or filtered data when the concrete algorithm needs them - Returns flag values indexed to match the filtered DataFrame - Does not need to reapply data quality or custom-filter masks

4. Unfiltering

Purpose: Restore complete time series alignment by reintegrating filtered-out timestamps.

Why This Matters: - Output maintains the same temporal coverage as input where a source timestamp exists - Makes it explicit where the algorithm could not evaluate a point, using the fill value - Allows downstream algorithms to distinguish a valid false or zero flag from a discarded point

Restoration Logic: - Previously filtered points are set to the algorithm fill value, normally -1 - Remaining missing values are filled with the same fill value - Temporal alignment with input is preserved - Output collection can be directly added to or merged with the input collection

Concrete Algorithm Requirements

To implement a new flagging algorithm, subclasses must define:

Required Implementations

  • sensor_required_signals: List of raw sensor inputs needing data quality validation
  • output_signal_category: Identifier for the output flag signal
  • compute_flag: Core algorithm logic operating on filtered data
  • run: Orchestration method that executes the pipeline

Optional Overrides

  • derived_required_signals: Computed upstream flags required by the algorithm, without data quality validation
  • get_turbine_parameters: Turbine configuration values passed to compute_flag
  • get_additional_filters: Algorithm-specific filters beyond data quality