FAQ and Troubleshooting

Common issues, frequently asked questions, and troubleshooting guidance for the CreativeDynamics library.

FAQ: Could this be used to understand the “why” behind channel performance?

Not directly. As implemented, CreativeDynamics is not a causal inference system.

It is best framed as diagnostic + decision support, not causal explanation.

What it can do well

1. Detect statistically and structurally meaningful regime shifts (change points) in irregular KPI paths.
2. Summarise behaviour within each detected segment (stable / improving / declining).
3. Quantify a decision-relevant impact metric using benchmarking (for example actual_overspend_gbp versus a benchmark period).
4. Support monitoring and operational workflows: “Something changed here; performance is now in a worse regime; consider intervention.”

What it cannot do (by itself)

1. Identify the “why” (attribution): competitor action, auction dynamics, creative fatigue, targeting changes, reporting artefacts, and so on.
2. Make claims like “X caused Y” without an additional identification strategy (for example a controlled experiment, causal design, or instrument).

Is “predictive” the right word?

“Predictive” is often used loosely to mean “actionable signal for decisions”.

If your audience includes data scientists, more precise terms are usually better:

1. Change-point detection / regime detection
2. Retrospective inference (on historical data)
3. Monitoring / early warning (if run routinely)
4. Counterfactual-style benchmarking for impact metrics (“what if we stayed at benchmark?”)

If you run this routinely and treat detected regime shifts as a leading indicator that performance is likely to remain degraded without intervention, then it can be described as predictive in an early-warning sense.

It is not forecasting in the usual time-series sense (there is no explicit model producing future values).

Important nuance about benchmarking impact

Impact metrics such as actual_overspend_gbp and engagement_gap_clicks are based on a practical, quasi-counterfactual assumption:

“Benchmark performance is a plausible reference you could achieve again; impact is the gap while you are in a worse regime.”

This is not causal identification, but it can still be operationally useful if the benchmark period is representative and achievable.

Common Issue 1: Understanding Trend Line (slope: nan) in analysis output

If visualisation legend shows “Trend Line (slope: nan)” in creative analysis output, indicates no clear linear trend could be computed for that segment.

Statistical interpretation:

1. Insufficient data points: Not enough valid data points to establish trend
2. Too many missing values: Many data points have missing values or zeros
3. Perfectly horizontal data: All data points have exactly same value (no change)
4. Computational singularity: Division by zero or other mathematical issues in calculation

Business interpretation:

1. Creative’s performance is erratic or random with no discernible pattern
2. Too many days with zero impressions, clicks, or spend
3. Creative might have irregular serving patterns (turned on/off repeatedly)
4. Data could contain tracking issues or anomalies

Important note about “Improving” creatives with NaN trends:

May occasionally see creative classified as “improving” despite having NaN trend line. This occurs because:

1. Signature-based analysis: Classification comes from signature pattern analysis, independent of trend line calculation
2. Pattern vs. trend: Signature patterns can capture complex, non-linear improvements that linear trend lines cannot
3. Data characteristics: Performance might improve in specific time windows even if overall trend is undefined
4. Metric stability: Consistent improvement in aspects like reduced variance can be detected without clear trend

Solution: For creatives with “nan” slopes:

1. Consider them neither definitively improving nor declining based solely on linear trend
2. Examine signature patterns and detailed metrics for deeper insights
3. Treat “improving” classifications with NaN trends as cautiously positive signals
4. Consider manual review to understand actual performance patterns

Common Issue 2: Interpreting different performance/trend categories

When analysing time-series data, segments between detected change points classified based on trends:

1. Improving Performance: Metric shows favourable trend (e.g., decreasing CPC, increasing CTR) with stable signature patterns
2. Stable Performance: Metric does not show significant trend, signature patterns also stable
3. Declining Performance: Metric shows unfavourable trend (e.g., increasing CPC, decreasing CTR), potentially accompanied by significant signature pattern changes

Solution: Exact interpretation depends on metric being analysed and specific analysis goals. Always consider metric context (e.g., for costs, lower is better; for rates, higher is often better).

Common Issue 3: Understanding distance calculations in output

Distance calculations represent mathematical distance between signature patterns calculated for consecutive time windows in creative’s timeline. Larger distances indicate more significant changes in performance pattern, may signal creative fatigue or other performance shifts.

Solution: Distance based on mathematical signatures derived from rough path theory, providing more nuanced insights than simple trend analysis. High distances flag segments worthy of investigation for pattern changes.

Common Issue 4: Installation issues with dependencies

Library now uses roughpy which provides pre-compiled wheels, eliminating most installation issues. If problems occur, ensure you have latest pip version.

Solution: Refer to the “System prerequisites” section in the Installation guide for detailed instructions specific to your operating system.

Common Issue 5: Determining supported Python versions

Library tested with Python 3.10 and newer.

Solution: For specific compatibility details and recommendations, see the “Python prerequisites” section in the Installation guide.

Common Issue 6: Whether to use a Python virtual environment

Yes, highly recommended.

Solution: Install and use CreativeDynamics within a Python virtual environment to avoid conflicts with other projects or system-wide Python packages. See the “Create and activate a virtual environment” steps in the Installation guide.

Common Issue 7: Expected format for input data

Library typically expects time-series data in CSV file or DuckDB database table. Data should include columns for creative identifier, date/timestamp, and metrics you wish to analyse (e.g., impressions, clicks, spend). All column names standardised to lowercase immediately after CSV ingestion to simplify data handling throughout pipeline.

Solution: Refer to examples in the Usage guide and the Schema page for more details on data structure.

Common Issue 8: Differences between CLI, Python API, and FastAPI usage

1. CLI: Recommended approach for most analyses, offering standardised workflows with YAML configuration
2. Python API: Suitable for integration into custom Python scripts, enabling programmatic analysis and deeper customisation of parameters
3. FastAPI: Provides web service interface for analysis, suitable for integration with other applications or building interactive dashboards

Solution: Consult the Usage guide for detailed instructions on each interface, including examples for CLI, Python API, and FastAPI.

Common Issue 9: Recommended minimum days of data for analysis

The default minimum is often around 14 days, but for more reliable analysis:

1. 14-30 days: Basic analysis (minimum viable).
2. 30-60 days: Good reliability.
3. 60+ days: High reliability, capable of detecting subtle pattern changes.

Solution: Setting min_days too low may result in premature pattern detection. Setting it too high might exclude too many creatives from analysis. Adjust based on your data volume and analysis goals.

Common Issue 10: Signature computation method used

The CreativeDynamics library uses the roughpy library for robust and efficient signature calculation.

Solution: This is the standard method used by the library for its core signature-based analyses.

Common Issue 11: Interpreting the threshold parameter for change point detection

The threshold parameter (default: 1.0 or 1.5 depending on the interface/function) determines sensitivity to pattern changes detected via signature distance:

1. Lower values (e.g., 0.5-0.9): More sensitive, may detect subtle changes earlier but with potentially more false positives.
2. Default (1.0 or 1.5): Balanced approach for most use cases.
3. Higher values (e.g., >1.5-2.0): Less sensitive, only detects major pattern changes, fewer false positives.

Solution: Adjust the threshold based on how sensitive you need the detection to be for your specific use case. Experimentation might be needed.

Common Issue 12: ModuleNotFoundError when trying to import creativedynamics

This typically indicates that the CreativeDynamics library is not installed in your current Python environment, or you are not running your script from within the correct virtual environment.

Solution:

1. Ensure you have activated the virtual environment where you installed CreativeDynamics.
2. Verify that the library was installed correctly. From the project root directory, you can run pip install . (or pip install -e . for an editable/developer installation).
3. Double-check your import statement: from creativedynamics import analyzer (or other relevant modules).
4. Refer to the Installation guide for detailed setup instructions.

Common Issue 13: Seeing “nan” distances in output logs

“nan” (Not a Number) distances in computation logs can occur when:

1. There are divisions by zero (e.g., zero clicks when calculating CPC).
2. The analysis window contains days with missing or invalid data.
3. The input data has extreme outliers or inconsistencies.
4. CTR values are not properly normalised.

Solution: These “nan” values are generally handled by the system (e.g., by not flagging a change point if distances are NaN). However, if they represent a significant portion of the data, it may affect overall analysis reliability and indicates potential data quality issues to investigate.

Common Issue 14: No creatives meeting the minimum days requirement

If no creatives in your dataset meet the min_days requirement:

Solution:

1. Reduce the min_days parameter to a lower value (if appropriate for your analysis goals).
2. Extend the date range in your input data to gather more history.
3. Verify that your creative IDs are consistently tracked (inconsistent IDs may fragment the time series for what should be a single item).

Common Issue 15: Handling very large datasets

For very large datasets:

Solution:

1. Filter the input dataset beforehand to analyse only the most relevant items or time periods.
2. Consider summarising or aggregating data if appropriate for your analysis goals, though this might lose some granularity that signature analysis can capture.
3. Increase the min_days (or equivalent data length filter) parameter to focus on items with substantial data history.
4. Ensure your system has sufficient memory, as loading and processing large time-series can be memory-intensive. Consider processing in batches if needed.
5. Use the CLI with YAML configuration for more efficient processing of large datasets.

Common Issue 16: A declining trend showing zero reported impact in an example application

In applications calculating actual_overspend_gbp (financial) or engagement_gap_clicks (operational), a declining performance trend might still result in zero calculated impact.

Solution: This can be due to the specific methodology:

1. No suitable benchmark found: If the analysis couldn’t find a long enough “Stable” or “Improving” period earlier in the creative’s life to set a reliable benchmark, impact calculation defaults to zero.
2. Insufficient data after benchmark: If there are insufficient data points after the benchmark period, impact may not be calculated.
3. Decline never exceeds benchmark: The CPC trend might be statistically declining within a segment, but the actual CPC values during that decline might never rise above the benchmark CPC established from a previous, better-performing period.

Common Issue 17: Understanding “excluded” creatives in reports

Creatives listed as “excluded” in the report summary or logs are filtered out before the main fatigue analysis. This is due to failing data quality checks based on parameters like:

• Minimum Days (min_days)
• Maximum Gap (max_gap_days)
• Minimum Density (min_density)

Solution: The standard reports currently only show the reason for exclusion. The raw data for these creatives exists in your input file. To analyse them, you might need to adjust filter parameters or investigate their data quality.

Common Issue 18: What is a “path signature” in simple terms?

A path signature is a mathematical tool that effectively “summarises” the essential shape and order of events in a time series (like the performance of an ad over time).

Solution: It’s designed to be more descriptive and robust than simple averages or trends, capturing how the data stream evolves. For a deeper dive, see the Concepts section.

Common Issue 19: Why use path signatures for creative fatigue analysis?

Signatures can detect complex, non-linear patterns and changes in how a creative performs over time, which simpler methods (like linear trend analysis) might miss.

Solution: This allows for a more nuanced understanding of performance shifts, helping to identify genuine fatigue beyond simple declines. Learn more in the Concepts section.

Common Issue 20: How to contribute to CreativeDynamics

We welcome contributions.

Solution: Please see Contributing for details on setting up your development environment, coding standards, running tests, and the overall contribution workflow.

Common Issue 21: Customising the metrics being analysed

Yes, the library allows analysing metrics beyond the default CPC and CTR, provided they exist in your input data and are specified correctly during analysis setup (for example via CLI -m flag or API parameters).

Solution: Ensure your custom metric is present in the data. When analysing, ensure the interpretation of “improving” vs. “declining” trends makes sense for your custom metric (e.g., for conversion rate, higher is better; for cost per acquisition, lower is better).

Common Issue 22: CTR values appear to be in percentage format

If your dataset contains CTR values stored as percentages (e.g., 0.37 for 37%) instead of proper fractions (e.g., 0.0037 for 0.37%), this can cause impossibly high CTR benchmarks and unrealistic impact calculations.

Solution:

1. The library now automatically detects and normalises CTR values that appear to be in percentage format by dividing them by 100.
2. You can verify this by checking the logs during data loading, which will show CTR normalisation range information.
3. If you still see unrealistic CTR values (e.g., CTR > 0.1 or 10%), you may need to manually preprocess your data before analysis.

Common Issue 23: Understanding financial vs operational metrics

CreativeDynamics reports two distinct types of metrics that should not be combined:

Financial metrics (actual costs):

• actual_overspend_gbp: Actual excess spending due to higher CPC than benchmark
• Total Spend: Actual money spent on the creative

Operational metrics (performance indicators):

• engagement_gap_clicks: Lost clicks relative to benchmark (primary)
• CTR Decline %: Performance drop from peak CTR
• Creative Health Score: Composite performance indicator (0.0-1.0)

Why are these separate?

In accounting, actual losses (money already spent) cannot be combined with potential losses (theoretical opportunity costs) in a single number. These are conceptually distinct and must be reported separately.

Solution: Use financial metrics for budget analysis and operational metrics for creative refresh decisions. Do not add them together.

Common Issue 24: Understanding correlation risk (do not combine metrics)

When CPC and CTR metrics are negatively correlated (when CTR drops, CPC rises), both metrics may reflect the same underlying performance issue.

Solution:

1. The library provides correlation context between CPC and CTR to inform interpretation.
2. Risk levels are categorised as:
   • High Risk: correlation < -0.5 (strong negative correlation)
   • Medium Risk: correlation < -0.2 (moderate negative correlation)
   • Low Risk: weak or positive correlation
3. Do not compute any combined total. Interpret engagement_gap_clicks (operational) and actual_overspend_gbp (financial) separately regardless of risk level.
4. The visualisation dashboard includes a correlation risk indicator to aid interpretation.

Common Issue 25: Does bot traffic invalidate Creative Fatigue measurement?

If a substantial portion of digital advertising traffic consists of bots (some estimates suggest 50-80%), does this make Creative Fatigue analysis meaningless?

Short answer: No. CreativeDynamics measures platform behaviour economics, not pure human psychology. The analysis remains valid for its primary use case: financial optimisation and spend efficiency.

Detailed explanation:

1. Platform filtering is imperfect

   • Meta and Google filter known bot traffic before reporting impressions and clicks
   • However, industry experts confirm that some bot traffic still leaks through these filters
   • Your analysis uses the platform’s reported metrics, which include this residual bot contamination
   • The exact bot ratio in your data is unknowable but likely consistent across campaigns

2. What you’re actually measuring

   • You’re not measuring “pure human creative response”
   • You’re measuring what you actually pay for - the platform’s reported performance
   • Whether that’s 70% human + 30% bot, 50/50, or any other ratio doesn’t change the financial reality
   • CreativeDynamics optimises platform performance as reported, which directly drives your costs

3. The analysis measures system fatigue

   • Temporal decline patterns are real (something in the system is changing)
   • CPC increases when “performance” declines (you pay more regardless of audience composition)
   • Financial impact is real whether audiences are bots, humans, or a mix
   • You’re optimising your position in the platform ecosystem as it exists

4. Why relative performance still matters

   • If bots are systematically distributed across campaigns:
     • Creative A declining faster than Creative B remains a meaningful signal
     • Relative performance guides budget allocation decisions
     • Rate of change indicates optimal refresh timing
   • If bots are not systematically distributed:
     • That’s a bigger problem than creative fatigue
     • CreativeDynamics would help detect unusual patterns suggesting this

5. Where bot traffic would invalidate analysis

   • Understanding human psychology for creative development
   • Informing brand strategy based on “what resonates with people”
   • Making qualitative creative decisions about emotional response
   • Academic research on consumer behaviour

6. Where analysis remains valid (your use case)

   • Budget allocation and spend efficiency (primary use case)
   • Identifying when to rotate creatives to minimise costs
   • Detecting anomalies and performance shifts
   • Financial optimisation decisions
   • Comparative performance assessment across creatives

Solution:

CreativeDynamics is measuring platform ecosystem efficiency, not human preference. In a world where you pay platforms regardless of bot ratios, optimising spend based on observable performance patterns is rational and necessary—whether you’re advertising to humans, bots, or a mixture.

The practical reality: You’re optimising financial performance in the advertising system as it exists. The “true audience composition” is unknowable and, for budget allocation decisions, largely irrelevant. What matters is identifying when reported performance declines trigger higher costs, and responding accordingly.

Bottom line: CreativeDynamics optimises what you can measure and control (platform-reported performance and associated costs), not what you cannot (exact audience composition). For its primary application—spend efficiency—bot contamination does not invalidate the analysis; it is already factored into the metrics you’re optimising.

Common Issue 26: Understanding Fatigue Status labels (Terminal Decline vs Audience Exhausted vs Replace)

CreativeDynamics produces a human-readable Fatigue Status label per item in the analysis report. These labels are derived from the fatigue detector (creativedynamics/core/fatigue_detection.py) and then mapped to the report labels in the report generator (creativedynamics/reporting/generator.py).

At a high level:

1. The detector runs separately for CTR and CPC.
2. Each metric produces boolean flags:
   • is_terminal_decline
   • audience_exhausted
   • replacement_recommended
   • plus a confidence and reason string.
3. The report generator combines CTR/CPC signals and assigns a single Fatigue Status.

What window is analysed?

The detector evaluates the most recent min_decline_days rows of the item time series (default: 14 days). If there are fewer than 14 valid (non-missing) values for the metric in the recent window, the detector returns “Insufficient data” and does not flag fatigue.

What is “below benchmark”?

Each metric is evaluated relative to a benchmark value for that metric (e.g. ctr_benchmark for CTR, benchmark_cpc for CPC). The detector uses a performance threshold of 0.9 (90%) by default.

• CTR (higher is better)

  • Compute performance_ratio = mean(CTR_recent) / CTR_benchmark.
  • Define “below benchmark” as performance_ratio < 0.9.

• CPC (lower is better)

  • Compute performance_ratio = mean(CPC_recent) / CPC_benchmark.
  • Define “below benchmark” as performance_ratio > 1 / 0.9 (i.e. > ~1.11× benchmark).

What is “sustained poor performance”?

Within the recent 14-day window, the detector requires the majority of days to be worse than benchmark:

• CTR: at least 70% of recent days have CTR < CTR_benchmark.
• CPC: at least 70% of recent days have CPC > CPC_benchmark.

This is deliberately stricter than a single-day dip: it is intended to represent an item that is consistently under-performing, not temporarily noisy.

What is “declining trend”?

The detector fits a simple linear slope over the recent window:

• CTR: a negative slope indicates a decline (slope < 0).
• CPC: a positive slope indicates a decline in efficiency (slope > 0, because rising CPC is worse).

What is “recent volatility”?

The detector computes volatility as a coefficient of variation (CV) in the recent window:

recent_volatility = std(metric_recent) / mean(metric_recent)

It uses a threshold of 0.15 by default.

Status/flag definitions

The detector emits two core signals and one action signal:

1) Terminal Decline Detected (is_terminal_decline = True)

This flag means the recent window is structurally bad and still getting worse.

In code terms (per metric):

• is_terminal_decline is true when all three are true:
  • “below benchmark” (as defined above)
  • “sustained poor performance” (≥70% of recent days worse than benchmark)
  • “declining trend” (CTR slope < 0, CPC slope > 0)

Interpretation: this is a strong pattern-based warning. It indicates the item is not just below benchmark, but is also trending in the wrong direction across most of the recent window.

Why it is not automatically a replacement recommendation: terminal decline alone does not say whether the system is still actively exploring audiences (high volatility) or has settled into a stable under-performing regime (low volatility). That distinction is captured by the audience exhaustion flag.

2) Audience Exhausted (audience_exhausted = True)

This flag means performance is below benchmark and volatility has collapsed.

In code terms (per metric):

• audience_exhausted is true when both are true:
  • “below benchmark” (as defined above)
  • recent_volatility < 0.15

Interpretation: low volatility is treated as a proxy for an exhausted search/exploration process. In practice, it often corresponds to the platform repeatedly delivering into similar audiences where incremental performance no longer varies much day-to-day.

Important nuance: audience exhausted can be true even without a clear terminal trend slope. For example, an item can stabilise at a poor level (low volatility + below benchmark) without still declining day-by-day.

3) Creative Fatigued - Replace (replacement_recommended = True)

This is the strongest signal and is the only one treated as a direct replacement recommendation in the report.

In code terms (per metric):

• replacement_recommended is true when both are true:
  • is_terminal_decline is true
  • audience_exhausted is true

Interpretation: the item is below benchmark, is in sustained decline, and volatility is low. This combination is used to represent a “spent” creative: the system is not only performing poorly, but is also no longer showing signs of meaningful exploration or recovery.

How CTR and CPC are combined into a single Fatigue Status

Fatigue flags are computed per metric (CTR and CPC). The report then maps to a single human-readable label using a priority order:

1. If replacement is recommended in either metric → Creative Fatigued - Replace
2. Else if terminal decline is detected in either metric → Terminal Decline Detected
3. Else if audience exhaustion is detected in either metric → Audience Exhausted
4. Else if volatility is high in either metric (CV > 0.15) → High Volatility (Healthy)
5. Else → Healthy Performance

Solution: If you want replacement recommendations to be triggered only in very specific cases, use the Creative Fatigued - Replace label (i.e. replacement_recommended = True) rather than the broader early-warning labels.

See also

• Installation
• Usage
• Concepts
• Architecture
• Visualisation Guide