Development Challenges & Limitations

CreativeDynamics Library v0.9.8.1

This document outlines current challenges, limitations, and areas for consideration regarding the CreativeDynamics library and its application to time-series analysis.

Data Dependency and Granularity

1. Reliance on Aggregated Data: Core analysis operates on daily summaries, limiting intra-day dynamics modelling.
2. Platform Learning Phase: Cannot reliably identify platform-specific phases without event-level data.
3. Data Quality Filters: Items excluded based on thresholds; exclusion reasons are shown but not performance impact.
4. Input Schema Sensitivity: Expects specific lowercase column names; deviations require configuration through mapping files.

Methodology Assumptions and Limitations

1. Rough Path Signature Analysis: Requires parameter tuning; interpretation may be complex.
2. Change Point Definition: Based on signature thresholds, may not align with visual shifts in noisy data.
3. Segment Trend Classification: Linear regression slope simplifies non-linear segment trends.
4. Benchmark Definition: Uses longest stable/improving period; may not apply if no such segment exists or if it’s outdated.
5. Dual Metric Approach: Analyses both CTR and CPC; correlation risk is reported as context. Operational (engagement_gap_clicks) and financial (actual_overspend_gbp) metrics are reported separately and not combined.

Metric Interpretation and Reporting

1. Impact Metrics: Reports actual_overspend_gbp (financial inefficiency) and engagement_gap_clicks (operational impact) during ‘Declining’ periods.
2. Correlation Risk Context: Provides correlation-based risk context when CPC and CTR are negatively correlated; metrics are not added.
3. Undetermined Trends: Requires clear reporting for ‘undetermined’ or error segments.
4. Report Detail: Needs segment breakdowns and benchmark identification in tables.

Platform Integration

1. Analysis vs. Platform State: Analysis lacks platform context (e.g., dynamic algorithms, budget changes).

Technical Aspects

1. Performance: C++ backend via roughpy offers speed, but large datasets may challenge performance.
2. Dependency Management: Relies on standard packaging; consistent environments require diligence.
3. Dual Entry Points: The library provides two entry points (CLI with YAML configs and script with JSON mappings) which may cause confusion.

Summary of Key Challenges

1. Data granularity limitations.
2. Dual-metric interpretation and correlation risk context (metrics not combined).
3. Benchmark sensitivity and definition.
4. Trend classification simplifications.
5. Dual entry point confusion.
6. CTR normalisation requirements.

Addressing these challenges requires methodological refinements, clear documentation, and consistent implementation practices.