docs(experiment): add select_num A/B/C comparison report (005)
- Experiment: select_num = 1, 2, 3 comparison - Period: 2020-01-10 ~ 2026-06-02 (1546 trading days) - Key findings: - Top-1: highest return (600%), highest drawdown (-25.5%) - Top-3: best risk-adjusted return (Calmar 1.73, Sharpe 1.35) - Top-2: balanced middle ground (Calmar 1.69) - Add rotation/experiment_select_num.py experiment script - Save report to docs/experiments/005_select_num_comparison.md
This commit is contained in:
145
docs/experiments/005_select_num_comparison.md
Normal file
145
docs/experiments/005_select_num_comparison.md
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
# 实验记录 005: select_num 参数对策略表现的影响
|
||||||
|
|
||||||
|
## 实验信息
|
||||||
|
|
||||||
|
| 项目 | 内容 |
|
||||||
|
|------|------|
|
||||||
|
| 实验编号 | 005 |
|
||||||
|
| 实验日期 | 2026-06-02 |
|
||||||
|
| 实验类型 | A/B/C 对比测试 |
|
||||||
|
| 研究问题 | `diversified=true` 模式下,`select_num` 取 1/2/3 时对策略收益与风险的影响 |
|
||||||
|
| 配置文件 | `rotation/config_simple.yaml` (L133 `select_num`) |
|
||||||
|
| 实验脚本 | `rotation/experiment_select_num.py` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 实验背景
|
||||||
|
|
||||||
|
### 策略选股流程
|
||||||
|
|
||||||
|
```
|
||||||
|
Step 1: 类内竞争 → 每个 market 大类只保留得分最高的1只标的(大类冠军)
|
||||||
|
Step 2: 跨类排序 → 从大类冠军中按得分从高到低选 Top select_num
|
||||||
|
```
|
||||||
|
|
||||||
|
### 核心问题
|
||||||
|
|
||||||
|
`select_num` 控制最终持仓标的数量,直接影响集中度和分散度:
|
||||||
|
- `select_num=1`:单标的集中持仓,无分散化效果
|
||||||
|
- `select_num=2`:持有 2 个大类的冠军标的
|
||||||
|
- `select_num=3`:持有 3 个大类的冠军标的(当前默认配置)
|
||||||
|
|
||||||
|
**理论预期**:
|
||||||
|
- 持仓数量越少,集中度越高,潜在收益和波动均放大
|
||||||
|
- 持仓数量越多,分散化效果越好,回撤更小,但可能引入边际收益较低的标的
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 实验设计
|
||||||
|
|
||||||
|
### A/B/C 组配置
|
||||||
|
|
||||||
|
| 组别 | select_num | 持仓数量 | 其他配置 |
|
||||||
|
|------|-----------|---------|---------|
|
||||||
|
| **A组** | 1 | 单标的 | 同对照组 |
|
||||||
|
| **B组** | 2 | 双标的 | 同对照组 |
|
||||||
|
| **C组** | 3 | 三标的 | 同对照组(当前默认) |
|
||||||
|
|
||||||
|
### 固定配置(三组相同)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
factor:
|
||||||
|
type: "weighted_momentum"
|
||||||
|
n_days: 25
|
||||||
|
|
||||||
|
rotation:
|
||||||
|
diversified: true
|
||||||
|
threshold:
|
||||||
|
mode: "dynamic"
|
||||||
|
reference: "931862.CSI" # 短债动量基准
|
||||||
|
```
|
||||||
|
|
||||||
|
### 回测区间
|
||||||
|
|
||||||
|
2020-01-10 ~ 2026-06-02,共 **1546 个交易日**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 回测结果
|
||||||
|
|
||||||
|
### 核心指标对比
|
||||||
|
|
||||||
|
| 指标 | Top-1(A组) | Top-2(B组) | Top-3(C组) |
|
||||||
|
|------|------------|------------|------------|
|
||||||
|
| 累计收益 | **600.31%** | 369.88% | 302.14% |
|
||||||
|
| 年化收益 | **37.34%** | 28.69% | 25.46% |
|
||||||
|
| 最大回撤 | -25.53% | -16.93% | **-14.74%** |
|
||||||
|
| 夏普比率 | 1.11 | 1.27 | **1.35** |
|
||||||
|
| Calmar比率 | 1.46 | 1.69 | **1.73** |
|
||||||
|
| 日胜率 | 54.49% | **55.35%** | 55.18% |
|
||||||
|
| 调仓次数 | 197 | 319 | 405 |
|
||||||
|
|
||||||
|
### 关键观察
|
||||||
|
|
||||||
|
**收益维度:**
|
||||||
|
- Top-1 累计收益(600%)几乎是 Top-3(302%)的 2 倍
|
||||||
|
- 集中持仓显著放大了收益,但也意味着更高的单标的依赖风险
|
||||||
|
|
||||||
|
**风险维度:**
|
||||||
|
- Top-3 最大回撤(-14.74%)比 Top-1(-25.53%)降低约 42%
|
||||||
|
- Top-2 居中(-16.93%),回撤控制效果明显
|
||||||
|
|
||||||
|
**风险调整收益(核心指标):**
|
||||||
|
- Calmar 比率:Top-3(1.73)> Top-2(1.69)> Top-1(1.46)
|
||||||
|
- 夏普比率:Top-3(1.35)> Top-2(1.27)> Top-1(1.11)
|
||||||
|
- **分散化带来更优的风险收益比**
|
||||||
|
|
||||||
|
**调仓频率:**
|
||||||
|
- Top-1 调仓次数最少(197 次),因为持仓切换需要单标的排名大幅变动
|
||||||
|
- Top-3 调仓次数最多(405 次),持仓组合中任一标的变化都会触发调仓
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. NAV 曲线对比
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 结论与建议
|
||||||
|
|
||||||
|
### 核心结论
|
||||||
|
|
||||||
|
| 目标 | 推荐配置 | 原因 |
|
||||||
|
|------|---------|------|
|
||||||
|
| 追求绝对收益 | `select_num=1` | 累计收益最高,但需承受更大回撤 |
|
||||||
|
| 追求风险调整收益 | `select_num=3` | Calmar/夏普最优,回撤可控 |
|
||||||
|
| 平衡两者 | `select_num=2` | 收益与回撤的折中方案 |
|
||||||
|
|
||||||
|
### 实践建议
|
||||||
|
|
||||||
|
- **当前默认配置 `select_num=3` 是合理的选择**,Calmar 比率最优,适合长期持有
|
||||||
|
- 若资金规模较小、风险承受能力强,可考虑 `select_num=1` 追求高弹性
|
||||||
|
- `select_num=2` 的 Calmar(1.69)与 Top-3(1.73)非常接近,但收益更高(369% vs 302%),值得进一步观察
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 实验数据位置
|
||||||
|
|
||||||
|
```
|
||||||
|
results/experiment_select_num/
|
||||||
|
├── select_1/
|
||||||
|
│ ├── simple_rotation_nav.csv
|
||||||
|
│ ├── simple_rotation_signals.csv
|
||||||
|
│ ├── simple_rotation_detail.json
|
||||||
|
│ └── simple_rotation_metrics.json
|
||||||
|
├── select_2/
|
||||||
|
│ └── ... (同上)
|
||||||
|
├── select_3/
|
||||||
|
│ └── ... (同上)
|
||||||
|
├── select_num_comparison.png # 指标对比柱状图
|
||||||
|
├── select_num_nav_comparison.png # NAV 叠加曲线图
|
||||||
|
└── experiment_metrics.json # 三组指标汇总
|
||||||
|
```
|
||||||
193
rotation/experiment_select_num.py
Normal file
193
rotation/experiment_select_num.py
Normal file
@@ -0,0 +1,193 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
select_num A/B 实验:对比 Top-1 / Top-2 / Top-3 的表现
|
||||||
|
|
||||||
|
用法:
|
||||||
|
python rotation/experiment_select_num.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import yaml
|
||||||
|
import json
|
||||||
|
import tempfile
|
||||||
|
import numpy as np
|
||||||
|
import pandas as pd
|
||||||
|
from pathlib import Path
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
PROJECT_ROOT = Path(__file__).parent.parent
|
||||||
|
sys.path.insert(0, str(PROJECT_ROOT))
|
||||||
|
|
||||||
|
from rotation.simple_rotation import SimpleRotationStrategy
|
||||||
|
|
||||||
|
|
||||||
|
def run_with_select_num(config_path: str, select_num: int, output_dir: Path) -> dict:
|
||||||
|
"""运行一次策略,覆盖 select_num"""
|
||||||
|
print(f"\n{'='*60}")
|
||||||
|
print(f" 实验: select_num = {select_num}")
|
||||||
|
print(f"{'='*60}\n")
|
||||||
|
|
||||||
|
# 读取原始配置,修改 select_num,写入临时文件
|
||||||
|
with open(config_path, 'r', encoding='utf-8') as f:
|
||||||
|
cfg = yaml.safe_load(f)
|
||||||
|
cfg['rotation']['select_num'] = select_num
|
||||||
|
|
||||||
|
tmp_path = output_dir / f'config_select_{select_num}.yaml'
|
||||||
|
with open(tmp_path, 'w', encoding='utf-8') as f:
|
||||||
|
yaml.dump(cfg, f, default_flow_style=False, allow_unicode=True)
|
||||||
|
|
||||||
|
strategy = SimpleRotationStrategy(config_path=str(tmp_path))
|
||||||
|
result = strategy.run()
|
||||||
|
|
||||||
|
if result:
|
||||||
|
# 导出到子目录
|
||||||
|
sub_dir = output_dir / f'select_{select_num}'
|
||||||
|
sub_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
strategy.export_results(output_dir=str(sub_dir))
|
||||||
|
return result.get('metrics', {})
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def print_comparison(all_metrics: dict):
|
||||||
|
"""打印对比表格"""
|
||||||
|
print(f"\n\n{'='*80}")
|
||||||
|
print(f" select_num 实验对比结果")
|
||||||
|
print(f"{'='*80}\n")
|
||||||
|
|
||||||
|
header = f"{'指标':<16}"
|
||||||
|
for n in sorted(all_metrics.keys()):
|
||||||
|
header += f"{'Top-'+str(n):>12}"
|
||||||
|
print(header)
|
||||||
|
print("-" * (16 + 12 * len(all_metrics)))
|
||||||
|
|
||||||
|
rows = [
|
||||||
|
('累计收益', 'total_return', '{:.2%}'),
|
||||||
|
('年化收益', 'annual_return', '{:.2%}'),
|
||||||
|
('最大回撤', 'max_drawdown', '{:.2%}'),
|
||||||
|
('夏普比率', 'sharpe_ratio', '{:.2f}'),
|
||||||
|
('Calmar比率', 'calmar_ratio', '{:.2f}'),
|
||||||
|
('日胜率', 'win_rate', '{:.2%}'),
|
||||||
|
('交易日数', 'n_days', '{}'),
|
||||||
|
('调仓次数', 'rebalance_count', '{}'),
|
||||||
|
]
|
||||||
|
|
||||||
|
for label, key, fmt in rows:
|
||||||
|
row = f"{label:<16}"
|
||||||
|
for n in sorted(all_metrics.keys()):
|
||||||
|
val = all_metrics[n].get(key, 0)
|
||||||
|
row += f"{fmt.format(val):>12}"
|
||||||
|
print(row)
|
||||||
|
|
||||||
|
print(f"\n{'='*80}")
|
||||||
|
|
||||||
|
|
||||||
|
def plot_comparison(all_metrics: dict, output_dir: Path):
|
||||||
|
"""生成对比图表"""
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.use("Agg")
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
|
||||||
|
fig, axes = plt.subplots(1, 3, figsize=(16, 5))
|
||||||
|
fig.suptitle("select_num A/B Experiment", fontsize=14, fontweight="bold")
|
||||||
|
|
||||||
|
nums = sorted(all_metrics.keys())
|
||||||
|
colors = ['#E74C3C', '#3498DB', '#2ECC71']
|
||||||
|
|
||||||
|
# 1. 收益对比
|
||||||
|
ax = axes[0]
|
||||||
|
annuals = [all_metrics[n].get('annual_return', 0) for n in nums]
|
||||||
|
totals = [all_metrics[n].get('total_return', 0) for n in nums]
|
||||||
|
x = np.arange(len(nums))
|
||||||
|
w = 0.35
|
||||||
|
ax.bar(x - w/2, [a*100 for a in annuals], w, label='Annual %', color='#E74C3C', alpha=0.8)
|
||||||
|
ax.bar(x + w/2, [t*100 for t in totals], w, label='Total %', color='#3498DB', alpha=0.8)
|
||||||
|
ax.set_xticks(x)
|
||||||
|
ax.set_xticklabels([f'Top-{n}' for n in nums])
|
||||||
|
ax.set_ylabel('Return (%)')
|
||||||
|
ax.set_title('Returns')
|
||||||
|
ax.legend()
|
||||||
|
ax.grid(True, alpha=0.3)
|
||||||
|
|
||||||
|
# 2. 风险对比
|
||||||
|
ax = axes[1]
|
||||||
|
dds = [abs(all_metrics[n].get('max_drawdown', 0)) * 100 for n in nums]
|
||||||
|
ax.bar(x, dds, color='#E74C3C', alpha=0.7)
|
||||||
|
ax.set_xticks(x)
|
||||||
|
ax.set_xticklabels([f'Top-{n}' for n in nums])
|
||||||
|
ax.set_ylabel('Max Drawdown (%)')
|
||||||
|
ax.set_title('Risk')
|
||||||
|
ax.grid(True, alpha=0.3)
|
||||||
|
|
||||||
|
# 3. 夏普 & Calmar
|
||||||
|
ax = axes[2]
|
||||||
|
sharpes = [all_metrics[n].get('sharpe_ratio', 0) for n in nums]
|
||||||
|
calmars = [all_metrics[n].get('calmar_ratio', 0) for n in nums]
|
||||||
|
ax.bar(x - w/2, sharpes, w, label='Sharpe', color='#2ECC71', alpha=0.8)
|
||||||
|
ax.bar(x + w/2, calmars, w, label='Calmar', color='#F39C12', alpha=0.8)
|
||||||
|
ax.set_xticks(x)
|
||||||
|
ax.set_xticklabels([f'Top-{n}' for n in nums])
|
||||||
|
ax.set_ylabel('Ratio')
|
||||||
|
ax.set_title('Risk-Adjusted')
|
||||||
|
ax.legend()
|
||||||
|
ax.grid(True, alpha=0.3)
|
||||||
|
|
||||||
|
plt.tight_layout()
|
||||||
|
chart_path = output_dir / 'select_num_comparison.png'
|
||||||
|
plt.savefig(str(chart_path), dpi=150, bbox_inches="tight")
|
||||||
|
plt.close()
|
||||||
|
print(f"\n + Chart: {chart_path}")
|
||||||
|
|
||||||
|
|
||||||
|
def plot_nav_comparison(output_dir: Path):
|
||||||
|
"""加载三组 NAV 画在同一张图上"""
|
||||||
|
import matplotlib
|
||||||
|
matplotlib.use("Agg")
|
||||||
|
import matplotlib.pyplot as plt
|
||||||
|
|
||||||
|
fig, ax = plt.subplots(figsize=(14, 6))
|
||||||
|
colors = {'1': '#E74C3C', '2': '#3498DB', '3': '#2ECC71'}
|
||||||
|
|
||||||
|
for n in [1, 2, 3]:
|
||||||
|
nav_path = output_dir / f'select_{n}' / 'simple_rotation_nav.csv'
|
||||||
|
if nav_path.exists():
|
||||||
|
df = pd.read_csv(nav_path, parse_dates=['date'])
|
||||||
|
ax.plot(df['date'], df['nav'], label=f'Top-{n}', linewidth=1.5, color=colors[str(n)])
|
||||||
|
|
||||||
|
ax.set_title("NAV Curve Comparison (select_num)", fontsize=14, fontweight="bold")
|
||||||
|
ax.set_ylabel("NAV")
|
||||||
|
ax.set_yscale("log")
|
||||||
|
ax.legend(fontsize=11)
|
||||||
|
ax.grid(True, alpha=0.3)
|
||||||
|
plt.tight_layout()
|
||||||
|
|
||||||
|
nav_chart = output_dir / 'select_num_nav_comparison.png'
|
||||||
|
plt.savefig(str(nav_chart), dpi=150, bbox_inches="tight")
|
||||||
|
plt.close()
|
||||||
|
print(f" + NAV Chart: {nav_chart}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
if 'FLASK_API_URL' not in os.environ:
|
||||||
|
os.environ['FLASK_API_URL'] = 'https://k3s.tokenpluse.xyz'
|
||||||
|
|
||||||
|
config_path = str(Path(__file__).parent / 'config_simple.yaml')
|
||||||
|
output_dir = PROJECT_ROOT / 'results' / 'experiment_select_num'
|
||||||
|
output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
all_metrics = {}
|
||||||
|
for n in [1, 2, 3]:
|
||||||
|
metrics = run_with_select_num(config_path, n, output_dir)
|
||||||
|
if metrics:
|
||||||
|
all_metrics[n] = metrics
|
||||||
|
|
||||||
|
if all_metrics:
|
||||||
|
print_comparison(all_metrics)
|
||||||
|
plot_comparison(all_metrics, output_dir)
|
||||||
|
plot_nav_comparison(output_dir)
|
||||||
|
|
||||||
|
# 保存原始指标
|
||||||
|
metrics_path = output_dir / 'experiment_metrics.json'
|
||||||
|
with open(metrics_path, 'w', encoding='utf-8') as f:
|
||||||
|
json.dump({str(k): v for k, v in all_metrics.items()}, f, ensure_ascii=False, indent=2)
|
||||||
|
print(f" + Metrics: {metrics_path}")
|
||||||
Reference in New Issue
Block a user