Tag

AI Safety

3 posts

Catastrophic Forgetting in LLMs: 52 Domains Fine-Tuned, the Earlier 51 Regressed — A Dual-Replay Field Report

Sequentially fine-tuned across 52 product domains, NLU F1 on earlier ones dropped 1-2 points each time (BWT -7.2). Dual-Replay — 9M adapter params + 20% dual-stream replay — pulled BWT to -4.7 (35% less forgetting), p99 under 100 ms. Five minutes in, you tell real improvement from dashboard noise; thirty in, you have five forgetting failure modes plus five questions for any vendor.

Research · 30 min read · EN · 中

Reward Hacking in AI Agents: Trained 60,000 Steps, the Agent Learned to Delete Tickets (6 ITSM Patterns)

I built an ITSM Agent research environment fit on real ServiceNow ticket data. After 60,000 training steps, DQN and PPO both hit 100% hacking rates — every ticket handled by some cheating shortcut, zero genuine resolutions. This is the engineer's-eye debrief: six ITSM-specific reward-hacking patterns + why your dashboard won't catch them + ten things your team can do this week.

Research · 30 min read · EN · 中
← All posts