Thinking in Agent v2
ONEMooncake

Unified
Troubleshooting
Harness

Break down product silos.
Make everyone a cross-product expert.

Troubleshoot all issues in ONE harness
Know-it-all → Learn-it-all

Jerry He github.com/Jer-y
MicrosoftUnified Troubleshooting Harness

One critical incident, three products

One continuous failure chain

Zone · IoT team
Zone · App Service team
Zone · SQL team
IoT Hub
App Service
SQL DB
Direct Method failing
Upstream caller
Storage full · shrink blocked
COLLAB TASK
COLLAB TASK

One failure chain. Why cut it into collab tasks?

01 / 13
MicrosoftUnified Troubleshooting Harness

One method, many silos

The reasoning is the same everywhere. The org chart is not.

The essence of troubleshooting · expected vs actual
GAP
EXPECTED - - -　　ACTUAL ——
Close the gap with evidenceSame reasoning loop for every product
Org reality · split by product
Pod A?IoT
Pod B?App Service
Pod C?SQL
Step outside, open a ticketUnknown tables, tools and repos next door
02 / 13
MicrosoftUnified Troubleshooting Harness

The harness makes everyone a cross-product expert

To customers and call chains, the cloud is ONE

Know-it-all
Cross-product expert
Learn-it-all
Enablement · how learning happens

Agents deliver a verifiable evidence trail

Every hop: lead · table · code path · proof level
Boundary · what goes, what stays

Remove knowledge & tooling silos

Access & compliance boundaries stay
03 / 13
MicrosoftUnified Troubleshooting Harness

A Skill is a capability, not content

Why not write SOPs as Skills
01Context explodesMore SOPs → split finer → still noisy
02Descriptions overlapCross-product case → the wrong Skill fires
Content stays where it lives
ADO WikiEngHubSharePointOneNote→ fetched by a tool
Keeping knowledge fresh is a separate loop · cf. Karpathy's LLM Wiki
What a Skill is really for
01Progressive disclosureLoad only what's needed · trade time for space
name + descriptionalways
SKILL.mdon trigger
references/on demand
02 · scripts/DeterminismFixed steps run as code · same result every time
03 · references/ & assets/Known factsLook them up · never guess
04 / 13
MicrosoftUnified Troubleshooting Harness

High cohesion, low coupling, pluggable

AGENTS.mdRoutes by evidence gap · no hard-wired paths
missing telemetry → kustomissing code → bluebirdmissing TSG → enghub
Evidence & content × 8Lab repro · identity
01One Skill, one jobCohesive by tool or content source
02Pluggable & composablePull one out, another slots in
03Minimal change, no side effectsNew product or need touches one place
05 / 13
MicrosoftUnified Troubleshooting Harness

Trace across, dig down

ACROSS →↓ DOWN
Product A
Product B
Product C
…
Symptom
Change
Build
Source
Runtime
Still on Azure?
Keep going →
correlate
Root causeproven down to this layer
Evidence boundaryPROVENPARTIALCANDIDATE
06 / 13
MicrosoftUnified Troubleshooting Harness

Compliant, runs as you, never stuck

01Compliant by designEvery tool in the harness is approved
→
02Runs as youYour identity · access differs per person
→
03Never stuck403 → where to request · the rest keeps going
Engineer
Kusto
ADO
Source code
⚑ GAP PRESERVEDmay miss findings here
Everything else keeps going
EngHubRequest Access
where to request + target path → engineer
07 / 13
MicrosoftUnified Troubleshooting Harness

Real case: one call, three products

IoT Hub
App Service
SQL DB
SQL source
resource discovery
after rollback:
login fails on a DB error
why can't it shrink?
→ read the code
TLS

Direct Method failing
certificate_unknown

8u504

Java process changed
rolled back

Full

No auto-grow · storage maxed
pinned the table filling it

Shrink

Used ↓, allocated still over
mechanism verified in source

Fix all problems in one callAgent + engineer stitched every hop · no need to ask what the customer ran
08 / 13
MicrosoftUnified Troubleshooting Harness

E2E eval: root cause and cost

Any change that may affect the troubleshooting path gets a full run

ChangeSkill · prompt · MCP
kusto
enghub
dgrep
bluebird
Root cause found?
DURATIONend to end
TOOLSwhich tool, where
STEPSevery step counted
CREDITwhat it really cost
Every run, fully recorded→ compare models · tune cost · analyze paths
09 / 13
MicrosoftUnified Troubleshooting Harness

Eval discipline: no memorized answers

Exam mode · isolated run
MemoryPERSISTED CONTEXT
Write operationsWRITE
Questions & answersANSWER KEY
01
Real cases only≥2 services · engineer-verified · 15-day rolling
02
Pre-exam probesstale / blocked reported separately, never scored
03
Objective gradingjudges quote the answer · same question set
10 / 13
MicrosoftUnified Troubleshooting Harness

Roadmap: harness-level RSI

Recursive self-improvement · every real case makes the harness better

RSIskills · facts · eval cases
Real casestroubleshoot daily
Gaps surfacedunproven · unreachable
BackfillNEXTfacts · skills · eval cases
E2E evalre-verify
Why it's within reach
Gaps are already explicitEvery run flags what it couldn't prove or reach
Every run is already measuredSteps · tools · credit · time
Eval cases already come from real casesRolling · isolated · engineer-verified
NextClose the loopGaps → backfilled facts & Skills → re-verified by E2E eval
11 / 13
MicrosoftUnified Troubleshooting Harness

The harness sets the floor. People set the ceiling.

↑
CeilingPeople
TroubleshootLearnQuery
Same harness, any use · limited by imagination, not the model
FloorThe harnessCross-product capability for everyone
The role, redrawn
</>
Dev teamline of code · deploy · rollback
Customerreal business · real architecture
Engineercross-pod · cross-product
Deep tech × business senseEvery customer's architecture becomes your experience
12 / 13
MicrosoftUnified Troubleshooting Harness

Thank you

Questions & discussion

Jerry He github.com/Jer-y
13 / 13