Skip to content

Conversation

@amikofalvy
Copy link
Collaborator

Summary

  • Adds per-activity context breakdown data to AI model streamed text activities in the trace timeline
  • Builds a spanIdcontextBreakdown map from agent.generate spans
  • Looks up context breakdown from parent span for each AI generation activity
  • Adds contextBreakdown field to ActivityItem type
  • Removes debug console.log from copilot-token action

Context

Previously, context breakdown was only available at the conversation level. This change associates context breakdown data with individual AI generation activities, enabling more granular token usage analysis in the trace viewer timeline.

Test plan

  • Verify context breakdown appears on AI generation activities in trace timeline
  • Confirm context breakdown data matches expected values from telemetry

…trace viewer

- Add per-span context breakdown data to AI model streamed text activities
- Build spanId-to-contextBreakdown map from agent.generate spans
- Look up context breakdown from parent span for each AI generation activity
- Add contextBreakdown field to ActivityItem type
- Remove debug console.log from copilot-token action
@changeset-bot
Copy link

changeset-bot bot commented Dec 2, 2025

⚠️ No Changeset found

Latest commit: b2d30b4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@vercel
Copy link

vercel bot commented Dec 2, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
agents-manage-api Ready Ready Preview Comment Dec 2, 2025 5:15pm
agents-manage-ui Error Error Dec 2, 2025 5:15pm
agents-run-api Ready Ready Preview Comment Dec 2, 2025 5:15pm
1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
agents-docs Skipped Skipped Dec 2, 2025 5:15pm

@claude
Copy link

claude bot commented Dec 2, 2025

Claude finished @amikofalvy's task —— View job


Code Review: feat(manage-ui): add context breakdown per AI generation activity in trace viewer

Review Findings

Key Changes Analysis

1. Route.ts Context Breakdown Implementation (lines 1110-1160)

  • ✅ Well-structured: Creates a proper ContextBreakdownData type with comprehensive token breakdown fields
  • ❌ Critical Type Issue: ContextBreakdownData is defined locally but ActivityItem references undefined ContextBreakdown type
  • ✅ Good separation: Builds spanId→contextBreakdown map from agent.generate spans with proper parent-child relationships
  • ✅ Proper lookup logic: getContextBreakdownForSpan() checks both self and parent spans
  • ✅ Smart assignment: Only assigns context breakdown to ai_model_streamed_text activities via parent span lookup

2. Types.ts Extension (line 114)

  • ❌ Missing type definition: References undefined ContextBreakdown type - this will cause TypeScript compilation errors

3. Console.log Removal (copilot-token.ts)

  • ✅ Clean cleanup: Removes debug logging appropriately

Critical Issues

🚨 Type Definition Missing (High Priority)

The ContextBreakdown type is referenced in types.ts:114 but not defined anywhere. This will cause TypeScript compilation failures.

Required Fix:

// Add to types.ts
export interface ContextBreakdown {
  systemPromptTemplate: number;
  coreInstructions: number;
  agentPrompt: number;
  toolsSection: number;
  artifactsSection: number;
  dataComponents: number;
  artifactComponents: number;
  transferInstructions: number;
  delegationInstructions: number;
  thinkingPreparation: number;
  conversationHistory: number;
  total: number;
}

Architectural Assessment

Strengths

  1. Logical data flow: Context breakdown flows from agent.generate spans → ai_model_streamed_text activities via parent relationship
  2. Performance conscious: Uses Map for O(1) lookup of context breakdown data
  3. Defensive programming: Proper null checks and fallback handling
  4. Granular tracking: Provides detailed token breakdown by component type

⚠️ Areas for Consideration

Memory Efficiency: The spanIdToContextBreakdown Map could grow large with many activities. Consider:

  • Is cleanup needed after response construction?
  • Could we stream/process this data instead of holding it all in memory?

Error Boundary: Context breakdown parsing doesn't have explicit error handling:

// Current (line 1128)
if (data['context.breakdown.total_tokens'] !== undefined) {
  // Could fail silently if data structure changes
}

// Consider adding validation

Data Consistency: No validation that breakdown components sum to total - could lead to confusing UI states.

System Design Considerations

Extensibility

The implementation is well-positioned for future enhancements:

  • New breakdown categories can be added easily
  • The lookup pattern can extend to other activity types

Performance ⚠️

  • Single-pass creation of breakdown map is efficient
  • Parent span traversal is O(depth) per activity - acceptable for typical trace depths
  • Consider caching getContextBreakdownForSpan results if called frequently

Maintainability ⚠️

The hardcoded field mapping (lines 1130-1142) creates maintenance overhead:

  • Field names are string literals susceptible to typos
  • Changes to telemetry field names require updates in multiple places
  • Consider extracting to constants or configuration

Recommendations

Must Fix 🚨

  1. Add ContextBreakdown type definition to prevent compilation errors
  2. Export the type from types.ts for reusability

Should Consider ⚠️

  1. Extract field mapping constants for better maintainability
  2. Add validation for breakdown data consistency
  3. Consider error handling for malformed breakdown data

Nice to Have

  1. Add JSDoc comments explaining the context breakdown feature
  2. Consider unit tests for the breakdown lookup logic

Overall Assessment

This is a solid implementation that follows good architectural patterns. The main blocker is the missing type definition which will prevent compilation. Once fixed, this feature provides valuable granular insights into AI token usage patterns.

The code demonstrates thoughtful design with proper separation of concerns and efficient data structures. The parent-span lookup approach is elegant and maintains the existing activity model while extending it with rich context data.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants