Files
openobserve/tests
1d3618ad94 chore: system-managed SRE Agent service account provisioning (#11130)
## Summary

Implement automatic provisioning and lifecycle management of
per-organization system service accounts for the AI-powered SRE Agent.
These accounts serve as the identity for RCA (Root Cause Analysis) agent
API interactions and are created automatically for each organization.

## Key Features

### Service Account Provisioning
- Database migration creates SRE Agent accounts for all existing
organizations
- Deterministic email format:
`o2-sre-agent.org-<ORG_ID>@openobserve.internal`
- System accounts are automatically created on organization creation
- Idempotent operations ensure safe re-runs and self-healing

### System Account Protection
- API endpoints prevent modification/deletion of system service accounts
(403 Forbidden)
- User responses include `is_system` flag to identify system accounts
- Protection enforced at HTTP handler layer

### RCA Agent Client Refactoring
- Singleton pattern: global client initialized once at startup
- Per-request user credential forwarding for better audit trails and
scoping
- Lazy initialization: returns None if agent service not configured
- Updated all query methods to accept and forward auth credentials

### Incident Correlation Enhancement
- Query `ServiceIdentityConfig` to get configured semantic dimension
groups
- Extract only `distinguish_by` fields from matched identity sets
- Use parallel correlation execution (Service Discovery + semantic
extraction)
- Improved accuracy by using configured groups instead of fallback label
extraction

### Improved Token Redaction
- Consistent 12-character format: first 4 visible characters + 8
asterisks
- Better security for token display in UI
- Updated tests for new redaction format

## Technical Details

### Database Changes
- New migration:
`m20260331_000001_create_sys_rca_agent_service_accounts`
- Creates accounts with organization and RUM tokens
- Safe pagination (100 orgs at a time) with idempotency checks
- System accounts use role type 6 (SreAgent, distinct from
ServiceAccount)

### Authorization
- System accounts added to OpenFGA with standard org membership tuples
- SRE-specific read-only role for permission control
- Self-healing: runtime ensures OpenFGA state is consistent with
database state

### API Changes
- GET /users: Returns `is_system` and `description` fields
- PUT/DELETE /users: Reject modification/deletion of system accounts

## Test Coverage
- Token redaction tests updated for new format
- System service account protection verified
- User list pagination tested with system accounts

## Feature Flag
This is an enterprise feature gated behind `#[cfg(feature =
"enterprise")]`. System accounts are always created, but RCA integration
only works if the agent service is configured.

## Deployment
- Zero-downtime: migration is additive
- Opt-in for OSS deployments
- Automatic provisioning on upgrade and org creation

---------

Co-authored-by: Shrinath Rao <shnath@openobserve.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2026-04-06 14:30:20 +00:00
..
2024-09-22 17:57:47 +08:00