-
Notifications
You must be signed in to change notification settings - Fork 907
feat: add terminal session monitoring and AI audit #4904
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
huanghongbo-hhb
wants to merge
36
commits into
koderover:main
Choose a base branch
from
huanghongbo-hhb:feat/terminal-session-audit-v2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
1b5eb5d
feat: add terminal session monitoring and AI audit
huanghongbo-hhb 15edf18
docs: remove terminal audit API documentation
huanghongbo-hhb 39a2235
Merge remote-tracking branch 'upstream/main' into feat/terminal-sessi…
huanghongbo-hhb 333ec44
refactor(llm): share completion retry handling
huanghongbo-hhb ae3eec5
fix(terminal-audit): retry transient model responses
huanghongbo-hhb 358ae07
fix(terminal-audit): keep audit records atomic
huanghongbo-hhb 8056321
chore(terminal-audit): add ai diagnostic logs
huanghongbo-hhb ef320e9
fix(terminal-audit): analyze chunks concurrently
huanghongbo-hhb 7a5139c
chore(terminal-audit): improve model diagnostics
huanghongbo-hhb 42d5d52
refactor(terminal-audit): clarify terminal sequences
huanghongbo-hhb 599a40a
refactor(terminal-audit): name terminal protocol bytes
huanghongbo-hhb 1dc57db
refactor(terminal-audit): parse wrapped commands
huanghongbo-hhb c3469a8
fix(terminal-audit): expire stale ai analyses
huanghongbo-hhb 148118c
fix(terminal-audit): scope ai timeouts per chunk
huanghongbo-hhb 3f4f953
fix(terminal-audit): avoid echoing commands in ai output
huanghongbo-hhb 6f7f911
fix(terminal-audit): scope findings to chunk commands
huanghongbo-hhb cbacdb7
fix(terminal-audit): harden analysis and recording
huanghongbo-hhb fbc0f01
refactor(terminal-audit): remove incidental behavior changes
huanghongbo-hhb cd88d06
refactor(podexec): preserve terminal error behavior
huanghongbo-hhb 5db8b63
chore(terminal-audit): remove temporary ai diagnostics
huanghongbo-hhb 321b915
fix(terminal-audit): align backend pagination limits
huanghongbo-hhb 1e546b4
fix(podexec): close terminal session on disconnect
huanghongbo-hhb 6f946a3
fix(terminal-audit): ignore shell startup as standalone risk
huanghongbo-hhb f70f606
fix(terminal-audit): fix prompt string syntax
huanghongbo-hhb e988ab1
fix(terminal-audit): discard tui input after exit
huanghongbo-hhb 1631d43
fix(terminal-audit): restore pod service metadata
huanghongbo-hhb 74d19b7
refactor: simplify terminal audit flow and preserve APIs
huanghongbo-hhb 1f1662b
fix: preserve terminal audit result defaults
huanghongbo-hhb 0d12350
fix: preserve terminal option masking behavior
huanghongbo-hhb 6c68a52
refactor: remove redundant terminal audit guards
huanghongbo-hhb b520831
refactor: remove unreachable terminal audit fallbacks
huanghongbo-hhb 5d1f224
refactor: simplify terminal audit data flow
huanghongbo-hhb 3bb2c36
fix: rename terminal audit permission label
huanghongbo-hhb 1eb1bf7
refactor: group terminal session fields
huanghongbo-hhb 4128922
refactor: clarify terminal session field groups
huanghongbo-hhb 90af0bd
fix: avoid blocking terminal session close
huanghongbo-hhb File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
44 changes: 44 additions & 0 deletions
44
pkg/microservice/aslan/core/common/repository/models/terminal_audit_ai_result.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| package models | ||
|
|
||
| import "go.mongodb.org/mongo-driver/bson/primitive" | ||
|
|
||
| type TerminalAuditAIStatus string | ||
|
|
||
| const ( | ||
| TerminalAuditAIStatusRunning TerminalAuditAIStatus = "running" | ||
| TerminalAuditAIStatusSucceeded TerminalAuditAIStatus = "succeeded" | ||
| TerminalAuditAIStatusFailed TerminalAuditAIStatus = "failed" | ||
| ) | ||
|
|
||
| type TerminalAuditAIFinding struct { | ||
| Seq int64 `bson:"seq" json:"seq"` | ||
| Command string `bson:"command" json:"command"` | ||
| Risk string `bson:"risk" json:"risk"` | ||
| Reason string `bson:"reason" json:"reason"` | ||
| Suggestion string `bson:"suggestion" json:"suggestion"` | ||
| } | ||
|
|
||
| type TerminalAuditAIResult struct { | ||
| ID primitive.ObjectID `bson:"_id,omitempty" json:"id,omitempty"` | ||
| SessionID string `bson:"session_id" json:"session_id"` | ||
| Status TerminalAuditAIStatus `bson:"status" json:"status"` | ||
| RiskLevel string `bson:"risk_level" json:"risk_level"` | ||
| Summary string `bson:"summary" json:"summary"` | ||
| Findings []TerminalAuditAIFinding `bson:"findings" json:"findings"` | ||
| Coverage string `bson:"coverage" json:"coverage"` | ||
| Model string `bson:"model" json:"model"` | ||
| TokenNum int `bson:"token_num" json:"token_num"` | ||
| AnalyzedCommandCount int64 `bson:"analyzed_command_count" json:"analyzed_command_count"` | ||
| TotalCommandCount int64 `bson:"total_command_count" json:"total_command_count"` | ||
| ErrorMessage string `bson:"error_message" json:"error_message,omitempty"` | ||
| RunID string `bson:"run_id" json:"-"` | ||
| LeaseExpiresAt int64 `bson:"lease_expires_at" json:"-"` | ||
| StartedAt int64 `bson:"started_at" json:"started_at"` | ||
| FinishedAt int64 `bson:"finished_at" json:"finished_at"` | ||
| CreatedAt int64 `bson:"created_at" json:"created_at"` | ||
| UpdatedAt int64 `bson:"updated_at" json:"updated_at"` | ||
| } | ||
|
|
||
| func (TerminalAuditAIResult) TableName() string { | ||
| return "terminal_audit_ai_result" | ||
| } |
38 changes: 38 additions & 0 deletions
38
pkg/microservice/aslan/core/common/repository/models/terminal_command.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| package models | ||
|
|
||
| import "go.mongodb.org/mongo-driver/bson/primitive" | ||
|
|
||
| type TerminalCommand struct { | ||
| ID primitive.ObjectID `bson:"_id,omitempty" json:"id,omitempty"` | ||
| SessionID string `bson:"session_id" json:"session_id"` | ||
| Seq int64 `bson:"seq" json:"seq"` | ||
| Command string `bson:"command" json:"command"` | ||
| UserID string `bson:"user_id" json:"user_id"` | ||
| Username string `bson:"username" json:"username"` | ||
| Account string `bson:"account" json:"account"` | ||
| ProjectName string `bson:"project_name" json:"project_name"` | ||
| EnvName string `bson:"env_name" json:"env_name"` | ||
| TargetName string `bson:"target_name" json:"target_name"` | ||
| Protocol string `bson:"protocol" json:"protocol"` | ||
| RemoteAddr string `bson:"remote_addr" json:"remote_addr"` | ||
| LoginAccount string `bson:"login_account" json:"login_account"` | ||
| TimeOffsetMS int64 `bson:"time_offset_ms" json:"time_offset_ms"` | ||
| CreatedAt int64 `bson:"created_at" json:"created_at"` | ||
| } | ||
|
|
||
| func (TerminalCommand) TableName() string { | ||
| return "terminal_command" | ||
| } | ||
|
|
||
| type TerminalCommandListArgs struct { | ||
| SessionID string `form:"sessionID" json:"sessionID"` | ||
| ProjectName string `form:"projectName" json:"projectName"` | ||
| Username string `form:"username" json:"username"` | ||
| TargetName string `form:"targetName" json:"targetName"` | ||
| RemoteAddr string `form:"remoteAddr" json:"remoteAddr"` | ||
| Command string `form:"command" json:"command"` | ||
| StartTime int64 `form:"startTime" json:"startTime"` | ||
| EndTime int64 `form:"endTime" json:"endTime"` | ||
| PageNum int64 `form:"pageNum" json:"pageNum"` | ||
| PageSize int64 `form:"pageSize" json:"pageSize"` | ||
| } |
107 changes: 107 additions & 0 deletions
107
pkg/microservice/aslan/core/common/repository/models/terminal_session.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| package models | ||
|
|
||
| import "go.mongodb.org/mongo-driver/bson/primitive" | ||
|
|
||
| type TerminalSessionType string | ||
|
|
||
| const ( | ||
| TerminalSessionTypeSSH TerminalSessionType = "ssh" | ||
| TerminalSessionTypePodExec TerminalSessionType = "podexec" | ||
| TerminalSessionTypeWorkflowDebug TerminalSessionType = "workflow_debug" | ||
| ) | ||
|
|
||
| type TerminalSessionStatus string | ||
|
|
||
| const ( | ||
| TerminalSessionStatusRunning TerminalSessionStatus = "running" | ||
| TerminalSessionStatusFinished TerminalSessionStatus = "finished" | ||
| TerminalSessionStatusAborted TerminalSessionStatus = "aborted" | ||
| TerminalSessionStatusFailed TerminalSessionStatus = "failed" | ||
| ) | ||
|
|
||
| type TerminalSessionContext struct { | ||
| ProjectName string `bson:"project_name" json:"project_name"` | ||
| EnvName string `bson:"env_name" json:"env_name"` | ||
| ServiceName string `bson:"service_name" json:"service_name"` | ||
| } | ||
|
|
||
| type TerminalSessionWorkflowContext struct { | ||
| WorkflowName string `bson:"workflow_name" json:"workflow_name"` | ||
| JobName string `bson:"job_name" json:"job_name"` | ||
| TaskID int64 `bson:"task_id" json:"task_id"` | ||
| } | ||
|
|
||
| type TerminalSessionTarget struct { | ||
| TargetName string `bson:"target_name" json:"target_name"` | ||
| Protocol string `bson:"protocol" json:"protocol"` | ||
| RemoteAddr string `bson:"remote_addr" json:"remote_addr"` | ||
| } | ||
|
|
||
| type TerminalSessionSSHTarget struct { | ||
| LoginAccount string `bson:"login_account" json:"login_account"` | ||
| HostID string `bson:"host_id" json:"host_id"` | ||
| HostName string `bson:"host_name" json:"host_name"` | ||
| HostIP string `bson:"host_ip" json:"host_ip"` | ||
| } | ||
|
|
||
| type TerminalSessionKubernetesTarget struct { | ||
| ClusterID string `bson:"cluster_id" json:"cluster_id"` | ||
| Namespace string `bson:"namespace" json:"namespace"` | ||
| PodName string `bson:"pod_name" json:"pod_name"` | ||
| ContainerName string `bson:"container_name" json:"container_name"` | ||
| } | ||
|
|
||
| type TerminalSessionRecording struct { | ||
| StartedAt int64 `bson:"started_at" json:"started_at"` | ||
| EndedAt int64 `bson:"ended_at" json:"ended_at"` | ||
| DurationSeconds int64 `bson:"duration_seconds" json:"duration_seconds"` | ||
| LastActivityAt int64 `bson:"last_activity_at" json:"last_activity_at"` | ||
| CommandCount int64 `bson:"command_count" json:"command_count"` | ||
| StorageID string `bson:"storage_id" json:"storage_id"` | ||
| Bucket string `bson:"bucket" json:"bucket"` | ||
| ObjectKey string `bson:"object_key" json:"object_key"` | ||
| FileSize int64 `bson:"file_size" json:"file_size"` | ||
| ErrorMessage string `bson:"error_message" json:"error_message"` | ||
| CreatedAt int64 `bson:"created_at" json:"created_at"` | ||
| UpdatedAt int64 `bson:"updated_at" json:"updated_at"` | ||
| } | ||
|
|
||
| type TerminalSession struct { | ||
| ID primitive.ObjectID `bson:"_id,omitempty" json:"id,omitempty"` | ||
| SessionID string `bson:"session_id" json:"session_id"` | ||
| SessionType TerminalSessionType `bson:"session_type" json:"session_type"` | ||
| Status TerminalSessionStatus `bson:"status" json:"status"` | ||
| UserID string `bson:"user_id" json:"user_id"` | ||
| Username string `bson:"username" json:"username"` | ||
| Account string `bson:"account" json:"account"` | ||
|
|
||
| TerminalSessionContext `bson:",inline" json:",inline"` | ||
| TerminalSessionWorkflowContext `bson:",inline" json:",inline"` | ||
| TerminalSessionTarget `bson:",inline" json:",inline"` | ||
| TerminalSessionSSHTarget `bson:",inline" json:",inline"` | ||
| TerminalSessionKubernetesTarget `bson:",inline" json:",inline"` | ||
|
|
||
| ClientIP string `bson:"client_ip" json:"client_ip"` | ||
| UserAgent string `bson:"user_agent" json:"user_agent"` | ||
|
|
||
| TerminalSessionRecording `bson:",inline" json:",inline"` | ||
| } | ||
|
|
||
| func (TerminalSession) TableName() string { | ||
| return "terminal_session" | ||
| } | ||
|
|
||
| type TerminalSessionListArgs struct { | ||
| Status string `form:"status" json:"status"` | ||
| SessionType string `form:"sessionType" json:"sessionType"` | ||
| ProjectName string `form:"projectName" json:"projectName"` | ||
| EnvName string `form:"envName" json:"envName"` | ||
| ServiceName string `form:"serviceName" json:"serviceName"` | ||
| Username string `form:"username" json:"username"` | ||
| TargetName string `form:"targetName" json:"targetName"` | ||
| RemoteAddr string `form:"remoteAddr" json:"remoteAddr"` | ||
| StartTime int64 `form:"startTime" json:"startTime"` | ||
| EndTime int64 `form:"endTime" json:"endTime"` | ||
| PageNum int64 `form:"pageNum" json:"pageNum"` | ||
| PageSize int64 `form:"pageSize" json:"pageSize"` | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
154 changes: 154 additions & 0 deletions
154
pkg/microservice/aslan/core/common/repository/mongodb/terminal_audit_ai_result.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| package mongodb | ||
|
|
||
| import ( | ||
| "context" | ||
| "errors" | ||
| "fmt" | ||
| "time" | ||
|
|
||
| "go.mongodb.org/mongo-driver/bson" | ||
| "go.mongodb.org/mongo-driver/mongo" | ||
| "go.mongodb.org/mongo-driver/mongo/options" | ||
|
|
||
| "github.com/koderover/zadig/v2/pkg/microservice/aslan/config" | ||
| "github.com/koderover/zadig/v2/pkg/microservice/aslan/core/common/repository/models" | ||
| mongotool "github.com/koderover/zadig/v2/pkg/tool/mongo" | ||
| ) | ||
|
|
||
| type TerminalAuditAIResultColl struct { | ||
| *mongo.Collection | ||
|
|
||
| coll string | ||
| } | ||
|
|
||
| var ErrTerminalAuditAIAlreadyRunning = errors.New("terminal audit ai analysis is already running") | ||
|
|
||
| func NewTerminalAuditAIResultColl() *TerminalAuditAIResultColl { | ||
| name := models.TerminalAuditAIResult{}.TableName() | ||
| return &TerminalAuditAIResultColl{ | ||
| Collection: mongotool.Database(config.MongoDatabase()).Collection(name), | ||
| coll: name, | ||
| } | ||
| } | ||
|
|
||
| func (c *TerminalAuditAIResultColl) GetCollectionName() string { return c.coll } | ||
|
|
||
| func (c *TerminalAuditAIResultColl) EnsureIndex(ctx context.Context) error { | ||
| index := mongo.IndexModel{ | ||
| Keys: bson.D{{Key: "session_id", Value: 1}}, | ||
| Options: options.Index().SetUnique(true), | ||
| } | ||
| _, err := c.Indexes().CreateOne(ctx, index, mongotool.CreateIndexOptions(ctx)) | ||
| return err | ||
| } | ||
|
|
||
| func (c *TerminalAuditAIResultColl) TryStart(sessionID, runID string, startedAt, leaseExpiresAt int64) (*models.TerminalAuditAIResult, error) { | ||
| filter := bson.M{ | ||
| "session_id": sessionID, | ||
| "$or": bson.A{ | ||
| bson.M{"status": bson.M{"$ne": models.TerminalAuditAIStatusRunning}}, | ||
| bson.M{"lease_expires_at": bson.M{"$lte": startedAt}}, | ||
| bson.M{"lease_expires_at": bson.M{"$exists": false}}, | ||
| }, | ||
| } | ||
| update := bson.M{ | ||
| "$set": bson.M{ | ||
| "status": models.TerminalAuditAIStatusRunning, | ||
| "risk_level": "", | ||
| "summary": "", | ||
| "findings": []models.TerminalAuditAIFinding{}, | ||
| "coverage": "", | ||
| "model": "", | ||
| "token_num": 0, | ||
| "analyzed_command_count": 0, | ||
| "total_command_count": 0, | ||
| "error_message": "", | ||
| "run_id": runID, | ||
| "lease_expires_at": leaseExpiresAt, | ||
| "started_at": startedAt, | ||
| "finished_at": 0, | ||
| "updated_at": startedAt, | ||
| }, | ||
| "$setOnInsert": bson.M{ | ||
| "session_id": sessionID, | ||
| "created_at": startedAt, | ||
| }, | ||
| } | ||
| opts := options.FindOneAndUpdate().SetUpsert(true).SetReturnDocument(options.After) | ||
| ctx, cancel := context.WithTimeout(context.Background(), terminalAuditMongoTimeout) | ||
| defer cancel() | ||
| result := new(models.TerminalAuditAIResult) | ||
| // A running session with a valid lease does not match the filter, so the upsert | ||
| // attempts an insert and hits the unique session_id index established above. | ||
| err := c.FindOneAndUpdate(ctx, filter, update, opts).Decode(result) | ||
| if mongo.IsDuplicateKeyError(err) { | ||
| return nil, ErrTerminalAuditAIAlreadyRunning | ||
| } | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return result, nil | ||
| } | ||
|
|
||
| func (c *TerminalAuditAIResultColl) UpdateLease(sessionID, runID string, leaseExpiresAt int64) error { | ||
| now := time.Now().Unix() | ||
| ctx, cancel := context.WithTimeout(context.Background(), terminalAuditMongoTimeout) | ||
| defer cancel() | ||
| result, err := c.UpdateOne(ctx, bson.M{ | ||
| "session_id": sessionID, | ||
| "run_id": runID, | ||
| "status": models.TerminalAuditAIStatusRunning, | ||
| }, bson.M{ | ||
| "$max": bson.M{"lease_expires_at": leaseExpiresAt}, | ||
| "$set": bson.M{"updated_at": now}, | ||
| }) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| if result.MatchedCount == 0 { | ||
| return fmt.Errorf("terminal audit ai run %s no longer owns session %s", runID, sessionID) | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| func (c *TerminalAuditAIResultColl) Finish(result *models.TerminalAuditAIResult) error { | ||
| now := time.Now().Unix() | ||
| result.UpdatedAt = now | ||
| result.FinishedAt = now | ||
| update := bson.M{"$set": bson.M{ | ||
| "status": result.Status, | ||
| "risk_level": result.RiskLevel, | ||
| "summary": result.Summary, | ||
| "findings": result.Findings, | ||
| "coverage": result.Coverage, | ||
| "model": result.Model, | ||
| "token_num": result.TokenNum, | ||
| "analyzed_command_count": result.AnalyzedCommandCount, | ||
| "total_command_count": result.TotalCommandCount, | ||
| "error_message": result.ErrorMessage, | ||
| "lease_expires_at": 0, | ||
| "finished_at": result.FinishedAt, | ||
| "updated_at": result.UpdatedAt, | ||
| }} | ||
| ctx, cancel := context.WithTimeout(context.Background(), terminalAuditMongoTimeout) | ||
| defer cancel() | ||
| writeResult, err := c.UpdateOne(ctx, bson.M{"session_id": result.SessionID, "run_id": result.RunID}, update) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| if writeResult.MatchedCount == 0 { | ||
| return fmt.Errorf("terminal audit ai run %s no longer owns session %s", result.RunID, result.SessionID) | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| func (c *TerminalAuditAIResultColl) FindBySessionID(sessionID string) (*models.TerminalAuditAIResult, error) { | ||
| resp := new(models.TerminalAuditAIResult) | ||
| ctx, cancel := context.WithTimeout(context.Background(), terminalAuditMongoTimeout) | ||
| defer cancel() | ||
| err := c.FindOne(ctx, bson.M{"session_id": sessionID}).Decode(resp) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return resp, nil | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.