Skip to content

Bug: SSE connection does not automatically reconnect after disconnection #44

@ZeroClover

Description

@ZeroClover

Description

When using instantUpdates: true, the SSE connection for real-time secret updates does not automatically reconnect after being disconnected. Once the connection is lost, the operator continuously logs errors but never attempts to re-establish the connection.

Environment

  • Operator Version: v0.10.19

Steps to Reproduce

  1. Deploy the operator with an InfisicalSecret resource that has instantUpdates: true
  2. Wait for the SSE connection to be established (secrets sync successfully)
  3. Wait for the connection to be interrupted (e.g., by network issues, proxy timeout, or server restart)
  4. Observe that the operator logs errors continuously but never reconnects

Expected Behavior

When the SSE connection is lost, the operator should:

  1. Detect the disconnection
  2. Attempt to reconnect with exponential backoff
  3. Re-authenticate if the access token has expired
  4. Resume listening for real-time updates

Actual Behavior

The operator logs the error and exits the goroutine without any reconnection attempt:

ERROR Error occurred {"infisicalsecret": {"name":"my-secret","namespace":"my-namespace"}, "error": "stream error: stream ID 1; INTERNAL_ERROR; received from peer"}

This error repeats continuously for each InfisicalSecret resource with instantUpdates enabled.

After reviewing the source code, I identified the issue in internal/services/infisicalsecret/reconciler.go (lines 877-893):

go func() {
outer:
for {
select {
case ev := <-events:
logger.Info("Received SSE Event", "event", ev)
eventCh <- event.TypedGenericEvent[client.Object]{
Object: infisicalSecret,
}
case err := <-errors:
logger.Error(err, "Error occurred")
break outer
case <-ctx.Done():
break outer
}
}
}()

  1. No reconnection mechanism: When an error occurs, the goroutine exits without attempting to reconnect
  2. No error classification: All errors are treated the same - there's no distinction between recoverable (network issues) and non-recoverable errors (auth failures)
  3. No token refresh: The access token obtained during initial authentication is cached indefinitely without refresh logic, which could cause issues if the token expires while the SSE connection is active

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions