Distributed Caching: Distributed Session Management

Overview

A banking portal on OpenShift runs behind a load balancer that distributes requests across multiple pods. HTTP is stateless. Somewhere between the load balancer and the database, the platform must maintain the notion of a logged-in user — their identity, their active authorizations, and the state of any in-progress workflow (an IBAN lookup, an incomplete payment form, a pending PKCE authorization code exchange). This is the session problem.

In a single-pod deployment there is no session problem: session state lives in the pod’s memory, and every request goes to the same pod. In a multi-pod deployment the problem appears the moment a second pod starts. The original pod’s in-memory session store knows about the logged-in user; the second pod does not. Without a solution, a request routed to the second pod arrives as an anonymous user, and the customer is logged out.

Three solutions exist. The first is to do nothing and rely on the load balancer to route each customer’s requests to the same pod forever — sticky sessions. The second is to store session state in a shared store that every pod can read — distributed sessions. The third is to eliminate server-side session state entirely by encoding all state in the access token — stateless JWT sessions. In practice, a banking portal needs all three patterns applied to different parts of the system: JWT for API-to-API calls, distributed sessions for the web portal, and sticky sessions as a fallback for legacy integration points that cannot be refactored.

Sticky Sessions vs Distributed Session

Sticky sessions work by configuring the load balancer to route each client to the same backend pod on every request. OpenShift Routes support cookie-based affinity: the Route injects a cookie (typically named INGRESSCOOKIE) containing the pod name, and the OpenShift HAProxy router reads it on subsequent requests to route to the same pod. IP hash affinity also works but breaks when clients share an NAT IP.

The failure modes of sticky sessions are predictable and frequent in a containerised platform. Pod restart during a rolling deployment: the pod is replaced with a new pod at a new address; all sessions tied to the old pod are lost; customers are logged out mid-session. Pod eviction by the OpenShift scheduler during node pressure: same result. Scale-down from 5 pods to 3 during off-peak hours: two-fifths of active sessions are lost. Sticky sessions appear to work in development (single pod), appear to work in staging (low traffic, no failures), and fail in production at the worst possible time (pod restart under load at month-end).

Spring Session with Redis

Spring Session provides a transparent replacement for the servlet container’s HttpSession. Add the spring-session-data-redis dependency and annotate your configuration class with @EnableRedisHttpSession. Spring Session intercepts every getSession() call and reads/writes to Redis instead of the servlet container’s in-memory store. Application code does not change.

Session serialization format matters more than it appears. Java’s default serialization (the default in Spring Session) ties every serialised session to the exact class version that created it. A rolling deployment that changes a class in the session model will fail to deserialise sessions created by the old version — customers get an InvalidClassException and are logged out. Always configure JSON serialization for session data. JSON is forward-compatible: old sessions with fewer fields deserialise correctly in new code; new sessions with extra fields are ignored by old code.

spring:
  session:
    store-type: redis
    redis:
      # Namespace prefix for all session keys in Redis.
      # Allows multiple applications to share one Redis cluster safely.
      namespace: saib:portal:session
      # flush-mode: on-save (default) writes session to Redis only at response end;
      # immediate writes on every attribute change — slower but safer for payment flows.
      flush-mode: on-save
      # Configure-action: NONE means Spring Session will not attempt to configure
      # keyspace notifications on startup — required if Redis is a managed service
      # where CONFIG SET is not permitted.
      configure-action: notify-keyspace-events
    # Idle session timeout. SAMA: 900 seconds (15 minutes) for retail banking portal.
    timeout: 900s

  data:
    redis:
      host: redis-session.payments.svc
      port: 6379
      # Dedicated Redis instance for sessions — separate from cache Redis.
      # Session store failure must not be masked by cache-layer failures.
      client-name: portal-session-client
      connect-timeout: 2s
      timeout: 1s
      lettuce:
        pool:
          max-active: 32
          min-idle: 8

@Configuration
@EnableRedisHttpSession(maxInactiveIntervalInSeconds = 900)
public class SessionConfig {

    // Use Jackson2JsonRedisSerializer for forward-compatible session serialization.
    // Java default serialization will cause InvalidClassException during rolling deploys.
    @Bean
    public RedisSerializer<Object> springSessionDefaultRedisSerializer() {
        return new GenericJackson2JsonRedisSerializer();
    }

    // Session cookie: Secure + HttpOnly + SameSite=Strict + domain-scoped
    @Bean
    public CookieSerializer cookieSerializer() {
        DefaultCookieSerializer serializer = new DefaultCookieSerializer();
        serializer.setCookieName("SAIB_SESS");
        serializer.setSameSite("Strict");
        serializer.setUseSecureCookie(true);
        serializer.setUseHttpOnlyCookie(true);
        serializer.setDomainName("portal.saib.com.sa");
        return serializer;
    }
}

OAuth2 / OIDC Session Management

In a FAPI 2.0-compliant banking portal, the authorization flow introduces specific session requirements. PKCE (Proof Key for Code Exchange) generates a code_verifier on the client before redirecting to the authorization server. The code_challenge (a SHA-256 hash of the verifier) is sent to the authorization server. When the authorization code returns, the client sends the original code_verifier to prove it initiated the flow. Both values, along with the OAuth2 state parameter and nonce, must survive the redirect round trip — which means they must be stored somewhere between the initial redirect and the callback. That somewhere is the session.

What must NOT be in the session: the access token, the refresh token, or the ID token. Tokens belong in a secure HttpOnly cookie (for single-page applications) or in the application’s server-side token store (keyed by session ID, separate from the session data). Storing raw tokens in the Redis session keyspace means any engineer with Redis access can read them. Under SAMA’s Technology Risk Management framework, that is a privilege access control failure.

SAMA Session Timeout Requirements

SAMA’s retail banking and payment service guidelines specify session lifetime constraints that must be enforced at the platform level, not left to application discretion. These are examination items: an examiner will test idle timeout by opening a session, waiting, and attempting to transact.

The idle timeout maps directly to the Spring Session maxInactiveIntervalInSeconds and the Redis key TTL. The absolute timeout requires additional tracking: store the session creation time in the session itself, check it on every request, and invalidate if the age exceeds 8 hours. This check is not automatic in Spring Session — implement it as a HandlerInterceptor or a custom SessionManagementFilter.

@Configuration
@EnableWebSecurity
public class SecurityConfig {

    @Bean
    public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
        http
            .sessionManagement(session -> session
                // Prevent session fixation: new session ID issued after authentication.
                // migrateSession copies attributes to the new session; newSession discards them.
                .sessionFixation().migrateSession()
                // One session per user. A second login invalidates the first.
                // Required under SAMA concurrent-session controls.
                .maximumSessions(1)
                .maxSessionsPreventsLogin(false) // new login wins; old session invalidated
                .expiredUrl("/session-expired")
            )
            // CSRF protection enabled (default). Do not disable.
            // Spring Security stores the CSRF token in the session by default.
            .csrf(csrf -> csrf
                .csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())
            )
            // For payment initiation endpoints: require re-authentication.
            .authorizeHttpRequests(auth -> auth
                .requestMatchers("/payments/initiate/**").authenticated()
                .anyRequest().authenticated()
            );

        return http.build();
    }
}

Redis Session Configuration

Session data requires different Redis configuration from cache data. Cache misses are acceptable; session misses log users out. Cache data can be evicted under memory pressure; session data cannot. Run a dedicated Redis instance (or a dedicated Redis Cluster shard) for sessions, separate from the application cache Redis. This allows independent scaling, separate memory policies, and cleaner failure isolation.

Key sizing: a Spring Session entry serialised to JSON for a banking portal session typically runs 2–8 KB — user identity, CSRF token, OAuth2 state/nonce, navigation state. Budget 10 KB per session as a ceiling. At 100,000 concurrent sessions, that is 1 GB of session data — comfortably within a single Redis instance but large enough that you need a memory monitoring alert.

The Redis key structure Spring Session uses: saib:portal:session:sessions:{sessionId} (the Hash containing session attributes) and saib:portal:session:sessions:expires:{sessionId} (a String key used for TTL-based expiry notification). Do not manually modify these keys outside of the Spring Session lifecycle; the expiry notification mechanism relies on the TTL of the expires key.

Session Fixation and CSRF

Session fixation is an attack where an adversary pre-sets a session ID (by getting the victim to click a crafted link) and then authenticates as a legitimate user — at which point the adversary owns the now-authenticated session. The defence is to issue a new session ID after every authentication. Spring Security’s sessionFixation().migrateSession() does exactly this: on successful login, it creates a new session, copies all pre-authentication attributes (CSRF token, PKCE state), and invalidates the old session. The session cookie in the browser is updated to the new ID.

CSRF (Cross-Site Request Forgery) protection requires a secret token tied to the user’s session. On every state-changing request (POST, PUT, DELETE), Spring Security validates the CSRF token from the request against the token stored in the session. The CookieCsrfTokenRepository pattern stores the token in a non-HttpOnly cookie so that JavaScript can read and send it, while the validation still happens server-side against the session value. This is compatible with single-page application architectures without weakening the protection.

Multi-Application Session Sharing

A Saudi bank’s digital portal typically spans multiple subdomains: app.saib.com.sa (retail banking), biz.saib.com.sa (corporate banking), api.saib.com.sa (API portal). A customer authenticated on the retail portal should not be forced to re-authenticate on the corporate portal if they have access to both — single sign-on should handle the session continuation.

SSO across subdomains using Redis-backed sessions requires a shared Redis keyspace (all applications read the same session data) and a session cookie scoped to the parent domain (.saib.com.sa). Each application accesses the same session record using its own namespace prefix (to avoid key collisions) but reads the user identity attributes written by the SSO authentication service. In practice, this is complex to implement correctly — a shared session format, a shared attribute naming convention, and version-compatible serialization across all applications. For most banks, the cleaner architecture is to not share the Redis session across applications and instead rely on the OAuth2 authorization server (Keycloak or PingFederate) for SSO — each application has its own session, but the user only authenticates once at the IAM layer.

# OpenShift Route with cookie-based affinity — use ONLY as a fallback
# for legacy applications that cannot adopt distributed sessions.
# Do not use this as the primary session mechanism for new applications.
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: legacy-portal
  annotations:
    # Cookie affinity: HAProxy sets INGRESSCOOKIE with the backend pod name.
    haproxy.router.openshift.io/balance: source
    haproxy.router.openshift.io/disable_cookies: "false"
    router.openshift.io/cookie_name: INGRESSCOOKIE
    router.openshift.io/cookie.samesite: Strict
spec:
  tls:
    termination: edge
    insecureEdgeTerminationPolicy: Redirect

Pitfalls

Steps to Migrate from Sticky Sessions to Redis-Backed Distributed Sessions

1. Audit what is in the current session

   Enumerate every session.setAttribute() call in the codebase. Classify each attribute: is it safe to store in Redis (is it non-sensitive)? Is it necessary (could the data be derived on each request instead of stored)? Remove everything that is either sensitive or unnecessary before migrating.

2. Deploy a dedicated Redis Sentinel for sessions

   Provision a 3-node Redis Sentinel cluster (1 primary, 1 replica, 1 arbiter) in the same OpenShift namespace as the portal. This is separate from the application cache Redis. Configure persistence (appendonly yes) so session data survives Redis restart. Set maxmemory-policy noeviction — sessions must not be silently evicted.

3. Add Spring Session with JSON serialization

   Add spring-session-data-redis dependency, configure GenericJackson2JsonRedisSerializer as the session serializer, and set the namespace to an application-specific prefix. Deploy to a canary pod. Verify that sessions are visible in Redis and that the canary pod correctly reads sessions created by the old pods.

4. Remove OpenShift Route cookie affinity

   Delete the router.openshift.io/cookie_name annotation from the Route (or set haproxy.router.openshift.io/disable_cookies: "true"). The Route will now distribute requests round-robin. Verify that existing sessions survive routing to different pods.

5. Implement SAMA-compliant timeout enforcement

   Set maxInactiveIntervalInSeconds=900 for the idle timeout. Add a HandlerInterceptor that reads the session creation timestamp and invalidates sessions older than 8 hours. Test with a session aged to the limit in a staging environment.

6. Implement correct logout

   In the logout handler, call session.invalidate() (Spring deletes the Redis key) AND call the authorization server’s revocation endpoint for the access and refresh tokens. Clear the session cookie explicitly by setting its Max-Age=0. Add a regression test that verifies the session key is absent in Redis after logout.