How To Design A Real Time Chat Application - Architecture & Best Practices

How to Design a Real time Chat Application?

Real-time chat applications like WhatsApp, Telegram, and Slack have transformed how we communicate. They enable instant messaging across devices and locations. These messaging platforms must handle millions of concurrent connections, deliver messages with minimal latency, and provide features like message synchronization, notifications, and media sharing. In this section, we’ll design a scalable real-time chat application that delivers messages instantly while maintaining reliability and consistency.

Table of Contents

Problem Statement

Design a real-time chat application that:

– Supports one-on-one and group messaging
– Delivers messages instantly with minimal latency
– Synchronizes messages across multiple devices
– Supports media sharing (images, videos, files)
– Provides message status indicators (sent, delivered, read)
– Scales to millions of concurrent users
– Works reliably even with intermittent connectivity

This is a complex system design challenge that requires balancing real-time performance with reliability and scale.

Requirements Analysis

Functional Requirements

User Management: Registration, authentication, profile Users need accounts to identify themselves.
Contact Management: Add, remove, and block Users need to control who they communicate with.
One-on-One Messaging: Send and receive messages between two This is the core functionality.
Group Messaging: Create groups, add/remove members, send messages to groups. This extends communication to multiple participants.
Message Types: Support text, images, videos, files, location Modern chat apps support rich media.
Message Status: Show sent, delivered, and read Users want to know if their messages were received.
Notifications: Push notifications for new Users need to know when they receive messages while away.
Presence Indicators: Show online/offline status and last This helps users know when others are available.
Message Synchronization: Sync messages across multiple Users expect seamless experiences across phone, tablet, and computer.
Offline Support: Queue messages when offline and send when Communication should work despite connectivity issues.

Non-Functional Requirements

Low Latency: Message delivery in under Real-time communication requires minimal delays.
High Availability: 99% uptime. Communication tools must be reliable.
Scalability: Support millions of concurrent Popular chat applications have massive user bases.
Reliability: No message loss, even during service Messages must not disappear.
Security: End-to-end encryption for Privacy is essential for communication.
Consistency: Consistent message ordering across Messages should appear in the same order everywhere.

System Components and Architecture

High-Level Design

Our real-time chat application consists of these key components:

Client Applications: Mobile, web, and desktop apps
API Gateway: Entry point for all client requests
Authentication Service: Manages user authentication
User Service: Handles user profiles and relationships
Chat Service: Manages one-on-one and group conversations
Presence Service: Tracks online/offline status
Notification Service: Sends push notifications
Media Service: Handles media file storage and processing
WebSocket Service: Maintains persistent connections for real-time communication

Here’s a simplified architecture diagram:

This microservices architecture allows each component to scale independently based on demand.

Data Model Design

We need several data models to support our chat application:

User Model:

Table: users

```
user_id (PK): string
```
```
phone_number: string
```
```
username: string (optional)
```
```
name: string
```
```
profile_picture: string
```
```
status: string
```
```
created_at: timestamp
```
```
last_active: timestamp
```

Contact Model:

Table: contacts

```
contact_id (PK): string
```
```
user_id (FK): string
```
```
contact_user_id (FK): string
```
```
contact_name: string
```
```
blocked: boolean
```
```
created_at: timestamp
```

Conversation Model:

Table: conversations

```
conversation_id (PK): string
```
```
type: enum (one_on_one, group)
```
```
created_at: timestamp
```
```
updated_at: timestamp
```
```
last_message_id: string
```

Conversation Participant Model:

Table: conversation_participants

```
participant_id (PK): string
```
```
conversation_id (FK): string
```
```
user_id (FK): string
```
```
role: enum (member, admin)
```
```
joined_at: timestamp
```
```
last_read_message_id: string
```

Message Model:

Table: messages

```
message_id (PK): string
```
```
conversation_id (FK): string
```
```
sender_id (FK): string
```

type: enum (text, image, video, file, location)

```
content: text
```
```
media_url: string
```
```
sent_at: timestamp
```
```
delivered_at: timestamp
```
```
read_at: timestamp
```
```
deleted: boolean
```
```
reply_to_message_id: string (optional)
```

Device Model:

Table: devices

```
device_id (PK): string
```
```
user_id (FK): string
```

device_type: enum (ios, android, web, desktop)

```
push_token: string
```
```
last_active: timestamp
```

These models provide the foundation for tracking users, conversations, and messages.

Real-time Communication Mechanisms

WebSocket Architecture

For real-time messaging, we’ll use WebSockets:

1. Connection Establishment:

Client establishes WebSocket connection with server
Connection is authenticated using JWT or similar token
Server maintains mapping of user_id to active connections

Think of this as opening a dedicated phone line that stays open.

2. Connection Management:

Heartbeat mechanism to detect disconnections
Reconnection strategy with exponential backoff
Connection pooling for scalability

This ensures the connection remains stable and recovers from interruptions.

3. Message Routing:

Messages are routed based on conversation_id
Server looks up active connections for all participants
Messages are delivered to all connected devices of recipients

This is like a telephone switchboard routing calls to the right recipients.

Alternative Approaches

Although WebSockets are our primary mechanism, we’ll implement fallbacks:

HTTP Long Polling: For environments where WebSockets aren’t The client repeatedly asks the server for updates.
Server-Sent Events (SSE): For one-way server-to-client Good for notifications but not full chat.
Push Notifications: For delivering messages to offline This wakes up mobile apps to retrieve messages.

Having multiple communication methods ensures reliability across different environments.

Message Delivery and Synchronization

Message Flow

The complete message flow works as follows:

1. Message Creation:

Sender creates message locally
Message gets temporary ID and “sending” status
Message is sent to server via WebSocket

This provides immediate feedback to the sender.

2. Server Processing:

Server validates message
Server assigns permanent ID and timestamp
Server stores message in database
Server acknowledges receipt to sender

This ensures the message is properly recorded.

3. Message Delivery:

Server identifies recipient’s active connections
Server sends message to all connected devices
Recipients acknowledge receipt
Server updates message status to “delivered”

This gets the message to all of the recipient’s devices.

4. Read Receipts:

When recipient reads message, client sends read receipt
Server updates message status to “read”
Server notifies sender of read status

This lets the sender know their message was seen.

Offline Message Handling

For offline recipients:

1. Message Queuing:

Messages for offline users are stored in a queue
When user comes online, queued messages are delivered
Push notifications alert users of new messages

This ensures messages reach users even when they’re offline.

2. Message Synchronization:

Clients track last received message ID
On reconnection, clients request all messages since last received
Server sends missing messages in batches

This keeps all devices in sync, even after disconnections.

Group Chat Implementation

Group chats introduce additional complexity:

1. Group Creation and Management:

Any user can create a group
Creator becomes admin by default
Admins can add/remove members and other admins

This establishes the social structure of the group.

2. Message Distribution:

Messages are sent to all group members
Server fans out messages to all recipients
Delivery and read receipts are aggregated

This efficiently delivers messages to multiple recipients.

3. Scalability Challenges:

Large groups (1000+ members) require special handling
For very large groups, read receipts may be disabled
Messages may be delivered in batches

This addresses the challenges of very large groups.

Media Handling

For sharing media files:

1. Upload Process:

Client compresses media before upload
Media is uploaded to cloud storage (S3, GCS)
Server generates thumbnails for images and videos
Message contains media URL and metadata

This efficiently handles potentially large media files.

2. Download Process:

Thumbnails are downloaded automatically
Full media is downloaded on demand or based on settings
Progressive loading for large files

This optimizes bandwidth usage.

3. Storage Optimization:

Deduplication for identical files
Multiple resolution versions for images and videos
Automatic deletion of old media (configurable)

This reduces storage costs and improves performance.

Presence Indicators

To show online status:

1. Status Tracking:

Clients send heartbeats to server
Server updates presence database
Status changes are broadcast to relevant contacts

This shows who’s currently available.

2. Last Seen:

Timestamp of last activity is recorded
Shown as “last seen at [time]” when offline
Privacy settings can restrict visibility

This helps users know when someone was last active.

3. Typing Indicators:

Client sends typing event when user starts typing
Server broadcasts to conversation participants
Typing indicator expires after short timeout

This creates a more interactive feeling conversation.

Security Considerations

End-to-End Encryption

To ensure message privacy:

1. Key Exchange:

Each client generates public/private key pair
Public keys are exchanged through server
Signal Protocol or similar for key management

This establishes secure communication channels.

2. Message Encryption:

Messages encrypted with recipient’s public key
Server cannot decrypt message content
Group messages use group key management

This ensures only intended recipients can read messages.

3. Authentication:

Verify identity through phone number verification
Two-factor authentication for account recovery
Device verification for new logins

This prevents unauthorized access to accounts.

Scaling Considerations

Connection Scaling

To handle millions of concurrent connections:

1. Connection Pooling:

Distribute connections across multiple WebSocket servers
Use consistent hashing to route users to servers
Implement server-to-server communication for message delivery

This distributes the connection load.

2. Horizontal Scaling:

Add more WebSocket servers as user base grows
Use load balancers with sticky sessions
Implement auto-scaling based on connection count

This allows the system to grow with demand.

Database Scaling

To handle high message volume:

1. Database Sharding:

Shard by conversation_id or user_id
Use NoSQL databases for message storage
Implement read replicas for high-read scenarios

This distributes database load.

2. Caching Strategy:

Cache recent conversations and messages
Cache user presence information
Use Redis or similar for distributed caching

This reduces database load for frequent operations.

Fault Tolerance and Recovery

To ensure reliability:

1. Message Persistence:

Store messages durably before acknowledgment
Implement message deduplication
Use write-ahead logging for recovery

This prevents message loss.

2. Service Redundancy:

Deploy services across multiple regions
Implement automatic failover
Use circuit breakers to prevent cascading failures

This maintains availability during partial outages.

3. Disaster Recovery:

Regular backups of critical data
Cross-region replication
Documented recovery procedures

This protects against major failures.

Solution Walkthrough

Let’s walk through the complete flow of our chat application:

User Registration and Authentication

1. Registration:

User downloads app and enters phone number
System sends verification code via SMS
User enters code to verify identity
System creates user account and generates authentication tokens

This securely establishes the user’s identity.

2. Authentication:

User logs in with phone number
System verifies identity with SMS code or stored token
System issues new authentication token
User connects to WebSocket server with token

This secures ongoing access to the system.

One-on-One Chat

1. Starting a Conversation:

User selects contact from contact list
Client checks if conversation exists, creates if not
Client displays conversation history

This initiates or resumes a conversation.

2. Sending a Message:

User types and sends message
Client displays message with “sending” status
Client sends message to server via WebSocket
Server processes and stores message
Server sends acknowledgment to sender
Client updates message status to “sent”
Server delivers message to recipient
Recipient sends delivery receipt
Server updates message status to “delivered”
Sender client updates message status

This ensures reliable message delivery with status tracking.

3. Reading a Message:

Recipient opens conversation
Client marks messages as read locally
Client sends read receipts to server
Server updates message status
Server notifies sender
Sender client updates message status to “read”

This completes the message status lifecycle.

Group Chat

1. Creating a Group:

User selects “New Group” option
User selects contacts to add
User sets group name and optional image
Client sends group creation request to server
Server creates group and adds members
Server notifies all members

This establishes a new group conversation.

2. Group Messaging:

Similar to one-on-one messaging
Server fans out messages to all members
Delivery and read receipts are aggregated
Group updates are broadcast to all members

This efficiently handles multi-participant conversations.

Performance Optimization

To ensure sub-500ms message delivery:

1. Connection Optimization:

Keep WebSocket connections alive
Implement connection pooling
Use binary protocols for message transmission

This minimizes connection overhead.

2. Message Prioritization:

Prioritize message delivery over status updates
Process messages in parallel
Use separate queues for different message types

This ensures important operations happen first.

3. Geographic Distribution:

Deploy servers in multiple regions
Route users to nearest server
Implement cross-region message delivery

This reduces network latency.

Common Pitfalls and How to Avoid Them

Connection Management: Implement robust reconnection Connection issues are the most common source of problems.
Message Ordering: Use logical timestamps for consistent Different network paths can cause messages to arrive out of order.
Group Chat Scaling: Special handling for large Group chats can create fan- out challenges.
Media Handling: Optimize for different network Large media files can cause performance issues.
Security Vulnerabilities: Regular security audits and Chat applications are high-value targets for attackers.

By addressing these challenges, we can build a real-time chat application that delivers messages instantly while maintaining reliability, security, and scalability for millions of users.

This system design solution demonstrates how to handle real-time communication, message delivery, and synchronization at scale. The principles here apply to many real- time communication systems beyond chat applications.

You may also go through a separate article on System Design Core Concepts.

Additionally, test your knowledge by attempting System Design Interview Questions Practice MCQs.

Interested in a series of articles on System Design?, kindly check System Design Tutorials.