From Chaos to Cosmos: Fixing a SaaS Architecture That Spawned EC2 Instances Like Rabbits
Last month, a client came to me with a problem. Their AWS bill was astronomical, their backend was constantly crashing, and their DevOps engineer had developed a nervous twitch every time someone mentioned "new customer onboarding."
After reviewing their architecture, I understood why. They'd built what I call a "Schrodinger's Scale" system - simultaneously over-engineered and under-architected.
The Original Architecture (Or: How to Burn Money at Scale)
Here's what I walked into:
┌─────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Users │────▶│ React App │────▶│ Backend API │────▶│ Master DB │
└─────────┘ │ (EC2+Nginx) │ │ (Single EC2) │ │(EC2+MongoDB) │
└──────────────┘ └──────┬───────┘ └──────────────┘
│
├────────────▶┌──────────────┐
│ │ Org 1 DB │
│ │(EC2+MongoDB) │
│ └──────────────┘
│
├────────────▶┌──────────────┐
│ │ Org 2 DB │
│ │(EC2+MongoDB) │
│ └──────────────┘
│
└────────────▶┌──────────────┐
│ Org N DB │
│(EC2+MongoDB) │
└──────────────┘
🔥 Problem: New EC2 for each organization = $$$
🔥 Problem: Backend creates new connections per request
🔥 Problem: Static site served from EC2 instead of S3
Let me translate this disaster:
- One EC2 instance per organization for MongoDB (because "isolation" and "security")
- Static React app served from EC2 (because S3 is... scary?)
- Single backend EC2 handling all traffic (the bottleneck special)
- Two-hop database pattern: Query master DB → Find org DB → Query org DB
This is like building a separate house for each book in your library, then wondering why you're broke and can't find anything.
The Problems (Besides the Obvious)
1. Connection Pool Exhaustion
// Their code (simplified)
async function getOrgData(userId, dataId) {
// Step 1: Connect to master DB
const masterConn = await MongoClient.connect(MASTER_DB_URL);
const user = await masterConn.db('master').collection('users').findOne({ userId });
// Step 2: Get org DB URL
const orgDbUrl = user.organizationDbUrl; // "mongodb://org-47-db.internal:27017"
// Step 3: Create NEW connection to org DB
const orgConn = await MongoClient.connect(orgDbUrl); // 💀 NEW CONNECTION
const data = await orgConn.db('org').collection('data').findOne({ dataId });
// Step 4: Close connections (sometimes... when they remembered)
await masterConn.close();
await orgConn.close();
return data;
}
Every. Single. Request. New connections. The backend was spending more time establishing connections than serving data.
2. The AWS Bill of Doom
- 50 organizations = 50 EC2 instances
- Minimum EC2 cost: ~$20/month
- Database servers alone: $1000/month
- Most instances at <5% CPU utilization
They were essentially paying for 50 houses to store 50 notebooks.
3. Operational Nightmare
- Deployments? Update 50+ instances
- Monitoring? 50+ dashboards
- Backups? 50+ backup jobs
- Security patches? See you next month
The New Architecture (Or: How Normal People Build SaaS)
┌─────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Users │────▶│ CloudFront │────▶│ ALB │────▶│ Backend 1 │
└─────────┘ │ (CDN) │ │(Load Balancer)│ │(Auto-scaling)│
└──────┬───────┘ └──────────────┘ └──────┬───────┘
│ │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ S3 Bucket │ │ Backend N │
│ (React App) │ │(Auto-scaling)│
└──────────────┘ └──────┬───────┘
│
▼
┌─────────────────────────┐
│ MongoDB Replica Set │
│ ┌─────────────────┐ │
│ │ Primary │ │
│ └────────┬────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ ▼ ▼ │
│┌──────────┐ ┌──────────┐│
││Secondary │ │Secondary ││
│└──────────┘ └──────────┘│
└─────────────────────────┘
✅ Success: 3 DB instances total (with replication)
✅ Success: Persistent connection pooling
✅ Success: Static assets on CDN
The Implementation Details
1. Database Architecture Overhaul
Instead of one database per organization, we use one database with logical separation:
// New schema design
{
// All collections now have orgId as a partition key
users: {
_id: ObjectId(),
orgId: "org_123", // Partition key
email: "[email protected]",
// ... other fields
},
// Compound indexes for performance
indexes: [
{ orgId: 1, email: 1 }, // Org-specific queries
{ orgId: 1, createdAt: -1 } // Recent items per org
]
}
2. Connection Pool Management
// Singleton connection manager
class DatabaseConnection {
constructor() {
this.connection = null;
this.connectionPromise = null;
}
async connect() {
if (this.connection) {
return this.connection;
}
if (this.connectionPromise) {
return this.connectionPromise;
}
this.connectionPromise = MongoClient.connect(MONGODB_URL, {
maxPoolSize: 100,
minPoolSize: 10,
maxIdleTimeMS: 30000,
// Replica set configuration
replicaSet: 'rs0',
readPreference: 'secondaryPreferred' // Read from secondaries when possible
});
this.connection = await this.connectionPromise;
return this.connection;
}
}
// Usage - ONE connection, reused
const db = await DatabaseConnection.connect();
const data = await db
.collection('data')
.find({ orgId: req.user.orgId, ...query }) // Always filter by orgId
.toArray();
3. Row-Level Security
Every query now includes the organization filter:
// Middleware to enforce org isolation
function enforceOrgIsolation(req, res, next) {
const originalFind = db.collection.prototype.find;
db.collection.prototype.find = function(query, options) {
// Automatically inject orgId into every query
const secureQuery = {
...query,
orgId: req.user.orgId
};
return originalFind.call(this, secureQuery, options);
};
next();
}
4. Static Asset Optimization
# Old way: Nginx on EC2
location / {
root /var/www/react-app;
try_files $uri /index.html;
}
# New way: S3 + CloudFront
aws s3 sync build/ s3://app-bucket --delete
aws cloudfront create-invalidation --distribution-id ABCD --paths "/*"
Cost reduction: 95% (from $40/month to $2/month)
The Results
Performance Improvements
- API response time: 450ms → 45ms (90% reduction)
- Database connection time: 200ms → 0ms (connection pooling)
- Time to first byte: 800ms → 150ms (CloudFront caching)
Cost Savings
- Monthly AWS bill: $2,400 → $400 (83% reduction)
- Database costs: $1,000 → $150
- Static hosting: $40 → $2
Operational Wins
- Deployment time: 2 hours → 5 minutes
- Monitoring dashboards: 50+ → 3
- On-call incidents: 15/month → 1/month
Lessons for Your Architecture
1. Isolation Doesn't Require Separation
You don't need separate infrastructure for multi-tenancy. Logical isolation with proper indexes and query filters is sufficient for 99% of SaaS applications.
2. Connection Pools Are Not Optional
Database connections are expensive. Treat them like the scarce resource they are. One connection pool, properly managed, beats 50 connection pools every time.
3. Use the Right Tool
- Static files? S3 + CloudFront
- Dynamic API? Auto-scaling EC2/ECS
- Database? Managed services or proper replica sets
- Isolation? Application-level, not infrastructure-level
4. Monitor Actual Usage
Those 50 EC2 instances? Average CPU usage was 3%. They were paying for capacity they'd never use. Right-size based on actual metrics, not imagined scale.
The Code Patterns That Scale
// ❌ Bad: Infrastructure isolation
const getOrgDatabase = (orgId) => {
return new DatabaseConnection(`mongodb://org-${orgId}.internal:27017`);
};
// ✅ Good: Logical isolation
const getOrgData = async (orgId, query) => {
return db.collection('data').find({ orgId, ...query });
};
// ❌ Bad: New connections per request
app.get('/api/data', async (req, res) => {
const conn = await createNewConnection();
const data = await conn.find({});
await conn.close();
res.json(data);
});
// ✅ Good: Shared connection pool
app.get('/api/data', async (req, res) => {
const data = await db.collection('data').find({ orgId: req.user.orgId });
res.json(data);
});
Conclusion
Sometimes the best architecture is the boring one. Not every problem needs a distributed solution. Not every customer needs their own server. Not every static file needs a web server.
The client saved $2,000/month and improved performance by 10x. All by doing less, not more.
Remember: Your architecture should be as complex as necessary, but no more. And for most SaaS applications, "necessary" is a lot simpler than you think.
Now if you'll excuse me, I need to go explain to another client why their microservices don't need their own Kubernetes clusters. That's a story for another post.
P.S. - If your architecture diagram looks like a spider web and your AWS bill looks like a phone number, we should talk.