From Chaos to Cosmos: Fixing a SaaS Architecture That Spawned EC2 Instances Like Rabbits
A client's SaaS spun up a new EC2 instance for every customer, and the fix was mostly deleting things. This is what we changed, and why the boring architecture usually wins.
Adeen Shukla6 min read
Last month, a client came to us with a problem. Their AWS bill kept climbing, their backend kept falling over, and onboarding a new customer had become something the team visibly braced for.
Once I looked at the architecture, it made sense. It was over-engineered and under-architected at the same time, which is a surprisingly easy combination to end up with when a product grows faster than anyone planned.
What I walked into
┌─────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Users │────▶│ React App │────▶│ Backend API │────▶│ Master DB │
└─────────┘ │ (EC2+Nginx) │ │ (Single EC2) │ │(EC2+MongoDB) │
└──────────────┘ └──────┬───────┘ └──────────────┘
│
├────────────▶┌──────────────┐
│ │ Org 1 DB │
│ │(EC2+MongoDB) │
│ └──────────────┘
│
├────────────▶┌──────────────┐
│ │ Org 2 DB │
│ │(EC2+MongoDB) │
│ └──────────────┘
│
└────────────▶┌──────────────┐
│ Org N DB │
│(EC2+MongoDB) │
└──────────────┘
Every organization got its own EC2 instance running its own MongoDB, in the name of "isolation." A master database kept track of which customer lived on which server, so every request did two hops: ask the master DB where the org's database is, then go query that. The React app was served by Nginx on yet another EC2 box, and a single backend instance sat in the middle of all of it.
It was like building a separate house for each book in your library, and then wondering why you are broke and cannot find anything.
What was actually wrong
The cost problem is the obvious one. N customers means N database servers, and since most customers are small, most of those servers spend their lives idling while you pay for them around the clock.
What was actually crashing the backend, though, was connections. In simplified form, every request did this:
// The original pattern (simplified)
async function getOrgData(userId, dataId) {
// New connection to the master DB, on every request
const masterConn = await MongoClient.connect(MASTER_DB_URL);
const user = await masterConn.db('master').collection('users').findOne({ userId });
// Then a brand new connection to this org's own server
const orgConn = await MongoClient.connect(user.organizationDbUrl);
const data = await orgConn.db('org').collection('data').findOne({ dataId });
// Closed afterwards... when someone remembered to
await masterConn.close();
await orgConn.close();
return data;
}
Opening a MongoDB connection means a TCP handshake, TLS, auth and a bit of server-side setup. Because this happened twice per request, the backend spent more time saying hello to databases than actually reading from them, and under load, connections piled up faster than they closed until things fell over.
Then there is the operational side. Every deploy, backup job, monitoring dashboard and security patch was multiplied by the number of customers, and nobody wants to be the person patching a server per customer on a Friday evening.
The boring version
┌─────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Users │────▶│ CloudFront │────▶│ ALB │────▶│ Backend 1 │
└─────────┘ │ (CDN) │ │(Load Balancer)│ │(Auto-scaling)│
└──────┬───────┘ └──────────────┘ └──────┬───────┘
│ │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ S3 Bucket │ │ Backend N │
│ (React App) │ │(Auto-scaling)│
└──────────────┘ └──────┬───────┘
│
▼
┌─────────────────────────┐
│ MongoDB Replica Set │
│ ┌─────────────────┐ │
│ │ Primary │ │
│ └────────┬────────┘ │
│ ┌─────┴─────┐ │
│ ▼ ▼ │
│┌──────────┐ ┌──────────┐│
││Secondary │ │Secondary ││
│└──────────┘ └──────────┘│
└─────────────────────────┘
One replica set for everyone, an auto-scaling backend behind a load balancer, and the static app on S3 behind CloudFront. That is three database servers in total, no matter how many customers sign up. Nothing in this diagram would surprise anyone, which is exactly what I like about it.
How we got there
The big change was moving from one database per customer to one database where every document carries an orgId, and every index leads with it:
// Every document has an orgId, and every index starts with it
db.collection('users').createIndex({ orgId: 1, email: 1 }, { unique: true });
db.collection('data').createIndex({ orgId: 1, createdAt: -1 });
Next, we created one MongoDB client per process, once at startup, and shared it across every request. Since the driver already manages a connection pool for you, all you have to do is stop throwing it away.
// db.js: one client, one pool, reused by every request
const client = new MongoClient(MONGODB_URL, {
maxPoolSize: 100,
minPoolSize: 10,
maxIdleTimeMS: 30000,
replicaSet: 'rs0',
// Reads can go to secondaries. Great for dashboards,
// careful anywhere you need to read your own writes.
readPreference: 'secondaryPreferred',
});
const ready = client.connect();
export async function getDb() {
await ready;
return client.db('app');
}
The real risk with shared tables is one customer seeing another customer's data. Since you cannot rely on every developer remembering to add orgId to every query, we made sure they do not have to. Every data access goes through a small helper scoped to the request's org:
// The only way handlers touch data: already filtered to the caller's org
export function orgScoped(db, req) {
const orgId = req.user.orgId;
return {
find: (name, query = {}, opts) =>
db.collection(name).find({ ...query, orgId }, opts), // orgId last, so it can't be overridden
findOne: (name, query = {}, opts) =>
db.collection(name).findOne({ ...query, orgId }, opts),
insertOne: (name, doc) =>
db.collection(name).insertOne({ ...doc, orgId }),
};
}
Paired with a lint rule or a code review habit of "no raw db.collection() in route handlers," this makes isolation the default rather than something people have to remember.
The static site was the easiest win of all. A React build is just files, and files belong on a CDN rather than on a server running Nginx 24/7:
aws s3 sync build/ s3://app-bucket --delete
aws cloudfront create-invalidation --distribution-id ABCD --paths "/*"
What changed
I am not going to put a wall of before-and-after percentages in front of you. In short, the database fleet went from one server per customer to three, the connection errors that were taking the backend down went away, the bill came down considerably, and deploys went from touching every box to running one pipeline. Onboarding a new customer now means inserting a row instead of provisioning a server.
The lesson I keep relearning is that isolation does not require separation. For most SaaS products, logical isolation with good indexes and a query layer that enforces the tenant filter is enough. If one big customer genuinely needs physical isolation for compliance, give that customer a dedicated cluster, but don't make it the default for everyone on day one.
Since database connections are expensive things, a single shared, long-lived pool beats a fresh connection per request every time.
Sometimes the best architecture is the boring one. The client got a faster, cheaper and calmer system, and almost all of it came from doing less.
Someone else is now waiting to explain to me why their three microservices each need their own Kubernetes cluster, but that is a story for another post.
P.S. If your architecture diagram looks like a spider web and your AWS bill looks like a phone number, we should talk.
Notes from the building room
Get the next essay in your inbox. Occasional, never spam.
Keep reading
Working4 min read
The career roadmap I never followed
Students keep asking me what they should learn. I never had an answer at 19, and I'm not sure I have one now. What I do have is a different question.
Thinking5 min read
Painting an owl (or why "creating" matters)
Try creating something with your hands, without distractions. It might unblock you and show you a completely different side of things.