Day 44: Multiple Services per Host

Multiple Services per Host means running several independently built and versioned services on the same server (VM, App Service plan, or physical machine) instead of giving each its own VM or container host.

Intro

Multiple Services per Host means running several independently built and versioned services on the same server (VM, App Service plan, or physical machine) instead of giving each its own VM or container host. You trade some isolation for much lower cost and simpler operations. It is often the right first step for a system with many small, low-traffic services, such as the internal services of our insurance claims platform.

Why we need this

Every deployment unit has a fixed overhead: an OS, patching, monitoring agents, a load balancer slot, and a minimum billable VM size. If Claims.Api, Policy.Api, Documents.Api, Notifications.Worker and eight other small services each get their own VM, we pay for eleven mostly idle machines and patch eleven operating systems.

• Cost: small services often use 2-10% of a VM’s CPU. Packing them raises utilisation.
• Operational simplicity: one host to patch, monitor, back up and secure.
• Migration path: teams moving from a monolith can split code into services first and defer the container/orchestrator investment.
• Constraints: some organisations cannot use containers yet (licensing, on-premises Windows Server, compliance approvals).

What problem it solves

Problem from the topic list: per-service VM/container overhead is too costly for many small services. The resolution is that several services share one host, at the price of a higher blast radius.

Without it, in our claims system: 12 services x 1 VM each = 12 VMs, 12 patch cycles, 12 sets of agents and certificates, and a bill dominated by idle capacity. Small teams spend time on infrastructure instead of features.

With it, the same 12 services run on 2-3 hosts (for redundancy) with process-level separation. The cost drops, but a runaway service can now hurt its neighbours; the rest of this lesson is about controlling that risk.

When it is needed (and when it is NOT)

Good fit

• Many small, low-to-moderate-traffic services (internal tools, back-office APIs, workers).
• Dev, test and staging environments where cost matters more than isolation.
• Early migration from a monolith, before a platform team exists.
• Services owned by the same team with similar security requirements.

Wrong choice

• A service with a spiky or heavy workload (claims-document OCR) that would starve neighbours.
• Services with different compliance zones (payment card data next to a public marketing API).
• Services that need different OS versions, runtimes, or conflicting native dependencies.
• Strict SLOs per service where one service’s crash or memory leak must never affect another.
• When you already run Kubernetes or Azure Container Apps; there, bin-packing with resource limits is the better version of this idea.

How to identify the problem (key signals)

Signals that you need this pattern (too many hosts):

1. Fleet CPU averages under 10-15% on most VMs and memory is mostly unused.
2. The cloud bill grows linearly with service count, not with traffic.
3. Patching and certificate renewal are tracked per VM in spreadsheets.
4. New small services wait weeks for a VM to be provisioned.

Signals that it is already hurting (too many services per host): 5. One service’s memory leak triggers an OOM kill that takes down unrelated services. 6. Latency spikes in Policy.Api coincide with a batch job in Reports.Worker (noisy neighbour). 7. Deploying service A restarts the host or IIS/app pool and briefly interrupts service B. 8. Port or dependency conflicts (“service X needs .NET 8 runtime patch, service Y breaks”).

Flow Diagram

Several services share one host as separate processes with hard resource limits.

flowchart TB
    PX["Reverse proxy :443"]
    subgraph H1["Host 1"]
      A1["Claims.Api :5001 - MemoryMax 800M"]
      B1["Policy.Api :5002 - CPUQuota 50%"]
    end
    subgraph H2["Host 2"]
      A2["Claims.Api :5001"]
      B2["Policy.Api :5002"]
    end
    PX --> A1
    PX --> B1
    PX --> A2
    PX --> B2

Level 1: Beginner

Analogy: an apartment building. Each service is a tenant with its own locked flat (process), but they share the building’s water, power and lift (CPU, memory, network). Cheaper than separate houses, but if one tenant floods the bathroom, the neighbours notice.

Minimal example: two ASP.NET Core services on one Linux VM, on different ports, each a separate process.

// Claims.Api/Program.cs  (listens on 5001)
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/claims/{id}", (string id) => Results.Ok(new { id, status = "Open" }));
app.MapGet("/health", () => Results.Ok("healthy"));
app.Run();

// Policy.Api/Program.cs  (listens on 5002)
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/policies/{number}", (string number) => Results.Ok(new { number, active = true }));
app.MapGet("/health", () => Results.Ok("healthy"));
app.Run();

Start each with its own port: ASPNETCORE_URLS=http://127.0.0.1:5001 dotnet Claims.Api.dll and ASPNETCORE_URLS=http://127.0.0.1:5002 dotnet Policy.Api.dll. A reverse proxy on port 443 routes /claims/* and /policies/* to them.

Rule for beginners: one service = one process = one port = one deployment folder. Never load two services into one process.

Level 2: Intermediate

In a real .NET + Angular + SQL Server app, the host runs each service as a managed unit (systemd on Linux, Windows Service or separate IIS app pools on Windows), and a reverse proxy fronts them.

systemd unit with resource limits (Linux)

[Unit]
Description=Claims API
After=network.target

[Service]
WorkingDirectory=/opt/claims-api
ExecStart=/usr/bin/dotnet /opt/claims-api/Claims.Api.dll
Environment=ASPNETCORE_URLS=http://127.0.0.1:5001
Environment=ASPNETCORE_ENVIRONMENT=Production
User=claims-api
Restart=always
RestartSec=5
# Noisy-neighbour protection (cgroups v2)
MemoryHigh=600M
MemoryMax=800M
CPUQuota=50%
TasksMax=200

[Install]
WantedBy=multi-user.target

Each service has its own OS user, folder and limits. MemoryMax means a leaking service is killed and restarted alone instead of exhausting the host.

nginx routing (one public entry point)

server {
    listen 443 ssl;
    server_name claims.example.com;

    location /api/claims/   { proxy_pass http://127.0.0.1:5001/; }
    location /api/policies/ { proxy_pass http://127.0.0.1:5002/; }
    location /              { root /var/www/claims-ui; try_files $uri /index.html; }
}

ASP.NET Core side: forwarded headers and a readiness check

using Microsoft.AspNetCore.HttpOverrides;
using Microsoft.Extensions.Diagnostics.HealthChecks;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHealthChecks()
    .AddSqlServerCheck(builder.Configuration.GetConnectionString("ClaimsDb")!);

builder.Services.Configure<ForwardedHeadersOptions>(o =>
    o.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto);

var app = builder.Build();
app.UseForwardedHeaders();
app.MapHealthChecks("/health/ready");
app.Run();

static class SqlHealthExtensions
{
    // Minimal hand-rolled check to avoid extra packages
    public static IHealthChecksBuilder AddSqlServerCheck(this IHealthChecksBuilder b, string cs) =>
        b.AddCheck("sql", () =>
        {
            try
            {
                using var conn = new Microsoft.Data.SqlClient.SqlConnection(cs);
                conn.Open();
                return HealthCheckResult.Healthy();
            }
            catch (Exception ex)
            {
                return HealthCheckResult.Unhealthy("SQL unreachable", ex);
            }
        });
}

Angular side (current Angular major, standalone + inject): the UI does not know the services share a host. It calls relative URLs, so the routing can change later without frontend changes.

import { Injectable, inject } from '@angular/core';
import { HttpClient } from '@angular/common/http';

@Injectable({ providedIn: 'root' })
export class ClaimsService {
  private http = inject(HttpClient);
  getClaim(id: string) {
    return this.http.get<{ id: string; status: string }>(`/api/claims/claims/${id}`);
  }
}

Database rule: co-hosting does not mean sharing a database. Each service keeps its own database or at least its own schema and SQL login (see Day 5 and Day 6).

Level 3: Advanced

Failure modes

• Noisy neighbour: CPU or memory contention. Mitigate with cgroup limits (CPUQuota, MemoryMax), Windows Job Objects, or separate app pools with private memory limits.
• Blast radius of the host: a host failure or a bad OS patch takes down every co-hosted service. Always run at least two hosts and spread each service across both.
• Shared fate deployments: restarting a host-level agent, reverse proxy or IIS site affects everyone. Use graceful reloads (nginx -s reload) and per-service restarts.
• Port and file-system collisions: allocate ports from a documented registry; give each service its own directory, user and temp path.
• .NET runtime coupling: framework-dependent deployments share one installed runtime. A runtime patch updates all services at once (good for security, risky for regressions). Self-contained deployments remove the coupling but cost disk and patch effort per service.

Performance and scalability

• You can only scale by adding hosts, and every service on the host scales with it. A service that needs 10x capacity drags its neighbours along; that is the signal to move it out.
• Leave headroom: plan total requested CPU/memory at 60-70% of the host, and alert on host-level saturation, not just per-service metrics.
• Set Kestrel limits so one service cannot exhaust connections:

builder.WebHost.ConfigureKestrel(k =>
{
    k.Limits.MaxConcurrentConnections = 500;
    k.Limits.MaxRequestBodySize = 10 * 1024 * 1024; // 10 MB claim attachments
});

Security

• Run each service under its own low-privilege OS account; never share a service account or file permissions.
• Give each service its own secrets (Key Vault reference or managed identity), never a shared config file on disk.
• Bind services to 127.0.0.1 so only the reverse proxy is reachable from outside.
• A compromised service can attempt lateral movement on the same OS. Do not co-host services from different trust zones.

Common mistakes

1. No resource limits, so the first leak takes down the host.
2. Running only one host, so the pattern has no redundancy.
3. Co-hosting stateful/heavy jobs (report generation) with latency-sensitive APIs.
4. Sharing one database login across services “because they are on the same box”.
5. Manual deployment by copying files, with no rollback path.

Level 4: Expert and Architect view

Trade-off comparison

Combines with: Service per Team (Day 4, ownership stays per service even on shared hosts), Health Check API (Day 36), Externalized Configuration (Day 39), API Gateway/reverse proxy (Day 19), Log Aggregation and Metrics (Days 32-33, tag every log and metric with service name), and Strangler Fig (Day 49) during migration.

ADR (architecture-review style)

• Title: ADR-044 Host low-traffic claims back-office services on shared hosts
• Status: Proposed
• Context: We have 9 back-office services (Policy lookup, Notifications, Document index, and others) averaging under 10% CPU each. Dedicated VMs per service cost about 9x the compute needed. No platform team exists to run Kubernetes yet.
• Decision: Run these 9 services as separate processes on two shared Linux hosts behind a reverse proxy, each with its own OS user, systemd unit, cgroup limits, database and secrets. Payment and document-OCR services stay on dedicated hosts.
• Consequences: Lower cost and patching effort. Higher blast radius per host, mitigated by two hosts and resource limits. Coarse scaling. We accept manual capacity planning.
• Exit criteria: Move a service to its own container/host when it needs independent scaling, sustains over 30% of host CPU, or falls under stricter compliance. Revisit when the team adopts Azure Container Apps or AKS.

Azure implementation

Services that implement this pattern

• Azure App Service plan: the cleanest Azure form. One plan is the shared host (a set of VM instances); each web app in the plan is a service. All apps in a plan share the plan’s CPU and memory, and you pay for the plan, not per app.
• Azure Virtual Machines / VM Scale Sets: run multiple systemd or Windows services yourself, as in Level 2. Full control, full responsibility.
• Azure Container Apps (dedicated workload profiles) and AKS node pools: many containers share a node. This is the container-era equivalent, with real resource requests and limits.

Configuring App Service for shared hosting

1. Create one App Service plan (Linux, Premium v3 or newer for production; Basic/Standard tiers are older and usually less cost-effective; Free/Shared are for experiments only). Verify current tier names and prices on the Azure pricing page before committing.
2. Deploy each service as a separate web app in the plan, each with its own deployment slot(s), managed identity and app settings.
3. Enable Always On so apps are not unloaded when idle.
4. Enable “Per-app scaling” if you want to limit how many plan instances a specific app uses (Premium tiers).
5. Use Key Vault references for secrets, and give each app its own managed identity with least privilege on its own database.
6. Place Azure Front Door or Application Gateway in front for routing and WAF if you need a single public entry point.

Azure CLI sketch

az group create -n rg-claims -l westeurope
az appservice plan create -g rg-claims -n plan-claims-backoffice --is-linux --sku P0V3 --number-of-workers 2
az webapp create -g rg-claims -p plan-claims-backoffice -n app-claims-api --runtime "DOTNETCORE:10.0"
az webapp create -g rg-claims -p plan-claims-backoffice -n app-policy-api --runtime "DOTNETCORE:10.0"
az webapp config set -g rg-claims -n app-claims-api --always-on true

Check the exact runtime string and SKU names with az webapp list-runtimes --os-type linux and az appservice list-locations --sku P0V3 because they change over time.

Pricing considerations

• App Service: cost is per plan instance-hour, independent of how many apps run in it. Density is the savings lever; adding apps to an underused plan is free until resources run out.
• VMs: per-VM hourly cost plus disks; reserved instances or savings plans reduce cost for steady workloads.
• Always run at least 2 instances in production for availability, which doubles the base cost but is what makes the shared host safe.

Monitoring

• Application Insights per service (separate resource or distinct cloud role name) so telemetry stays attributable.
• Azure Monitor metrics at plan level: CPU percentage, memory percentage. Alert at 70-80% sustained.
• Use the plan-level “CPU per app” diagnostics (App Service Diagnose and solve problems) to find the noisy neighbour.

Reference architecture (text): Users reach the Angular SPA on Azure Static Web Apps or Blob static hosting, served through Azure Front Door with WAF. Front Door routes /api/claims/* and /api/policies/* to two web apps inside one App Service plan (2 instances, zone redundant where the tier supports it). Each app uses its own managed identity to reach its own Azure SQL database and Key Vault secrets, and publishes events to Azure Service Bus. All apps send telemetry to a shared Log Analytics workspace, tagged by cloud role name. Payment and OCR services live in a separate plan (or Container Apps) to isolate them.

Teaching guide for my team

2-minute beginner explanation: “Think of a shared office. Each team has its own desk and lockable drawer (its own process, folder and database), but we share the electricity and the internet. It is cheaper than renting a separate office per team. The risk: if one team plugs in a giant heater, the lights flicker for everyone. So we set limits per desk and we never put the vault (payments) in the shared office.”

5-minute intermediate explanation: Walk through the three layers: (1) process separation, one service per process with its own OS user; (2) resource controls, cgroup limits, Kestrel limits, app pools; (3) shared entry point via reverse proxy. Show the systemd unit and explain MemoryMax. Then explain the blast radius: host failure, shared runtime patches, coarse scaling. Finish with the exit criteria from the ADR: when a service grows, it graduates to its own container or host.

Hands-on exercise

1. On a Linux VM (or WSL), publish Claims.Api and Policy.Api from Level 1 on ports 5001 and 5002.
2. Create two systemd units with MemoryMax=200M and CPUQuota=25%.
3. Add an endpoint to Claims.Api that allocates memory in a loop (/leak).
4. Call /leak, then watch systemctl status claims-api and call Policy.Api.

• Expected outcome: Claims.Api is killed and restarted by systemd once it hits the limit; Policy.Api keeps answering throughout. Repeat with the limits removed to see the whole host slow down or the OOM killer pick a victim, which proves why limits matter.

Interview-style questions

1. What is the main risk of Multiple Services per Host? Larger blast radius and noisy neighbours: one service’s failure or resource hunger can affect co-hosted services, and the host itself is a shared point of failure.
2. How do you reduce that risk without moving to containers? Separate processes and OS users, cgroup/app-pool resource limits, at least two hosts, health checks with automatic restart, and keeping high-risk or heavy services on dedicated hosts.
3. When do you move a service off a shared host? When it needs independent scaling, consumes a large share of the host, has stricter security or compliance needs, or its incidents repeatedly affect neighbours.

Mastery checklist

• I can explain the trade-off: lower cost and overhead versus weaker isolation and higher blast radius.
• I can deploy two .NET services as separate processes on one host with distinct ports, users and folders.
• I can configure resource limits (systemd cgroups or IIS app pool limits) and prove they contain a memory leak.
• I can design routing through a reverse proxy while keeping services bound to localhost.
• I can decide, with metrics, when a service must leave the shared host.
• I can map the pattern to an Azure App Service plan and explain how billing works (per plan, not per app).
• I can compare it against Service per VM, Service per Container and an orchestrator in an ADR.
• I keep databases and secrets separate per service even when the host is shared.

Key takeaway

Multiple Services per Host buys density and low cost by sharing a host, so it only works safely with per-service process separation, hard resource limits, and at least two hosts. Use it for many small services, and graduate a service out of the shared host as soon as its scale or risk outgrows it.