[service/Instance]ToSlotRatio implementation across SSM and ego service

In Morgan Stanley clusters, SSM hosts are configured as part of the mgmt servers specifically grouped into SSM resource group(s). The SSM hosts are grouped by their hardware configurations including:

• Maximum physical memory, eg: ssm instances of the applications consumes more memory than others due to its flooded sessions and tasks, and it requires not to put too many of such ssm instance on the same SSM host.

• Bandwidth of the network card, eg: IRD/ZCS application is a bandwidth intensive application which requires more interactions among clinets --> ssm instance --> tasks. Limiting total number of concurrent such ssm instance on an SSM host becomes very critical.

• Size of the NVMe drive when SSM BEV boundary limit is reached the ssm instance requires to flush out the excessive info from memory to the local NVMe drive on the SSM host

• etc

Our current implementation is very cumbersome:

• we have to group SSM host resource groups with a max number slots set per SSM host

iapp5106.devin1.ms.com andwang 2023-11-03 12:17:20 prod grid-dd-pd-3 andwang $ egosh-bin rg | egrep 'NAME|SSM'

NAME HOSTS SLOTS FREE ALLOCATED

SSM1TB25G_5slot 1 5 3 2 <-- this is a new type hardware with 25GB network card and 1TB physical memory

SSM500GB10G_1slot 9 9 2 7 <-- this SSM host resource group is created for IRD/ZCS applications (esp: for interactive app)

SSM500GB10G_2slot 22 44 8 36 <-- this SSM host resource group is created for IRD/ZCS and CMD applications that are less intensive in terms of network traffic and memory

SSM500GB10G_30slot 2 60 22 38 <-- this SSM host resource group is created for non-RID/CMD applications

SSM500GB10G_5slot 3 15 2 13 <-- this SSM host resource group is created for our MTCA KDB application

SSM500GB10G_80slot 4 320 186 134 <-- this SSM host resource group is created for FRM applications which require less network bandwidth and memory.

iapp5106.devin1.ms.com andwang 2023-11-03 12:17:31 prod grid-dd-pd-3 andwang $

Our requirement is very simple, since we can set the serviceToSlotRatio to 1:N for the task and whether we can do a slot based serviceToSlotRatio or instanceToSlotRatio to 1:N for the ssm instance in application profile. So that we can:

• Weight application ssm instance by specifying the serviceToSlotRatio or instanceToSlotRatio to 1:N in ssm section of the application profile. This will tell symphony that the ssm instance of the application will require minimum N slots on a SSM host.

• We can simplify the SSM host resource group by setting only 1 resource group, the slots per SSM hosts ==> ncpus = nprocs * ncores

• We can provision new servers based on availability in the cluster and capacity required in the worst case scenario

We also like to apply the same concept to ego service instances (1) weight ego service instance (2) start each instance and consume the footprint based on [service/instance]ToSlotRatio = 1:N