ccNUMA Effect on Solaris

The memcpy test that was used on the 8-socket AMD64 system running Linux was run on a 4-socket AMD64 Solaris system. I was interested in seeing the effect of strong lgrp affinity on Solaris.

[sbahra@numa ~/ccnuma/tests/malloc] lgrpinfo
lgroup 0 (root):
Children: 5-8
CPUs: 0-7
Memory: installed 16G, allocated 1.5G, free 15G
Lgroup resources: 1-4 (CPU); 1-4 (memory)
Latency: 120
lgroup 1 (leaf):
Children: none, Parent: 5
CPUs: 0 1
Memory: installed 3.5G, allocated 290M, free 3.2G
Lgroup resources: 1 (CPU); 1 (memory)
Load: 0.000198
Latency: 50
lgroup 2 (leaf):
Children: none, Parent: 6
CPUs: 2 3
Memory: installed 4.0G, allocated 295M, free 3.7G
Lgroup resources: 2 (CPU); 2 (memory)
Load: 0
Latency: 50
lgroup 3 (leaf):
Children: none, Parent: 7
CPUs: 4 5
Memory: installed 4.0G, allocated 248M, free 3.8G
Lgroup resources: 3 (CPU); 3 (memory)
Load: 0.5
Latency: 50
lgroup 4 (leaf):
Children: none, Parent: 8
CPUs: 6 7
Memory: installed 4.0G, allocated 689M, free 3.3G
Lgroup resources: 4 (CPU); 4 (memory)
Load: 0
Latency: 50
lgroup 5 (intermediate):
Children: 1, Parent: 0
CPUs: 0-5
Memory: installed 12G, allocated 832M, free 11G
Lgroup resources: 1-3 (CPU); 1-3 (memory)
Latency: 83
lgroup 6 (intermediate):
Children: 2, Parent: 0
CPUs: 0-7
Memory: installed 16G, allocated 1.5G, free 15G
Lgroup resources: 1-4 (CPU); 1-4 (memory)
Latency: 83
lgroup 7 (intermediate):
Children: 3, Parent: 0
CPUs: 0-7
Memory: installed 16G, allocated 1.5G, free 15G
Lgroup resources: 1-4 (CPU); 1-4 (memory)
Latency: 83
lgroup 8 (intermediate):
Children: 4, Parent: 0
CPUs: 2-7
Memory: installed 13G, allocated 1.2G, free 11G
Lgroup resources: 2-4 (CPU); 2-4 (memory)
Latency: 83
[sbahra@numa ~/ccnuma/tests/malloc]




The following two plots have memory size in bytes on x-axis and CPU ticks (returned from rdtsc) on y-axis. This is a simple memcpy test with a hot cache.

Explicit lgrp affinity

No explicit lgrp affinity.