You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-14 - 21:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Mon Jul 14, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2184121200,6sleep021843-21strings10:55:020
2184121200,6sleep021843-21strings10:55:020
205779910123,66cyclictest0-21swapper/309:39:583
2057799924,27cyclictest0-21swapper/310:43:313
2057799924,25cyclictest0-21swapper/306:35:053
2057799914,26cyclictest0-21swapper/307:10:053
2057799913,27cyclictest0-21swapper/311:40:103
2057799894,26cyclictest0-21swapper/307:40:123
2057799894,25cyclictest0-21swapper/309:39:153
20577998922,13cyclictest0-21swapper/308:10:013
2057799874,28cyclictest0-21swapper/310:29:543
2057799874,27cyclictest0-21swapper/308:04:433
2057799874,26cyclictest0-21swapper/309:14:563
20577998727,52cyclictest31127-21gltestperf09:00:003
2057799867,21cyclictest0-21swapper/310:24:393
2057799864,26cyclictest0-21swapper/310:55:443
2057799864,26cyclictest0-21swapper/310:55:443
2057799864,25cyclictest0-21swapper/311:00:003
2057799864,25cyclictest0-21swapper/309:08:153
2057799864,24cyclictest1160-21snmpd06:50:093
2057799864,24cyclictest1160-21snmpd06:50:093
20577998630,49cyclictest19144-21gltestperf08:35:023
20577998618,12cyclictest0-21swapper/309:25:133
20577998618,12cyclictest0-21swapper/309:25:133
2057799854,27cyclictest0-21swapper/308:15:033
2057799835,26cyclictest0-21swapper/311:09:473
2057799835,24cyclictest0-21swapper/306:24:563
2057799834,70cyclictest0-21swapper/309:55:073
2057799834,17cyclictest0-21swapper/307:50:153
20577998311,9cyclictest106250irq/46-eth009:50:153
2057799824,70cyclictest0-21swapper/310:00:143
2057799824,26cyclictest3667-21yum-updatesd-he10:19:383
2057799814,28cyclictest0-21swapper/311:14:503
2057799814,25cyclictest0-21swapper/311:24:063
2057799814,14cyclictest0-21swapper/306:48:183
2057799814,14cyclictest0-21swapper/306:48:183
2057799806,17cyclictest0-21swapper/307:39:443
2057799804,26cyclictest0-21swapper/309:48:223
2057799804,25cyclictest0-21swapper/308:08:053
2057799804,25cyclictest0-21swapper/308:08:053
20577998026,47cyclictest23385-21cpuspeed_turbos08:44:583
20577998026,47cyclictest23385-21cpuspeed_turbos08:44:583
20577998024,10cyclictest16583-21sort08:30:003
2057799794,68cyclictest0-21swapper/311:30:553
2057799794,26cyclictest0-21swapper/307:19:453
2057799794,22cyclictest0-21swapper/310:24:573
20577997924,9cyclictest20474-21hwlatdetect07:30:043
20577997923,10cyclictest1265-21runrttasks06:40:173
20577997922,10cyclictest12858-21unixbench_multi08:20:173
20577997913,12cyclictest0-21swapper/307:55:123
2057799785,15cyclictest0-21swapper/311:09:373
2057799784,12cyclictest0-21swapper/310:45:053
20577997824,9cyclictest942-21crond10:14:383
20577997824,9cyclictest942-21crond10:14:383
20577997822,9cyclictest3948-21needreboot09:10:053
20577997819,11cyclictest29585-21unixbench_multi08:55:163
20577997818,10cyclictest9307-21unixbench_singl09:20:153
20577997817,10cyclictest19118-21gltestperf10:49:593
20577997814,13cyclictest0-21swapper/310:05:023
2057799774,65cyclictest0-21swapper/309:30:053
20577997722,9cyclictest9349-21unixbench_singl07:05:183
20577997721,10cyclictest3293-21fschecks_count11:24:563
20577997718,9cyclictest15072-21cpuspeed_turbos07:20:003
20577997715,10cyclictest27850-21missed_timers07:45:103
2057799764,21cyclictest18863-21grep07:25:213
2057799764,21cyclictest18863-21grep07:25:213
2057799764,14cyclictest0-21swapper/311:49:583
20577997622,9cyclictest15886-21latency_hist08:29:423
2057799754,26cyclictest0-21swapper/308:54:423
2057799754,15cyclictest0-21swapper/306:34:473
2057799754,14cyclictest0-21swapper/307:00:083
2057799744,19cyclictest0-21swapper/311:35:283
2057799744,19cyclictest0-21swapper/311:35:283
2057799744,13cyclictest12299-21aten2.4-expect11:44:493
2057799734,12cyclictest0-21swapper/310:34:533
20577997214,12cyclictest0-21swapper/308:39:563
20577997020,8cyclictest4323-21switchtime06:55:173
281682660,4sleep128273-21timerandwakeup07:45:171
2057699667,44cyclictest0-21swapper/210:19:352
2057699643,42cyclictest12368-21needreboot08:20:092
30262633,13sleep10-21swapper/108:00:171
2057699637,41cyclictest0-21swapper/207:19:422
8782584,13sleep00-21swapper/010:11:380
8782584,13sleep00-21swapper/010:11:380
89662564,45sleep00-21swapper/009:20:110
223099560,6rtkit-daemon2229-21rtkit-daemon11:19:421
172812563,13sleep00-21swapper/010:45:070
265182554,43sleep00-21swapper/008:50:050
2057699554,31cyclictest0-21swapper/208:40:082
20576995539,10cyclictest3063-21tr11:24:532
2057599553,44cyclictest0-21swapper/111:19:351
20575995523,13cyclictest0-21swapper/110:14:381
20575995523,13cyclictest0-21swapper/110:14:381
2057399556,29cyclictest0-21swapper/009:59:530
188312550,5sleep218834-21unixbench_singl09:40:152
80382543,9sleep00-21swapper/011:34:520
80382543,9sleep00-21swapper/011:34:520
324062543,43sleep00-21swapper/007:55:080
223099540,6rtkit-daemon2229-21rtkit-daemon10:29:520
2057699546,41cyclictest0-21swapper/206:55:072
2057699544,30cyclictest0-21swapper/207:35:192
20576995420,13cyclictest0-21swapper/210:40:042
2057399543,45cyclictest0-21swapper/007:08:340
195872543,10sleep20-21swapper/210:50:092
36292533,10sleep00-21swapper/008:03:220
223099530,6rtkit-daemon2229-21rtkit-daemon11:24:571
213752533,10sleep20-21swapper/209:46:172
2057699534,30cyclictest0-21swapper/208:25:132
2057699534,29cyclictest0-21swapper/208:19:442
20576995336,10cyclictest1961-21unixbench_singl11:20:092
2057399535,29cyclictest0-21swapper/011:24:350
27442523,41sleep00-21swapper/009:09:480
261552522,9sleep20-21swapper/209:56:252
209002523,10sleep00-21swapper/009:45:140
2057699525,28cyclictest0-21swapper/211:35:052
2057699525,28cyclictest0-21swapper/211:35:052
2057699524,41cyclictest0-21swapper/209:29:582
20575995234,11cyclictest7513-21tail10:25:041
20575995234,11cyclictest15376-21cut11:49:531
2057399524,28cyclictest0-21swapper/006:35:120
20573995234,11cyclictest5446-21tr11:29:530
20573995220,12cyclictest18370-21tail10:49:520
221072512,41sleep10-21swapper/107:34:571
2057699515,27cyclictest0-21swapper/210:54:482
2057699515,27cyclictest0-21swapper/210:54:482
2057699515,25cyclictest3743-21munin-run08:04:432
2057699514,28cyclictest0-21swapper/211:04:582
2057699514,28cyclictest0-21swapper/210:25:102
2057699514,27cyclictest0-21swapper/209:49:582
20576995121,12cyclictest0-21swapper/209:20:082
20576995120,13cyclictest0-21swapper/210:30:102
20576995120,12cyclictest0-21swapper/208:49:552
2057599514,40cyclictest0-21swapper/111:45:061
2057399515,29cyclictest0-21swapper/011:05:090
2057399515,28cyclictest0-21swapper/009:04:560
2057399515,28cyclictest0-21swapper/008:45:120
2057399515,28cyclictest0-21swapper/008:45:120
2057399514,29cyclictest0-21swapper/007:50:100
20573995134,11cyclictest8072-21grep08:10:160
2057399513,42cyclictest0-21swapper/010:40:120
20573995123,23cyclictest0-21swapper/008:15:070
177472513,10sleep20-21swapper/210:46:182
143222513,42sleep00-21swapper/009:32:490
118052513,42sleep00-21swapper/011:42:060
223099500,4rtkit-daemon2229-21rtkit-daemon09:53:080
2057699507,34cyclictest26-21rcuc/207:09:572
2057699505,25cyclictest0-21swapper/210:05:562
2057699504,28cyclictest0-21swapper/210:35:072
2057699504,28cyclictest0-21swapper/210:20:082
2057699504,28cyclictest0-21swapper/209:24:592
2057699504,28cyclictest0-21swapper/209:24:592
2057699504,28cyclictest0-21swapper/208:35:102
2057699504,28cyclictest0-21swapper/207:25:222
2057699504,28cyclictest0-21swapper/207:25:222
2057699504,28cyclictest0-21swapper/207:00:162
2057699504,27cyclictest0-21swapper/209:10:032
2057699504,27cyclictest0-21swapper/208:45:142
2057699504,27cyclictest0-21swapper/208:45:142
2057699504,27cyclictest0-21swapper/208:04:562
2057699504,27cyclictest0-21swapper/208:04:562
2057699504,27cyclictest0-21swapper/207:45:032
2057699504,27cyclictest0-21swapper/207:05:072
2057699504,12cyclictest0-21swapper/211:48:402
20576995033,11cyclictest1514-21strings06:50:142
20576995033,11cyclictest1514-21strings06:50:142
20576995020,12cyclictest0-21swapper/210:10:002
20576995020,12cyclictest0-21swapper/210:10:002
2057699502,35cyclictest0-21swapper/206:25:172
2057599504,40cyclictest0-21swapper/107:55:351
2057399505,27cyclictest0-21swapper/009:25:120
2057399505,27cyclictest0-21swapper/009:25:120
2057399505,27cyclictest0-21swapper/008:59:540
2057399505,27cyclictest0-21swapper/008:20:130
2057399505,27cyclictest0-21swapper/006:40:030
2057399504,27cyclictest12007-21hwlatdetect10:35:010
2057399504,27cyclictest0-21swapper/010:04:570
20573995034,10cyclictest16341-21unixbench_multi11:50:080
20573995033,11cyclictest28748-21gltestperf08:55:020
180672503,10sleep00-21swapper/007:25:090
180672503,10sleep00-21swapper/007:25:090
180452503,40sleep00-21swapper/008:34:130
170942504,38sleep00-21swapper/009:39:540
165992503,40sleep00-21swapper/007:20:340
120392502,9sleep10-21swapper/107:13:371
71552493,40sleep00-21swapper/007:02:120
60352493,39sleep00-21swapper/008:08:340
60352493,39sleep00-21swapper/008:08:340
321562493,9sleep20-21swapper/211:17:092
297512492,9sleep10-21swapper/111:12:081
250292493,40sleep00-21swapper/006:30:560
21732493,9sleep10-21swapper/109:05:461
2057699495,27cyclictest0-21swapper/206:45:032
2057699495,27cyclictest0-21swapper/206:45:032
2057699494,28cyclictest0-21swapper/207:50:172
2057699494,27cyclictest0-21swapper/210:00:052
2057699494,27cyclictest0-21swapper/209:15:012
2057699494,27cyclictest0-21swapper/209:04:412
2057699494,27cyclictest0-21swapper/208:09:562
2057699494,27cyclictest0-21swapper/206:35:082
2057699494,26cyclictest0-21swapper/211:30:102
2057699494,26cyclictest0-21swapper/207:40:082
2057699493,27cyclictest0-21swapper/211:40:022
20576994921,11cyclictest0-21swapper/211:49:462
20576994919,10cyclictest1265-21runrttasks10:59:492
2057399494,27cyclictest0-21swapper/009:15:080
2057399494,27cyclictest0-21swapper/007:30:060
2057399494,26cyclictest0-21swapper/006:45:080
2057399494,26cyclictest0-21swapper/006:45:080
20573994919,9cyclictest24095-21latency_hist07:39:450
20573994919,10cyclictest20479-21latency_hist08:39:420
165542493,40sleep00-21swapper/009:35:470
130202492,10sleep10-21swapper/110:36:541
95472482,9sleep10-21swapper/111:38:441
95472482,9sleep10-21swapper/111:38:441
309492482,10sleep10-21swapper/110:06:431
297852483,10sleep00-21swapper/011:12:360
263472483,39sleep00-21swapper/009:58:560
25612483,39sleep00-21swapper/006:54:340
25612483,39sleep00-21swapper/006:54:340
238312482,40sleep10-21swapper/109:52:121
213752483,9sleep20-21swapper/207:30:272
2057699484,27cyclictest0-21swapper/208:55:132
2057699484,26cyclictest0-21swapper/208:30:042
2057699484,26cyclictest0-21swapper/206:32:502
20576994831,10cyclictest28163-21tr11:09:502
20576994817,12cyclictest0-21swapper/209:05:082
20575994835,8cyclictest20148-21fschecks_count07:30:011
20573994818,12cyclictest0-21swapper/006:55:080
158442482,9sleep10-21swapper/110:44:401
322422472,39sleep10-21swapper/109:02:311
322322473,10sleep10-21swapper/106:46:331
322322473,10sleep10-21swapper/106:46:331
226962472,38sleep10-21swapper/108:42:281
223099470,4rtkit-daemon2229-21rtkit-daemon10:24:590
2057699475,25cyclictest0-21swapper/209:38:202
2057699473,27cyclictest0-21swapper/207:19:472
2057599473,37cyclictest0-21swapper/108:04:571
2057599473,37cyclictest0-21swapper/108:04:571
2057599473,36cyclictest5367-21tail07:00:011
2057399478,14cyclictest0-21swapper/010:20:070
2057399475,12cyclictest22956-21chrt06:29:460
20573994732,10cyclictest14420-21grep08:25:010
20573994728,13cyclictest0-21swapper/007:19:450
20573994720,12cyclictest0-21swapper/011:24:510
20573994717,12cyclictest0-21swapper/011:14:490
20573994715,14cyclictest0-21swapper/008:40:130
191142472,9sleep10-21swapper/107:27:121
191142472,9sleep10-21swapper/107:27:121
168622472,40sleep10-21swapper/107:24:011
154372472,38sleep10-21swapper/108:26:161
33912462,38sleep10-21swapper/110:15:541
298502462,9sleep10-21swapper/108:57:231
275222462,10sleep10-21swapper/111:09:101
270682462,9sleep10-21swapper/109:59:541
267062463,10sleep00-21swapper/007:44:480
264212462,9sleep10-21swapper/107:44:041
251062463,38sleep00-21swapper/011:03:490
2057699464,25cyclictest0-21swapper/206:42:512
2057599465,33cyclictest0-21swapper/107:19:451
2057599464,36cyclictest0-21swapper/108:30:031
20575994633,7cyclictest13322-21munin-run08:24:421
20575994632,9cyclictest29403-21cpuspeed_turbos07:50:011
2057399465,35cyclictest0-21swapper/010:14:500
2057399465,32cyclictest0-21swapper/007:09:560
94602452,37sleep10-21swapper/107:06:041
83522452,9sleep10-21swapper/108:12:381
60512453,36sleep10-21swapper/110:24:251
273592453,36sleep10-21swapper/108:51:031
229102452,9sleep10-21swapper/106:29:241
226452452,37sleep10-21swapper/110:57:501
226452452,37sleep10-21swapper/110:57:501
20576994517,11cyclictest0-21swapper/207:58:032
2057599454,35cyclictest0-21swapper/109:29:521
2057599454,34cyclictest0-21swapper/109:19:531
2057399454,34cyclictest0-21swapper/007:44:450
117632452,9sleep10-21swapper/109:25:321
117632452,9sleep10-21swapper/109:25:321
299742442,10sleep10-21swapper/106:43:141
25202442,9sleep10-21swapper/106:54:021
25202442,9sleep10-21swapper/106:54:021
250972442,9sleep10-21swapper/106:31:501
223099440,4rtkit-daemon2229-21rtkit-daemon11:30:041
2057599444,35cyclictest0-21swapper/108:19:331
20573994427,10cyclictest12591-21grep11:44:510
202992444,33sleep00-21swapper/006:21:380
202232444,33sleep20-21swapper/206:20:392
61662433,7sleep16264-21memory09:15:051
213992432,9sleep10-21swapper/109:46:361
2057599434,34cyclictest0-21swapper/111:40:041
2057599433,36cyclictest0-21swapper/109:58:091
2057599433,33cyclictest0-21swapper/110:59:561
202012432,10sleep10-21swapper/110:52:091
47132422,10sleep10-21swapper/109:12:151
223099420,5rtkit-daemon2229-21rtkit-daemon09:37:541
223099420,4rtkit-daemon2229-21rtkit-daemon09:40:011
2057599424,32cyclictest0-21swapper/108:48:121
2057599424,32cyclictest0-21swapper/108:48:121
20575994216,9cyclictest5070-21cat06:59:461
107262422,10sleep10-21swapper/110:33:071
254602412,31sleep10-21swapper/106:34:481
20575994121,14cyclictest0-21swapper/110:44:361
2057599403,32cyclictest0-21swapper/108:39:421
2057599364,11cyclictest0-21swapper/106:24:471
20577993419,8cyclictest20589-21sort06:24:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional