You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 12:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Sun Nov 16, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2172721100,5sleep334-21rcuc/319:44:543
243799720,5rtkit-daemon2436-21rtkit-daemon20:05:163
2649996945,19cyclictest0-21swapper/120:10:121
243799680,6rtkit-daemon0-21swapper/200:28:583
134832663,12sleep30-21swapper/322:50:013
265199658,48cyclictest4110-21proc_pri21:20:183
264999654,52cyclictest0-21swapper/122:13:331
123092622,9sleep00-21swapper/022:45:150
294702613,13sleep20-21swapper/220:00:092
129092610,3sleep013031-21fschecks_time23:55:030
265199605,48cyclictest31701-21rt-features21:10:213
265199605,48cyclictest31701-21rt-features21:10:213
41312593,12sleep30-21swapper/322:30:023
2651995934,11cyclictest28809-21yum-updatesd-he19:58:503
2649995935,11cyclictest0-21swapper/121:07:331
2649995833,12cyclictest0-21swapper/119:34:541
2648995830,13cyclictest0-21swapper/021:25:120
243799580,6rtkit-daemon2436-21rtkit-daemon20:50:223
2651995729,12cyclictest0-21swapper/322:49:483
2649995740,12cyclictest0-21swapper/119:50:291
2649995738,14cyclictest0-21swapper/123:05:131
2649995738,12cyclictest0-21swapper/122:00:181
264999573,48cyclictest0-21swapper/119:39:551
2651995628,12cyclictest0-21swapper/300:24:293
2649995635,12cyclictest0-21swapper/123:20:031
2649995634,10cyclictest0-21swapper/121:20:161
2649995633,11cyclictest0-21swapper/123:55:111
264999563,48cyclictest0-21swapper/121:15:091
2649995621,12cyclictest0-21swapper/121:40:041
264999562,48cyclictest0-21swapper/121:59:491
264999562,48cyclictest0-21swapper/121:59:491
218922563,46sleep321920-21telnet22:00:003
156162563,45sleep20-21swapper/221:45:122
297402553,16sleep00-21swapper/023:21:500
2651995527,13cyclictest0-21swapper/322:15:263
2651995524,12cyclictest0-21swapper/323:14:463
2650995538,10cyclictest16272-21cpuspeed_turbos20:40:092
2650995520,28cyclictest6782-21cstates20:20:092
2649995532,11cyclictest0-21swapper/123:10:061
2649995522,24cyclictest9475-21fschecks_time20:25:161
264999552,48cyclictest0-21swapper/120:45:101
243799550,6rtkit-daemon2436-21rtkit-daemon20:28:153
243799550,5rtkit-daemon2436-21rtkit-daemon20:58:523
265199543,9cyclictest19087-21sshd00:08:203
265199543,9cyclictest19087-21sshd00:08:203
2651995426,12cyclictest0-21swapper/320:14:533
2651995425,12cyclictest0-21swapper/319:06:423
2651995425,12cyclictest0-21swapper/319:06:423
2649995432,11cyclictest0-21swapper/122:20:041
264999542,46cyclictest0-21swapper/123:15:091
264999542,46cyclictest0-21swapper/100:19:441
2648995428,8cyclictest31287-21latency_hist20:04:540
174732543,44sleep10-21swapper/100:05:031
174732543,44sleep10-21swapper/100:05:031
327572534,6sleep30-21swapper/319:00:103
2651995324,13cyclictest0-21swapper/319:34:403
265099535,38cyclictest0-21swapper/219:05:202
265099535,38cyclictest0-21swapper/219:05:202
265099535,32cyclictest0-21swapper/221:16:202
2650995338,10cyclictest27483-21grep00:25:052
2650995337,10cyclictest9231-21unixbench_singl23:45:182
264999535,31cyclictest0-21swapper/100:01:461
2649995332,11cyclictest0-21swapper/123:35:011
2649995331,10cyclictest0-21swapper/100:20:061
2649995330,12cyclictest0-21swapper/123:25:081
2649995330,12cyclictest0-21swapper/123:25:081
264999533,46cyclictest0-21swapper/120:19:521
2649995324,14cyclictest0-21swapper/122:45:231
2648995334,13cyclictest0-21swapper/023:43:200
2648995331,11cyclictest0-21swapper/019:20:100
2648995330,11cyclictest0-21swapper/020:10:240
243799530,4rtkit-daemon2436-21rtkit-daemon20:35:223
13502532,43sleep00-21swapper/023:30:120
2651995222,12cyclictest0-21swapper/321:09:513
2650995237,10cyclictest11869-21grep22:45:072
2650995236,9cyclictest9099-21grep20:25:112
2650995232,13cyclictest17952-21head21:50:102
264999526,28cyclictest0-21swapper/120:51:151
264999524,41cyclictest0-21swapper/121:50:021
2649995227,8cyclictest9883-21cat19:19:561
2649995226,10cyclictest15866-21munin-node22:55:161
2649995224,14cyclictest0-21swapper/120:40:301
2649995224,12cyclictest0-21swapper/122:05:011
2649995223,14cyclictest0-21swapper/119:41:041
2649995221,13cyclictest0-21swapper/100:10:181
2648995237,10cyclictest0-21swapper/021:05:200
2648995225,8cyclictest15633-21latency_hist22:54:470
2648995223,10cyclictest27590-21latency_hist23:19:460
243799520,5rtkit-daemon2436-21rtkit-daemon21:19:513
243799520,5rtkit-daemon2436-21rtkit-daemon20:45:153
243799520,5rtkit-daemon2436-21rtkit-daemon20:15:083
213592524,41sleep20-21swapper/220:50:152
178552522,43sleep00-21swapper/020:41:500
167152523,41sleep00-21swapper/019:32:020
96072512,43sleep00-21swapper/023:49:270
81892513,42sleep00-21swapper/022:36:080
60932513,42sleep00-21swapper/020:19:010
306822510,4sleep1201ktimersoftd/100:30:171
2651995136,9cyclictest3031-21cpuspeed_turbos23:34:593
265199513,9cyclictest11281-21munin-node19:20:263
265099514,42cyclictest0-21swapper/222:20:192
265099514,16cyclictest0-21swapper/219:15:142
2650995136,9cyclictest6808-21grep23:40:202
2650995136,10cyclictest22339-21unixbench-2d23:05:202
2650995135,10cyclictest20281-21grep21:55:112
2650995135,10cyclictest20281-21grep21:55:112
2650995134,9cyclictest22589-21taskset00:15:032
264999514,30cyclictest0-21swapper/119:21:011
2649995133,12cyclictest0-21swapper/119:46:171
2649995133,12cyclictest0-21swapper/119:46:171
2649995123,13cyclictest0-21swapper/120:00:171
2648995136,10cyclictest0-21swapper/020:32:570
2648995136,10cyclictest0-21swapper/020:32:570
264899513,42cyclictest9980-21telnet21:35:040
264899513,41cyclictest5413-21telnet19:10:110
2648995129,11cyclictest0-21swapper/021:10:140
2648995129,11cyclictest0-21swapper/021:10:140
2648995125,7cyclictest2548-21latency_hist21:19:510
2648995112,32cyclictest0-21swapper/019:45:110
2648995112,32cyclictest0-21swapper/019:45:110
243799510,5rtkit-daemon2436-21rtkit-daemon21:34:133
243799510,5rtkit-daemon2436-21rtkit-daemon20:30:453
243799510,5rtkit-daemon2436-21rtkit-daemon20:30:453
191212512,42sleep00-21swapper/019:37:180
166122512,43sleep00-21swapper/021:46:350
144132512,42sleep00-21swapper/019:28:070
58642502,42sleep00-21swapper/022:31:570
48102502,9sleep00-21swapper/019:07:190
48102502,9sleep00-21swapper/019:07:190
39262502,10sleep20-21swapper/222:29:492
321102502,9sleep20-21swapper/223:26:362
321102502,9sleep20-21swapper/223:26:362
309112502,9sleep30-21swapper/300:31:543
265199509,35cyclictest0-21swapper/322:40:173
265199509,35cyclictest0-21swapper/322:40:173
2651995036,9cyclictest30055-21hwlatdetect20:00:153
265099506,39cyclictest0-21swapper/200:10:132
265099504,41cyclictest0-21swapper/222:15:202
265099504,40cyclictest0-21swapper/220:30:262
265099504,40cyclictest0-21swapper/220:30:262
265099504,40cyclictest0-21swapper/219:45:172
265099504,40cyclictest0-21swapper/219:45:172
2650995034,10cyclictest454-21cpuspeed_turbos23:30:012
264999503,40cyclictest0-21swapper/121:25:041
2649995029,12cyclictest0-21swapper/123:45:071
2649995025,11cyclictest0-21swapper/100:26:501
2649995024,11cyclictest0-21swapper/120:20:071
2649995024,10cyclictest32595-21munin-node23:29:581
2649995023,12cyclictest0-21swapper/122:34:481
2649995023,11cyclictest0-21swapper/123:00:551
2649995022,12cyclictest0-21swapper/122:40:151
2649995022,12cyclictest0-21swapper/122:40:151
2649995022,12cyclictest0-21swapper/119:28:201
2648995031,13cyclictest0-21swapper/020:05:110
2648995028,11cyclictest0-21swapper/021:31:590
2648995027,7cyclictest25201-21latency_hist23:14:460
2648995027,13cyclictest0-21swapper/021:40:030
2648995026,20cyclictest0-21swapper/000:25:200
2648995022,10cyclictest21661-21latency_hist21:59:490
2648995022,10cyclictest21661-21latency_hist21:59:490
179032502,9sleep30-21swapper/320:42:303
73282492,9sleep30-21swapper/319:14:033
47762493,9sleep10-21swapper/119:06:521
47762493,9sleep10-21swapper/119:06:521
323092493,39sleep10-21swapper/121:13:111
323092493,39sleep10-21swapper/121:13:111
321082493,41sleep00-21swapper/023:26:340
321082493,41sleep00-21swapper/023:26:340
299872492,9sleep20-21swapper/221:09:012
299292492,9sleep30-21swapper/323:24:193
288132492,9sleep00-21swapper/019:58:500
288072493,40sleep10-21swapper/119:58:471
284022493,8sleep228442-21tune2fs23:20:072
2651994933,10cyclictest13211-21gltestperf21:40:113
265199493,27cyclictest0-21swapper/323:50:053
2651994922,12cyclictest0-21swapper/319:54:543
2651994920,12cyclictest0-21swapper/322:58:593
265199492,11cyclictest0-21swapper/321:35:233
265099497,36cyclictest0-21swapper/220:35:192
265099495,39cyclictest0-21swapper/220:45:202
265099494,40cyclictest0-21swapper/223:55:062
265099494,40cyclictest0-21swapper/200:20:132
2650994934,10cyclictest27269-21grep22:10:052
2650994932,11cyclictest12529-21aten2.4-expect19:25:072
2649994933,12cyclictest0-21swapper/121:04:421
2649994925,11cyclictest0-21swapper/122:35:001
2649994924,12cyclictest0-21swapper/119:10:321
2649994923,11cyclictest0-21swapper/122:50:071
2649994921,12cyclictest0-21swapper/120:35:501
264999492,42cyclictest0-21swapper/121:47:171
264899493,41cyclictest0-21swapper/000:18:050
2648994927,11cyclictest0-21swapper/022:15:190
263932492,9sleep30-21swapper/322:09:193
251762492,40sleep00-21swapper/020:58:390
243799490,5rtkit-daemon2436-21rtkit-daemon20:23:153
192352492,9sleep30-21swapper/319:38:463
95702482,41sleep00-21swapper/019:17:110
9252483,39sleep10-21swapper/120:06:091
81812482,9sleep20-21swapper/222:36:022
48412483,9sleep00-21swapper/021:23:380
45932482,40sleep00-21swapper/023:36:250
40042482,9sleep20-21swapper/220:14:562
311042483,9sleep00-21swapper/000:34:240
2651994834,9cyclictest16006-21wc00:00:133
2651994832,9cyclictest22256-21cpuspeed_turbos19:45:083
2651994832,9cyclictest22256-21cpuspeed_turbos19:45:083
2651994832,10cyclictest13260-21cut19:25:143
265199482,11cyclictest0-21swapper/322:21:363
265199482,11cyclictest0-21swapper/322:10:083
265199482,10cyclictest0-21swapper/323:48:573
265099486,37cyclictest0-21swapper/221:40:052
265099485,39cyclictest0-21swapper/221:30:192
265099485,39cyclictest0-21swapper/200:30:102
265099485,36cyclictest1243-21snmpd20:55:072
2650994833,9cyclictest20246-21grep19:40:142
2650994833,8cyclictest24401-21aten2.4-expect19:50:092
265099483,38cyclictest0-21swapper/222:55:062
264999484,24cyclictest12019-21crond23:54:431
2649994822,12cyclictest0-21swapper/121:34:501
2648994830,13cyclictest0-21swapper/023:09:460
2648994829,13cyclictest0-21swapper/019:50:290
212742482,38sleep00-21swapper/020:50:130
15252482,9sleep20-21swapper/222:25:022
285442472,39sleep00-21swapper/022:11:170
265199474,26cyclictest0-21swapper/321:49:493
265199473,38cyclictest0-21swapper/321:25:033
265099474,38cyclictest0-21swapper/221:20:212
265099474,38cyclictest0-21swapper/220:10:142
265099474,14cyclictest0-21swapper/221:25:112
2650994731,10cyclictest17736-21grep00:05:062
2650994731,10cyclictest17736-21grep00:05:062
2650994716,23cyclictest5417-21cpu19:10:112
264999475,37cyclictest0-21swapper/121:39:491
2649994730,11cyclictest2310-21gltestperf22:25:081
2649994723,11cyclictest0-21swapper/120:31:111
2649994723,11cyclictest0-21swapper/120:31:111
2649994722,9cyclictest5866-21perl23:40:061
2648994730,12cyclictest0-21swapper/021:50:120
264899473,38cyclictest19565-21munin-node19:40:070
2648994725,11cyclictest0-21swapper/000:14:440
202852472,9sleep20-21swapper/223:03:062
189832472,9sleep00-21swapper/000:07:000
189832472,9sleep00-21swapper/000:07:000
118292472,9sleep20-21swapper/221:36:252
107082473,9sleep00-21swapper/020:26:590
105992472,9sleep20-21swapper/222:41:302
105992472,9sleep20-21swapper/222:41:302
105972473,39sleep00-21swapper/022:41:280
105972473,39sleep00-21swapper/022:41:280
69742462,9sleep30-21swapper/323:41:243
30022462,38sleep00-21swapper/022:25:200
273242462,39sleep00-21swapper/021:00:330
2651994632,8cyclictest361ktimersoftd/322:35:093
2651994631,10cyclictest2482-21ls22:25:093
265099466,34cyclictest0-21swapper/219:58:512
265099464,25cyclictest26447-21turbostat22:09:482
265099464,14cyclictest0-21swapper/221:10:082
265099464,14cyclictest0-21swapper/221:10:082
2650994631,9cyclictest18316-21chrt19:35:212
2649994622,12cyclictest0-21swapper/120:55:031
2649994621,12cyclictest0-21swapper/122:15:101
2648994631,10cyclictest11331-21swap23:50:160
2648994629,12cyclictest0-21swapper/020:38:340
2648994625,14cyclictest0-21swapper/023:00:150
264899462,37cyclictest31506-21telnet22:20:010
2648994619,7cyclictest16815-21latency_hist00:04:440
220662462,37sleep30-21swapper/323:05:173
210202464,34sleep30-21swapper/321:55:253
210202464,34sleep30-21swapper/321:55:253
201732462,39sleep00-21swapper/020:45:580
83412452,38sleep00-21swapper/020:22:160
2651994531,9cyclictest27230-21tail21:00:253
265199453,25cyclictest0-21swapper/323:00:173
2651994529,10cyclictest18167-21ls21:50:143
2651994511,21cyclictest0-21swapper/323:24:583
2651994511,21cyclictest0-21swapper/323:24:583
265099456,35cyclictest0-21swapper/221:00:052
265099456,29cyclictest0-21swapper/222:00:032
265099455,35cyclictest0-21swapper/219:21:012
265099454,36cyclictest0-21swapper/223:35:092
265099454,36cyclictest0-21swapper/223:19:452
265099454,35cyclictest32373-21head20:05:152
264899453,37cyclictest0-21swapper/022:00:100
264899452,26cyclictest0-21swapper/022:05:010
249502452,9sleep20-21swapper/223:11:372
97092442,9sleep30-21swapper/319:18:593
265199443,36cyclictest0-21swapper/300:15:153
265199443,25cyclictest0-21swapper/300:10:143
265199442,12cyclictest0-21swapper/323:17:413
2648994427,12cyclictest0-21swapper/022:55:230
20202442,35sleep30-21swapper/323:30:363
166352442,9sleep20-21swapper/200:02:312
2651994318,12cyclictest0-21swapper/323:54:493
265099434,32cyclictest0-21swapper/222:50:102
2650994330,8cyclictest15095-21cpuspeed_turbos19:30:112
2648994325,13cyclictest0-21swapper/000:20:300
2648994321,10cyclictest0-21swapper/019:04:560
265099423,9cyclictest0-21swapper/223:53:332
264999424,32cyclictest0-21swapper/119:04:561
22892424,31sleep20-21swapper/219:01:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional