You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 09:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Wed Dec 17, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2789621230,6sleep027898-21grep19:41:560
2345121130,9sleep20-21swapper/200:06:122
926799974,75cyclictest0-21swapper/022:36:400
107112900,6sleep310710-21awk21:21:473
9267998816,49cyclictest0-21swapper/022:41:270
926799874,65cyclictest0-21swapper/021:41:450
926799864,66cyclictest0-21swapper/022:56:290
926799864,59cyclictest0-21swapper/022:31:310
926799863,67cyclictest0-21swapper/020:41:200
926799854,74cyclictest0-21swapper/021:22:000
926799854,66cyclictest0-21swapper/020:51:470
926799854,64cyclictest0-21swapper/000:23:430
9267998515,55cyclictest0-21swapper/023:01:310
9267998512,9cyclictest121850irq/46-eth019:21:500
926799844,72cyclictest0-21swapper/023:36:350
926799844,72cyclictest0-21swapper/023:36:350
926799844,64cyclictest0-21swapper/022:16:340
926799843,66cyclictest0-21swapper/023:06:430
926799833,22cyclictest0-21swapper/000:03:400
9267998316,8cyclictest28585-21munin-node19:46:490
9267998315,8cyclictest1419-21runrttasks22:28:380
926799824,14cyclictest0-21swapper/019:36:550
926799823,57cyclictest0-21swapper/020:12:040
926799823,21cyclictest0-21swapper/020:48:080
926799814,56cyclictest0-21swapper/020:16:390
310182810,5sleep041ktimersoftd/022:06:290
926799803,56cyclictest0-21swapper/020:01:380
926799803,19cyclictest0-21swapper/022:05:140
9267998023,51cyclictest1419-21runrttasks23:58:500
9267998021,52cyclictest12700-21grep19:11:420
9267998020,53cyclictest14776-21grep20:21:540
9267998018,7cyclictest1287-21snmpd21:28:150
926799793,66cyclictest7706-21cut21:16:370
9267997915,49cyclictest0-21swapper/023:46:260
9267997914,8cyclictest25894-21sadc21:56:170
926799784,67cyclictest0-21swapper/021:01:410
926799784,14cyclictest0-21swapper/023:41:320
926799784,13cyclictest0-21swapper/000:11:390
926799784,12cyclictest0-21swapper/019:06:500
926799783,68cyclictest6661-21grep20:06:420
926799783,68cyclictest6661-21grep20:06:420
926799783,60cyclictest0-21swapper/000:16:410
926799783,60cyclictest0-21swapper/000:16:410
9267997829,44cyclictest5758-21needreboot23:26:390
9267997827,45cyclictest22869-21tr20:41:330
9267997827,45cyclictest22869-21tr20:41:330
9267997825,47cyclictest15743-21cpuspeed_turbos20:26:350
9267997821,50cyclictest20121-21munin-node22:51:440
9267997821,50cyclictest20121-21munin-node22:51:440
926799774,66cyclictest0-21swapper/020:31:480
926799774,13cyclictest0-21swapper/000:26:290
926799773,68cyclictest0-21swapper/020:59:250
926799773,23cyclictest0-21swapper/022:11:430
926799773,23cyclictest0-21swapper/022:11:430
9267997726,45cyclictest1419-21runrttasks19:28:390
9267997726,45cyclictest1419-21runrttasks19:28:390
9267997722,49cyclictest1419-21runrttasks23:13:490
9267997711,12cyclictest0-21swapper/019:32:540
926799766,8cyclictest214181telnet21:46:300
926799764,15cyclictest0-21swapper/000:06:340
926799764,12cyclictest0-21swapper/022:46:490
9267997625,44cyclictest31310-21cpuspeed_turbos19:51:350
9267997623,48cyclictest1419-21runrttasks21:59:050
9267997620,50cyclictest17265-21fschecks_time23:51:320
926799754,15cyclictest0-21swapper/021:36:180
926799754,15cyclictest0-21swapper/021:36:180
926799753,65cyclictest14261-21munin-node19:16:470
926799744,14cyclictest0-21swapper/021:36:510
926799744,13cyclictest0-21swapper/022:21:490
926799744,13cyclictest0-21swapper/019:06:230
926799743,64cyclictest0-21swapper/023:31:340
9267997421,48cyclictest1419-21runrttasks00:35:370
9267997413,8cyclictest1419-21runrttasks23:17:280
108812740,4sleep118-21rcuc/123:36:451
108812740,4sleep118-21rcuc/123:36:451
926799734,63cyclictest0-21swapper/023:21:310
926799734,27cyclictest0-21swapper/021:11:360
9267997314,8cyclictest1287-21snmpd21:08:110
9267997118,7cyclictest1141-21cpuspeed19:56:340
248599710,8rtkit-daemon0-21swapper/019:06:581
248599680,7rtkit-daemon0-21swapper/019:24:581
2142526713,47sleep30-21swapper/319:31:373
248599650,8rtkit-daemon0-21swapper/019:16:581
248599630,9rtkit-daemon0-21swapper/019:13:431
248599620,8rtkit-daemon0-21swapper/019:26:281
248599620,8rtkit-daemon0-21swapper/019:26:281
6512613,49sleep20-21swapper/219:56:222
9269995943,10cyclictest26402-21cpuspeed_turbos00:11:242
926899594,37cyclictest0-21swapper/100:16:391
926899594,37cyclictest0-21swapper/100:16:391
9268995917,23cyclictest25497-21unixbench_multi19:36:561
926899584,20cyclictest0-21swapper/120:26:331
268562582,18sleep20-21swapper/220:48:052
9268995716,14cyclictest0-21swapper/123:41:251
293052572,11sleep10-21swapper/119:46:421
115032573,12sleep20-21swapper/223:41:142
115032573,12sleep20-21swapper/223:41:142
926899564,32cyclictest0-21swapper/122:26:271
9268995622,13cyclictest0-21swapper/121:41:191
297152563,9sleep129810-21sh23:11:271
159592563,46sleep10-21swapper/123:47:141
926899555,30cyclictest0-21swapper/100:31:431
926899554,44cyclictest0-21swapper/122:21:461
926899553,27cyclictest0-21swapper/121:46:481
9268995524,10cyclictest17863-21telnet22:46:271
9268995518,13cyclictest0-21swapper/120:41:451
9268995518,13cyclictest0-21swapper/120:41:451
9269995437,10cyclictest24561-21grep21:51:372
926899546,28cyclictest0-21swapper/100:21:251
926899545,30cyclictest0-21swapper/119:46:091
926899545,29cyclictest0-21swapper/122:36:311
926899545,29cyclictest0-21swapper/121:16:431
926899544,28cyclictest0-21swapper/100:11:131
926899544,27cyclictest0-21swapper/121:26:391
926899544,27cyclictest0-21swapper/120:11:221
926899544,27cyclictest0-21swapper/120:11:221
926899544,26cyclictest0-21swapper/123:31:441
926899544,26cyclictest0-21swapper/121:58:201
926899544,25cyclictest0-21swapper/122:06:341
9268995421,13cyclictest0-21swapper/123:26:331
9268995421,13cyclictest0-21swapper/122:51:371
9268995421,13cyclictest0-21swapper/122:51:371
9268995420,13cyclictest0-21swapper/120:36:481
9268995420,13cyclictest0-21swapper/120:16:571
9268995419,13cyclictest0-21swapper/123:51:441
9268995419,13cyclictest0-21swapper/122:41:391
9268995419,13cyclictest0-21swapper/119:31:491
9268995419,13cyclictest0-21swapper/100:26:291
9268995419,12cyclictest0-21swapper/123:21:321
9268995418,13cyclictest0-21swapper/121:51:351
9268995417,13cyclictest0-21swapper/120:01:581
100452543,9sleep30-21swapper/322:27:363
927099535,40cyclictest9216-21cat23:36:143
9270995338,10cyclictest4351-21sed20:01:423
927099533,42cyclictest1287-21snmpd22:23:243
926899535,29cyclictest0-21swapper/121:31:351
926899535,29cyclictest0-21swapper/121:31:351
926899535,29cyclictest0-21swapper/121:11:521
926899535,29cyclictest0-21swapper/121:01:531
926899535,29cyclictest0-21swapper/120:11:521
926899534,30cyclictest0-21swapper/122:11:341
926899534,30cyclictest0-21swapper/122:11:341
926899534,27cyclictest0-21swapper/123:06:151
926899534,27cyclictest0-21swapper/121:46:051
926899534,26cyclictest0-21swapper/120:53:091
926899534,26cyclictest0-21swapper/120:36:201
9268995322,13cyclictest0-21swapper/122:01:381
9268995322,12cyclictest0-21swapper/100:01:331
9268995320,15cyclictest0-21swapper/121:21:471
9268995320,10cyclictest13594-21diskstats20:21:371
9268995316,13cyclictest0-21swapper/119:56:541
79792533,9sleep30-21swapper/320:11:063
79792533,9sleep30-21swapper/320:11:063
64932532,10sleep30-21swapper/323:28:303
64402532,10sleep20-21swapper/221:11:592
248599530,5rtkit-daemon2484-21rtkit-daemon21:01:312
248599530,5rtkit-daemon2484-21rtkit-daemon19:36:352
240112534,41sleep324039-21ls20:41:503
240112534,41sleep324039-21ls20:41:503
126032533,10sleep10-21swapper/122:34:521
927099525,41cyclictest1287-21snmpd21:43:173
9270995237,10cyclictest1471-21df19:56:363
9270995236,9cyclictest13380-21cpuspeed_turbos20:21:353
9270995236,10cyclictest29518-21kernelversion00:16:353
9270995236,10cyclictest29518-21kernelversion00:16:353
927099523,42cyclictest16579-21munin-node21:36:383
926999525,40cyclictest21088-21yum-updatesd-he21:46:182
9269995236,10cyclictest27395-21tail19:41:482
926999523,30cyclictest0-21swapper/220:51:332
926899525,27cyclictest0-21swapper/122:56:521
926899525,27cyclictest0-21swapper/122:16:451
926899524,28cyclictest0-21swapper/123:11:151
9268995220,13cyclictest0-21swapper/123:16:491
9268995212,20cyclictest0-21swapper/121:06:321
248599520,5rtkit-daemon2484-21rtkit-daemon23:48:553
101612522,9sleep20-21swapper/222:29:092
9270995137,8cyclictest30680-21sh22:06:173
9270995135,10cyclictest18041-21cpuspeed_turbos22:46:293
9270995117,27cyclictest31518-21df_abs19:51:373
926899515,27cyclictest0-21swapper/119:51:331
926899515,27cyclictest0-21swapper/100:11:411
926899514,26cyclictest0-21swapper/123:58:391
926899514,25cyclictest0-21swapper/120:48:081
42192512,9sleep30-21swapper/323:24:593
307242512,10sleep30-21swapper/319:51:203
138742512,9sleep20-21swapper/219:14:312
927099503,39cyclictest11843-21telnet23:41:253
926999505,39cyclictest0-21swapper/220:26:432
926999504,40cyclictest0-21swapper/223:11:342
9269995036,9cyclictest30484-21grep20:56:392
9269995033,12cyclictest9045-21grep20:11:422
926899504,27cyclictest0-21swapper/120:56:521
282702503,10sleep30-21swapper/319:45:313
248599500,5rtkit-daemon2484-21rtkit-daemon19:35:202
15132502,9sleep30-21swapper/323:17:303
927099494,38cyclictest0-21swapper/323:56:123
9270994933,9cyclictest26772-21fschecks_count21:56:353
9269994935,9cyclictest23024-21unixbench-2d21:46:542
9269994935,8cyclictest12303-21unixbench_singl22:31:492
9269994932,10cyclictest13792-21grep20:21:392
248599490,5rtkit-daemon2484-21rtkit-daemon20:15:033
155182492,10sleep30-21swapper/319:16:493
9270994833,9cyclictest4981-21cut21:11:323
9270994832,10cyclictest13273-21grep22:36:293
9270994817,25cyclictest0-21swapper/322:16:493
926999484,10cyclictest24735-21munin-run20:46:202
926999484,10cyclictest24735-21munin-run20:46:202
9269994834,9cyclictest7327-21cpuspeed_turbos21:16:332
9269994833,9cyclictest14546-21cpuspeed_turbos21:31:332
9269994833,9cyclictest14546-21cpuspeed_turbos21:31:332
9269994832,9cyclictest20100-21needreboot23:56:382
9269994832,10cyclictest24427-21tune2fs00:06:302
926999483,39cyclictest0-21swapper/219:21:382
9269994821,9cyclictest17997-21chrt22:46:282
30022482,39sleep20-21swapper/221:06:392
256392482,9sleep30-21swapper/319:37:173
234082482,38sleep30-21swapper/300:05:413
19182482,10sleep20-21swapper/222:11:402
19182482,10sleep20-21swapper/222:11:402
150992482,9sleep20-21swapper/222:41:142
9270994731,9cyclictest31620-21grep22:06:353
927099473,38cyclictest3179-21munin-run22:16:163
927099473,38cyclictest3179-21munin-run22:16:163
926999474,37cyclictest0-21swapper/200:21:292
926999474,36cyclictest21528-21proc_pri20:36:522
9269994733,8cyclictest26844-21grep21:56:352
9269994731,10cyclictest29739-21needreboot22:01:432
9269994718,7cyclictest9242-21latency_hist19:06:252
9269994714,27cyclictest281ktimersoftd/200:16:242
9269994714,27cyclictest281ktimersoftd/200:16:242
29422473,37sleep30-21swapper/300:29:113
19592472,9sleep20-21swapper/223:21:152
114282472,10sleep30-21swapper/323:40:353
114282472,10sleep30-21swapper/323:40:353
9270994632,9cyclictest24441-21cut00:06:303
9270994632,9cyclictest15677-21cpuspeed_turbos22:41:293
9270994630,9cyclictest17918-21aten_r1power_po20:31:313
926999465,14cyclictest0-21swapper/223:16:402
9269994632,9cyclictest6558-21tune2fs22:21:362
9269994629,11cyclictest4932-21cat00:31:432
9269994620,9cyclictest1419-21runrttasks19:26:452
9269994620,9cyclictest1419-21runrttasks19:26:452
296822462,9sleep30-21swapper/323:11:253
138082463,9sleep30-21swapper/319:13:393
113612462,9sleep20-21swapper/221:23:482
927099455,35cyclictest0-21swapper/322:31:333
927099455,34cyclictest22212-21gltestperf21:46:413
927099454,35cyclictest0-21swapper/320:46:323
927099454,35cyclictest0-21swapper/320:36:503
927099454,26cyclictest0-21swapper/320:51:363
9270994530,9cyclictest30782-21kernelversion20:56:433
927099452,38cyclictest0-21swapper/321:51:303
9270994519,20cyclictest20421-21cpuspeed_turbos22:51:293
9270994519,20cyclictest20421-21cpuspeed_turbos22:51:293
926999455,34cyclictest14135-21aten2.4-expect23:46:232
926999455,34cyclictest0-21swapper/223:28:572
926999455,33cyclictest0-21swapper/220:01:352
9269994530,9cyclictest1290-21cpuspeed_turbos19:56:352
926999453,37cyclictest0-21swapper/220:06:362
926999453,37cyclictest0-21swapper/220:06:362
926999453,37cyclictest0-21swapper/219:46:562
43722452,37sleep30-21swapper/321:11:083
185202452,9sleep20-21swapper/221:38:392
137962452,37sleep20-21swapper/223:45:212
927099445,34cyclictest0-21swapper/321:01:473
927099444,35cyclictest20730-21unixbench_multi19:26:553
927099444,35cyclictest20730-21unixbench_multi19:26:553
927099444,33cyclictest0-21swapper/319:06:243
9270994430,9cyclictest29449-21crond23:11:153
926999444,33cyclictest0-21swapper/220:31:312
9269994426,13cyclictest0-21swapper/219:21:232
9269994418,8cyclictest9486-21munin-node19:06:362
9269994418,8cyclictest9097-21latency_hist23:36:142
90732442,36sleep30-21swapper/321:20:103
292852443,9sleep20-21swapper/223:09:102
280902442,10sleep30-21swapper/300:14:283
267432442,36sleep30-21swapper/323:02:103
248599440,4rtkit-daemon2484-21rtkit-daemon20:16:333
203382444,34sleep20-21swapper/222:51:292
203382444,34sleep20-21swapper/222:51:292
175432442,35sleep30-21swapper/320:31:093
116022442,35sleep30-21swapper/319:11:013
927099434,32cyclictest0-21swapper/300:01:133
927099432,35cyclictest30975-21telnet00:21:243
926999438,20cyclictest10728-21head20:16:342
926999434,34cyclictest0-21swapper/222:56:372
926999434,33cyclictest0-21swapper/221:26:352
9269994329,9cyclictest4748-21proc_pri22:16:462
9269994327,11cyclictest1538-21grep00:26:302
89592434,33sleep10-21swapper/119:02:501
927099424,32cyclictest0-21swapper/321:36:193
927099424,32cyclictest0-21swapper/321:36:193
926999423,34cyclictest0-21swapper/223:01:322
926999423,34cyclictest0-21swapper/222:06:292
926999423,33cyclictest17473-21yum-updatesd-he22:46:162
9269994217,7cyclictest0-21swapper/223:51:382
136532422,34sleep30-21swapper/321:27:323
927099414,31cyclictest0-21swapper/300:31:243
9270994127,9cyclictest22874-21cpuspeed_turbos22:56:303
185722402,10sleep30-21swapper/319:23:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional