You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 18:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Sat Jun 20, 2026 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
115579911216,27cyclictest4078-21munin-plugin-st11:58:390
1302521100,15sleep21156399cyclictest11:23:492
1155799994,28cyclictest0-21swapper/011:43:530
1155799975,28cyclictest0-21swapper/010:33:590
1155799965,26cyclictest0-21swapper/012:11:250
1155799955,23cyclictest23170-21ssh09:14:100
1155799955,23cyclictest23170-21ssh09:14:100
1155799954,28cyclictest0-21swapper/011:58:510
1155799954,28cyclictest0-21swapper/011:58:510
1155799944,27cyclictest0-21swapper/009:59:150
1155799944,27cyclictest0-21swapper/009:09:170
1155799935,27cyclictest0-21swapper/008:09:020
1155799934,28cyclictest0-21swapper/008:59:120
1155799934,28cyclictest0-21swapper/008:19:130
1155799934,28cyclictest0-21swapper/007:59:150
1155799934,27cyclictest0-21swapper/007:24:220
1155799924,27cyclictest0-21swapper/011:39:020
1155799924,27cyclictest0-21swapper/010:12:420
1155799924,27cyclictest0-21swapper/008:29:040
1155799924,27cyclictest0-21swapper/006:54:240
1155799924,24cyclictest0-21swapper/007:34:000
11557999215,24cyclictest1080-21lldpd09:22:530
11557999211,25cyclictest0-21swapper/010:04:000
11557999210,26cyclictest0-21swapper/010:39:480
1155799915,24cyclictest7647-21yum-updatesd-he10:28:080
1155799915,13cyclictest0-21swapper/008:54:460
1155799904,28cyclictest0-21swapper/007:14:190
11557999021,13cyclictest0-21swapper/008:49:190
1155799893,26cyclictest29662-21grep10:59:070
1155799884,27cyclictest0-21swapper/011:33:560
1155799874,29cyclictest0-21swapper/010:54:100
11557998735,11cyclictest30179-21df08:34:030
11557998735,11cyclictest30179-21df08:34:030
11557998721,13cyclictest0-21swapper/011:28:570
11557998633,9cyclictest4215-21tail09:34:040
1155799854,28cyclictest0-21swapper/010:19:100
1155799845,13cyclictest0-21swapper/011:11:040
1155799844,28cyclictest0-21swapper/009:39:150
1155799844,27cyclictest0-21swapper/011:49:010
1155799844,27cyclictest0-21swapper/009:29:070
11563997122,8cyclictest247932timerwakeupswit11:39:122
1156399654,53cyclictest0-21swapper/210:28:052
11561996513,44cyclictest0-21swapper/109:23:401
11561996128,27cyclictest0-21swapper/109:49:571
11563996018,36cyclictest3936-21irqstats11:09:052
1156199593,36cyclictest0-21swapper/109:24:211
244725810,40sleep20-21swapper/211:54:092
1156399584,47cyclictest0-21swapper/210:59:152
1156199587,28cyclictest0-21swapper/110:49:131
1156199585,35cyclictest0-21swapper/110:54:541
1156199583,25cyclictest0-21swapper/111:29:051
11561995817,20cyclictest27778-21netstat11:44:081
1155799586,30cyclictest22728-21irqstats10:49:050
38052573,46sleep20-21swapper/210:21:072
1156399577,12cyclictest0-21swapper/211:49:092
1156399573,46cyclictest9008-21ssh12:04:042
1156199579,30cyclictest0-21swapper/109:14:001
1156199579,30cyclictest0-21swapper/109:14:001
11561995731,9cyclictest13585-21ssh11:23:551
123442563,10sleep30-21swapper/311:22:183
123442563,10sleep30-21swapper/311:22:183
1156399565,44cyclictest0-21swapper/212:18:402
1156399564,46cyclictest0-21swapper/209:44:192
1156199569,26cyclictest0-21swapper/109:32:121
11561995610,39cyclictest32672-21gltestperf11:04:001
1156499558,41cyclictest0-21swapper/309:42:023
11564995528,12cyclictest0-21swapper/307:09:023
11564995517,27cyclictest0-21swapper/310:34:133
1156399556,42cyclictest0-21swapper/211:04:512
1156399554,44cyclictest0-21swapper/209:39:092
11563995534,12cyclictest0-21swapper/209:53:592
11563995534,12cyclictest0-21swapper/209:53:592
11563995521,28cyclictest1227-21runrttasks10:15:032
1156199554,32cyclictest0-21swapper/108:55:101
11561995538,10cyclictest9371-21cut07:49:141
11561995538,10cyclictest9371-21cut07:49:141
11561995512,26cyclictest12261-21gltestperf08:59:051
11561995511,26cyclictest2893-21fschecks_time08:44:051
287682542,10sleep30-21swapper/311:45:403
1156399549,22cyclictest0-21swapper/210:42:472
1156399545,41cyclictest0-21swapper/210:44:012
1156399543,45cyclictest0-21swapper/211:29:052
11563995415,33cyclictest26-21rcuc/212:00:142
11563995415,33cyclictest26-21rcuc/212:00:142
11563995411,8cyclictest26-21rcuc/210:34:132
1156199544,44cyclictest3985-21latency11:09:051
1156199544,31cyclictest0-21swapper/111:22:201
1156199544,31cyclictest0-21swapper/111:22:201
1156199544,30cyclictest0-21swapper/110:27:591
1156199543,39cyclictest0-21swapper/111:56:221
11561995422,26cyclictest11533-21munin-node10:34:081
11561995411,26cyclictest0-21swapper/109:37:321
11561995410,26cyclictest1124-21snmpd09:44:571
11557995436,11cyclictest4217-21head07:39:110
44402533,9sleep30-21swapper/310:23:253
27852533,10sleep20-21swapper/209:32:502
1156399538,38cyclictest0-21swapper/209:50:112
1156399535,42cyclictest0-21swapper/211:13:562
1156199539,25cyclictest0-21swapper/110:46:431
1156199538,28cyclictest0-21swapper/109:58:441
1156199538,28cyclictest0-21swapper/109:58:441
1156199538,26cyclictest0-21swapper/111:16:221
1156199537,28cyclictest7279-21df_abs09:39:001
1156199536,28cyclictest16086-21ssh12:14:111
1156199534,29cyclictest0-21swapper/112:09:011
1156199534,29cyclictest0-21swapper/107:29:241
1156199534,21cyclictest0-21swapper/110:09:021
1156199533,44cyclictest0-21swapper/107:18:201
11561995324,12cyclictest0-21swapper/108:51:211
11561995320,26cyclictest19961-21cpuspeed11:33:541
11561995311,24cyclictest0-21swapper/109:09:201
11561995310,28cyclictest21449-21aten_r1power_po07:09:031
11557995334,12cyclictest29715-21gltestperf09:24:030
1155799533,44cyclictest0-21swapper/012:04:220
11557995322,13cyclictest0-21swapper/011:06:510
11557995322,12cyclictest0-21swapper/011:23:500
11557995321,25cyclictest30593-21cpuspeed10:13:570
33602522,9sleep30-21swapper/308:44:143
284672522,9sleep20-21swapper/209:23:222
17232522,9sleep20-21swapper/208:43:002
117492523,10sleep20-21swapper/211:20:072
117492523,10sleep20-21swapper/211:20:072
1156499526,39cyclictest0-21swapper/312:03:513
1156499522,45cyclictest0-21swapper/309:22:113
1156399526,9cyclictest13918-21kworker/2:010:53:532
1156399525,41cyclictest0-21swapper/210:03:582
1156399524,42cyclictest0-21swapper/210:53:422
11563995215,31cyclictest0-21swapper/207:34:042
1156199529,26cyclictest5187-21tail11:58:581
1156199529,26cyclictest5187-21tail11:58:581
1156199528,27cyclictest0-21swapper/108:34:161
1156199528,27cyclictest0-21swapper/108:34:161
1156199525,28cyclictest0-21swapper/107:39:241
1156199524,40cyclictest0-21swapper/109:59:121
1156199524,28cyclictest0-21swapper/108:23:591
1156199524,28cyclictest0-21swapper/108:18:481
1156199524,26cyclictest0-21swapper/110:18:541
1156199523,28cyclictest0-21swapper/108:06:431
11561995214,14cyclictest0-21swapper/110:41:221
11561995211,26cyclictest0-21swapper/110:59:061
1155799525,27cyclictest0-21swapper/011:19:140
1155799525,27cyclictest0-21swapper/011:19:140
1155799525,24cyclictest11164-21iostat_ios09:44:080
1155799524,30cyclictest0-21swapper/010:29:060
11557995235,11cyclictest16129-21tail08:04:090
64422512,42sleep20-21swapper/208:49:372
280142513,41sleep228106-21timerandwakeup11:44:142
1156499516,37cyclictest34-21rcuc/309:08:463
11564995123,23cyclictest0-21swapper/311:15:333
11564995120,24cyclictest361ktimersoftd/309:28:553
1156499512,43cyclictest0-21swapper/310:49:353
11564995119,24cyclictest14734-21needreboot09:49:103
1156399518,9cyclictest10020-21users10:29:182
1156399514,42cyclictest0-21swapper/211:34:102
1156399514,39cyclictest0-21swapper/207:24:052
11563995136,10cyclictest0-21swapper/209:23:592
1156399513,43cyclictest0-21swapper/209:04:122
1156199516,26cyclictest0-21swapper/107:24:151
1156199514,30cyclictest0-21swapper/112:03:531
1156199514,29cyclictest31579-21missed_timers10:14:081
11561995119,13cyclictest0-21swapper/111:48:581
11561995114,11cyclictest0-21swapper/109:04:121
1155799514,25cyclictest0-21swapper/007:54:010
1155799513,35cyclictest0-21swapper/007:10:290
102672517,35sleep00-21swapper/006:44:100
76332502,9sleep30-21swapper/310:27:493
248392503,8sleep324883-21needreboot10:04:113
198032503,9sleep30-21swapper/309:58:103
198032503,9sleep30-21swapper/309:58:103
173142502,41sleep20-21swapper/208:06:532
160292503,9sleep30-21swapper/306:55:123
1156499503,41cyclictest0-21swapper/309:03:223
11564995024,13cyclictest0-21swapper/309:09:203
11564995021,23cyclictest361ktimersoftd/308:52:043
11564995016,7cyclictest361ktimersoftd/312:01:043
11564995016,7cyclictest361ktimersoftd/312:01:043
1156399503,41cyclictest0-21swapper/209:09:232
1156399503,41cyclictest0-21swapper/208:54:062
11563995026,11cyclictest0-21swapper/208:30:032
1156199507,25cyclictest0-21swapper/107:19:051
1156199505,35cyclictest8799-21fschecks_count10:29:011
1156199505,28cyclictest0-21swapper/107:04:131
1156199505,28cyclictest0-21swapper/107:04:131
1156199505,28cyclictest0-21swapper/106:54:231
1156199504,28cyclictest0-21swapper/110:04:021
1156199504,28cyclictest0-21swapper/108:19:151
1156199504,28cyclictest0-21swapper/106:49:191
1156199504,27cyclictest0-21swapper/111:43:071
1156199503,28cyclictest0-21swapper/107:48:501
1156199503,28cyclictest0-21swapper/107:48:501
1155799509,25cyclictest0-21swapper/006:49:040
1155799505,27cyclictest1227-21runrttasks12:16:490
1155799505,13cyclictest0-21swapper/009:51:550
1155799504,28cyclictest0-21swapper/010:44:180
1155799504,28cyclictest0-21swapper/007:49:070
1155799504,28cyclictest0-21swapper/007:49:070
326972492,9sleep20-21swapper/207:29:372
306252493,9sleep30-21swapper/310:13:583
1156499494,39cyclictest0-21swapper/308:28:033
1156499493,42cyclictest0-21swapper/310:09:223
1156499493,39cyclictest22354-21cpuspeed_turbos09:14:003
1156499493,39cyclictest22354-21cpuspeed_turbos09:14:003
1156499493,16cyclictest0-21swapper/310:00:293
1156399498,36cyclictest0-21swapper/210:01:042
1156399494,14cyclictest0-21swapper/209:38:452
1156399493,9cyclictest26-21rcuc/209:17:182
1156399493,9cyclictest26-21rcuc/209:17:182
1156199498,34cyclictest10869-21cpuspeed_turbos07:54:021
1156199494,28cyclictest0-21swapper/108:29:171
1155799495,27cyclictest0-21swapper/009:54:110
1155799495,27cyclictest0-21swapper/009:54:110
1155799494,28cyclictest0-21swapper/011:14:060
1155799494,27cyclictest0-21swapper/007:19:050
1155799494,26cyclictest0-21swapper/007:44:100
1155799494,26cyclictest0-21swapper/007:44:100
309992482,39sleep30-21swapper/311:01:503
302892482,39sleep30-21swapper/307:24:263
163632482,38sleep30-21swapper/311:28:513
14442482,9sleep30-21swapper/308:39:423
125762482,40sleep30-21swapper/307:57:183
124982482,40sleep20-21swapper/207:56:172
11564994833,10cyclictest29745-21cut09:24:033
11564994823,11cyclictest0-21swapper/311:48:533
11564994820,7cyclictest361ktimersoftd/308:55:503
1156399487,33cyclictest0-21swapper/208:18:472
1156399484,11cyclictest0-21swapper/208:34:162
1156399484,11cyclictest0-21swapper/208:34:162
1156399483,12cyclictest0-21swapper/212:12:392
11563994823,11cyclictest0-21swapper/208:59:152
11563994811,32cyclictest0-21swapper/208:09:002
1156199485,26cyclictest0-21swapper/108:09:041
1155799484,27cyclictest0-21swapper/009:04:010
1155799484,27cyclictest0-21swapper/008:28:480
1155799484,27cyclictest0-21swapper/008:14:230
1155799484,27cyclictest0-21swapper/007:04:190
1155799484,27cyclictest0-21swapper/007:04:190
1155799484,27cyclictest0-21swapper/006:59:040
1155799484,26cyclictest0-21swapper/008:46:500
77522473,9sleep30-21swapper/307:46:423
77522473,9sleep30-21swapper/307:46:423
244602473,9sleep30-21swapper/308:21:413
221832473,9sleep30-21swapper/308:18:063
203762472,37sleep20-21swapper/207:04:222
203762472,37sleep20-21swapper/207:04:222
186542473,9sleep30-21swapper/307:03:183
1156499477,12cyclictest0-21swapper/310:48:343
11564994725,17cyclictest37-21ksoftirqd/311:04:143
11564994719,22cyclictest0-21swapper/311:27:493
11563994729,13cyclictest0-21swapper/210:12:202
1155799473,31cyclictest0-21swapper/007:31:420
30952462,9sleep30-21swapper/307:38:323
255242462,9sleep20-21swapper/207:14:292
136512462,37sleep20-21swapper/207:59:072
1156499467,13cyclictest0-21swapper/311:38:273
1156499464,12cyclictest0-21swapper/312:18:283
1156499463,38cyclictest0-21swapper/312:10:463
1156499463,38cyclictest0-21swapper/310:30:133
1156499463,36cyclictest0-21swapper/310:38:573
11564994618,7cyclictest361ktimersoftd/310:57:193
1156399464,35cyclictest265962sleep208:24:212
1156199464,13cyclictest0-21swapper/108:39:091
1155799465,25cyclictest0-21swapper/008:41:490
102262462,37sleep20-21swapper/207:52:502
102262462,37sleep20-21swapper/207:52:502
77212453,9sleep20-21swapper/207:46:172
77212453,9sleep20-21swapper/207:46:172
256792452,9sleep30-21swapper/307:16:323
141532453,9sleep20-21swapper/206:53:542
125752452,37sleep30-21swapper/306:49:123
1156499454,36cyclictest0-21swapper/311:54:113
1156499454,11cyclictest0-21swapper/311:11:303
1156499453,37cyclictest0-21swapper/309:33:563
1156499453,36cyclictest0-21swapper/307:23:493
11564994523,10cyclictest0-21swapper/311:41:243
11564994521,7cyclictest5619-21cat07:43:503
1156499452,8cyclictest12269-21taskset09:45:193
11563994523,9cyclictest0-21swapper/206:59:162
11561994513,25cyclictest18415-21taskset07:00:091
5562442,36sleep30-21swapper/307:33:493
11564994430,9cyclictest16003-21fschecks_count08:04:083
1156399448,8cyclictest26-21rcuc/207:09:232
1156399445,34cyclictest0-21swapper/206:49:182
1156399444,33cyclictest0-21swapper/206:48:522
11563994422,9cyclictest0-21swapper/207:19:112
1156399442,10cyclictest0-21swapper/208:19:002
11563994415,23cyclictest4296-21idleruntime-cro08:48:472
1156199448,20cyclictest0-21swapper/107:36:391
129282432,9sleep30-21swapper/307:58:563
1156399435,32cyclictest0-21swapper/207:39:042
1156199434,21cyclictest0-21swapper/107:58:591
111942434,6sleep10-21swapper/106:44:311
293032422,34sleep30-21swapper/308:32:293
196262422,34sleep30-21swapper/308:10:533
317852412,10sleep30-21swapper/308:38:463
317852412,10sleep30-21swapper/308:38:463
1156499412,33cyclictest0-21swapper/307:48:583
1156499412,33cyclictest0-21swapper/307:48:583
11564994119,17cyclictest1227-21runrttasks07:08:213
11564994119,17cyclictest1227-21runrttasks07:08:213
1156499403,30cyclictest0-21swapper/306:48:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional