You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 08:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Mar 16, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29526992522,2cyclictest607-21in:imuxsock22:00:013
2952699250,24cyclictest22773-21diskmemload22:59:253
2952699250,24cyclictest22773-21diskmemload22:25:373
29523992523,1cyclictest0-21swapper/200:03:042
2952399252,2cyclictest12969-21sed22:32:152
2952399250,25cyclictest0-21swapper/222:46:032
2952399250,24cyclictest13261-21ssh00:37:092
2952399250,24cyclictest0-21swapper/221:16:122
2952099250,23cyclictest313-21systemd-journal21:10:011
29519992523,1cyclictest0-21swapper/000:32:240
2951999250,24cyclictest0-21swapper/021:10:200
2951999250,24cyclictest0-21swapper/000:18:260
2952699240,2cyclictest5743-21ssh23:58:383
2952699240,24cyclictest22773-21diskmemload00:10:363
2952699240,23cyclictest22773-21diskmemload23:51:133
2952699240,23cyclictest22773-21diskmemload23:32:213
2952699240,23cyclictest22773-21diskmemload22:54:113
2952699240,1cyclictest0-21swapper/300:38:353
2952699240,0cyclictest22773-21diskmemload22:40:303
29523992423,0cyclictest28185-21apt-get22:17:062
29523992422,0cyclictest0-21swapper/223:34:272
29523992421,2cyclictest3512-21ssh23:25:552
29523992421,2cyclictest2343-21ssh23:55:592
29523992421,2cyclictest15259-21ssh22:35:032
2952399240,24cyclictest0-21swapper/223:45:292
2952399240,1cyclictest27662-21ssh21:14:242
29520992424,0cyclictest0-21swapper/122:29:071
29520992423,0cyclictest0-21swapper/119:12:051
29520992422,1cyclictest11627-21sh00:35:091
29520992422,1cyclictest0-21swapper/121:13:331
2952099240,23cyclictest22773-21diskmemload00:31:381
2952099240,23cyclictest0-21swapper/119:17:061
29519992422,1cyclictest15813-21ssh23:37:130
2951999240,24cyclictest0-21swapper/022:38:040
2951999240,23cyclictest22773-21diskmemload21:53:380
2951999240,23cyclictest22773-21diskmemload21:53:380
2951999240,23cyclictest22773-21diskmemload21:40:080
2951999240,23cyclictest22773-21diskmemload00:27:210
2951999240,23cyclictest0-21swapper/000:22:070
29526992323,0cyclictest22773-21diskmemload22:02:473
29526992323,0cyclictest22773-21diskmemload21:19:093
29526992323,0cyclictest22773-21diskmemload21:10:103
29526992323,0cyclictest0-21swapper/300:15:343
29526992321,1cyclictest24489-21sh23:14:423
29526992320,2cyclictest31039-21ssh21:47:393
29526992320,2cyclictest16189-21rm21:34:363
2952699230,2cyclictest4476-21ssh21:22:333
2952699230,2cyclictest31141-21dump-pmu-power20:20:013
2952699230,23cyclictest22773-21diskmemload23:40:303
2952699230,23cyclictest0-21swapper/323:25:273
2952699230,23cyclictest0-21swapper/323:00:203
2952699230,23cyclictest0-21swapper/322:20:243
2952699230,23cyclictest0-21swapper/321:40:413
2952699230,22cyclictest31466-21apt-get23:22:063
2952699230,22cyclictest22625-21ssh22:11:343
2952699230,22cyclictest15484-21runrttasks00:07:273
2952699230,0cyclictest22773-21diskmemload23:07:433
2952699230,0cyclictest0-21swapper/321:36:563
2952699230,0cyclictest0-21swapper/300:29:473
29523992323,0cyclictest22773-21diskmemload22:53:452
29523992323,0cyclictest0-21swapper/222:04:452
29523992323,0cyclictest0-21swapper/221:58:542
29523992323,0cyclictest0-21swapper/221:47:522
29523992323,0cyclictest0-21swapper/221:26:122
29523992323,0cyclictest0-21swapper/200:24:142
29523992323,0cyclictest0-21swapper/200:07:222
29523992321,1cyclictest1009-21ssh23:54:132
29523992321,1cyclictest0-21swapper/221:37:112
29523992321,1cyclictest0-21swapper/200:27:222
29523992320,0cyclictest0-21swapper/221:40:202
2952399230,2cyclictest4006-21ssh22:55:342
2952399230,23cyclictest0-21swapper/222:10:312
2952399230,23cyclictest0-21swapper/200:15:332
2952399230,22cyclictest23721-21rm23:13:332
2952399230,22cyclictest22773-21diskmemload22:09:342
2952399230,22cyclictest21480-21awk20:00:002
2952399230,22cyclictest0-21swapper/223:08:322
2952399230,22cyclictest0-21swapper/222:40:102
2952399230,0cyclictest22773-21diskmemload21:23:362
2952399230,0cyclictest0-21swapper/223:36:452
2952399230,0cyclictest0-21swapper/200:11:372
29520992323,0cyclictest0-21swapper/123:46:081
29520992323,0cyclictest0-21swapper/100:26:471
29520992321,1cyclictest75-21kswapd021:40:021
29520992321,1cyclictest17079-21ssh21:35:331
29520992321,1cyclictest0-21swapper/123:37:151
29520992321,1cyclictest0-21swapper/123:11:371
29520992321,1cyclictest0-21swapper/122:20:361
29520992321,1cyclictest0-21swapper/122:05:351
29520992321,1cyclictest0-21swapper/122:00:281
29520992321,1cyclictest0-21swapper/121:33:531
29520992321,1cyclictest0-21swapper/121:26:211
29520992321,1cyclictest0-21swapper/100:22:161
29520992321,1cyclictest0-21swapper/100:10:401
29520992320,2cyclictest12187-21sh23:02:251
2952099230,23cyclictest0-21swapper/122:19:201
2952099230,23cyclictest0-21swapper/121:47:501
2952099230,22cyclictest8509-21sh23:30:321
2952099230,22cyclictest7663-21sh00:01:041
2952099230,22cyclictest29860-21ssh22:48:421
2952099230,22cyclictest23072-21rm00:15:381
2952099230,22cyclictest22773-21diskmemload22:36:051
2952099230,22cyclictest22773-21diskmemload21:52:001
2952099230,22cyclictest22773-21diskmemload21:52:001
2952099230,22cyclictest10483-21ssh22:30:481
2952099230,22cyclictest0-21swapper/122:56:531
2952099230,22cyclictest0-21swapper/121:57:041
2952099230,0cyclictest0-21swapper/122:52:001
29519992323,0cyclictest22773-21diskmemload00:35:260
29519992323,0cyclictest0-21swapper/023:58:410
29519992323,0cyclictest0-21swapper/022:46:480
29519992323,0cyclictest0-21swapper/022:25:090
29519992322,1cyclictest22773-21diskmemload23:23:080
29519992322,1cyclictest22773-21diskmemload00:01:520
29519992322,1cyclictest0-21swapper/022:54:260
29519992321,1cyclictest0-21swapper/023:27:160
2951999230,2cyclictest32631-21sh22:20:530
2951999230,23cyclictest0-21swapper/000:05:100
2951999230,22cyclictest8493-21diskmemload23:30:300
2951999230,22cyclictest22773-21diskmemload22:06:240
2951999230,22cyclictest22773-21diskmemload00:10:320
2951999230,22cyclictest19049-21cron19:55:000
2951999230,22cyclictest0-21swapper/022:00:350
2951999230,22cyclictest0-21swapper/021:20:160
2951999230,21cyclictest0-21swapper/021:59:260
2951999230,0cyclictest22773-21diskmemload22:15:560
2951999230,0cyclictest22773-21diskmemload21:47:450
2951999230,0cyclictest0-21swapper/023:13:350
2951999230,0cyclictest0-21swapper/022:30:560
29526992222,0cyclictest0-21swapper/323:35:083
29526992222,0cyclictest0-21swapper/323:15:353
29526992222,0cyclictest0-21swapper/321:25:113
29526992222,0cyclictest0-21swapper/300:34:353
29526992222,0cyclictest0-21swapper/300:02:033
29526992220,1cyclictest29170-21ssh00:22:063
29526992220,1cyclictest0-21swapper/319:15:003
29526992219,2cyclictest15406-21ssh22:35:163
2952699220,2cyclictest2083-21expr21:52:083
2952699220,2cyclictest2083-21expr21:52:083
2952699220,21cyclictest9885-21apt-get19:37:063
2952699220,21cyclictest22773-21diskmemload22:15:573
2952699220,21cyclictest22773-21diskmemload22:05:483
2952699220,21cyclictest15546-21munin-run20:55:003
2952699220,21cyclictest0-21swapper/322:47:073
2952699220,21cyclictest0-21swapper/320:32:053
2952699220,1cyclictest606-21dbus-daemon20:50:003
2952699220,0cyclictest0-21swapper/322:30:513
29523992222,0cyclictest0-21swapper/223:40:132
29523992222,0cyclictest0-21swapper/219:32:062
29523992221,1cyclictest22773-21diskmemload23:16:182
29523992220,2cyclictest351-21ssh21:50:032
29523992220,2cyclictest351-21ssh21:50:032
29523992220,1cyclictest30251-21sh23:20:112
29523992220,1cyclictest0-21swapper/223:00:332
2952399220,22cyclictest0-21swapper/222:23:542
2952399220,21cyclictest8763-21apt-get20:42:072
2952399220,21cyclictest0-21swapper/221:30:362
2952399220,21cyclictest0-21swapper/220:07:072
2952399220,1cyclictest0-21swapper/222:25:172
2952399220,0cyclictest0-21swapper/220:22:072
29520992222,0cyclictest0-21swapper/123:05:121
29520992222,0cyclictest0-21swapper/119:27:061
29520992222,0cyclictest0-21swapper/119:27:061
29520992221,1cyclictest7093-21apt-get19:32:061
29520992221,1cyclictest22773-21diskmemload23:16:131
29520992220,1cyclictest0-21swapper/100:05:241
2952099220,22cyclictest0-21swapper/123:41:481
2952099220,21cyclictest30182-21ssh23:20:051
2952099220,21cyclictest22773-21diskmemload23:25:331
2952099220,21cyclictest22773-21diskmemload21:20:201
2952099220,21cyclictest18365-21apt-get21:02:061
2952099220,21cyclictest0-21swapper/121:15:061
2952099220,21cyclictest0-21swapper/120:07:071
2952099220,1cyclictest3677-21ssh23:57:091
2952099220,1cyclictest22409-21/usr/sbin/munin22:42:091
29519992222,0cyclictest22773-21diskmemload23:06:020
29519992221,1cyclictest12118-21kworker/0:020:47:050
29519992221,1cyclictest0-21swapper/022:56:550
29519992221,1cyclictest0-21swapper/021:25:360
29519992220,1cyclictest31168-21awk20:20:010
2951999220,22cyclictest0-21swapper/021:35:210
2951999220,21cyclictest28737-21idleruntime-cro20:15:010
2951999220,21cyclictest0-21swapper/023:41:130
2951999220,21cyclictest0-21swapper/021:15:060
2951999220,21cyclictest0-21swapper/021:07:050
2951999220,21cyclictest0-21swapper/019:27:070
2951999220,21cyclictest0-21swapper/019:27:070
2951999220,1cyclictest10069-21ssh23:01:530
2951999220,1cyclictest0-21swapper/023:45:400
2951999220,0cyclictest0-21swapper/022:41:250
29526992121,0cyclictest22773-21diskmemload23:45:033
29526992121,0cyclictest0-21swapper/321:07:063
2952699210,20cyclictest4697-21apt-get19:27:063
2952699210,20cyclictest4697-21apt-get19:27:063
2952699210,20cyclictest0-21swapper/320:22:073
29523992119,1cyclictest0-21swapper/219:40:012
2952399210,20cyclictest15947-21apt-get20:57:062
2952399210,20cyclictest0-21swapper/220:52:042
2952399210,20cyclictest0-21swapper/219:42:062
29520992120,0cyclictest0-21swapper/120:47:041
29520992119,1cyclictest0-21swapper/119:50:001
2952099210,0cyclictest0-21swapper/119:52:071
29519992121,0cyclictest0-21swapper/021:30:040
29519992120,0cyclictest0-21swapper/019:12:050
2951999210,1cyclictest4295-21munin-run19:25:010
29526992020,0cyclictest0-21swapper/320:57:063
29526992020,0cyclictest0-21swapper/320:27:073
2952699200,0cyclictest12305-21apt-get19:42:073
2952699200,0cyclictest0-21swapper/321:02:063
2952699200,0cyclictest0-21swapper/320:12:073
29520992020,0cyclictest0-21swapper/120:17:071
2952099200,20cyclictest0-21swapper/120:12:071
2952099200,0cyclictest0-21swapper/120:27:071
29519992019,1cyclictest1542-21apt-get20:27:060
2951999200,0cyclictest0-21swapper/020:32:050
2951999200,0cyclictest0-21swapper/019:17:060
29526991919,0cyclictest0-21swapper/320:37:073
29526991917,1cyclictest4268-21sh19:25:013
2952399190,0cyclictest0-21swapper/219:22:062
29520991919,0cyclictest0-21swapper/119:22:061
29520991917,1cyclictest8353-21munin-plugin-st20:40:001
29519991919,0cyclictest0-21swapper/020:02:070
29519991919,0cyclictest0-21swapper/019:35:020
29526991818,0cyclictest19489-21apt-get19:57:063
2952699180,17cyclictest32297-21apt-get19:17:063
29523991818,0cyclictest0-21swapper/220:17:072
29520991818,0cyclictest0-21swapper/119:42:071
29519991815,2cyclictest24357-21apt-get20:07:070
2951999180,17cyclictest15947-21apt-get20:57:060
29526991717,0cyclictest0-21swapper/319:52:073
29526991717,0cyclictest0-21swapper/319:47:063
29526991715,1cyclictest9087-21munin-plugin-st19:35:013
29523991717,0cyclictest17095-21apt-get19:52:072
29523991717,0cyclictest0-21swapper/219:17:062
29523991715,1cyclictest8320-21idleruntime-cro20:40:002
29519991717,0cyclictest0-21swapper/019:45:010
29519991715,1cyclictest13550-21apt-get20:52:040
29526991614,1cyclictest0-21swapper/320:07:093
29523991616,0cyclictest3949-21apt-get20:32:052
29523991616,0cyclictest0-21swapper/219:27:062
29523991616,0cyclictest0-21swapper/219:27:062
29523991614,1cyclictest2353-21irqstats20:27:102
29520991616,0cyclictest0-21swapper/120:42:071
29520991616,0cyclictest0-21swapper/119:57:061
29520991614,1cyclictest0-21swapper/120:25:011
2952099160,15cyclictest5940-21munin-run20:35:001
29519991616,0cyclictest0-21swapper/020:42:070
29519991616,0cyclictest0-21swapper/019:32:050
29526991515,0cyclictest0-21swapper/320:02:083
2952399150,1cyclictest20554-21cut21:05:012
29519991515,0cyclictest0-21swapper/021:02:070
29519991515,0cyclictest0-21swapper/020:37:070
2952699140,0cyclictest0-21swapper/320:40:263
29523991414,0cyclictest0-21swapper/220:10:142
29523991414,0cyclictest0-21swapper/219:45:352
2952399140,14cyclictest0-21swapper/221:06:332
2952399140,14cyclictest0-21swapper/220:45:452
2952399140,0cyclictest0-21swapper/220:00:032
2952399140,0cyclictest0-21swapper/219:10:492
2952099140,14cyclictest0-21swapper/120:55:171
2952099140,14cyclictest0-21swapper/120:00:091
2952099140,14cyclictest0-21swapper/119:35:291
2951999140,14cyclictest0-21swapper/020:20:150
2951999140,14cyclictest0-21swapper/019:45:090
2951999140,0cyclictest0-21swapper/019:55:320
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional