You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 08:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Sat Mar 14, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3220992523,1cyclictest0-21swapper/321:11:013
322099250,24cyclictest29295-21diskmemload23:07:003
321899250,24cyclictest7593-21cron21:20:001
321799251,1cyclictest29076-21diskmemload00:15:500
3220992422,1cyclictest22498-21sh00:09:373
3220992422,0cyclictest29295-21diskmemload00:27:253
322099240,23cyclictest0-21swapper/322:05:113
322099240,23cyclictest0-21swapper/321:36:163
322099240,23cyclictest0-21swapper/321:30:203
322099240,1cyclictest29295-21diskmemload21:54:073
3219992421,2cyclictest609-21rs:main2
321999240,2cyclictest23429-21rm22:37:032
321999240,23cyclictest4639-21/usr/sbin/munin23:22:002
321999240,23cyclictest29295-21diskmemload22:08:182
321999240,23cyclictest13524-21ls23:30:012
321999240,23cyclictest0-21swapper/221:40:022
321999240,23cyclictest0-21swapper/200:02:562
321999240,1cyclictest2748-21ssh21:15:082
321999240,1cyclictest22786-21apt-get00:10:022
3218992424,0cyclictest0-21swapper/123:32:451
3218992423,1cyclictest29295-21diskmemload23:05:121
321899240,2cyclictest21490-21ssh21:32:291
321899240,24cyclictest0-21swapper/122:04:211
321899240,23cyclictest29295-21diskmemload23:03:291
321899240,23cyclictest29295-21diskmemload21:35:491
321899240,23cyclictest0-21swapper/122:51:561
321899240,1cyclictest991-21/usr/sbin/munin22:46:581
3217992423,1cyclictest3352-21sh23:51:490
3217992423,1cyclictest0-21swapper/000:13:160
321799240,24cyclictest0-21swapper/023:01:550
321799240,23cyclictest29295-21diskmemload21:56:310
3220992323,0cyclictest29295-21diskmemload23:57:003
3220992323,0cyclictest0-21swapper/323:36:583
3220992323,0cyclictest0-21swapper/323:33:013
3220992323,0cyclictest0-21swapper/322:00:543
3220992323,0cyclictest0-21swapper/321:59:123
3220992322,1cyclictest29295-21diskmemload22:41:073
3220992321,1cyclictest23427-21apt-get00:10:023
3220992321,1cyclictest0-21swapper/300:20:253
3220992321,1cyclictest0-21swapper/300:16:003
3220992320,2cyclictest29520-21sh21:40:563
322099230,23cyclictest29295-21diskmemload22:53:073
322099230,23cyclictest0-21swapper/321:29:463
322099230,22cyclictest7218-21munin-run23:55:003
322099230,22cyclictest4789-21ssh21:47:103
322099230,22cyclictest31676-21ssh22:45:143
322099230,22cyclictest313-21systemd-journal19:50:003
322099230,22cyclictest25593-21rm23:10:543
322099230,22cyclictest14528-21sh22:28:063
322099230,22cyclictest0-21swapper/323:47:063
322099230,22cyclictest0-21swapper/322:21:553
322099230,22cyclictest0-21swapper/300:01:173
322099230,0cyclictest0-21swapper/323:26:503
322099230,0cyclictest0-21swapper/300:31:103
3219992323,0cyclictest0-21swapper/200:20:352
3219992322,1cyclictest20120-21apt-get20:51:552
3219992322,1cyclictest0-21swapper/200:17:512
3219992321,1cyclictest18023-21ssh23:02:312
3219992321,1cyclictest0-21swapper/223:35:142
3219992321,1cyclictest0-21swapper/222:00:112
3219992321,1cyclictest0-21swapper/221:21:512
321999230,23cyclictest0-21swapper/223:44:172
321999230,23cyclictest0-21swapper/221:11:182
321999230,23cyclictest0-21swapper/200:26:492
321999230,22cyclictest6323-21diskmemload23:53:472
321999230,22cyclictest5607-21ssh21:48:232
321999230,22cyclictest16458-21diskmemload22:30:452
321999230,22cyclictest10676-21/usr/sbin/munin22:57:052
321999230,22cyclictest0-21swapper/223:18:312
321999230,1cyclictest0-21swapper/223:57:382
321999230,1cyclictest0-21swapper/222:16:562
321999230,1cyclictest0-21swapper/200:32:532
321999230,0cyclictest29295-21diskmemload23:08:312
321999230,0cyclictest29295-21diskmemload22:46:312
3218992323,0cyclictest29295-21diskmemload21:50:391
3218992323,0cyclictest0-21swapper/122:08:521
3218992321,1cyclictest606-21dbus-daemon00:10:011
3218992321,1cyclictest26932-21ssh22:40:561
3218992321,1cyclictest20085-21munin-run19:45:001
3218992321,1cyclictest0-21swapper/100:20:031
3218992321,1cyclictest0-21swapper/100:01:311
321899230,2cyclictest4443-21sh22:18:371
321899230,2cyclictest24986-21ssh22:38:171
321899230,2cyclictest14079-21sh21:26:391
321899230,23cyclictest29295-21diskmemload23:52:261
321899230,23cyclictest0-21swapper/123:27:011
321899230,23cyclictest0-21swapper/123:12:081
321899230,23cyclictest0-21swapper/121:22:091
321899230,23cyclictest0-21swapper/121:13:211
321899230,22cyclictest29295-21diskmemload21:45:061
321899230,22cyclictest14837-21ssh00:32:111
321899230,22cyclictest12580-21ssh21:55:551
321899230,22cyclictest0-21swapper/122:20:491
321899230,1cyclictest19877-21sh23:36:251
3217992323,0cyclictest29295-21diskmemload23:33:210
3217992323,0cyclictest29295-21diskmemload00:23:260
3217992323,0cyclictest0-21swapper/023:45:100
3217992323,0cyclictest0-21swapper/023:27:240
3217992323,0cyclictest0-21swapper/023:10:210
3217992323,0cyclictest0-21swapper/022:01:580
3217992323,0cyclictest0-21swapper/021:51:560
3217992323,0cyclictest0-21swapper/021:41:100
3217992323,0cyclictest0-21swapper/021:15:240
3217992321,1cyclictest30333-21ssh21:11:200
3217992321,1cyclictest0-21swapper/023:16:580
3217992321,1cyclictest0-21swapper/021:35:090
3217992321,1cyclictest0-21swapper/020:55:010
3217992320,2cyclictest20341-21memory21:32:000
321799230,23cyclictest0-21swapper/022:44:150
321799230,22cyclictest29295-21diskmemload22:15:560
321799230,22cyclictest29295-21diskmemload21:47:490
321799230,22cyclictest0-21swapper/022:11:560
321799230,1cyclictest0-21swapper/022:20:310
321799230,1cyclictest0-21swapper/021:28:160
3220992222,0cyclictest0-21swapper/323:16:213
3220992222,0cyclictest0-21swapper/322:17:353
3220992222,0cyclictest0-21swapper/321:22:173
3220992221,1cyclictest0-21swapper/321:15:413
3220992221,0cyclictest0-21swapper/323:00:113
3220992220,1cyclictest0-21swapper/300:35:093
322099220,22cyclictest0-21swapper/322:36:303
322099220,21cyclictest0-21swapper/323:20:033
322099220,21cyclictest0-21swapper/320:46:563
322099220,21cyclictest0-21swapper/319:41:573
322099220,1cyclictest0-21swapper/322:30:453
322099220,0cyclictest0-21swapper/323:40:033
322099220,0cyclictest0-21swapper/322:13:103
3219992220,1cyclictest29295-21diskmemload22:25:052
3219992220,1cyclictest20871-21apt-get19:46:582
3219992220,1cyclictest10067-21apt-get20:31:562
3219992220,1cyclictest0-21swapper/223:10:132
3219992219,2cyclictest21186-21ssh00:39:272
321999220,2cyclictest15906-21cat21:27:052
321999220,22cyclictest0-21swapper/221:37:112
321999220,21cyclictest29295-21diskmemload22:51:002
321999220,21cyclictest29295-21diskmemload21:30:492
321999220,21cyclictest28075-21apt-get20:01:562
321999220,1cyclictest29295-21diskmemload23:30:242
321999220,1cyclictest19054-21apt-get00:06:542
321999220,1cyclictest0-21swapper/222:40:362
321999220,1cyclictest0-21swapper/222:20:352
3218992222,0cyclictest0-21swapper/122:56:331
3218992222,0cyclictest0-21swapper/122:11:501
3218992222,0cyclictest0-21swapper/100:16:061
3218992220,1cyclictest9033-21diskstats23:56:581
3218992219,2cyclictest22845-21cron19:50:001
321899220,22cyclictest0-21swapper/120:16:551
321899220,22cyclictest0-21swapper/100:37:021
321899220,21cyclictest31854-21apt-get23:16:571
321899220,21cyclictest28352-21apt-get21:40:021
321899220,21cyclictest25200-21perl00:12:011
321899220,21cyclictest0-21swapper/123:40:051
321899220,21cyclictest0-21swapper/123:20:351
321899220,21cyclictest0-21swapper/120:21:551
321899220,21cyclictest0-21swapper/100:25:391
321899220,20cyclictest313-21systemd-journal21:05:011
321899220,1cyclictest32633-21ssh23:47:271
321899220,0cyclictest0-21swapper/122:33:341
3217992222,0cyclictest6931-21ssh23:22:510
3217992222,0cyclictest29295-21diskmemload23:58:080
3217992222,0cyclictest15965-21ssh00:33:520
3217992222,0cyclictest0-21swapper/023:36:010
3217992222,0cyclictest0-21swapper/022:25:030
3217992222,0cyclictest0-21swapper/021:20:280
3217992221,1cyclictest16017-21ssh00:02:360
321799220,22cyclictest0-21swapper/023:05:270
321799220,21cyclictest4835-21cron20:20:010
321799220,21cyclictest30066-21grep20:05:010
321799220,21cyclictest29295-21diskmemload00:25:510
321799220,21cyclictest29295-21diskmemload00:06:410
321799220,21cyclictest18110-21memory22:32:000
321799220,21cyclictest0-21swapper/000:35:040
321799220,1cyclictest32710-21/usr/sbin/munin22:47:040
321799220,1cyclictest29295-21diskmemload22:50:470
321799220,1cyclictest23578-21ssh22:06:450
321799220,0cyclictest0-21swapper/022:56:190
321799220,0cyclictest0-21swapper/022:37:050
3220992121,0cyclictest0-21swapper/322:55:063
3220992120,0cyclictest0-21swapper/320:36:553
3220992118,2cyclictest24494-21cron21:00:013
322099210,0cyclictest0-21swapper/319:11:543
3219992120,0cyclictest2848-21apt-get20:16:562
3219992119,1cyclictest0-21swapper/219:35:012
3219992118,2cyclictest32448-21cron20:10:012
3219992118,2cyclictest32448-21cron20:10:012
321999210,0cyclictest0-21swapper/223:45:232
3218992121,0cyclictest0-21swapper/119:56:561
3218992119,1cyclictest29326-21munin-run21:10:011
321899210,21cyclictest0-21swapper/120:51:561
321899210,20cyclictest12463-21apt-get20:36:551
321899210,0cyclictest10817-21apt-get19:26:561
3217992121,0cyclictest6004-21apt-get19:16:560
3217992121,0cyclictest18029-21apt-get19:41:580
3217992121,0cyclictest0-21swapper/023:40:200
3217992120,1cyclictest20075-21apt-get19:45:020
321799210,21cyclictest9657-21latency_hist20:30:000
321799210,0cyclictest3605-21apt-get19:11:530
321799210,0cyclictest0-21swapper/020:46:560
321799210,0cyclictest0-21swapper/019:36:550
321799210,0cyclictest0-21swapper/019:21:550
3220992020,0cyclictest0-21swapper/319:51:543
3220992020,0cyclictest0-21swapper/319:16:563
3220992019,0cyclictest0-21swapper/320:21:553
322099200,0cyclictest0-21swapper/320:41:563
3219992020,0cyclictest0-21swapper/220:46:562
3218992020,0cyclictest0-21swapper/119:51:541
3217992020,0cyclictest0-21swapper/020:31:560
321799200,20cyclictest0-21swapper/021:02:040
321799200,0cyclictest0-21swapper/020:06:560
321799200,0cyclictest0-21swapper/020:06:560
3220991919,0cyclictest8422-21apt-get19:21:553
3220991918,1cyclictest0-21swapper/320:11:553
322099190,18cyclictest26936-21munin-run21:05:013
3218991919,0cyclictest0-21swapper/120:41:561
3218991919,0cyclictest0-21swapper/120:26:551
3217991919,0cyclictest0-21swapper/020:36:550
3217991919,0cyclictest0-21swapper/019:26:550
321799190,1cyclictest313-21systemd-journal20:45:000
3219991818,0cyclictest0-21swapper/220:41:562
3219991818,0cyclictest0-21swapper/220:36:552
3219991817,0cyclictest0-21swapper/221:01:562
3218991818,0cyclictest0-21swapper/119:21:551
3218991817,1cyclictest17672-21apt-get20:46:561
3218991817,0cyclictest0-21swapper/119:11:541
3218991816,1cyclictest0-21swapper/120:10:011
3218991816,1cyclictest0-21swapper/120:10:011
3217991818,0cyclictest0-21swapper/020:56:540
3217991818,0cyclictest0-21swapper/019:31:560
3217991817,1cyclictest23261-21apt-get19:51:540
3220991717,0cyclictest0-21swapper/320:06:553
3220991717,0cyclictest0-21swapper/320:06:553
3220991717,0cyclictest0-21swapper/319:36:553
3220991715,1cyclictest9670-21perf20:30:003
3220991715,1cyclictest21163-21ntp_states20:52:013
3219991717,0cyclictest0-21swapper/219:26:562
3218991717,0cyclictest0-21swapper/120:01:561
321899170,0cyclictest0-21swapper/120:31:561
321899170,0cyclictest0-21swapper/119:31:551
3217991716,1cyclictest0-21swapper/020:11:550
322099167,7cyclictest611-21irqbalance21:06:593
3220991615,1cyclictest0-21swapper/320:01:563
3220991613,2cyclictest27268-21grep19:57:043
3219991615,0cyclictest0-21swapper/219:36:552
321999160,15cyclictest28368-21ntp_states21:07:012
3218991616,0cyclictest0-21swapper/119:16:561
3217991614,1cyclictest27675-21sendmail-msp20:00:000
3220991514,1cyclictest13214-21apt-get19:31:553
3219991515,0cyclictest0-21swapper/219:21:552
3219991513,1cyclictest0-21swapper/220:26:572
321999150,1cyclictest0-21swapper/219:10:162
3220991414,0cyclictest0-21swapper/320:30:203
322099140,14cyclictest0-21swapper/320:15:343
322099140,14cyclictest0-21swapper/319:25:063
321999140,14cyclictest0-21swapper/220:58:582
321999140,14cyclictest0-21swapper/219:40:492
321999140,14cyclictest0-21swapper/219:15:352
321999140,0cyclictest0-21swapper/220:11:002
321999140,0cyclictest0-21swapper/219:55:372
321999140,0cyclictest0-21swapper/219:50:592
321899140,14cyclictest0-21swapper/120:10:191
321899140,0cyclictest0-21swapper/120:55:121
321799140,0cyclictest0-21swapper/021:05:100
321799140,0cyclictest0-21swapper/020:20:300
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional