You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 04:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Jun 29, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
118799270,26cyclictest0-21swapper/022:49:320
1198992523,1cyclictest0-21swapper/321:15:033
119899250,24cyclictest7905-21ssh22:47:033
119899250,24cyclictest25652-21perf22:35:003
119899250,24cyclictest0-21swapper/321:47:023
1193992522,2cyclictest29248-21ssh21:40:172
119399250,24cyclictest1979-21ssh23:40:522
119399250,24cyclictest0-21swapper/223:55:302
119399250,1cyclictest30725-21ssh21:11:412
119399250,1cyclictest0-21swapper/222:11:262
118899252,22cyclictest20214-21ssh22:58:371
118899250,24cyclictest0-21swapper/122:20:121
118799250,24cyclictest0-21swapper/021:58:410
119899240,24cyclictest0-21swapper/323:45:253
119899240,23cyclictest22934-21ssh00:26:553
1193992423,1cyclictest3876-21apt-get19:15:522
1193992423,1cyclictest27591-21diskmemload22:08:322
1193992422,1cyclictest1107-21apt-get20:15:532
1193992422,1cyclictest0-21swapper/222:29:582
1193992421,2cyclictest27591-21diskmemload00:00:532
119399240,23cyclictest27591-21diskmemload23:01:152
119399240,23cyclictest27591-21diskmemload22:22:412
119399240,23cyclictest0-21swapper/221:15:532
119399240,23cyclictest0-21swapper/200:30:022
1188992421,2cyclictest32419-21cstates22:40:541
118899240,24cyclictest0-21swapper/121:58:571
118899240,24cyclictest0-21swapper/121:48:551
118899240,23cyclictest5810-21ssh00:11:271
118899240,23cyclictest5810-21ssh00:11:271
118899240,23cyclictest27591-21diskmemload21:39:291
118899240,23cyclictest0-21swapper/123:05:531
118899240,23cyclictest0-21swapper/121:21:011
118899240,1cyclictest19704-21df23:55:541
118899240,0cyclictest0-21swapper/123:23:341
118899240,0cyclictest0-21swapper/123:17:321
1187992423,1cyclictest62550irq/127-eno100:30:450
1187992423,1cyclictest0-21swapper/022:37:570
1187992422,1cyclictest27759-21sh21:38:390
118799242,1cyclictest62550irq/127-eno100:00:380
1187992421,2cyclictest6870-21sh21:20:270
118799240,24cyclictest0-21swapper/022:27:020
118799240,23cyclictest27591-21diskmemload21:41:010
118799240,23cyclictest0-21swapper/023:37:540
118799240,23cyclictest0-21swapper/023:30:530
118799240,23cyclictest0-21swapper/023:17:260
118799240,23cyclictest0-21swapper/000:29:310
118799240,1cyclictest27591-21diskmemload21:25:110
118799240,0cyclictest27591-21diskmemload23:20:430
1198992323,0cyclictest0-21swapper/323:50:193
1198992323,0cyclictest0-21swapper/323:34:163
1198992323,0cyclictest0-21swapper/323:15:313
1198992323,0cyclictest0-21swapper/323:10:503
1198992323,0cyclictest0-21swapper/323:08:063
1198992323,0cyclictest0-21swapper/323:00:543
1198992323,0cyclictest0-21swapper/300:33:563
1198992321,1cyclictest18530-21ssh22:56:233
1198992321,1cyclictest0-21swapper/323:22:303
1198992321,1cyclictest0-21swapper/300:18:193
1198992321,1cyclictest0-21swapper/300:09:113
119899230,2cyclictest31162-21sh21:41:013
119899230,23cyclictest0-21swapper/322:52:213
119899230,23cyclictest0-21swapper/321:55:193
119899230,23cyclictest0-21swapper/321:36:203
119899230,22cyclictest9376-21ssh21:21:383
119899230,22cyclictest22616-21sh22:02:433
119899230,22cyclictest17943-21ssh22:27:043
119899230,22cyclictest16387-21ssh00:21:013
119899230,22cyclictest16284-21ssh21:28:003
119899230,22cyclictest12894-21ssh22:22:463
119899230,1cyclictest1974-21ssh23:40:523
119899230,0cyclictest0-21swapper/323:36:113
119899230,0cyclictest0-21swapper/322:10:333
1193992323,0cyclictest2926-21apt-get22:15:012
1193992323,0cyclictest27591-21diskmemload22:36:402
1193992321,1cyclictest12797-21ssh00:18:232
1193992320,2cyclictest6122-21ssh23:15:242
1193992320,2cyclictest10304-21sh21:51:082
119399230,23cyclictest0-21swapper/223:47:272
119399230,23cyclictest0-21swapper/223:38:062
119399230,23cyclictest0-21swapper/221:27:042
119399230,23cyclictest0-21swapper/200:20:452
119399230,22cyclictest777-21ssh23:10:412
119399230,22cyclictest5859-21ssh00:11:302
119399230,22cyclictest5859-21ssh00:11:302
119399230,22cyclictest2818-21munin-run00:10:002
119399230,22cyclictest24832-21latency_hist21:05:002
119399230,22cyclictest22545-21sh22:02:382
119399230,22cyclictest14746-21ssh22:53:492
119399230,22cyclictest10538-21sh21:23:162
119399230,22cyclictest0-21swapper/222:31:282
119399230,0cyclictest27591-21diskmemload21:57:462
119399230,0cyclictest27591-21diskmemload00:25:172
119399230,0cyclictest0-21swapper/223:05:302
119399230,0cyclictest0-21swapper/221:34:102
1188992323,0cyclictest0-21swapper/122:53:091
1188992321,1cyclictest606-21dbus-daemon23:45:011
1188992321,1cyclictest5986-21basename19:20:001
1188992321,1cyclictest27591-21diskmemload00:35:021
1188992320,2cyclictest24354-21sh00:00:101
118899230,2cyclictest28949-21ssh00:32:341
118899230,2cyclictest17328-21ssh23:25:251
118899230,23cyclictest0-21swapper/122:06:171
118899230,23cyclictest0-21swapper/121:26:521
118899230,22cyclictest30702-21sh23:36:051
118899230,22cyclictest29102-21sh22:37:011
118899230,22cyclictest25376-21ssh23:31:281
118899230,22cyclictest24897-21ssh23:02:271
118899230,22cyclictest23368-21ssh00:27:371
118899230,22cyclictest21011-21apt-get22:30:541
118899230,22cyclictest15348-21sh23:51:191
118899230,22cyclictest11251-21sh23:48:201
118899230,1cyclictest31916-21ssh21:41:251
118899230,1cyclictest0-21swapper/100:21:041
118899230,0cyclictest27591-21diskmemload21:13:371
118899230,0cyclictest0-21swapper/123:10:441
118899230,0cyclictest0-21swapper/122:18:211
118899230,0cyclictest0-21swapper/100:16:151
1187992323,0cyclictest0-21swapper/020:15:020
1187992322,1cyclictest0-21swapper/023:51:150
1187992322,1cyclictest0-21swapper/023:01:110
1187992322,1cyclictest0-21swapper/000:07:370
1187992322,0cyclictest0-21swapper/000:18:540
1187992321,1cyclictest0-21swapper/021:53:580
118799230,23cyclictest27591-21diskmemload21:14:270
118799230,23cyclictest0-21swapper/000:36:510
118799230,22cyclictest27668-21/usr/sbin/munin23:05:580
118799230,22cyclictest27591-21diskmemload23:10:570
118799230,22cyclictest22752-21grep19:55:000
118799230,22cyclictest21944-21sh23:57:170
118799230,22cyclictest20951-21ssh21:31:440
118799230,22cyclictest15585-21munin-plugin-st19:40:010
118799230,22cyclictest1535-21sh21:15:460
118799230,22cyclictest0-21swapper/022:00:280
118799230,1cyclictest7679-21ssh22:18:160
118799230,1cyclictest6767-21rm00:12:520
118799230,1cyclictest6767-21rm00:12:520
118799230,0cyclictest0-21swapper/022:55:310
118799230,0cyclictest0-21swapper/022:41:020
1198992222,0cyclictest27591-21diskmemload23:26:333
1198992222,0cyclictest0-21swapper/322:07:293
1198992222,0cyclictest0-21swapper/321:32:183
1198992220,1cyclictest28495-21ssh22:36:073
119899220,21cyclictest0-21swapper/323:55:023
119899220,21cyclictest0-21swapper/322:40:483
119899220,21cyclictest0-21swapper/322:15:143
119899220,21cyclictest0-21swapper/319:45:533
119899220,21cyclictest0-21swapper/300:10:213
119899220,21cyclictest0-21swapper/300:10:213
119899220,1cyclictest0-21swapper/321:10:153
119899220,1cyclictest0-21swapper/300:36:113
119899220,1cyclictest0-21swapper/300:00:343
119899220,0cyclictest0-21swapper/321:50:443
1193992222,0cyclictest27591-21diskmemload23:26:312
1193992221,0cyclictest0-21swapper/219:10:522
1193992219,3cyclictest0-21swapper/221:36:432
119399220,21cyclictest25353-21ssh23:31:262
119399220,21cyclictest16303-21sh23:52:412
119399220,21cyclictest15966-21ssh22:55:112
119399220,21cyclictest0-21swapper/200:35:202
119399220,1cyclictest27591-21diskmemload21:46:302
119399220,1cyclictest12808-21systemctl20:40:002
1188992221,1cyclictest25499-21apt-get20:00:531
1188992220,1cyclictest30385-21cron20:10:001
1188992220,1cyclictest24831-21apt-get21:05:001
118899220,22cyclictest0-21swapper/122:10:041
118899220,22cyclictest0-21swapper/122:00:401
118899220,21cyclictest964-21ssh21:15:031
118899220,21cyclictest606-21dbus-daemon21:10:011
118899220,21cyclictest606-21dbus-daemon21:10:011
118899220,21cyclictest27591-21diskmemload22:45:021
118899220,21cyclictest0-21swapper/100:05:201
118899220,1cyclictest23082-21apt-get19:55:521
118899220,0cyclictest27591-21diskmemload21:51:361
118899220,0cyclictest0-21swapper/122:25:541
1187992222,0cyclictest0-21swapper/022:05:550
1187992222,0cyclictest0-21swapper/000:20:300
1187992220,1cyclictest0-21swapper/023:40:190
1187992220,1cyclictest0-21swapper/022:50:430
118799220,21cyclictest9708-21ssh22:20:470
118799220,21cyclictest27591-21diskmemload22:30:510
118799220,21cyclictest12602-21munin-run23:50:000
118799220,21cyclictest0-21swapper/021:45:180
118799220,1cyclictest17152-21ssh23:25:120
118799220,0cyclictest27591-21diskmemload22:10:110
118799220,0cyclictest0-21swapper/020:05:530
1198992121,0cyclictest0-21swapper/319:25:523
1198992119,1cyclictest17941-21apt-get20:50:533
1193992121,0cyclictest13151-21apt-get20:40:512
1193992121,0cyclictest0-21swapper/222:42:132
1193992121,0cyclictest0-21swapper/219:50:532
1193992121,0cyclictest0-21swapper/219:40:522
1193992119,1cyclictest8692-21apt-get19:25:522
119399210,20cyclictest5941-21apt-get20:25:532
1188992121,0cyclictest0-21swapper/120:30:521
1188992121,0cyclictest0-21swapper/119:40:521
118899210,20cyclictest0-21swapper/120:35:531
118899210,0cyclictest0-21swapper/119:35:541
1187992120,0cyclictest0-21swapper/019:15:520
118799210,20cyclictest0-21swapper/019:20:530
118799210,0cyclictest0-21swapper/019:10:520
1198992020,0cyclictest0-21swapper/321:00:533
1198992020,0cyclictest0-21swapper/320:30:523
1193992020,0cyclictest0-21swapper/219:55:532
119399200,0cyclictest18293-21apt-get19:45:532
119399200,0cyclictest0-21swapper/220:50:532
119399200,0cyclictest0-21swapper/219:30:532
1188992020,0cyclictest0-21swapper/119:50:531
1187992020,0cyclictest0-21swapper/020:45:520
1187992020,0cyclictest0-21swapper/020:00:530
1187992020,0cyclictest0-21swapper/019:55:530
1187992020,0cyclictest0-21swapper/019:45:530
118799200,19cyclictest24829-21perf21:05:000
1198991919,0cyclictest0-21swapper/321:05:023
1198991919,0cyclictest0-21swapper/321:05:023
1198991918,0cyclictest339-21apt-get20:15:023
1198991917,1cyclictest17986-21munin-run19:45:013
1193991919,0cyclictest0-21swapper/220:10:542
1193991919,0cyclictest0-21swapper/220:05:522
1193991919,0cyclictest0-21swapper/219:35:532
1188991919,0cyclictest0-21swapper/120:45:521
1188991919,0cyclictest0-21swapper/120:40:511
1188991918,0cyclictest0-21swapper/120:55:531
1187991919,0cyclictest0-21swapper/020:50:530
1187991917,1cyclictest20337-21apt-get20:55:530
1187991917,1cyclictest15238-21munin-run20:45:000
118799190,18cyclictest0-21swapper/020:30:550
1198991818,0cyclictest0-21swapper/320:55:533
1198991818,0cyclictest0-21swapper/320:05:533
119899180,2cyclictest313-21systemd-journal20:50:003
119899180,17cyclictest12780-21cron20:40:003
1188991816,1cyclictest0-21swapper/120:15:021
1198991717,0cyclictest0-21swapper/320:25:533
1193991717,0cyclictest0-21swapper/221:05:022
1193991717,0cyclictest0-21swapper/221:05:022
1193991717,0cyclictest0-21swapper/220:30:522
1187991717,0cyclictest0-21swapper/020:20:530
1187991717,0cyclictest0-21swapper/020:10:540
118799170,1cyclictest10735-21apt-get20:35:520
1198991616,0cyclictest0-21swapper/320:20:533
1198991616,0cyclictest0-21swapper/319:55:523
1198991615,1cyclictest0-21swapper/320:00:533
1198991613,2cyclictest15557-21grep19:40:013
1198991612,3cyclictest13173-21perf19:35:013
119899160,15cyclictest14199-21ntp_states20:40:573
1193991616,0cyclictest15544-21apt-get20:45:522
119399160,15cyclictest27590-21http20:05:002
1188991616,0cyclictest0-21swapper/119:45:531
1188991616,0cyclictest0-21swapper/119:20:531
1188991614,1cyclictest11091-21apt-get19:30:531
1187991614,1cyclictest8024-21/usr/sbin/munin20:30:010
1187991614,1cyclictest0-21swapper/019:45:010
1198991515,0cyclictest0-21swapper/319:50:533
119899150,0cyclictest6299-21apt-get19:20:533
1193991513,1cyclictest3546-21apt-get20:20:532
1188991515,0cyclictest0-21swapper/120:10:541
1188991513,1cyclictest8694-21apt-get19:25:521
118799150,0cyclictest27594-21ssh21:10:010
118799150,0cyclictest27594-21ssh21:10:010
119899140,14cyclictest0-21swapper/320:10:083
119899140,14cyclictest0-21swapper/319:15:023
119399140,14cyclictest0-21swapper/220:55:212
119399140,14cyclictest0-21swapper/219:20:322
1188991414,0cyclictest0-21swapper/120:50:581
1188991414,0cyclictest0-21swapper/120:20:181
118799140,14cyclictest0-21swapper/019:30:350
118799140,14cyclictest0-21swapper/019:25:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional