You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 06:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Jun 21, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2912992523,1cyclictest14013-21ssh23:29:283
2910992520,4cyclictest0-21swapper/221:33:442
291099250,24cyclictest11180-21ssh00:27:392
291099250,24cyclictest0-21swapper/222:28:522
291099250,21cyclictest0-21swapper/221:25:012
2905992523,1cyclictest31174-21perf22:45:011
290599250,24cyclictest28553-21diskmemload22:30:151
290599250,24cyclictest17507-21awk21:30:001
290599250,24cyclictest0-21swapper/121:16:531
290599250,1cyclictest11894-21rm22:55:571
2904992523,1cyclictest0-21swapper/021:42:480
290499250,24cyclictest5073-21ssh21:17:550
290499250,24cyclictest0-21swapper/023:46:410
290499250,24cyclictest0-21swapper/022:50:510
2912992422,1cyclictest0-21swapper/321:33:583
291299240,24cyclictest0-21swapper/300:32:083
291299240,23cyclictest16578-21sh20:45:003
2910992420,1cyclictest4658-21cstates22:50:162
2910992420,0cyclictest0-21swapper/200:16:342
2910992419,0cyclictest0-21swapper/223:43:362
291099240,23cyclictest4932-21cron22:20:002
291099240,23cyclictest0-21swapper/222:38:062
291099240,23cyclictest0-21swapper/222:06:102
291099240,23cyclictest0-21swapper/221:19:192
291099240,1cyclictest28553-21diskmemload21:55:052
291099240,1cyclictest0-21swapper/223:50:182
290599240,23cyclictest31045-21sh22:13:211
290599240,23cyclictest0-21swapper/121:40:331
290599240,23cyclictest0-21swapper/121:31:521
290599240,1cyclictest20132-21perf19:45:011
290599240,1cyclictest0-21swapper/122:53:521
290599240,0cyclictest28553-21diskmemload21:59:381
290599240,0cyclictest0-21swapper/123:11:161
2904992423,0cyclictest0-21swapper/000:20:010
2904992422,1cyclictest28553-21diskmemload00:21:270
2904992421,2cyclictest622-21gdbus21:50:010
2904992421,2cyclictest0-21swapper/000:14:010
290499240,24cyclictest0-21swapper/023:56:510
290499240,24cyclictest0-21swapper/000:33:400
290499240,23cyclictest0-21swapper/021:30:190
290499240,1cyclictest30317-21sh22:43:450
2912992323,0cyclictest28553-21diskmemload22:15:223
2912992323,0cyclictest0-21swapper/323:24:003
2912992323,0cyclictest0-21swapper/323:07:463
2912992323,0cyclictest0-21swapper/322:47:043
2912992323,0cyclictest0-21swapper/321:53:043
2912992322,1cyclictest28553-21diskmemload21:35:363
2912992322,1cyclictest22050irq/143-nvme0q420:20:013
2912992321,1cyclictest0-21swapper/323:47:403
2912992321,1cyclictest0-21swapper/322:41:573
2912992321,1cyclictest0-21swapper/321:55:523
2912992321,1cyclictest0-21swapper/321:41:153
2912992321,1cyclictest0-21swapper/321:25:013
2912992320,2cyclictest28553-21diskmemload23:30:223
291299230,22cyclictest9913-21sh21:22:083
291299230,22cyclictest4840-21ssh21:17:163
291299230,22cyclictest28553-21diskmemload22:36:213
291299230,22cyclictest21011-21sh00:36:333
291299230,22cyclictest1198-21sh21:45:093
291299230,0cyclictest28553-21diskmemload23:15:243
291299230,0cyclictest28553-21diskmemload22:56:583
291299230,0cyclictest28553-21diskmemload22:21:173
291299230,0cyclictest28553-21diskmemload00:00:053
291299230,0cyclictest0-21swapper/323:51:133
291299230,0cyclictest0-21swapper/323:37:583
291299230,0cyclictest0-21swapper/322:02:213
2910992323,0cyclictest28553-21diskmemload22:10:082
2910992323,0cyclictest0-21swapper/223:16:112
2910992323,0cyclictest0-21swapper/200:24:062
2910992322,1cyclictest24522-21apt-get00:10:152
2910992322,0cyclictest0-21swapper/219:55:142
2910992321,1cyclictest3123-21uptime21:45:242
2910992321,1cyclictest0-21swapper/223:37:272
2910992321,1cyclictest0-21swapper/221:21:052
2910992319,4cyclictest0-21swapper/223:59:482
2910992319,3cyclictest4526-21basename20:20:012
2910992319,3cyclictest20166-21apt-get23:05:142
291099230,2cyclictest407-21ssh23:46:092
291099230,2cyclictest21372-21cron20:55:002
291099230,23cyclictest0-21swapper/223:30:222
291099230,23cyclictest0-21swapper/223:00:132
291099230,23cyclictest0-21swapper/221:40:442
291099230,23cyclictest0-21swapper/221:35:102
291099230,22cyclictest9389-21apt-get23:25:142
291099230,22cyclictest16976-21ssh00:02:022
291099230,22cyclictest13152-21ssh00:30:102
291099230,1cyclictest28553-21diskmemload22:48:372
291099230,0cyclictest0-21swapper/223:11:372
2905992323,0cyclictest0-21swapper/123:51:121
2905992323,0cyclictest0-21swapper/121:47:021
2905992323,0cyclictest0-21swapper/121:35:571
2905992323,0cyclictest0-21swapper/100:33:221
2905992323,0cyclictest0-21swapper/100:16:211
2905992322,1cyclictest28553-21diskmemload21:51:531
2905992320,2cyclictest31933-21/usr/sbin/munin23:15:181
290599230,23cyclictest0-21swapper/123:59:231
290599230,22cyclictest28553-21diskmemload22:15:541
290599230,22cyclictest2652-21apt-get00:20:161
290599230,22cyclictest23917-21ssh22:36:581
290599230,22cyclictest10457-21ssh00:26:271
290599230,0cyclictest0-21swapper/123:02:411
2904992323,0cyclictest0-21swapper/022:34:420
2904992323,0cyclictest0-21swapper/022:06:020
2904992323,0cyclictest0-21swapper/000:27:570
2904992322,1cyclictest62550irq/127-eno100:00:070
290499232,1cyclictest62550irq/127-eno122:48:160
2904992320,2cyclictest28553-21diskmemload22:56:420
2904992320,2cyclictest22164-21ssh22:35:180
2904992320,2cyclictest21311-21ssh22:04:390
2904992320,2cyclictest12771-21sh22:26:100
290499231,1cyclictest27272-21unixbench_singl23:10:210
290499230,2cyclictest6472-21ssh23:21:070
290499230,2cyclictest21247-21sh00:05:270
290499230,23cyclictest28553-21diskmemload22:12:250
290499230,23cyclictest0-21swapper/023:51:540
290499230,23cyclictest0-21swapper/023:27:230
290499230,22cyclictest28553-21diskmemload22:21:300
290499230,22cyclictest24458-21ssh23:08:590
290499230,22cyclictest22757-21sh00:39:220
290499230,22cyclictest16807-21ssh21:28:570
290499230,22cyclictest0-21swapper/021:10:010
290499230,1cyclictest606-21dbus-daemon23:00:210
290499230,1cyclictest28553-21diskmemload21:58:510
290499230,0cyclictest28553-21diskmemload23:37:120
290499230,0cyclictest0-21swapper/023:32:200
290499230,0cyclictest0-21swapper/021:35:450
2912992222,0cyclictest28553-21diskmemload23:56:073
2912992221,1cyclictest0-21swapper/300:05:033
2912992220,1cyclictest28553-21diskmemload00:13:243
2912992220,1cyclictest0-21swapper/322:25:143
291299220,21cyclictest29701-21ssh00:15:143
291299220,21cyclictest14968-21apt-get23:00:133
291299220,21cyclictest0-21swapper/323:40:363
291299220,21cyclictest0-21swapper/323:10:063
291299220,21cyclictest0-21swapper/322:50:183
291299220,21cyclictest0-21swapper/300:25:153
291299220,21cyclictest0-21swapper/300:20:313
291299220,1cyclictest31212-21rm21:11:143
291299220,1cyclictest24632-21ssh22:06:353
291299220,1cyclictest18979-21dump-pmu-power20:50:003
291299220,0cyclictest0-21swapper/319:20:123
2910992222,0cyclictest28553-21diskmemload22:40:032
2910992220,1cyclictest21640-21apt-get20:55:132
2910992220,1cyclictest0-21swapper/222:20:092
2910992220,1cyclictest0-21swapper/221:50:302
2910992219,3cyclictest0-21swapper/222:03:002
2910992219,2cyclictest611-21irqbalance00:05:242
291099220,21cyclictest0-21swapper/223:20:032
291099220,21cyclictest0-21swapper/222:30:362
291099220,21cyclictest0-21swapper/220:10:132
291099220,1cyclictest28889-21ssh21:10:082
2905992222,0cyclictest0-21swapper/123:42:281
2905992222,0cyclictest0-21swapper/123:25:211
2905992222,0cyclictest0-21swapper/122:47:571
2905992222,0cyclictest0-21swapper/100:10:491
2905992220,1cyclictest28999-21apt-get21:10:141
2905992220,1cyclictest27336-21munin-run20:00:001
2905992220,1cyclictest23189-21ssh22:05:231
2905992220,1cyclictest14476-21cstates00:00:151
2905992220,1cyclictest0-21swapper/123:45:151
290599220,22cyclictest0-21swapper/122:25:141
290599220,21cyclictest5234-21cron19:15:001
290599220,21cyclictest28553-21diskmemload23:05:061
290599220,21cyclictest26449-21apt-get21:05:121
290599220,21cyclictest21716-21rm00:06:221
290599220,21cyclictest18479-21apt-get00:35:141
290599220,21cyclictest0-21swapper/123:35:201
290599220,1cyclictest29990-21apt-get20:05:121
290599220,1cyclictest0-21swapper/122:00:281
2904992222,0cyclictest28553-21diskmemload21:23:120
2904992220,1cyclictest32121-21idleruntime-cro20:10:000
2904992220,1cyclictest0-21swapper/023:41:410
2904992220,1cyclictest0-21swapper/022:15:210
290499220,2cyclictest992-21rm23:15:500
290499220,21cyclictest10057-21cron19:25:000
290499220,21cyclictest0-21swapper/021:50:560
290499220,1cyclictest0-21swapper/020:30:110
2912992121,0cyclictest27314-21apt-get22:10:153
2912992121,0cyclictest0-21swapper/319:12:513
2912992119,1cyclictest7681-21munin-run19:20:003
291299210,20cyclictest0-21swapper/319:25:023
291299210,1cyclictest11709-21awk20:35:003
291299210,0cyclictest0-21swapper/322:30:153
2910992120,1cyclictest1649-21cron19:25:002
2910992120,0cyclictest17995-21apt-get19:40:112
291099210,20cyclictest7193-21apt-get20:25:142
2905992120,1cyclictest24916-21idleruntime-cro19:55:001
290599210,21cyclictest0-21swapper/123:20:211
290599210,20cyclictest15895-21ntp_states23:30:181
290599210,0cyclictest0-21swapper/119:30:131
2904992121,0cyclictest0-21swapper/021:05:120
2904992120,1cyclictest5527-21apt-get19:15:130
2904992120,1cyclictest0-21swapper/019:40:120
290499210,21cyclictest0-21swapper/020:35:130
2912992020,0cyclictest0-21swapper/320:10:133
2912992018,1cyclictest25181-21apt-get19:55:143
291299200,20cyclictest0-21swapper/320:50:133
291299200,19cyclictest14087-21sendmail20:40:003
291299200,0cyclictest0-21swapper/319:50:133
2910992020,0cyclictest0-21swapper/220:45:152
291099200,20cyclictest0-21swapper/220:35:132
291099200,1cyclictest0-21swapper/219:50:132
2905992020,0cyclictest0-21swapper/120:55:131
2905992020,0cyclictest0-21swapper/120:35:121
2905992018,1cyclictest26184-21perf21:05:011
2905992018,1cyclictest26184-21perf21:05:011
2904992020,0cyclictest22784-21apt-get19:50:130
2904992020,0cyclictest0-21swapper/020:20:130
2904992020,0cyclictest0-21swapper/019:25:130
290499200,0cyclictest16852-21apt-get20:45:140
2912991919,0cyclictest0-21swapper/321:00:133
2912991919,0cyclictest0-21swapper/321:00:133
2912991917,1cyclictest34-21ksoftirqd/321:00:003
2910991919,0cyclictest0-21swapper/219:15:132
2910991918,1cyclictest0-21swapper/220:40:132
2905991919,0cyclictest0-21swapper/120:20:141
2905991917,1cyclictest7688-21idleruntime-cro19:20:001
2905991917,1cyclictest0-21swapper/120:35:001
2904991919,0cyclictest0-21swapper/020:15:130
2904991919,0cyclictest0-21swapper/019:10:130
2904991918,0cyclictest21640-21apt-get20:55:130
2912991818,0cyclictest0-21swapper/320:25:143
2910991818,0cyclictest0-21swapper/219:35:142
2910991817,1cyclictest10735-21apt-get19:25:132
2910991817,0cyclictest3132-21apt-get19:10:132
2910991817,0cyclictest0-21swapper/220:00:122
2910991816,1cyclictest24054-21apt-get21:00:132
2910991816,1cyclictest24054-21apt-get21:00:132
2905991818,0cyclictest0-21swapper/120:45:141
2905991818,0cyclictest0-21swapper/120:25:141
2905991818,0cyclictest0-21swapper/119:25:131
290599180,17cyclictest0-21swapper/120:15:131
2904991818,0cyclictest0-21swapper/021:00:130
2904991818,0cyclictest0-21swapper/021:00:130
2904991816,1cyclictest13132-21apt-get19:30:130
2912991715,1cyclictest611-21irqbalance19:35:543
2910991717,0cyclictest0-21swapper/219:30:132
2910991716,1cyclictest4798-21apt-get20:20:132
2905991717,0cyclictest0-21swapper/119:45:121
2905991715,1cyclictest21378-21munin-run20:55:001
2904991717,0cyclictest0-21swapper/020:50:130
2910991615,0cyclictest20389-21apt-get19:45:122
291099160,15cyclictest313-21systemd-journal20:10:002
291099160,15cyclictest10010-21diskstats20:30:122
2905991614,1cyclictest2119-21ls20:15:011
2912991513,1cyclictest1-21systemd20:10:003
2910991515,0cyclictest0-21swapper/221:05:122
2905991515,0cyclictest0-21swapper/119:35:181
2904991514,0cyclictest0-21swapper/019:35:140
2912991414,0cyclictest0-21swapper/319:45:313
2912991413,0cyclictest0-21swapper/320:00:123
291299140,14cyclictest0-21swapper/320:20:013
291299140,0cyclictest0-21swapper/321:05:163
291299140,0cyclictest0-21swapper/319:35:013
2905991414,0cyclictest0-21swapper/120:41:171
2904991414,0cyclictest0-21swapper/020:40:100
2904991414,0cyclictest0-21swapper/019:55:140
290499140,14cyclictest0-21swapper/020:25:070
290499140,14cyclictest0-21swapper/020:02:160
290499140,0cyclictest0-21swapper/020:10:350
290499140,0cyclictest0-21swapper/019:45:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional