You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 08:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Fri Apr 10, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2653992822,5cyclictest28678-21diskmemload23:14:350
265399280,0cyclictest0-21swapper/000:22:010
2653992721,5cyclictest28678-21diskmemload23:42:180
265399270,23cyclictest0-21swapper/021:11:350
265399270,22cyclictest28678-21diskmemload23:35:590
266399260,23cyclictest151rcu_preempt21:28:023
2653992621,4cyclictest0-21swapper/023:21:000
2653992619,6cyclictest0-21swapper/022:59:130
265399260,21cyclictest0-21swapper/023:09:560
265399260,21cyclictest0-21swapper/022:22:090
265399260,1cyclictest8857-21ssh00:30:350
265399260,1cyclictest28678-21diskmemload23:03:550
2663992523,1cyclictest22975-21sh22:08:563
266399250,24cyclictest0-21swapper/322:19:163
266399250,24cyclictest0-21swapper/322:10:193
266399250,1cyclictest18150-21diskmemload21:31:393
2655992523,1cyclictest4279-21cron20:20:011
265599250,24cyclictest32563-21ssh23:19:251
265599250,24cyclictest28678-21diskmemload23:57:121
265599250,24cyclictest0-21swapper/121:25:391
2653992521,3cyclictest0-21swapper/021:51:360
2653992520,4cyclictest0-21swapper/022:19:240
2653992520,4cyclictest0-21swapper/021:41:080
2653992519,5cyclictest18242-21ssh00:39:360
265399250,24cyclictest0-21swapper/023:59:310
265399250,24cyclictest0-21swapper/022:28:310
265399250,24cyclictest0-21swapper/022:10:040
265399250,24cyclictest0-21swapper/022:08:310
265399250,24cyclictest0-21swapper/021:28:360
265399250,1cyclictest0-21swapper/021:58:120
265399250,0cyclictest0-21swapper/023:25:320
265399250,0cyclictest0-21swapper/021:39:360
2663992422,1cyclictest28678-21diskmemload22:24:503
2663992421,2cyclictest16759-21sh23:35:173
266399240,23cyclictest401-21ssh00:23:303
266399240,23cyclictest31867-21ssh23:49:413
266399240,23cyclictest313-21systemd-journal23:09:153
266399240,23cyclictest26304-21awk21:05:013
266399240,23cyclictest0-21swapper/322:35:343
266399240,23cyclictest0-21swapper/321:42:463
266399240,1cyclictest28678-21diskmemload00:00:083
266399240,0cyclictest0-21swapper/321:38:433
265999242,21cyclictest22014-21ssh23:40:342
2659992420,1cyclictest0-21swapper/200:24:402
265999240,23cyclictest28678-21diskmemload00:33:182
265999240,23cyclictest25891-21ssh00:15:322
265999240,23cyclictest13469-21ssh21:27:242
2655992424,0cyclictest0-21swapper/100:31:431
2655992423,1cyclictest28678-21diskmemload23:41:041
2655992421,2cyclictest13906-21perf20:40:001
265599240,24cyclictest0-21swapper/123:47:161
265599240,1cyclictest29432-21awk20:05:001
2653992420,4cyclictest0-21swapper/000:17:560
2653992420,3cyclictest0-21swapper/000:28:410
265399240,23cyclictest5024-21sh19:15:000
265399240,23cyclictest27631-21apt-get20:04:110
265399240,23cyclictest0-21swapper/022:30:120
265399240,23cyclictest0-21swapper/000:08:110
265399240,20cyclictest28678-21diskmemload23:17:320
265399240,20cyclictest0-21swapper/021:30:210
265399240,0cyclictest28678-21diskmemload21:20:420
265399240,0cyclictest0-21swapper/022:47:010
265399240,0cyclictest0-21swapper/000:02:410
2663992323,0cyclictest0-21swapper/323:26:123
2663992322,1cyclictest28678-21diskmemload23:17:123
2663992322,1cyclictest28678-21diskmemload22:50:453
2663992322,0cyclictest0-21swapper/320:39:133
2663992321,1cyclictest34-21ksoftirqd/323:42:463
2663992321,1cyclictest32476-21apt-get20:14:123
2663992321,1cyclictest32476-21apt-get20:14:123
2663992320,2cyclictest13974-21ssh23:34:003
2663992320,2cyclictest1055-21ssh21:46:333
2663992320,2cyclictest10072-21latency_hist21:55:013
266399230,23cyclictest28678-21diskmemload23:55:073
266399230,23cyclictest28678-21diskmemload00:37:253
266399230,23cyclictest0-21swapper/300:08:433
266399230,22cyclictest4563-21ssh00:26:543
266399230,22cyclictest3723-21ntp_states23:54:193
266399230,22cyclictest21456-21ssh00:11:293
266399230,22cyclictest0-21swapper/321:56:383
2659992323,0cyclictest28678-21diskmemload00:27:492
2659992323,0cyclictest0-21swapper/223:11:192
2659992323,0cyclictest0-21swapper/223:05:472
2659992323,0cyclictest0-21swapper/222:58:542
2659992323,0cyclictest0-21swapper/221:56:232
2659992323,0cyclictest0-21swapper/221:37:182
2659992323,0cyclictest0-21swapper/200:00:312
2659992322,0cyclictest0-21swapper/223:58:282
2659992321,2cyclictest27540-21ssh22:12:232
2659992321,1cyclictest0-21swapper/222:44:162
2659992321,1cyclictest0-21swapper/200:36:382
2659992320,1cyclictest0-21swapper/223:50:132
265999230,23cyclictest13618-1kworker/u9:021:33:102
265999230,23cyclictest0-21swapper/200:14:442
265999230,22cyclictest22315-21ssh22:07:472
265999230,22cyclictest17440-21rm22:03:152
265999230,22cyclictest16350-21latency_hist23:35:012
265999230,1cyclictest0-21swapper/221:14:472
265999230,0cyclictest0-21swapper/223:28:222
265999230,0cyclictest0-21swapper/223:22:122
2655992323,0cyclictest0-21swapper/122:57:241
2655992323,0cyclictest0-21swapper/121:18:521
2655992323,0cyclictest0-21swapper/100:18:021
2655992323,0cyclictest0-21swapper/100:06:231
2655992322,1cyclictest5620-21apt-get19:19:121
2655992321,2cyclictest736-21ssh23:20:021
2655992321,1cyclictest7582-21sh21:21:141
2655992321,1cyclictest0-21swapper/100:10:091
2655992320,2cyclictest26335-21ssh22:42:581
265599230,2cyclictest21819-21cron19:50:001
265599230,23cyclictest28678-21diskmemload00:36:201
265599230,23cyclictest0-21swapper/123:34:031
265599230,23cyclictest0-21swapper/122:22:561
265599230,23cyclictest0-21swapper/121:44:211
265599230,23cyclictest0-21swapper/121:13:571
265599230,22cyclictest352-21ssh23:50:431
265599230,22cyclictest28678-21diskmemload23:14:191
265599230,22cyclictest19916-21ssh23:08:331
265599230,22cyclictest15921-21ssh22:32:491
265599230,22cyclictest0-21swapper/119:55:001
265599230,22cyclictest0-21swapper/100:21:371
265599230,1cyclictest26999-21ssh22:11:261
265599230,0cyclictest0-21swapper/121:56:151
2653992323,0cyclictest0-21swapper/022:44:400
2653992320,3cyclictest0-21swapper/023:53:320
265399230,22cyclictest28678-21diskmemload21:15:350
265399230,22cyclictest14659-21munin-run19:35:010
265399230,22cyclictest0-21swapper/022:04:340
265399230,22cyclictest0-21swapper/020:44:130
265399230,22cyclictest0-21swapper/000:13:540
265399230,1cyclictest0-21swapper/022:37:380
2663992222,0cyclictest0-21swapper/323:11:573
2663992220,1cyclictest0-21swapper/322:30:273
266399220,21cyclictest28678-21diskmemload00:30:063
266399220,21cyclictest28678-21diskmemload00:15:063
266399220,21cyclictest0-21swapper/321:20:113
266399220,21cyclictest0-21swapper/321:10:353
266399220,21cyclictest0-21swapper/320:55:023
266399220,21cyclictest0-21swapper/320:34:133
266399220,21cyclictest0-21swapper/320:09:133
266399220,21cyclictest0-21swapper/319:39:133
266399220,0cyclictest0-21swapper/323:20:393
2659992222,0cyclictest0-21swapper/222:37:032
2659992222,0cyclictest0-21swapper/221:16:082
2659992220,1cyclictest0-21swapper/219:25:012
265999220,22cyclictest0-21swapper/222:30:082
265999220,22cyclictest0-21swapper/222:28:172
265999220,22cyclictest0-21swapper/222:21:442
265999220,22cyclictest0-21swapper/221:50:212
265999220,22cyclictest0-21swapper/200:05:012
265999220,21cyclictest30714-21ssh22:46:462
265999220,21cyclictest28678-21diskmemload23:45:432
265999220,21cyclictest28678-21diskmemload22:50:492
265999220,21cyclictest0-21swapper/221:49:022
265999220,21cyclictest0-21swapper/221:21:332
265999220,1cyclictest28160-21ssh23:15:172
265999220,1cyclictest13892-21turbostat20:40:002
265999220,0cyclictest0-21swapper/223:00:302
2655992222,0cyclictest0-21swapper/122:25:041
265599220,22cyclictest4111-21ssh00:26:141
265599220,22cyclictest22513-21ssh21:35:291
265599220,22cyclictest0-21swapper/123:00:311
265599220,22cyclictest0-21swapper/122:09:141
265599220,22cyclictest0-21swapper/122:00:091
265599220,22cyclictest0-21swapper/121:31:331
265599220,21cyclictest2411-21ssh22:50:041
265599220,21cyclictest0-21swapper/123:35:031
265599220,21cyclictest0-21swapper/123:27:311
265599220,21cyclictest0-21swapper/121:45:481
265599220,21cyclictest0-21swapper/120:44:131
265599220,1cyclictest20923-21sh22:37:321
265599220,1cyclictest0-21swapper/100:01:071
2653992221,1cyclictest0-21swapper/023:46:380
2653992221,1cyclictest0-21swapper/022:53:520
2653992220,2cyclictest0-21swapper/023:33:050
2653992220,1cyclictest606-21dbus-daemon20:00:000
2653992219,2cyclictest0-21swapper/021:45:270
2653992215,6cyclictest21845-21turbostat19:50:000
265399220,2cyclictest6864-21sed20:25:010
265399220,21cyclictest0-21swapper/019:29:120
265399220,1cyclictest4298-21latency_hist20:20:010
2663992119,1cyclictest12829-21apt-get19:34:123
266399210,21cyclictest0-21swapper/322:57:073
266399210,21cyclictest0-21swapper/322:46:573
266399210,21cyclictest0-21swapper/322:25:093
266399210,21cyclictest0-21swapper/322:00:073
266399210,21cyclictest0-21swapper/321:16:013
266399210,20cyclictest26886-21apt-get21:09:143
266399210,20cyclictest0-21swapper/322:41:113
266399210,1cyclictest22431-21apt-get19:54:123
266399210,1cyclictest0-21swapper/323:01:183
2659992120,0cyclictest7282-21apt-get20:29:132
2659992119,1cyclictest19464-21munin-plugin-st19:45:002
265999210,21cyclictest0-21swapper/222:15:242
265999210,1cyclictest24493-21apt-get21:04:142
2655992119,1cyclictest5048-21munin-run19:15:001
2655992119,1cyclictest31460-21ssh22:15:191
2655992119,1cyclictest1892-21basename20:15:011
2655992119,1cyclictest1892-21basename20:15:011
265599210,21cyclictest0-21swapper/122:49:181
265599210,20cyclictest4889-21apt-get20:24:131
265599210,20cyclictest21100-21apt-get20:55:021
265599210,20cyclictest0-21swapper/119:59:101
2653992121,0cyclictest0-21swapper/020:59:120
265399210,21cyclictest0-21swapper/020:09:130
265399210,21cyclictest0-21swapper/019:24:130
265399210,20cyclictest0-21swapper/021:09:140
265399210,1cyclictest15857-21tail19:39:160
265399210,1cyclictest0-21swapper/020:54:140
265399210,0cyclictest0-21swapper/019:44:110
2663992020,0cyclictest0-21swapper/319:19:123
266399200,0cyclictest0-21swapper/320:54:133
266399200,0cyclictest0-21swapper/320:44:133
266399200,0cyclictest0-21swapper/319:49:133
2659992020,0cyclictest0-21swapper/220:34:122
2659992020,0cyclictest0-21swapper/220:09:132
2659992020,0cyclictest0-21swapper/219:14:122
265999200,20cyclictest0-21swapper/221:09:142
265999200,0cyclictest0-21swapper/220:14:122
265999200,0cyclictest0-21swapper/220:14:122
2655992020,0cyclictest0-21swapper/120:49:131
2655992019,0cyclictest0-21swapper/119:34:121
2655992017,2cyclictest19469-21idleruntime-cro19:45:001
265599200,20cyclictest0-21swapper/121:04:141
2653992020,0cyclictest0-21swapper/020:49:130
2653992020,0cyclictest0-21swapper/020:39:130
265399200,0cyclictest0-21swapper/020:34:120
2663991919,0cyclictest0-21swapper/320:29:133
2663991917,1cyclictest0-21swapper/319:15:003
2659991919,0cyclictest0-21swapper/219:59:102
2659991918,1cyclictest27279-21apt-get20:00:022
2655991919,0cyclictest0-21swapper/120:09:131
2655991919,0cyclictest0-21swapper/119:39:131
2655991918,0cyclictest8039-21apt-get19:24:131
2653991919,0cyclictest0-21swapper/021:04:140
2653991919,0cyclictest0-21swapper/019:19:110
2663991818,0cyclictest0-21swapper/319:59:103
2659991818,0cyclictest0-21swapper/220:24:132
2659991817,1cyclictest22073-21apt-get20:59:122
2659991817,1cyclictest16890-21apt-get20:49:132
2659991816,1cyclictest15226-21apt-get19:39:132
2663991717,0cyclictest0-21swapper/319:44:113
266399170,16cyclictest10431-21apt-get19:29:133
2659991717,0cyclictest0-21swapper/219:29:122
2659991614,1cyclictest5566-21chrt19:18:562
2659991614,1cyclictest0-21swapper/219:49:222
265999160,15cyclictest14690-21cpuspeed_turbos20:44:132
2655991614,1cyclictest0-21swapper/119:29:121
265599160,15cyclictest21112-21munin-run20:55:001
2663991514,0cyclictest0-21swapper/320:24:133
2663991513,1cyclictest9082-21ntp_states19:24:183
2663991512,2cyclictest18680-21cron20:50:003
2655991515,0cyclictest0-21swapper/121:09:131
2663991414,0cyclictest0-21swapper/320:15:183
2663991412,1cyclictest0-21swapper/320:00:023
2659991414,0cyclictest0-21swapper/219:50:202
265999140,14cyclictest0-21swapper/220:15:172
265999140,0cyclictest0-21swapper/219:30:202
2655991414,0cyclictest0-21swapper/120:30:161
2655991414,0cyclictest0-21swapper/120:25:391
2653991414,0cyclictest0-21swapper/020:26:130
265399140,14cyclictest0-21swapper/020:10:120
265399140,14cyclictest0-21swapper/020:10:120
265399140,14cyclictest0-21swapper/019:50:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional