You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 19:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sun May 03, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
734699250,24cyclictest1890-21diskmemload09:43:413
734699250,24cyclictest0-21swapper/311:17:043
734599250,24cyclictest1890-21diskmemload10:01:252
734499250,24cyclictest19859-21ssh11:21:031
734499250,24cyclictest17540-21sh11:46:221
7343992525,0cyclictest0-21swapper/010:49:490
734399250,24cyclictest0-21swapper/011:32:500
734399250,24cyclictest0-21swapper/011:05:060
734399250,24cyclictest0-21swapper/009:45:440
734399250,24cyclictest0-21swapper/009:40:440
734399250,1cyclictest29541-21/usr/sbin/munin11:01:160
734399250,1cyclictest0-21swapper/009:10:360
7346992423,1cyclictest9884-21ssh12:09:473
7346992423,0cyclictest1890-21diskmemload12:31:203
7346992421,2cyclictest15994-21ssh10:48:183
734699240,23cyclictest1890-21diskmemload11:03:353
734699240,23cyclictest1890-21diskmemload10:12:453
734699240,23cyclictest0-21swapper/309:54:533
7345992424,0cyclictest0-21swapper/211:04:002
7345992422,1cyclictest7902-21munin-run09:15:012
734599240,2cyclictest3472-21sendmail09:40:002
734599240,23cyclictest6462-21ssh10:40:182
734599240,23cyclictest1890-21diskmemload09:31:122
7344992422,1cyclictest29081-21ssh11:57:021
7344992422,1cyclictest0-21swapper/109:24:271
734499240,24cyclictest1890-21diskmemload12:23:171
734499240,24cyclictest0-21swapper/111:28:571
734499240,24cyclictest0-21swapper/111:08:491
734499240,23cyclictest0-21swapper/110:48:321
734499240,1cyclictest13238-21timerandwakeup10:16:201
7343992424,0cyclictest1890-21diskmemload10:56:560
7343992423,1cyclictest1890-21diskmemload10:17:500
7343992423,1cyclictest13-21ksoftirqd/010:23:510
7343992422,1cyclictest0-21swapper/011:51:140
734399240,24cyclictest1890-21diskmemload12:39:330
734399240,24cyclictest0-21swapper/009:39:410
734399240,23cyclictest29519-21munin-run09:00:000
734399240,23cyclictest1890-21diskmemload12:22:560
734399240,1cyclictest29721-21diskmemload11:29:300
734399240,1cyclictest26485-21hddtemp_smartct09:31:140
734399240,0cyclictest0-21swapper/010:39:440
7346992323,0cyclictest1890-21diskmemload10:50:543
7346992323,0cyclictest1890-21diskmemload10:44:123
7346992323,0cyclictest0-21swapper/311:43:543
7346992323,0cyclictest0-21swapper/311:35:213
7346992323,0cyclictest0-21swapper/311:10:303
7346992321,1cyclictest5377-21ssh09:11:353
7346992321,1cyclictest0-21swapper/311:59:563
7346992320,2cyclictest21620-21diskmemload12:20:323
734699230,2cyclictest22134-21ssh09:26:273
734699230,23cyclictest0-21swapper/312:18:403
734699230,23cyclictest0-21swapper/311:47:383
734699230,23cyclictest0-21swapper/310:59:483
734699230,23cyclictest0-21swapper/310:15:473
734699230,22cyclictest31989-21ssh12:00:373
734699230,22cyclictest31382-21ssh12:28:473
734699230,22cyclictest23437-21sh09:56:293
734699230,22cyclictest1890-21diskmemload11:53:213
734699230,22cyclictest10873-21sh12:10:583
734699230,22cyclictest0-21swapper/311:25:023
734699230,0cyclictest0-21swapper/311:33:303
734699230,0cyclictest0-21swapper/310:36:533
734699230,0cyclictest0-21swapper/310:21:373
7345992323,0cyclictest0-21swapper/212:06:432
7345992323,0cyclictest0-21swapper/211:50:332
7345992323,0cyclictest0-21swapper/211:26:472
7345992323,0cyclictest0-21swapper/211:11:052
7345992323,0cyclictest0-21swapper/209:41:442
7345992321,1cyclictest0-21swapper/210:26:162
7345992320,1cyclictest6688-21sh11:36:562
734599230,2cyclictest9478-21ssh10:13:402
734599230,2cyclictest3447-21ssh12:03:392
734599230,23cyclictest0-21swapper/212:32:472
734599230,23cyclictest0-21swapper/212:21:172
734599230,23cyclictest0-21swapper/210:30:282
734599230,23cyclictest0-21swapper/209:51:592
734599230,23cyclictest0-21swapper/209:17:142
734599230,22cyclictest9716-21apt07:15:002
734599230,22cyclictest606-21dbus-daemon12:00:002
734599230,22cyclictest31968-21/usr/sbin/munin11:31:152
734599230,22cyclictest27373-21sh12:25:462
734599230,22cyclictest0-21swapper/211:20:012
734599230,0cyclictest0-21swapper/211:18:452
734599230,0cyclictest0-21swapper/210:50:212
7344992323,0cyclictest1890-21diskmemload12:31:291
7344992323,0cyclictest0-21swapper/112:12:491
7344992323,0cyclictest0-21swapper/112:02:511
7344992323,0cyclictest0-21swapper/111:36:091
7344992323,0cyclictest0-21swapper/111:00:321
7344992322,1cyclictest1890-21diskmemload10:14:451
734499230,2cyclictest9463-21ssh09:45:101
734499230,23cyclictest1890-21diskmemload10:00:041
734499230,23cyclictest0-21swapper/112:28:281
734499230,23cyclictest0-21swapper/111:18:051
734499230,23cyclictest0-21swapper/110:52:071
734499230,23cyclictest0-21swapper/109:40:221
734499230,23cyclictest0-21swapper/109:15:561
734499230,22cyclictest622-21gdbus09:05:011
734499230,22cyclictest622-21gdbus09:05:011
734499230,22cyclictest2894-21ssh11:34:191
734499230,22cyclictest2319-21sh10:06:331
734499230,22cyclictest15075-21munin-run08:30:011
734499230,22cyclictest0-21swapper/110:42:141
734499230,0cyclictest0-21swapper/110:59:101
7343992323,0cyclictest0-21swapper/011:55:420
7343992321,2cyclictest0-21swapper/010:25:150
7343992321,1cyclictest0-21swapper/012:25:090
7343992321,1cyclictest0-21swapper/011:10:050
7343992321,1cyclictest0-21swapper/010:02:040
7343992320,2cyclictest10153-21ssh11:41:130
734399230,23cyclictest0-21swapper/012:16:590
734399230,23cyclictest0-21swapper/009:15:030
734399230,22cyclictest9731-21perf07:15:000
734399230,22cyclictest1890-21diskmemload11:37:370
734399230,22cyclictest0-21swapper/012:30:030
734399230,22cyclictest0-21swapper/010:50:460
734399230,1cyclictest2296-21cut09:10:010
734399230,0cyclictest0-21swapper/011:17:090
734399230,0cyclictest0-21swapper/010:33:180
734399230,0cyclictest0-21swapper/010:06:010
7346992222,0cyclictest0-21swapper/310:03:103
7346992220,1cyclictest12186-21ssh09:18:173
7346992220,1cyclictest1039-21cat10:06:133
734699220,21cyclictest0-21swapper/312:40:003
734699220,21cyclictest0-21swapper/310:30:203
734699220,21cyclictest0-21swapper/310:25:403
734699220,21cyclictest0-21swapper/309:30:043
734699220,21cyclictest0-21swapper/308:51:113
734699220,21cyclictest0-21swapper/308:46:113
734699220,1cyclictest9821-21ssh09:45:373
734699220,1cyclictest19784-21apt-get07:36:113
734699220,1cyclictest0-21swapper/311:20:173
734699220,1cyclictest0-21swapper/311:08:033
734699220,1cyclictest0-21swapper/309:39:413
734699220,0cyclictest0-21swapper/309:20:213
7345992222,0cyclictest1890-21diskmemload10:16:342
7345992222,0cyclictest0-21swapper/210:57:182
7345992222,0cyclictest0-21swapper/209:45:312
7345992221,1cyclictest0-21swapper/211:06:362
7345992220,2cyclictest32670-21ssh10:06:002
734599220,22cyclictest0-21swapper/212:15:262
734599220,21cyclictest20878-21ssh09:55:172
734599220,21cyclictest1890-21diskmemload09:20:052
734599220,21cyclictest16937-21perf07:30:012
734599220,1cyclictest9871-21ssh12:38:362
734599220,1cyclictest930-21ssh10:35:232
734599220,1cyclictest21780-21basename10:25:002
734599220,1cyclictest12935-21ssh11:42:442
734599220,0cyclictest0-21swapper/212:14:102
734599220,0cyclictest0-21swapper/211:45:092
7344992222,0cyclictest19282-21ssh09:53:231
7344992222,0cyclictest1890-21diskmemload12:35:201
7344992222,0cyclictest1890-21diskmemload12:06:351
7344992222,0cyclictest0-21swapper/111:11:361
734499220,21cyclictest20705-21ssh09:55:051
734499220,21cyclictest19852-21sh09:25:471
734499220,1cyclictest3011-21apt-get08:06:101
734499220,1cyclictest25604-21sh10:27:481
734499220,1cyclictest20738-21ssh11:50:281
734499220,1cyclictest1432-21ssh10:36:021
734499220,1cyclictest1322-21diskmemload09:36:561
734499220,1cyclictest0-21swapper/112:15:311
734499220,1cyclictest0-21swapper/111:44:091
734499220,0cyclictest0-21swapper/109:30:311
7343992222,0cyclictest0-21swapper/010:40:140
7343992222,0cyclictest0-21swapper/007:41:110
7343992221,0cyclictest0-21swapper/007:16:110
734399220,22cyclictest5556-21apt-get12:06:110
734399220,22cyclictest0-21swapper/011:20:480
734399220,21cyclictest31728-21sh12:00:170
734399220,21cyclictest1890-21diskmemload11:45:060
734399220,21cyclictest0-21swapper/010:12:230
734399220,21cyclictest0-21swapper/008:01:110
734399220,1cyclictest20485-21cstates09:26:120
734399220,1cyclictest18148-21ssh09:51:570
734399220,1cyclictest10250-21idleruntime-cro08:20:010
734399220,0cyclictest0-21swapper/012:10:540
7346992120,0cyclictest10073-21apt-get07:16:113
7346992120,0cyclictest0-21swapper/307:56:113
7346992119,1cyclictest31926-21turbostat.cron09:05:013
7346992119,1cyclictest31926-21turbostat.cron09:05:013
7346992119,1cyclictest17469-21cat08:35:013
734699210,20cyclictest0-21swapper/308:16:113
734699210,1cyclictest606-21dbus-daemon08:45:003
734699210,0cyclictest10628-21apt-get08:21:133
7345992121,0cyclictest30487-21apt-get07:55:022
7345992120,1cyclictest572-21apt-get08:01:112
7345992119,1cyclictest13019-21apt-get08:26:112
734599210,21cyclictest1890-21diskmemload10:45:072
734599210,21cyclictest0-21swapper/209:25:212
734599210,20cyclictest0-21swapper/208:41:122
734599210,20cyclictest0-21swapper/207:36:112
734599210,0cyclictest32276-21apt-get09:06:112
734599210,0cyclictest0-21swapper/209:05:012
734599210,0cyclictest0-21swapper/209:05:012
7344992121,0cyclictest5409-21apt-get08:11:111
7344992121,0cyclictest0-21swapper/110:30:181
7344992121,0cyclictest0-21swapper/110:20:361
7344992118,2cyclictest313-21systemd-journal08:45:001
7343992121,0cyclictest0-21swapper/009:56:150
7343992121,0cyclictest0-21swapper/007:55:020
7343992121,0cyclictest0-21swapper/007:46:110
7343992120,1cyclictest0-21swapper/007:21:120
7343992119,1cyclictest27055-21latency_hist08:55:000
7346992020,0cyclictest0-21swapper/308:06:103
734699200,19cyclictest0-21swapper/307:45:023
734699200,0cyclictest0-21swapper/307:41:113
7345992020,0cyclictest0-21swapper/208:56:112
7345992020,0cyclictest0-21swapper/207:21:122
7345992018,1cyclictest27012-21sh07:50:002
7344992020,0cyclictest30612-21apt-get07:56:111
7344992020,0cyclictest24255-21apt-get07:45:021
7344992020,0cyclictest22201-21apt-get07:41:121
7344992020,0cyclictest10073-21apt-get07:16:101
7344992020,0cyclictest0-21swapper/108:15:021
7344992018,1cyclictest19851-21basename08:40:001
734499200,0cyclictest0-21swapper/108:46:111
7343992020,0cyclictest0-21swapper/008:46:110
7343992020,0cyclictest0-21swapper/008:41:120
7343992020,0cyclictest0-21swapper/008:11:110
7343992020,0cyclictest0-21swapper/007:31:100
734399200,0cyclictest0-21swapper/008:31:110
734399200,0cyclictest0-21swapper/008:06:100
734699190,18cyclictest0-21swapper/308:56:113
7345991919,0cyclictest0-21swapper/207:31:102
7344991919,0cyclictest15416-21apt-get08:31:111
7344991919,0cyclictest14546-21perf07:25:011
7344991919,0cyclictest0-21swapper/109:06:111
7346991818,0cyclictest0-21swapper/308:26:103
7346991817,1cyclictest17280-21apt-get07:31:093
7345991818,0cyclictest0-21swapper/208:31:112
7345991817,0cyclictest0-21swapper/207:51:112
734599180,17cyclictest0-21swapper/208:10:002
7343991818,0cyclictest0-21swapper/008:36:120
7343991818,0cyclictest0-21swapper/007:26:120
7346991717,0cyclictest17810-21apt-get08:36:113
7346991717,0cyclictest0-21swapper/307:11:123
7345991717,0cyclictest0-21swapper/208:51:112
7345991717,0cyclictest0-21swapper/208:15:022
7345991717,0cyclictest0-21swapper/208:11:112
7345991717,0cyclictest0-21swapper/207:41:122
7345991716,1cyclictest12141-21dump-pmu-power07:20:002
7344991716,1cyclictest0-21swapper/107:31:091
7343991717,0cyclictest0-21swapper/008:26:110
7346991614,1cyclictest2666-21apt08:05:003
7346991614,1cyclictest16951-21basename07:30:013
7344991616,0cyclictest0-21swapper/107:11:121
7344991614,1cyclictest19595-21basename07:35:101
7344991515,0cyclictest0-21swapper/107:50:011
734499150,1cyclictest29190-21chrt08:56:571
734399150,1cyclictest27020-21apt-get07:50:020
7346991414,0cyclictest0-21swapper/309:05:443
734699140,0cyclictest0-21swapper/308:12:083
734699140,0cyclictest0-21swapper/307:50:493
734699140,0cyclictest0-21swapper/307:20:413
734599140,14cyclictest0-21swapper/208:47:082
734599140,14cyclictest0-21swapper/208:21:092
734599140,0cyclictest0-21swapper/208:35:232
7344991414,0cyclictest0-21swapper/107:26:091
734499140,14cyclictest0-21swapper/108:50:011
734499140,0cyclictest0-21swapper/108:20:181
734399140,14cyclictest0-21swapper/008:20:130
734399140,14cyclictest0-21swapper/007:37:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional