You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 01:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Mon May 25, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
214699300,29cyclictest34-21ksoftirqd/312:11:233
213399290,22cyclictest22444-21rm12:23:030
2133992820,7cyclictest4876-21apt-get07:15:020
213399280,27cyclictest31106-21latency_hist09:40:010
213399280,21cyclictest0-21swapper/009:43:030
2133992723,4cyclictest1271-21ssh11:07:470
213399270,0cyclictest28275-21diskmemload09:28:580
214199260,25cyclictest28275-21diskmemload11:35:572
2136992624,1cyclictest23277-21irqstats12:23:061
213699260,25cyclictest19111-21ssh09:29:311
2133992620,5cyclictest28631-21ssh10:34:100
2133992620,5cyclictest0-21swapper/012:08:080
2133992619,6cyclictest0-21swapper/010:13:010
213399260,25cyclictest25709-21ssh10:03:180
213399260,1cyclictest11332-21ssh10:18:550
213399260,0cyclictest0-21swapper/011:35:490
213399260,0cyclictest0-21swapper/010:52:070
213399260,0cyclictest0-21swapper/010:41:510
214199250,24cyclictest0-21swapper/209:45:222
214199250,24cyclictest0-21swapper/209:45:222
2136992523,1cyclictest23122-21cut07:53:001
2136992523,1cyclictest0-21swapper/110:22:451
2133992523,2cyclictest0-21swapper/011:22:530
2133992522,3cyclictest0-21swapper/009:22:530
2133992520,4cyclictest29228-21rm10:07:420
2133992520,4cyclictest28275-21diskmemload11:13:380
2133992519,5cyclictest5407-21ssh12:35:450
2133992519,0cyclictest0-21swapper/011:48:020
213399250,24cyclictest28275-21diskmemload12:10:390
213399250,21cyclictest976-21ssh12:32:310
213399250,21cyclictest28275-21diskmemload10:55:150
213399250,21cyclictest28275-21diskmemload09:56:550
213399250,21cyclictest12104-21rm11:17:040
213399250,21cyclictest0-21swapper/011:44:280
213399250,20cyclictest0-21swapper/009:30:560
213399250,0cyclictest31291-21apt-get09:13:030
2146992422,1cyclictest8720-21sh10:45:203
2146992422,1cyclictest34-21ksoftirqd/310:38:513
2146992420,3cyclictest75-21kswapd011:55:023
214699240,23cyclictest30425-21sh09:11:593
214699240,1cyclictest28275-21diskmemload10:44:183
214699240,1cyclictest0-21swapper/309:58:053
2141992422,1cyclictest106242sleep209:23:012
214199240,23cyclictest28275-21diskmemload09:33:192
214199240,23cyclictest0-21swapper/211:44:032
214199240,23cyclictest0-21swapper/211:11:542
214199240,23cyclictest0-21swapper/210:48:022
214199240,23cyclictest0-21swapper/210:32:582
214199240,23cyclictest0-21swapper/209:29:172
2136992423,1cyclictest0-21swapper/112:19:471
2136992423,0cyclictest10616-21apt-get09:23:011
213699240,23cyclictest28275-21diskmemload11:51:511
213699240,23cyclictest0-21swapper/111:36:081
213699240,1cyclictest0-21swapper/110:48:211
213699240,0cyclictest0-21swapper/110:30:561
2133992421,3cyclictest0-21swapper/012:15:160
2133992419,4cyclictest8320-21ssh09:48:120
2133992419,4cyclictest8320-21ssh09:48:120
213399240,24cyclictest28275-21diskmemload10:28:470
213399240,21cyclictest0-21swapper/011:50:540
213399240,1cyclictest28275-21diskmemload11:56:550
213399240,1cyclictest27136-21ssh12:26:420
213399240,0cyclictest28275-21diskmemload10:35:150
213399240,0cyclictest0-21swapper/011:01:320
2146992323,0cyclictest331rcuc/310:15:443
2146992323,0cyclictest0-21swapper/311:45:483
2146992323,0cyclictest0-21swapper/311:10:113
2146992323,0cyclictest0-21swapper/309:25:023
2146992322,1cyclictest34-21ksoftirqd/311:01:503
2146992321,1cyclictest0-21swapper/309:33:063
2146992320,2cyclictest0-21swapper/309:23:013
214699231,21cyclictest331rcuc/312:01:403
214699230,2cyclictest3943-21ssh11:09:033
214699230,23cyclictest34-21ksoftirqd/310:12:213
214699230,23cyclictest331rcuc/309:48:593
214699230,23cyclictest331rcuc/309:48:593
214699230,23cyclictest0-21swapper/312:25:413
214699230,23cyclictest0-21swapper/312:15:293
214699230,23cyclictest0-21swapper/309:44:323
214699230,22cyclictest27891-21sh10:05:573
214699230,22cyclictest25496-21ssh10:32:223
214699230,22cyclictest13104-21sh10:20:563
214699230,22cyclictest12124-21ssh09:52:373
214699230,22cyclictest1131-21sh11:35:073
214699230,22cyclictest0-21swapper/310:28:073
214699230,1cyclictest331rcuc/312:20:193
214699230,1cyclictest0-21swapper/312:37:423
214699230,0cyclictest28275-21diskmemload11:27:523
214699230,0cyclictest28275-21diskmemload09:35:393
214699230,0cyclictest0-21swapper/311:50:583
214699230,0cyclictest0-21swapper/311:32:393
214699230,0cyclictest0-21swapper/311:20:493
214699230,0cyclictest0-21swapper/309:16:343
214699230,0cyclictest0-21swapper/308:23:033
2141992323,0cyclictest28275-21diskmemload12:25:192
2141992323,0cyclictest28275-21diskmemload11:28:132
2141992323,0cyclictest28275-21diskmemload09:44:202
2141992323,0cyclictest0-21swapper/211:23:012
2141992322,1cyclictest28275-21diskmemload10:41:182
2141992322,1cyclictest13841-21apt08:40:002
2141992322,1cyclictest0-21swapper/210:38:022
214199231,21cyclictest32327-21apt-get08:13:032
214199231,21cyclictest32327-21apt-get08:13:032
214199230,2cyclictest28275-21diskmemload09:54:582
214199230,23cyclictest0-21swapper/212:37:042
214199230,23cyclictest0-21swapper/212:04:522
214199230,23cyclictest0-21swapper/211:00:362
214199230,22cyclictest28275-21diskmemload12:13:442
214199230,22cyclictest21526-21ssh10:57:192
214199230,22cyclictest0-21swapper/211:47:082
214199230,1cyclictest0-21swapper/210:23:202
214199230,0cyclictest28275-21diskmemload10:51:082
214199230,0cyclictest28275-21diskmemload10:29:252
214199230,0cyclictest0-21swapper/212:32:452
214199230,0cyclictest0-21swapper/210:17:012
214199230,0cyclictest0-21swapper/209:36:302
2136992323,0cyclictest0-21swapper/111:33:231
2136992323,0cyclictest0-21swapper/111:20:131
2136992323,0cyclictest0-21swapper/110:55:091
2136992323,0cyclictest0-21swapper/110:00:071
2136992323,0cyclictest0-21swapper/109:51:471
2136992323,0cyclictest0-21swapper/109:47:161
2136992323,0cyclictest0-21swapper/109:47:161
2136992323,0cyclictest0-21swapper/109:34:481
2136992322,1cyclictest23438-21latency_hist09:00:011
2136992321,1cyclictest5899-21ssh10:14:521
2136992321,1cyclictest0-21swapper/112:04:011
2136992321,1cyclictest0-21swapper/111:45:511
2136992321,1cyclictest0-21swapper/111:08:001
2136992321,1cyclictest0-21swapper/107:15:021
2136992320,2cyclictest6478-21sh11:12:071
2136992320,2cyclictest32479-21ssh09:41:491
213699230,23cyclictest17364-21ssh09:57:081
213699230,23cyclictest0-21swapper/112:27:161
213699230,23cyclictest0-21swapper/110:50:381
213699230,22cyclictest783-21snmpd11:41:171
213699230,22cyclictest28228-21sh10:06:221
213699230,22cyclictest0-21swapper/112:08:231
213699230,0cyclictest0-21swapper/112:33:351
213699230,0cyclictest0-21swapper/109:15:331
2133992323,0cyclictest0-21swapper/012:00:510
2133992322,1cyclictest28275-21diskmemload11:34:340
2133992320,3cyclictest0-21swapper/011:26:320
213399230,2cyclictest12551-21ssh10:48:130
213399230,22cyclictest13031-21sh10:20:510
2146992222,0cyclictest28275-21diskmemload10:55:093
2146992222,0cyclictest0-21swapper/312:32:083
2146992222,0cyclictest0-21swapper/311:15:363
2146992221,1cyclictest34-21ksoftirqd/310:01:443
2146992220,1cyclictest16736-21apt-get08:48:033
214699220,22cyclictest0-21swapper/311:40:593
214699220,21cyclictest0-21swapper/310:50:193
214699220,1cyclictest609-21rs:main3
214699220,1cyclictest20281-21apt-get07:48:033
214699220,0cyclictest0-21swapper/307:43:013
2141992220,1cyclictest0-21swapper/209:55:212
214199220,21cyclictest14339-21apt-get08:43:012
214199220,21cyclictest13775-21sh11:18:062
214199220,21cyclictest0-21swapper/212:15:052
214199220,21cyclictest0-21swapper/211:52:042
214199220,21cyclictest0-21swapper/211:05:492
214199220,21cyclictest0-21swapper/210:06:062
214199220,21cyclictest0-21swapper/209:03:022
214199220,21cyclictest0-21swapper/207:15:022
214199220,1cyclictest29076-21ssh11:31:452
214199220,1cyclictest23527-21apt-get11:55:022
214199220,1cyclictest0-21swapper/212:24:322
214199220,1cyclictest0-21swapper/210:10:532
214199220,1cyclictest0-21swapper/209:15:532
214199220,1cyclictest0-21swapper/209:12:122
214199220,0cyclictest0-21swapper/212:05:262
2136992222,0cyclictest28275-21diskmemload12:10:051
2136992222,0cyclictest0-21swapper/112:35:011
2136992222,0cyclictest0-21swapper/111:57:181
2136992222,0cyclictest0-21swapper/110:42:051
2136992222,0cyclictest0-21swapper/110:25:261
2136992221,1cyclictest28275-21diskmemload11:15:411
2136992220,1cyclictest4740-21apt-get08:23:031
2136992220,1cyclictest0-21swapper/108:40:001
213699220,21cyclictest0-21swapper/110:35:051
213699220,21cyclictest0-21swapper/109:10:381
213699220,1cyclictest7129-21apt-get08:28:011
213699220,1cyclictest0-21swapper/111:00:531
213699220,0cyclictest28275-21diskmemload11:29:131
213699220,0cyclictest28275-21diskmemload09:38:521
2133992220,2cyclictest0-21swapper/009:54:170
213399220,21cyclictest21054-21munin-run08:55:000
213399220,21cyclictest0-21swapper/007:58:000
213399220,1cyclictest4198-21cron08:20:010
213399220,0cyclictest23942-21apt-get09:03:020
214699210,21cyclictest0-21swapper/308:33:033
214699210,20cyclictest0-21swapper/308:17:593
214699210,20cyclictest0-21swapper/307:52:593
214699210,0cyclictest0-21swapper/307:28:013
214199210,20cyclictest0-21swapper/208:28:012
2136992121,0cyclictest0-21swapper/107:28:021
2136992120,1cyclictest4876-21apt-get07:15:001
2136992120,1cyclictest21041-21ls08:55:001
213699210,0cyclictest0-21swapper/108:43:011
2133992121,0cyclictest0-21swapper/007:52:590
2133992120,1cyclictest2047-21apt-get07:10:020
2133992120,1cyclictest0-21swapper/008:13:020
2133992120,1cyclictest0-21swapper/008:13:020
213399210,20cyclictest0-21swapper/008:48:030
213399210,20cyclictest0-21swapper/007:28:010
213399210,0cyclictest0-21swapper/008:03:010
2146992020,0cyclictest0-21swapper/308:58:023
2146992020,0cyclictest0-21swapper/307:18:013
2146992018,1cyclictest27063-21basename08:00:003
2141992020,0cyclictest0-21swapper/208:17:592
2141992019,1cyclictest22669-21apt-get07:52:592
2141992018,2cyclictest19774-21idleruntime-cro07:45:002
2141992018,1cyclictest12480-21perf07:30:012
2136992020,0cyclictest0-21swapper/109:03:021
2136992020,0cyclictest0-21swapper/108:33:021
213699200,20cyclictest0-21swapper/108:50:001
213699200,19cyclictest25067-21apt-get07:58:001
2133992020,0cyclictest0-21swapper/008:38:010
213399200,20cyclictest0-21swapper/008:43:010
213399200,20cyclictest0-21swapper/007:43:010
2146991919,0cyclictest19129-21apt-get08:53:013
2146991919,0cyclictest0-21swapper/309:03:023
2146991919,0cyclictest0-21swapper/307:38:013
2146991919,0cyclictest0-21swapper/307:33:013
2141991919,0cyclictest0-21swapper/208:23:032
2141991919,0cyclictest0-21swapper/207:48:032
2141991919,0cyclictest0-21swapper/207:23:022
2141991918,1cyclictest15468-21apt-get07:38:012
2141991918,1cyclictest0-21swapper/208:48:032
2136991919,0cyclictest29932-21apt-get08:08:011
2133991919,0cyclictest0-21swapper/008:23:030
2133991917,1cyclictest17412-21awk07:40:010
2146991818,0cyclictest0-21swapper/308:13:023
2146991818,0cyclictest0-21swapper/308:13:023
2136991816,1cyclictest17885-21apt-get07:43:011
2133991818,0cyclictest0-21swapper/008:58:020
2146991716,1cyclictest0-21swapper/309:08:023
2141991717,0cyclictest2970-21apt-get07:13:012
2141991717,0cyclictest12969-21apt-get07:33:012
2141991717,0cyclictest0-21swapper/208:08:022
2141991717,0cyclictest0-21swapper/207:58:002
213699170,16cyclictest26390-21apt-get09:08:021
2133991717,0cyclictest0-21swapper/008:08:020
2133991717,0cyclictest0-21swapper/007:48:030
2133991717,0cyclictest0-21swapper/007:33:010
2146991610,4cyclictest331rcuc/308:43:043
2141991615,1cyclictest27538-21apt-get08:03:012
2141991614,1cyclictest9530-21apt-get08:33:032
2136991616,0cyclictest0-21swapper/107:48:031
2136991614,1cyclictest17359-21sh07:40:011
2133991616,0cyclictest0-21swapper/008:33:020
2133991615,1cyclictest8177-21apt-get07:23:020
2146991513,1cyclictest611-21irqbalance07:21:573
214699150,15cyclictest34-21ksoftirqd/308:03:013
213699150,8cyclictest0-21swapper/108:17:591
213399150,11cyclictest611-21irqbalance08:27:370
2146991414,0cyclictest0-21swapper/307:10:023
214699140,14cyclictest0-21swapper/308:35:503
214699140,0cyclictest0-21swapper/308:25:193
2141991414,0cyclictest0-21swapper/209:05:192
214199140,14cyclictest0-21swapper/208:57:002
214199140,0cyclictest0-21swapper/208:50:142
2136991414,0cyclictest0-21swapper/107:20:261
213699140,14cyclictest0-21swapper/108:10:171
213699140,14cyclictest0-21swapper/108:10:171
213699140,0cyclictest0-21swapper/108:00:491
213399140,14cyclictest0-21swapper/009:08:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional