You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 01:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Mon May 04, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1448199260,25cyclictest17712-21ssh12:08:102
1447699260,25cyclictest28318-21sh12:17:121
14482992523,1cyclictest14405-21ssh12:34:193
1448299250,24cyclictest7809-21diskmemload10:08:073
1448299250,24cyclictest0-21swapper/312:08:293
14481992520,4cyclictest8669-21ssh12:00:582
1448199250,24cyclictest15870-21smartctl09:16:222
1448199250,24cyclictest0-21swapper/209:52:422
14476992524,1cyclictest0-21swapper/109:16:591
1447699250,24cyclictest0-21swapper/111:10:121
1446899253,1cyclictest62550irq/127-eno111:59:450
1446899250,24cyclictest6808-21apt-get08:00:020
14482992422,1cyclictest11423-21apt-get10:10:023
14482992421,2cyclictest15440-21rm09:45:053
1448299240,24cyclictest0-21swapper/311:42:363
1448299240,23cyclictest7809-21diskmemload09:43:463
1448299240,1cyclictest2916-21rm10:01:383
1448299240,0cyclictest7809-21diskmemload12:19:523
14481992423,1cyclictest28-21ksoftirqd/212:33:362
14481992422,1cyclictest19241-21munin-run07:20:012
14481992422,0cyclictest0-21swapper/211:06:202
14481992419,4cyclictest0-21swapper/211:59:312
1448199240,23cyclictest7809-21diskmemload10:06:162
1448199240,23cyclictest7809-21diskmemload09:20:232
1448199240,23cyclictest28846-21latency_hist10:25:012
1448199240,1cyclictest7809-21diskmemload12:16:222
14476992423,1cyclictest0-21swapper/111:26:171
14476992421,2cyclictest26211-21sh10:51:121
14476992421,2cyclictest25102-21ssh09:53:041
1447699240,24cyclictest0-21swapper/109:24:211
1447699240,23cyclictest0-21swapper/111:06:421
1447699240,1cyclictest21244-21sh10:17:311
1446899240,24cyclictest0-21swapper/012:04:240
1446899240,24cyclictest0-21swapper/011:33:440
1446899240,24cyclictest0-21swapper/011:27:430
1446899240,23cyclictest0-21swapper/012:15:520
1446899240,0cyclictest0-21swapper/011:43:590
14482992323,0cyclictest7809-21diskmemload09:28:553
14482992323,0cyclictest0-21swapper/312:01:023
14482992323,0cyclictest0-21swapper/310:48:033
14482992323,0cyclictest0-21swapper/310:31:193
14482992321,1cyclictest0-21swapper/312:36:083
14482992321,1cyclictest0-21swapper/311:53:003
14482992321,1cyclictest0-21swapper/310:16:313
14482992320,2cyclictest27224-21ssh10:22:533
14482992320,2cyclictest14573-21ssh09:16:053
1448299230,2cyclictest7809-21diskmemload11:30:333
1448299230,2cyclictest2764-21ssh11:56:023
1448299230,23cyclictest0-21swapper/312:20:233
1448299230,23cyclictest0-21swapper/310:57:143
1448299230,23cyclictest0-21swapper/310:36:193
1448299230,23cyclictest0-21swapper/309:35:423
1448299230,22cyclictest5989-21ntp_states12:26:233
1448299230,22cyclictest16319-21ssh11:10:583
1448299230,22cyclictest1080-21apt-get11:26:173
1448299230,1cyclictest0-21swapper/310:50:213
1448299230,1cyclictest0-21swapper/308:40:013
1448299230,0cyclictest7809-21diskmemload11:37:363
1448299230,0cyclictest0-21swapper/311:01:153
1448299230,0cyclictest0-21swapper/309:55:053
1448299230,0cyclictest0-21swapper/309:50:143
1448299230,0cyclictest0-21swapper/309:14:283
14481992323,0cyclictest7809-21diskmemload11:18:042
14481992323,0cyclictest0-21swapper/211:36:102
14481992323,0cyclictest0-21swapper/210:59:272
14481992323,0cyclictest0-21swapper/210:33:222
14481992323,0cyclictest0-21swapper/210:16:402
14481992322,1cyclictest4053-1kworker/u9:011:23:182
14481992321,1cyclictest10770-21apt-get09:41:172
1448199230,23cyclictest0-21swapper/210:35:172
1448199230,22cyclictest7809-21diskmemload11:28:402
1448199230,22cyclictest0-21swapper/210:51:342
1448199230,22cyclictest0-21swapper/210:03:482
1448199230,22cyclictest0-21swapper/209:31:412
1448199230,0cyclictest0-21swapper/212:35:532
1448199230,0cyclictest0-21swapper/212:25:282
1448199230,0cyclictest0-21swapper/212:13:102
1448199230,0cyclictest0-21swapper/211:34:182
14476992323,0cyclictest0-21swapper/111:56:161
14476992323,0cyclictest0-21swapper/111:50:281
14476992323,0cyclictest0-21swapper/111:30:361
14476992323,0cyclictest0-21swapper/111:02:451
14476992323,0cyclictest0-21swapper/110:41:421
14476992323,0cyclictest0-21swapper/109:38:141
14476992323,0cyclictest0-21swapper/109:11:041
14476992321,1cyclictest31638-21apt-get12:21:181
14476992321,1cyclictest0-21swapper/112:36:261
14476992321,1cyclictest0-21swapper/112:30:201
14476992321,1cyclictest0-21swapper/112:26:221
14476992321,1cyclictest0-21swapper/109:31:211
14476992321,1cyclictest0-21swapper/107:30:001
1447699230,23cyclictest0-21swapper/111:48:511
1447699230,23cyclictest0-21swapper/110:04:441
1447699230,22cyclictest7188-21sh10:34:121
1447699230,22cyclictest13843-21ssh12:05:141
1447699230,22cyclictest0-21swapper/111:16:241
1447699230,22cyclictest0-21swapper/109:56:401
1447699230,1cyclictest17534-21munin-run10:15:011
1447699230,0cyclictest0-21swapper/110:23:301
14468992323,0cyclictest7809-21diskmemload10:40:380
14468992323,0cyclictest7809-21diskmemload09:10:500
14468992323,0cyclictest0-21swapper/012:25:260
14468992323,0cyclictest0-21swapper/011:36:110
14468992323,0cyclictest0-21swapper/010:53:410
14468992322,1cyclictest7809-21diskmemload11:05:540
14468992322,1cyclictest0-21swapper/009:53:020
14468992322,1cyclictest0-21swapper/009:16:040
14468992322,0cyclictest22934-21ssh10:19:440
14468992321,1cyclictest9130-21expr10:36:170
14468992321,1cyclictest0-21swapper/010:31:290
14468992320,2cyclictest5419-21apt09:05:000
1446899230,2cyclictest28341-21ssh09:26:540
1446899230,2cyclictest28242-21latency_hist08:45:010
1446899230,2cyclictest28242-21latency_hist08:45:010
1446899230,23cyclictest0-21swapper/012:37:570
1446899230,23cyclictest0-21swapper/010:47:460
1446899230,22cyclictest62550irq/127-eno112:05:200
1446899230,22cyclictest31673-21sh09:59:290
1446899230,22cyclictest30766-21sh12:20:140
1446899230,22cyclictest0-21swapper/010:56:170
1446899230,21cyclictest0-21swapper/012:11:370
1446899230,1cyclictest0-21swapper/009:32:090
1446899230,0cyclictest0-21swapper/011:45:180
14482992220,1cyclictest21043-21irqstats12:11:213
14482992220,1cyclictest10372-21sh11:05:373
14482992220,1cyclictest0-21swapper/308:10:013
1448299220,2cyclictest28247-21perf08:45:013
1448299220,2cyclictest28247-21perf08:45:013
1448299220,21cyclictest8994-21munin-run08:05:013
1448299220,21cyclictest27902-21apt-get11:21:163
1448299220,21cyclictest23919-21ssh09:23:363
1448299220,21cyclictest0-21swapper/310:41:063
1448299220,21cyclictest0-21swapper/310:25:013
1448299220,1cyclictest26818-21ssh11:47:363
1448299220,1cyclictest24775-21ssh11:17:253
14481992222,0cyclictest7809-21diskmemload10:10:322
14481992221,1cyclictest7809-21diskmemload09:35:112
14481992221,1cyclictest271rcuc/211:47:332
14481992220,2cyclictest25258-21ssh09:25:062
14481992220,1cyclictest16704-21ssh10:41:452
14481992220,1cyclictest0-21swapper/209:10:022
1448199220,2cyclictest11723-21apt-get08:11:162
1448199220,21cyclictest21338-21apt-get08:31:152
1448199220,21cyclictest0-21swapper/211:40:212
1448199220,21cyclictest0-21swapper/211:10:012
1448199220,21cyclictest0-21swapper/211:00:542
1448199220,1cyclictest7809-21diskmemload11:51:182
1448199220,1cyclictest28182-21perl09:56:192
1448199220,0cyclictest0-21swapper/210:46:222
1448199220,0cyclictest0-21swapper/209:46:142
14476992222,0cyclictest0-21swapper/109:28:451
14476992220,1cyclictest19589-21apt-get07:21:151
14476992220,1cyclictest0-21swapper/111:35:221
1447699220,22cyclictest14463-21cyclictest12:13:311
1447699220,22cyclictest0-21swapper/110:36:191
1447699220,21cyclictest7427-21apt-get12:00:021
1447699220,21cyclictest0-21swapper/111:20:071
1447699220,21cyclictest0-21swapper/109:40:561
1447699220,21cyclictest0-21swapper/108:46:141
1447699220,1cyclictest30987-21apt-get08:51:151
1447699220,1cyclictest0-21swapper/111:40:281
1447699220,1cyclictest0-21swapper/110:45:091
1447699220,1cyclictest0-21swapper/109:46:011
1447699220,1cyclictest0-21swapper/108:00:011
1447699220,1cyclictest0-21swapper/107:15:011
1447699220,0cyclictest0-21swapper/110:25:241
14468992221,0cyclictest11723-21apt-get08:11:160
14468992220,1cyclictest9991-21ssh12:30:390
14468992220,1cyclictest6161-21ssh10:05:120
14468992220,1cyclictest32195-21ssh10:26:530
14468992220,1cyclictest21343-21rm11:15:130
14468992220,1cyclictest0-21swapper/011:50:020
14468992219,2cyclictest20675-21cstates09:21:170
1446899220,22cyclictest7809-21diskmemload11:00:040
1446899220,21cyclictest31739-21apt-get07:46:160
1446899220,21cyclictest30631-21turbostat08:50:010
1446899220,21cyclictest15402-21ssh09:45:010
1446899220,21cyclictest0-21swapper/011:20:330
1446899220,21cyclictest0-21swapper/011:11:200
1446899220,21cyclictest0-21swapper/010:11:030
1446899220,21cyclictest0-21swapper/010:00:450
1446899220,21cyclictest0-21swapper/008:00:010
1446899220,21cyclictest0-21swapper/007:26:160
14482992118,2cyclictest605-21avahi-daemon08:15:003
1448299210,21cyclictest0-21swapper/309:30:023
14481992121,0cyclictest21982-21apt-get07:26:162
14481992120,1cyclictest4125-21apt-get07:56:162
1448199210,20cyclictest0-21swapper/208:56:152
1448199210,1cyclictest16197-21perf08:20:002
1448199210,0cyclictest0-21swapper/208:00:022
14476992121,0cyclictest0-21swapper/107:36:161
14476992118,2cyclictest25847-21awk08:40:011
1447699210,0cyclictest0-21swapper/108:41:181
1447699210,0cyclictest0-21swapper/108:41:181
1447699210,0cyclictest0-21swapper/107:31:151
1446899210,20cyclictest0-21swapper/007:41:170
1446899210,0cyclictest17168-21apt-get07:16:160
1446899210,0cyclictest14776-21apt-get07:11:160
1446899210,0cyclictest0-21swapper/008:16:170
1446899210,0cyclictest0-21swapper/008:06:160
14482992018,1cyclictest28592-21apt-get08:46:143
14482992017,2cyclictest26491-21munin-plugin-st07:35:003
1448299200,1cyclictest26825-21apt-get07:36:163
1448299200,0cyclictest0-21swapper/307:21:163
14481992020,0cyclictest0-21swapper/208:51:152
14481992018,1cyclictest18601-21munin-run08:25:002
1448199200,0cyclictest0-21swapper/208:46:142
1448199200,0cyclictest0-21swapper/208:06:172
14476992020,0cyclictest0-21swapper/107:46:161
1447699200,0cyclictest0-21swapper/108:11:161
14468992020,0cyclictest0-21swapper/008:21:150
14468992018,1cyclictest3031-21perf09:00:010
14468992018,1cyclictest26463-21cron07:35:000
14482991917,1cyclictest23440-21perf08:35:003
14482991917,1cyclictest16817-21apt07:15:013
14481991919,0cyclictest0-21swapper/208:41:172
14481991919,0cyclictest0-21swapper/208:41:172
14481991919,0cyclictest0-21swapper/207:21:162
14481991918,1cyclictest0-21swapper/209:06:182
14481991917,1cyclictest28-21ksoftirqd/208:30:002
14476991919,0cyclictest0-21swapper/108:16:171
14476991919,0cyclictest0-21swapper/107:41:171
14482991818,0cyclictest0-21swapper/307:46:163
14481991816,1cyclictest151rcu_preempt07:11:162
1446899180,17cyclictest25831-21latency_hist08:40:010
14482991715,1cyclictest1-21systemd08:20:003
14481991714,2cyclictest5646-21sed09:05:002
14476991717,0cyclictest0-21swapper/108:31:151
14476991717,0cyclictest0-21swapper/107:16:161
14476991716,1cyclictest6189-21apt-get08:00:021
14468991717,0cyclictest0-21swapper/008:31:160
14468991716,1cyclictest1719-21apt-get07:51:150
14482991616,0cyclictest0-21swapper/309:06:183
14481991616,0cyclictest0-21swapper/207:46:152
14481991616,0cyclictest0-21swapper/207:41:172
14476991616,0cyclictest0-21swapper/108:06:171
14476991614,1cyclictest21000-21basename08:30:001
14468991616,0cyclictest5788-21apt-get09:06:170
14468991614,1cyclictest195512sleep007:21:120
14482991513,1cyclictest6159-21dump-pmu-power08:00:013
1448299150,14cyclictest0-21swapper/307:41:543
14481991515,0cyclictest0-21swapper/207:36:152
14481991515,0cyclictest0-21swapper/207:31:152
1448199150,14cyclictest3767-21perf07:55:012
14468991515,0cyclictest18938-21apt-get08:26:150
14468991514,0cyclictest0-21swapper/007:36:150
1448299140,14cyclictest0-21swapper/309:00:243
1448299140,14cyclictest0-21swapper/308:20:013
1448299140,14cyclictest0-21swapper/307:50:323
1448299140,0cyclictest0-21swapper/308:56:593
1448299140,0cyclictest0-21swapper/308:51:073
1448299140,0cyclictest0-21swapper/308:25:503
1448299140,0cyclictest0-21swapper/307:26:503
1448299140,0cyclictest0-21swapper/307:15:023
14481991414,0cyclictest0-21swapper/208:35:462
14476991414,0cyclictest0-21swapper/109:00:101
14476991414,0cyclictest0-21swapper/108:55:341
14476991414,0cyclictest0-21swapper/108:20:161
1447699140,0cyclictest0-21swapper/109:05:261
1446899140,0cyclictest0-21swapper/008:51:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional