You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 14:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Apr 30, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2026923926,0sleep30-21swapper/306:52:003
2067123525,0sleep20-21swapper/206:54:072
2083023228,0sleep00-21swapper/006:55:570
2073023223,0sleep10-21swapper/106:54:461
209949976,0cyclictest3140-21munin-node07:32:092
339960,0migration/35078-21sh09:31:123
339960,0migration/33511-21sh10:49:073
339960,0migration/33511-21sh10:49:073
339960,0migration/328259-21sh09:43:443
339960,0migration/327435-21sh12:05:573
339960,0migration/326792-21sh11:22:463
339960,0migration/326277-21kthreadcore07:47:163
339960,0migration/321956-21kthreadcore08:32:173
339960,0migration/31970-21sh12:07:553
339960,0migration/311632-21sh10:33:363
29725060,0irq/27-eth0-tx-32312-21munin-node08:47:200
29715060,0irq/26-eth0-rx-6559-21runrttasks12:13:543
29715060,0irq/26-eth0-rx-26069-21df_inode08:42:083
29715060,0irq/26-eth0-rx-13801-21gltestperf10:37:103
209949960,0cyclictest0-21swapper/209:53:162
209939965,0cyclictest31018-21iostat_ios09:27:151
19160,0ktimersoftd/10-21swapper/111:50:001
149960,0migration/021075-21sh10:42:000
149960,0migration/012519-21sh09:55:110
8215050,0irq/24-0000:00:20845-21latency_hist10:41:592
8215050,0irq/24-0000:00:19703-21cat12:21:582
8215050,0irq/24-0000:00:10628-21idleruntime-cro08:01:592
339950,0migration/36425-21hddtemp_smartct09:32:123
339950,0migration/36311-21kthreadcore10:52:153
339950,0migration/33512-21munin-node07:32:113
339950,0migration/329801-21irqrtprio11:27:133
339950,0migration/323606-21kthreadcore07:22:143
339950,0migration/322797-21munin-node08:07:183
339950,0migration/320350-21latency_hist11:21:583
339950,0migration/315761-21cpuspeed_turbos07:17:113
339950,0migration/31471-21taskset07:02:283
339950,0migration/313947-21munin-node09:17:093
339950,0migration/31267-21kthreadcore08:47:153
339950,0migration/311491-21fschecks_time08:02:123
29725050,0irq/27-eth0-tx-3704-21munin-node12:12:152
29725050,0irq/27-eth0-tx-28386-21ssh12:07:092
29725050,0irq/27-eth0-tx-28275-21munin-node11:47:162
29725050,0irq/27-eth0-tx-0-21swapper/211:53:492
29725050,0irq/27-eth0-tx-0-21swapper/211:27:522
29725050,0irq/27-eth0-tx-0-21swapper/211:12:522
29725050,0irq/27-eth0-tx-0-21swapper/210:34:122
29725050,0irq/27-eth0-tx-0-21swapper/008:57:110
29725050,0irq/27-eth0-tx-0-21swapper/008:42:100
29725050,0irq/27-eth0-tx-0-21swapper/008:37:470
29725050,0irq/27-eth0-tx-0-21swapper/008:17:220
29725050,0irq/27-eth0-tx-0-21swapper/006:58:010
29715050,0irq/26-eth0-rx-5919-21fschecks_count09:12:123
29715050,0irq/26-eth0-rx-30866-21iostat_ios07:52:153
29715050,0irq/26-eth0-rx-30617-21if_eth1.209:47:133
29715050,0irq/26-eth0-rx-25550-21grep09:02:163
29715050,0irq/26-eth0-rx-20921-21aten_r1power_fr11:02:093
29715050,0irq/26-eth0-rx-17615-21latency11:37:163
29715050,0irq/26-eth0-rx-16975-21kthreadcore09:37:153
29715050,0irq/26-eth0-rx-10663-21ssh11:53:453
29715050,0irq/26-eth0-rx-10502-21ssh11:32:513
29715050,0irq/26-eth0-rx-0-21swapper/311:15:233
29715050,0irq/26-eth0-rx-0-21swapper/310:58:273
29715050,0irq/26-eth0-rx-0-21swapper/310:46:233
29715050,0irq/26-eth0-rx-0-21swapper/310:27:193
29715050,0irq/26-eth0-rx-0-21swapper/310:24:473
29715050,0irq/26-eth0-rx-0-21swapper/310:02:113
29715050,0irq/26-eth0-rx-0-21swapper/309:57:173
29715050,0irq/26-eth0-rx-0-21swapper/309:07:393
29715050,0irq/26-eth0-rx-0-21swapper/308:58:203
29715050,0irq/26-eth0-rx-0-21swapper/308:52:163
259950,0migration/26847-21kthreadcore09:12:162
259950,0migration/22868-21gltestperf07:07:122
259950,0migration/228192-21taskset09:22:352
259950,0migration/226628-21if_eth1.208:42:112
259950,0migration/22618-21sh11:07:352
259950,0migration/221067-21latency_hist10:41:592
259950,0migration/212374-21latency_hist11:16:592
259950,0migration/211407-21fschecks_time08:02:122
259950,0migration/211126-21sh11:34:412
249950,0watchdog/22345-21kthreadcore09:47:172
249950,0watchdog/211601-21timerwakeupswit09:32:212
209949955,0cyclictest13076-21diskmemload12:03:482
209949955,0cyclictest13076-21diskmemload10:10:552
209949955,0cyclictest0-21swapper/210:48:552
209949955,0cyclictest0-21swapper/210:48:552
209949950,0cyclictest0-21swapper/209:58:372
209949950,0cyclictest0-21swapper/209:43:202
209949950,0cyclictest0-21swapper/209:11:382
209939954,0cyclictest5165-21ps07:57:131
209929955,0cyclictest0-21swapper/010:18:020
209929954,0cyclictest30627-21cut09:07:190
209929954,0cyclictest26761-21cat11:02:200
209929952,0cyclictest28267-21munin-run11:26:580
179950,0migration/15016-21latency_hist08:21:591
169950,0watchdog/121525-21aten_r1power_fr10:02:091
149950,0migration/07468-21kthreadcore09:32:150
149950,0migration/06441-21kthreadcore07:32:150
149950,0migration/06122-21kthreadcore11:12:140
149950,0migration/05130-21df_abs10:52:100
149950,0migration/04040-21sh11:11:340
149950,0migration/03347-21irqrtprio07:07:150
149950,0migration/030938-21kthreadcore07:02:150
149950,0migration/029070-21munin-node12:07:120
149950,0migration/028618-21munin-node07:47:180
149950,0migration/028144-21sh10:04:010
149950,0migration/024291-21kthreadcore09:02:150
149950,0migration/023195-21kthreadcore12:22:150
149950,0migration/021912-21cpuspeed_turbos09:42:110
149950,0migration/021882-21latency_hist09:21:590
149950,0migration/018672-21irqrtprio08:07:160
149950,0migration/016041-21fschecks_count07:17:120
149950,0migration/015999-21kthreadcore09:17:180
149950,0migration/01567-21kthreadcore09:27:160
149950,0migration/015309-21kthreadcore11:37:150
149950,0migration/014896-21taskset07:12:430
149950,0migration/01440-21taskset11:47:180
149950,0migration/01286-21kthreadcore10:47:170
149950,0migration/01286-21kthreadcore10:47:170
149950,0migration/011934-21aten2.4-expect12:17:080
149950,0migration/010711-21sendmail_mailqu08:22:210
4140,0ktimersoftd/00-21swapper/011:56:060
339940,0migration/38325-21runrttasks07:08:243
339940,0migration/36611-21kthreadcore10:12:153
339940,0migration/322465-21irqrtprio06:57:113
339940,0migration/316693-21sh07:42:093
339940,0migration/315041-21ksmtuned07:14:243
339940,0migration/313840-21kthreadcore11:57:153
29725040,0irq/27-eth0-tx-21386-21munin-node09:02:102
29725040,0irq/27-eth0-tx-19451-21ssh10:58:452
29725040,0irq/27-eth0-tx-13076-21diskmemload11:37:042
29725040,0irq/27-eth0-tx-12027-21ssh11:57:052
29725040,0irq/27-eth0-tx-0-21swapper/211:42:032
29725040,0irq/27-eth0-tx-0-21swapper/210:02:312
29725040,0irq/27-eth0-tx-0-21swapper/209:37:162
29725040,0irq/27-eth0-tx-0-21swapper/209:27:322
29725040,0irq/27-eth0-tx-0-21swapper/209:17:042
29715040,0irq/26-eth0-rx-1653-21ssh11:47:203
29715040,0irq/26-eth0-rx-0-21swapper/312:22:273
29715040,0irq/26-eth0-rx-0-21swapper/312:17:283
29715040,0irq/26-eth0-rx-0-21swapper/311:44:193
29715040,0irq/26-eth0-rx-0-21swapper/311:09:373
29715040,0irq/26-eth0-rx-0-21swapper/310:17:083
29715040,0irq/26-eth0-rx-0-21swapper/310:07:283
29715040,0irq/26-eth0-rx-0-21swapper/309:52:243
29715040,0irq/26-eth0-rx-0-21swapper/309:22:013
29715040,0irq/26-eth0-rx-0-21swapper/307:27:103
259940,0migration/29858-21aten_r1power_cu07:37:092
259940,0migration/28621-21kthreadcore08:22:162
259940,0migration/228512-21kthreadcore08:12:162
259940,0migration/22748-21switchtime10:27:202
259940,0migration/226203-21mdstat11:02:162
259940,0migration/224266-21irqcore07:47:152
259940,0migration/218936-21kthreadcore07:42:142
259940,0migration/213481-21cpuspeed_turbos10:17:102
259940,0migration/212024-21cpuspeed_turbos08:27:102
259940,0migration/210250-21kthreadcore10:12:162
249940,0watchdog/222106-21kthreadcore08:07:172
249940,0watchdog/20-21swapper/208:57:092
209949943,0cyclictest5649-21awk10:52:122
209949943,0cyclictest3556-21lldpd08:51:322
209949943,0cyclictest3556-21lldpd07:56:322
209949943,0cyclictest3556-21lldpd07:28:022
209949943,0cyclictest28736-21awk07:02:132
209949943,0cyclictest28579-21idleruntime-cro07:26:592
209949943,0cyclictest21419-21awk11:22:142
209949943,0cyclictest12655-21awk12:17:112
209949943,0cyclictest0-21swapper/208:41:592
209949940,0cyclictest26191-21grep10:22:152
209939943,0cyclictest5408-21awk11:12:121
209939943,0cyclictest5072-21awk11:32:131
209939943,0cyclictest4664-21awk12:12:131
209939943,0cyclictest3556-21lldpd12:21:341
209939943,0cyclictest3556-21lldpd10:53:331
209939943,0cyclictest3556-21lldpd10:22:021
209939943,0cyclictest3556-21lldpd09:36:031
209939943,0cyclictest3514-21snmpd08:47:061
209939943,0cyclictest3514-21snmpd08:07:511
209939943,0cyclictest3241-21irqbalance12:07:161
209939943,0cyclictest3241-21irqbalance12:03:561
209939943,0cyclictest3241-21irqbalance09:42:071
209939943,0cyclictest3241-21irqbalance09:12:271
209939943,0cyclictest3241-21irqbalance08:57:571
209939943,0cyclictest3241-21irqbalance08:52:471
209939943,0cyclictest3241-21irqbalance08:37:371
209939943,0cyclictest3241-21irqbalance08:17:171
209939943,0cyclictest3241-21irqbalance08:12:271
209939943,0cyclictest3241-21irqbalance08:02:371
209939943,0cyclictest3241-21irqbalance07:42:181
209939943,0cyclictest3241-21irqbalance07:37:081
209939943,0cyclictest3241-21irqbalance07:27:081
209939943,0cyclictest3241-21irqbalance07:22:381
209939943,0cyclictest3241-21irqbalance07:17:281
209939943,0cyclictest3241-21irqbalance07:13:081
209939943,0cyclictest3241-21irqbalance07:07:481
209939943,0cyclictest3241-21irqbalance07:03:281
209939943,0cyclictest3241-21irqbalance06:57:581
209939943,0cyclictest30717-21awk07:52:141
209939943,0cyclictest21988-21awk10:42:131
209939943,0cyclictest21103-21awk11:42:121
209939943,0cyclictest19294-21awk08:32:141
209939943,0cyclictest14380-21awk09:57:151
209939943,0cyclictest12573-21awk08:27:121
209929943,0cyclictest5983-21awk10:32:140
209929943,0cyclictest5419-21grep10:12:100
209929943,0cyclictest3556-21lldpd12:14:340
209929943,0cyclictest3556-21lldpd09:47:330
209929943,0cyclictest3514-21snmpd08:29:360
209929943,0cyclictest3514-21snmpd08:14:210
209929943,0cyclictest3241-21irqbalance11:30:170
209929943,0cyclictest29665-21awk07:27:140
209929943,0cyclictest28583-21sadc07:26:590
209929943,0cyclictest17257-21sadc08:06:590
209929943,0cyclictest13921-21awk10:37:110
209929940,0cyclictest0-21swapper/012:06:210
209929940,0cyclictest0-21swapper/010:28:270
179940,0migration/19848-21kthreadcore10:32:161
179940,0migration/1638-21kthreadcore09:47:161
179940,0migration/15514-21kthreadcore11:52:141
179940,0migration/132296-21kthreadcore09:07:201
179940,0migration/130734-21kthreadcore11:07:131
179940,0migration/129224-21fschecks_time11:27:101
179940,0migration/122044-21aten_r1power_po09:22:091
179940,0migration/121269-21gltestperf11:22:131
179940,0migration/116510-21kthreadcore09:37:151
169940,0watchdog/128841-21cut08:42:131
169940,0watchdog/128525-21missed_timers07:47:171
169940,0watchdog/127678-21ssh11:04:291
169940,0watchdog/117628-21load11:37:161
169940,0watchdog/114782-21iostat_ios09:17:171
169940,0watchdog/113076-21diskmemload12:22:041
169940,0watchdog/113076-21diskmemload09:04:051
169940,0watchdog/11257-21kthreadcore10:47:171
169940,0watchdog/11257-21kthreadcore10:47:171
169940,0watchdog/10-21swapper/109:55:131
169940,0watchdog/10-21swapper/107:32:091
149940,0migration/02828-21sensors_volt10:07:180
149940,0migration/019152-21sh08:32:130
149940,0migration/015259-21kthreadcore09:57:170
339930,0migration/39872-21sh07:57:193
29725030,0irq/27-eth0-tx-0-21swapper/008:52:170
29725030,0irq/27-eth0-tx-0-21swapper/007:57:080
29715030,0irq/26-eth0-rx-0-21swapper/308:40:083
29715030,0irq/26-eth0-rx-0-21swapper/308:27:093
29715030,0irq/26-eth0-rx-0-21swapper/308:25:183
29715030,0irq/26-eth0-rx-0-21swapper/308:17:103
29715030,0irq/26-eth0-rx-0-21swapper/308:15:273
29715030,0irq/26-eth0-rx-0-21swapper/307:37:173
249930,0watchdog/221279-21timerandwakeup07:17:212
209949932,0cyclictest7645-21awk08:52:142
209949932,0cyclictest3609-21cat08:17:162
209949932,0cyclictest26855-21sensors06:57:162
209949930,0cyclictest0-21swapper/207:12:082
209939932,0cyclictest14005-21awk10:17:121
209929933,0cyclictest8490-21kworker/0:009:12:080
209929933,0cyclictest8490-21kworker/0:007:42:040
209929933,0cyclictest8490-21kworker/0:007:37:130
209929933,0cyclictest22065-21kworker/0:111:57:050
209929933,0cyclictest22065-21kworker/0:111:42:060
209929933,0cyclictest22065-21kworker/0:111:32:020
209929933,0cyclictest22065-21kworker/0:111:17:160
209929933,0cyclictest22065-21kworker/0:110:57:100
209929933,0cyclictest19520-21kworker/0:210:22:040
209929933,0cyclictest19520-21kworker/0:209:37:000
209929930,0cyclictest30259-21diskstats07:52:120
179930,0migration/12194-21kthreadcore10:27:161
179930,0migration/114088-21runrttasks10:37:121
169930,0watchdog/119168-21ssh10:57:571
169930,0watchdog/110968-21sort10:12:211
169930,0watchdog/10-21swapper/111:57:161
169930,0watchdog/10-21swapper/111:17:081
169930,0watchdog/10-21swapper/110:07:051
249920,0watchdog/20-21swapper/208:32:092
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional