You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 14:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat May 02, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351710,0ktimersoftd/315006-21perf06:56:373
2139323430,0sleep00-21swapper/006:54:430
2133623223,0sleep20-21swapper/206:54:012
2127023021,0sleep10-21swapper/106:53:161
339960,0migration/33242-21sh09:45:313
339960,0migration/326995-21sh09:38:493
339960,0migration/325826-21sh10:37:373
339960,0migration/324250-21munin-node07:47:013
339960,0migration/32071-21sh11:05:213
339960,0migration/319752-21sh09:36:033
339960,0migration/317997-21sh10:12:273
339960,0migration/312839-21kernelversion10:32:053
339960,0migration/312373-21munin-node09:12:003
339960,0migration/310834-21sh09:49:003
339960,0migration/310357-21latency_hist07:36:473
29725060,0irq/27-eth0-tx-11494-21munin-node08:02:030
216509966,0cyclictest19838-21diskmemload09:01:001
19160,0ktimersoftd/119838-21diskmemload10:49:181
149960,0migration/09557-21sh11:49:440
149960,0migration/03175-21sh10:06:090
149960,0migration/027564-21switchtime06:57:110
149960,0migration/023222-21kthreadcore11:17:040
149960,0migration/018028-21sh10:54:070
149960,0migration/01535-21sh12:26:250
8215050,0irq/24-0000:00:3432-21latency_hist10:06:472
339950,0migration/37254-21kthreadcore07:07:063
339950,0migration/34363-21taskset07:54:353
339950,0migration/34264-21cpuspeed_turbos09:07:003
339950,0migration/331507-21kthreadcore12:02:043
339950,0migration/331120-21kthreadcore10:22:043
339950,0migration/32895-21kthreadcore08:42:043
339950,0migration/325108-21munin-node08:12:013
339950,0migration/324772-21munin-node11:57:103
339950,0migration/321507-21irqcore09:17:043
339950,0migration/321488-21kthreadcore11:17:043
339950,0migration/319351-21sh09:55:343
339950,0migration/318140-21taskset11:15:073
339950,0migration/31594-21timerwakeupswit10:02:113
339950,0migration/311109-21missed_timers08:22:063
29725050,0irq/27-eth0-tx-5265-21tail09:27:032
29725050,0irq/27-eth0-tx-32260-21proc_pri12:02:072
29725050,0irq/27-eth0-tx-28509-21munin-node07:02:030
29725050,0irq/27-eth0-tx-27034-21munin-node11:22:062
29725050,0irq/27-eth0-tx-26789-21munin-node11:42:102
29725050,0irq/27-eth0-tx-26090-21ssh11:40:342
29725050,0irq/27-eth0-tx-2606-21ssh11:26:522
29725050,0irq/27-eth0-tx-25914-21ssh11:19:232
29725050,0irq/27-eth0-tx-11234-21ssh10:11:032
29725050,0irq/27-eth0-tx-0-21swapper/211:12:002
29725050,0irq/27-eth0-tx-0-21swapper/211:04:202
29725050,0irq/27-eth0-tx-0-21swapper/210:59:052
29725050,0irq/27-eth0-tx-0-21swapper/210:30:102
29725050,0irq/27-eth0-tx-0-21swapper/209:52:102
29725050,0irq/27-eth0-tx-0-21swapper/209:42:272
29725050,0irq/27-eth0-tx-0-21swapper/209:25:012
29725050,0irq/27-eth0-tx-0-21swapper/209:12:592
29725050,0irq/27-eth0-tx-0-21swapper/209:09:452
29725050,0irq/27-eth0-tx-0-21swapper/008:56:490
29725050,0irq/27-eth0-tx-0-21swapper/008:37:020
29725050,0irq/27-eth0-tx-0-21swapper/007:36:590
29725050,0irq/27-eth0-tx-0-21swapper/007:22:090
29715050,0irq/26-eth0-rx-7746-21missed_timers12:07:053
29715050,0irq/26-eth0-rx-4498-21kthreadcore10:47:043
29715050,0irq/26-eth0-rx-27970-21entropy10:41:593
29715050,0irq/26-eth0-rx-1934-21ssh09:02:183
29715050,0irq/26-eth0-rx-16437-21ssh12:12:123
29715050,0irq/26-eth0-rx-0-21swapper/312:22:153
29715050,0irq/26-eth0-rx-0-21swapper/312:17:583
29715050,0irq/26-eth0-rx-0-21swapper/311:47:043
29715050,0irq/26-eth0-rx-0-21swapper/311:44:023
29715050,0irq/26-eth0-rx-0-21swapper/311:27:093
29715050,0irq/26-eth0-rx-0-21swapper/311:07:103
29715050,0irq/26-eth0-rx-0-21swapper/310:52:533
29715050,0irq/26-eth0-rx-0-21swapper/310:27:103
29715050,0irq/26-eth0-rx-0-21swapper/308:58:473
29715050,0irq/26-eth0-rx-0-21swapper/307:37:013
259950,0migration/229856-21kthreadcore10:22:042
259950,0migration/229316-21latency_hist07:26:472
259950,0migration/227280-21sh09:18:592
259950,0migration/219974-21latency_hist09:56:472
259950,0migration/218095-21switchtime09:32:092
259950,0migration/216995-21munin-node10:52:102
216509955,0cyclictest0-21swapper/112:02:271
216509950,0cyclictest0-21swapper/111:54:341
216509950,0cyclictest0-21swapper/111:12:291
216509950,0cyclictest0-21swapper/110:42:501
216509950,0cyclictest0-21swapper/110:38:161
216509950,0cyclictest0-21swapper/110:31:481
216509950,0cyclictest0-21swapper/109:16:271
216489955,0cyclictest0-21swapper/010:22:330
216489950,0cyclictest0-21swapper/010:33:350
179950,0migration/19459-21sh11:28:521
179950,0migration/19445-21kthreadcore08:47:051
179950,0migration/127019-21fschecks_time12:22:011
179950,0migration/126431-21taskset09:57:531
179950,0migration/118543-21munin-node12:16:591
179950,0migration/115896-21munin-node12:12:081
179950,0migration/114148-21kthreadcore08:27:041
169950,0watchdog/127275-21grep07:22:061
149950,0migration/08489-21kthreadcore08:47:050
149950,0migration/06366-21kthreadcore07:32:050
149950,0migration/04734-21irqrtprio10:27:020
149950,0migration/031552-21runrttasks11:22:040
149950,0migration/030742-21munin-node07:52:030
149950,0migration/028335-21kthreadcore08:12:060
149950,0migration/027348-21aten_r1power_fr11:01:580
149950,0migration/02726-21latency_hist07:06:480
149950,0migration/027064-21kthreadcore07:47:050
149950,0migration/024225-21switchtime12:17:090
149950,0migration/022438-21kthreadcore11:37:040
149950,0migration/020820-21gltestperf09:57:000
149950,0migration/020077-21kthreadcore08:07:040
149950,0migration/019569-21if_eth1.211:57:020
149950,0migration/019343-21cpuspeed_turbos08:31:590
149950,0migration/019275-21kthreadcore07:42:040
149950,0migration/018789-21taskset08:30:550
149950,0migration/016228-21kthreadcore09:52:050
149950,0migration/015759-21munin-node12:12:060
149950,0migration/013293-21kthreadcore10:12:050
149950,0migration/012192-21taskset08:26:380
149950,0migration/012160-21latency_hist09:31:470
149950,0migration/011791-21irqcore11:32:030
339940,0migration/3894-21kthreadcore07:27:063
339940,0migration/328851-21taskset07:22:293
339940,0migration/322518-21kthreadcore09:57:033
339940,0migration/320186-21diskmemload10:17:003
339940,0migration/319202-21fschecks_count11:37:003
339940,0migration/316809-21taskset11:52:283
29725040,0irq/27-eth0-tx-29471-21munin-node07:27:030
29725040,0irq/27-eth0-tx-24611-21ssh12:17:292
29725040,0irq/27-eth0-tx-13665-21kthreadcore11:32:042
29725040,0irq/27-eth0-tx-0-21swapper/211:56:502
29725040,0irq/27-eth0-tx-0-21swapper/211:52:222
29725040,0irq/27-eth0-tx-0-21swapper/211:07:022
29725040,0irq/27-eth0-tx-0-21swapper/210:38:172
29725040,0irq/27-eth0-tx-0-21swapper/208:57:472
29725040,0irq/27-eth0-tx-0-21swapper/007:57:090
29715040,0irq/26-eth0-rx-0-21swapper/311:33:023
29715040,0irq/26-eth0-rx-0-21swapper/311:22:083
29715040,0irq/26-eth0-rx-0-21swapper/310:59:583
29715040,0irq/26-eth0-rx-0-21swapper/310:07:293
29715040,0irq/26-eth0-rx-0-21swapper/309:26:483
29715040,0irq/26-eth0-rx-0-21swapper/309:22:103
29715040,0irq/26-eth0-rx-0-21swapper/308:53:193
29715040,0irq/26-eth0-rx-0-21swapper/308:36:593
29715040,0irq/26-eth0-rx-0-21swapper/308:17:043
259940,0migration/28202-21kthreadcore08:22:042
259940,0migration/24640-21kthreadcore11:47:042
259940,0migration/24582-21fschecks_time09:47:002
259940,0migration/24373-21munin-node08:17:062
259940,0migration/232452-21kthreadcore10:42:042
259940,0migration/231702-21kthreadcore07:02:052
259940,0migration/231346-21kthreadcore07:27:052
259940,0migration/225952-21proc_pri09:37:072
259940,0migration/220193-21runrttasks10:17:012
259940,0migration/218336-21kthreadcore07:17:062
259940,0migration/214551-21kthreadcore10:12:052
259940,0migration/212506-21kthreadcore12:12:032
249940,0watchdog/29697-21chrt10:48:052
249940,0watchdog/228774-21tr08:12:062
249940,0watchdog/226358-21ssh12:21:532
249940,0watchdog/20-21swapper/210:32:052
249940,0watchdog/20-21swapper/208:52:022
216519943,0cyclictest5759-21awk07:57:042
216519943,0cyclictest3626-21awk07:07:032
216519943,0cyclictest3514-21snmpd12:08:452
216519943,0cyclictest3514-21snmpd08:49:592
216519943,0cyclictest3514-21snmpd08:41:142
216519943,0cyclictest3514-21snmpd08:09:292
216519943,0cyclictest3514-21snmpd07:41:152
216519943,0cyclictest3514-21snmpd07:36:452
216519943,0cyclictest3241-21irqbalance09:01:562
216519943,0cyclictest3241-21irqbalance08:27:062
216519943,0cyclictest22817-21awk06:57:032
216519943,0cyclictest10384-21awk07:12:032
216509944,0cyclictest0-21swapper/110:08:381
216509943,0cyclictest878-21awk08:42:021
216509943,0cyclictest3556-21lldpd08:21:491
216509943,0cyclictest3514-21snmpd11:57:151
216509943,0cyclictest3514-21snmpd11:49:151
216509943,0cyclictest3514-21snmpd10:29:451
216509943,0cyclictest3514-21snmpd10:12:441
216509943,0cyclictest3514-21snmpd09:26:591
216509943,0cyclictest3514-21snmpd08:39:441
216509943,0cyclictest3514-21snmpd08:36:441
216509943,0cyclictest3514-21snmpd07:29:451
216509943,0cyclictest3241-21irqbalance10:24:161
216509943,0cyclictest3241-21irqbalance09:41:561
216509943,0cyclictest3241-21irqbalance09:39:261
216509943,0cyclictest22729-21awk06:57:031
216509943,0cyclictest19900-21awk11:17:021
216509943,0cyclictest19000-21awk08:07:021
216509943,0cyclictest17174-21grep07:17:051
216509943,0cyclictest12449-21awk08:02:031
216509943,0cyclictest12148-21awk10:52:031
216509943,0cyclictest11376-21awk07:37:021
216489944,0cyclictest12940-21kworker/0:210:08:290
216489944,0cyclictest0-21swapper/011:07:290
216489943,0cyclictest4877-21awk09:47:010
216489943,0cyclictest3570-21awk11:27:020
216489943,0cyclictest3556-21lldpd09:44:190
216489943,0cyclictest3514-21snmpd10:38:300
216489943,0cyclictest3514-21snmpd09:06:590
216489943,0cyclictest3514-21snmpd08:52:440
216489943,0cyclictest3514-21snmpd08:43:590
216489943,0cyclictest3241-21irqbalance11:13:250
216489943,0cyclictest3241-21irqbalance11:00:250
216489943,0cyclictest21264-21awk09:17:030
216489943,0cyclictest20102-21sadc09:16:470
179940,0migration/130254-21kthreadcore11:02:041
179940,0migration/128208-21irqcore11:22:031
179940,0migration/124894-21aten_r1power_po08:12:001
179940,0migration/120334-21runrttasks09:16:491
179940,0migration/119654-21munin-node10:56:591
179940,0migration/11952-21swap09:22:111
179940,0migration/119188-21munin-node11:37:001
179940,0migration/118595-21sh09:53:221
179940,0migration/111678-21kthreadcore07:12:051
169940,0watchdog/19906-21ssh11:09:291
169940,0watchdog/15478-21tune2fs07:57:021
169940,0watchdog/14687-21perl09:07:021
169940,0watchdog/13118-21if_err_eth1.312:07:011
169940,0watchdog/110748-21munin-node11:32:011
169940,0watchdog/110461-21ssh09:47:541
169940,0watchdog/10-21swapper/110:17:051
169940,0watchdog/10-21swapper/109:32:061
169940,0watchdog/10-21swapper/108:52:021
149940,0migration/028214-21sh09:01:570
149940,0migration/02460-21diskmemload12:06:580
29725030,0irq/27-eth0-tx-0-21swapper/007:11:590
29715030,0irq/26-eth0-rx-0-21swapper/308:47:033
29715030,0irq/26-eth0-rx-0-21swapper/308:31:593
29715030,0irq/26-eth0-rx-0-21swapper/308:26:593
29715030,0irq/26-eth0-rx-0-21swapper/308:06:593
29715030,0irq/26-eth0-rx-0-21swapper/308:01:593
29715030,0irq/26-eth0-rx-0-21swapper/307:57:073
29715030,0irq/26-eth0-rx-0-21swapper/307:41:583
29715030,0irq/26-eth0-rx-0-21swapper/307:12:003
29715030,0irq/26-eth0-rx-0-21swapper/307:01:593
259930,0migration/223352-21runrttasks07:43:472
216519932,0cyclictest3556-21lldpd08:45:192
216519932,0cyclictest3556-21lldpd08:31:492
216519932,0cyclictest3514-21snmpd07:52:142
216519932,0cyclictest3514-21snmpd07:48:442
216509933,0cyclictest24761-21awk07:47:031
216509932,0cyclictest9486-21sensors07:32:091
216509932,0cyclictest8539-21sensors07:07:101
216509932,0cyclictest3514-21snmpd09:02:441
216509932,0cyclictest3241-21irqbalance10:02:161
216509932,0cyclictest2501-21crond07:06:471
216509932,0cyclictest18102-21awk07:42:021
216509930,0cyclictest31134-21perl07:52:041
216489933,0cyclictest22510-21kworker/0:008:16:530
216489933,0cyclictest22510-21kworker/0:007:16:530
216489933,0cyclictest17290-21kworker/0:011:41:540
216489933,0cyclictest17290-21kworker/0:010:42:060
216489933,0cyclictest12940-21kworker/0:210:17:090
216489933,0cyclictest12940-21kworker/0:209:37:010
216489933,0cyclictest12940-21kworker/0:209:28:090
216489933,0cyclictest11549-21awk11:52:030
216489933,0cyclictest0-21swapper/010:47:180
216489932,0cyclictest28498-21cat09:21:470
216489932,0cyclictest28369-21grep12:02:030
169930,0watchdog/10-21swapper/111:42:011
169930,0watchdog/10-21swapper/108:17:021
329920,0watchdog/30-21swapper/306:57:033
29715020,0irq/26-eth0-rx-0-21swapper/307:17:103
249920,0watchdog/20-21swapper/208:01:502
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional