You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 17:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Dec 16, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139524837,0sleep00-21swapper/007:07:530
139524837,0sleep00-21swapper/007:07:530
117423625,0sleep30-21swapper/307:05:153
117423625,0sleep30-21swapper/307:05:153
154223425,0sleep20-21swapper/207:09:352
154223425,0sleep20-21swapper/207:09:352
131123324,0sleep10-21swapper/107:06:541
131123324,0sleep10-21swapper/107:06:541
1626992121,0cyclictest16025-21kworker/2:010:59:362
1626991818,0cyclictest16025-21kworker/2:010:26:362
16239970,0cyclictest0-21swapper/011:38:530
339960,0migration/318656-21sh12:05:173
339960,0migration/312678-21sh12:21:153
29725060,0irq/27-eth0-tx-2780-21munin-node07:35:000
29715060,0irq/26-eth0-rx-4844-21irqrtprio10:44:573
29715060,0irq/26-eth0-rx-13566-21sh11:08:173
29715060,0irq/26-eth0-rx-13321-21cut11:44:573
16249960,0cyclictest9616-21kthreadcore12:19:551
149960,0migration/09371-21hwlatdetect07:14:560
149960,0migration/06674-21sh09:11:250
35150,0ktimersoftd/30-21swapper/311:26:573
35150,0ktimersoftd/30-21swapper/310:54:483
339950,0migration/39651-21runrttasks10:29:203
339950,0migration/39396-21pmu-power12:00:003
339950,0migration/37823-21kthreadcore07:34:593
339950,0migration/34274-21kthreadcore11:39:593
339950,0migration/332694-21kthreadcore10:59:573
339950,0migration/332480-21aten_r1power_cu09:09:523
339950,0migration/329552-21aten_r1power_cu07:54:523
339950,0migration/329515-21sh09:44:443
339950,0migration/326838-21kthreadcore08:14:593
339950,0migration/323976-21sh11:14:553
339950,0migration/323122-21cpuspeed_turbos07:49:533
339950,0migration/322255-21munin-node11:30:003
339950,0migration/321458-21fschecks_count12:09:533
339950,0migration/318734-21diskmemload11:49:413
339950,0migration/318054-21kthreadcore12:24:573
339950,0migration/315360-21idleruntime11:09:553
339950,0migration/311792-21unixbench_singl08:50:043
29725050,0irq/27-eth0-tx-2841-21proc_pri11:00:002
29725050,0irq/27-eth0-tx-21326-21ssh12:09:522
29725050,0irq/27-eth0-tx-17538-21munin-node09:20:072
29725050,0irq/27-eth0-tx-15359-21ssh10:12:332
29725050,0irq/27-eth0-tx-12637-21ssh12:21:092
29725050,0irq/27-eth0-tx-0-21swapper/212:34:512
29725050,0irq/27-eth0-tx-0-21swapper/211:26:482
29725050,0irq/27-eth0-tx-0-21swapper/210:41:382
29725050,0irq/27-eth0-tx-0-21swapper/210:15:012
29725050,0irq/27-eth0-tx-0-21swapper/210:01:162
29725050,0irq/27-eth0-tx-0-21swapper/209:47:232
29715050,0irq/26-eth0-rx-7908-21latency_hist10:09:403
29715050,0irq/26-eth0-rx-6559-21runrttasks10:40:123
29715050,0irq/26-eth0-rx-5162-21sensors10:05:043
29715050,0irq/26-eth0-rx-4626-21ssh12:17:313
29715050,0irq/26-eth0-rx-31727-21munin-node11:19:583
29715050,0irq/26-eth0-rx-23321-21ssh10:16:133
29715050,0irq/26-eth0-rx-12395-21cpuspeed_turbos09:34:533
29715050,0irq/26-eth0-rx-10261-21sh09:31:423
29715050,0irq/26-eth0-rx-0-21swapper/312:35:403
29715050,0irq/26-eth0-rx-0-21swapper/312:29:513
29715050,0irq/26-eth0-rx-0-21swapper/311:34:463
29715050,0irq/26-eth0-rx-0-21swapper/310:49:573
29715050,0irq/26-eth0-rx-0-21swapper/310:34:583
29715050,0irq/26-eth0-rx-0-21swapper/310:21:303
29715050,0irq/26-eth0-rx-0-21swapper/309:49:543
29715050,0irq/26-eth0-rx-0-21swapper/309:21:463
29715050,0irq/26-eth0-rx-0-21swapper/309:15:013
259950,0migration/26271-21munin-node11:04:542
259950,0migration/24261-21kthreadcore11:19:592
259950,0migration/231914-21sh10:21:142
259950,0migration/228685-21munin-node11:15:012
259950,0migration/22411-21kthreadcore12:14:592
259950,0migration/216805-21kthreadcore12:24:572
259950,0migration/214585-21kthreadcore08:29:582
259950,0migration/212418-21sh09:50:532
259950,0migration/210420-21sh09:31:592
249950,0watchdog/228878-21kthreadcore09:04:592
249950,0watchdog/20-21swapper/210:34:592
249950,0watchdog/20-21swapper/207:10:002
179950,0migration/19003-21kthreadcore08:24:591
179950,0migration/130311-21pmu-power08:40:011
179950,0migration/12928-21sendmail_mailqu11:00:011
179950,0migration/128668-21ntp_offset10:00:011
179950,0migration/126710-21kthreadcore11:34:581
179950,0migration/11769-21kthreadcore12:34:541
179950,0migration/117372-21taskset10:32:201
179950,0migration/116528-21latency_hist11:29:401
16269954,0cyclictest29321-21munin-run07:54:402
16249955,0cyclictest0-21swapper/112:18:391
16249955,0cyclictest0-21swapper/111:06:361
16249955,0cyclictest0-21swapper/109:55:491
169950,0watchdog/123469-21cut10:34:591
149950,0migration/0995-21kthreadcore12:34:540
149950,0migration/08713-21kthreadcore11:59:580
149950,0migration/06895-21sh11:22:490
149950,0migration/04984-21munin-node08:24:540
149950,0migration/04432-21fschecks_time07:59:550
149950,0migration/032033-21kthreadcore10:59:560
149950,0migration/030296-21kthreadcore11:54:580
149950,0migration/030001-21fschecks_count07:54:540
149950,0migration/029250-21needreboot08:15:010
149950,0migration/028320-21kthreadcore09:04:590
149950,0migration/026327-21latency_hist09:24:400
149950,0migration/02580-21taskset10:43:120
149950,0migration/025295-21kthreadcore12:29:570
149950,0migration/022576-21kthreadcore12:09:560
149950,0migration/022516-21timerandwakeup10:15:030
149950,0migration/02019-21kthreadcore08:19:580
149950,0migration/020077-21sh09:38:350
149950,0migration/019743-21fschecks_time08:59:540
149950,0migration/017939-21gltestperf08:09:540
149950,0migration/017828-21kthreadcore11:09:570
149950,0migration/016936-21gltestperf07:44:550
149950,0migration/016452-21kthreadcore09:54:580
149950,0migration/016363-21irqcore07:19:570
149950,0migration/016173-21sh09:17:380
149950,0migration/015040-21kthreadcore10:29:580
149950,0migration/011825-21cpuspeed_turbos08:29:530
149950,0migration/011413-21missed_timers12:19:560
149950,0migration/010151-21fschecks_time07:39:530
339940,0migration/330826-21kthreadcore09:05:003
339940,0migration/327930-21taskset07:25:383
339940,0migration/32531-21taskset07:34:383
339940,0migration/315437-21kthreadcore10:29:583
339940,0migration/315321-21kthreadcore08:54:583
339940,0migration/314225-21kthreadcore08:04:583
29725040,0irq/27-eth0-tx-18651-21ssh10:50:292
29725040,0irq/27-eth0-tx-0-21swapper/212:04:482
29725040,0irq/27-eth0-tx-0-21swapper/211:54:562
29725040,0irq/27-eth0-tx-0-21swapper/211:44:512
29725040,0irq/27-eth0-tx-0-21swapper/211:36:152
29725040,0irq/27-eth0-tx-0-21swapper/211:10:182
29725040,0irq/27-eth0-tx-0-21swapper/210:44:522
29725040,0irq/27-eth0-tx-0-21swapper/210:05:142
29725040,0irq/27-eth0-tx-0-21swapper/209:35:342
29725040,0irq/27-eth0-tx-0-21swapper/209:24:502
29725040,0irq/27-eth0-tx-0-21swapper/209:14:522
29725040,0irq/27-eth0-tx-0-21swapper/209:10:512
29725040,0irq/27-eth0-tx-0-21swapper/008:44:570
29725040,0irq/27-eth0-tx-0-21swapper/008:04:590
29715040,0irq/26-eth0-rx-0-21swapper/311:54:593
29715040,0irq/26-eth0-rx-0-21swapper/310:00:183
29715040,0irq/26-eth0-rx-0-21swapper/309:55:343
29715040,0irq/26-eth0-rx-0-21swapper/309:40:243
29715040,0irq/26-eth0-rx-0-21swapper/309:25:443
29715040,0irq/26-eth0-rx-0-21swapper/308:46:113
29715040,0irq/26-eth0-rx-0-21swapper/308:30:013
29715040,0irq/26-eth0-rx-0-21swapper/307:09:543
259940,0migration/24619-21if_err_eth1.311:59:552
259940,0migration/228669-21kthreadcore08:39:582
259940,0migration/226607-21kthreadcore12:29:572
259940,0migration/222174-21irqrtprio09:39:572
259940,0migration/219845-21kthreadcore08:34:572
259940,0migration/219585-21kthreadcore11:29:572
259940,0migration/214418-21timerwakeupswit07:15:052
249940,0watchdog/230425-21if_err_eth1.307:54:562
249940,0watchdog/226887-21kthreadcore08:14:592
249940,0watchdog/219006-21kthreadcore09:54:592
249940,0watchdog/217280-21aten_r1power_en08:09:512
249940,0watchdog/20-21swapper/208:24:592
19458240,0sleep30-21swapper/308:34:573
179940,0migration/15921-21aten_r1power_po09:49:521
179940,0migration/15857-21kthreadcore07:34:591
179940,0migration/15669-21kthreadcore07:59:581
179940,0migration/13404-21kthreadcore10:04:581
179940,0migration/127943-21taskset07:25:481
179940,0migration/127233-21latency_hist11:54:401
179940,0migration/125320-21kthreadcore10:54:591
179940,0migration/123324-21munin-node12:29:521
179940,0migration/122990-21munin-node07:49:521
179940,0migration/122706-21kthreadcore09:39:581
179940,0migration/118337-21unixbench-2d10:50:041
179940,0migration/116613-21kthreadcore07:19:581
179940,0migration/116326-21kthreadcore08:29:591
179940,0migration/114648-21kthreadcore12:04:571
179940,0migration/112938-21fschecks_count08:54:551
179940,0migration/110335-21kthreadcore08:49:591
16269943,0cyclictest6657-21awk08:49:562
16269943,0cyclictest4729-21awk07:59:572
16269943,0cyclictest3556-21lldpd08:46:352
16269943,0cyclictest3556-21lldpd07:40:362
16269943,0cyclictest3514-21snmpd08:21:462
16269943,0cyclictest3514-21snmpd07:47:012
16269943,0cyclictest3514-21snmpd07:36:012
16269943,0cyclictest3514-21snmpd07:22:012
16269943,0cyclictest29388-21awk07:29:562
16269943,0cyclictest13403-21awk08:54:572
16269943,0cyclictest10114-21munin-node10:29:522
16269943,0cyclictest0-21swapper/211:49:532
16269940,0cyclictest1460-21munin-node11:39:532
16249943,0cyclictest9677-21awk09:14:571
16249943,0cyclictest3514-21snmpd12:29:021
16249943,0cyclictest3514-21snmpd11:26:321
16249943,0cyclictest28230-21awk10:39:551
16249943,0cyclictest26744-21awk09:04:571
16249943,0cyclictest24585-21sadc08:39:401
16249943,0cyclictest22002-21awk12:09:551
16249943,0cyclictest17127-21awk07:44:571
16249943,0cyclictest15476-21awk11:09:551
16249943,0cyclictest13050-21awk09:34:551
16239943,0cyclictest6836-21awk09:49:560
16239943,0cyclictest4157-21awk09:29:560
16239943,0cyclictest3514-21snmpd08:44:310
16239943,0cyclictest2816-21sadc09:29:400
16239943,0cyclictest21868-21awk09:39:560
16239943,0cyclictest11174-21awk11:44:550
169940,0watchdog/16824-21kthreadcore10:44:591
169940,0watchdog/16178-21kthreadcore07:10:001
169940,0watchdog/14534-21netstat08:44:591
169940,0watchdog/14172-21kthreadcore11:39:591
169940,0watchdog/11966-21grep10:24:551
169940,0watchdog/116010-21nfs_client08:04:591
169940,0watchdog/114364-21ssh10:10:231
169940,0watchdog/113610-21cut07:15:001
169940,0watchdog/111629-21ssh12:03:391
169940,0watchdog/10-21swapper/111:14:551
169940,0watchdog/10-21swapper/109:24:591
149940,0migration/04155-21runrttasks12:16:300
149940,0migration/028175-21runrttasks10:57:260
149940,0migration/027104-21kthreadcore10:19:580
149940,0migration/025794-21kthreadcore09:59:580
149940,0migration/022413-21kthreadcore11:49:590
149940,0migration/022222-21kthreadcore08:34:580
149940,0migration/019296-21kthreadcore12:24:580
149940,0migration/017326-21kthreadcore10:49:580
4130,0ktimersoftd/029978-21runrttasks11:16:350
29725030,0irq/27-eth0-tx-0-21swapper/007:24:520
29715030,0irq/26-eth0-rx-10696-21kthreadcore07:14:593
29715030,0irq/26-eth0-rx-0-21swapper/309:00:373
29715030,0irq/26-eth0-rx-0-21swapper/308:39:593
29715030,0irq/26-eth0-rx-0-21swapper/308:25:003
29715030,0irq/26-eth0-rx-0-21swapper/308:19:523
29715030,0irq/26-eth0-rx-0-21swapper/308:01:003
29715030,0irq/26-eth0-rx-0-21swapper/307:48:163
29715030,0irq/26-eth0-rx-0-21swapper/307:39:563
29715030,0irq/26-eth0-rx-0-21swapper/307:19:533
259930,0migration/223355-21kthreadcore08:59:582
249930,0watchdog/210560-21munin-node08:04:592
179930,0migration/118703-21irqcore09:19:581
179930,0migration/117233-21runrttasks08:09:481
179930,0migration/113744-21kthreadcore07:39:571
16269932,0cyclictest22773-21awk07:24:562
16249933,0cyclictest3556-21lldpd07:56:051
16249933,0cyclictest30525-21awk09:44:551
16249932,0cyclictest3514-21snmpd09:12:011
16249932,0cyclictest32636-21awk11:19:561
16249932,0cyclictest25061-21grep08:14:591
16249932,0cyclictest20029-21awk08:59:561
16249932,0cyclictest17193-21grep10:14:541
16239932,0cyclictest9917-21sh10:45:140
16239932,0cyclictest8875-21awk10:09:540
16239932,0cyclictest5608-21kthreadcore11:39:590
16239932,0cyclictest3514-21snmpd12:05:470
16239932,0cyclictest3514-21snmpd11:25:170
16239932,0cyclictest3514-21snmpd09:47:140
16239932,0cyclictest2832-21awk07:09:580
16239932,0cyclictest27050-21sadc10:39:400
16239932,0cyclictest2297-21awk10:24:560
16239932,0cyclictest22457-21sensors11:30:020
16239932,0cyclictest15500-21kthreadcore08:54:580
16239930,0cyclictest1637-21cut10:04:580
16239930,0cyclictest0-21swapper/007:49:530
16239930,0cyclictest0-21swapper/007:29:590
169930,0watchdog/132195-21diskmemload10:22:431
169930,0watchdog/10-21swapper/111:54:551
169930,0watchdog/10-21swapper/111:44:551
169930,0watchdog/10-21swapper/109:29:591
29715020,0irq/26-eth0-rx-0-21swapper/308:10:173
16249921,0cyclictest28447-21latency_hist07:29:411
16249920,0cyclictest0-21swapper/108:19:401
16239921,0cyclictest5735-21wc08:49:400
16239920,0cyclictest0-21swapper/011:04:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional