You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 08:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Aug 27, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450700,0irq/25-eth10-21swapper/318:43:323
1112724536,0sleep10-21swapper/118:45:101
562923623,0sleep00-21swapper/018:40:570
1106423324,0sleep20-21swapper/218:44:272
339960,0migration/37192-21sh21:21:533
339960,0migration/34212-21sh23:01:223
339960,0migration/332133-21sh21:18:283
339960,0migration/328383-21missed_timers22:35:593
339960,0migration/322206-21sh22:34:053
339960,0migration/321965-21munin-node20:05:563
339960,0migration/315083-21sh20:25:413
339960,0migration/312113-21sh23:49:243
29715060,0irq/26-eth0-rx-29594-21kthreadcore20:55:593
112449966,0cyclictest0-21swapper/200:08:222
112449960,0cyclictest0-21swapper/223:51:022
339950,0migration/36694-21munin-node19:51:023
339950,0migration/328987-21cpuspeed_turbos00:00:533
339950,0migration/327791-21irqcore19:45:573
339950,0migration/325330-21kthreadcore20:31:003
339950,0migration/323534-21latency_hist22:15:413
339950,0migration/323234-21sendmail_mailqu20:51:023
339950,0migration/320348-21munin-node20:01:053
339950,0migration/318538-21fschecks_count18:50:543
339950,0migration/317862-21munin-node19:11:023
339950,0migration/316792-21irqcore22:10:573
339950,0migration/314419-21kthreadcore18:45:583
339950,0migration/313268-21latency_hist23:30:413
339950,0migration/312960-21taskset19:34:113
339950,0migration/312960-21taskset19:34:113
339950,0migration/310589-21kthreadcore21:45:583
29725050,0irq/27-eth0-tx-0-21swapper/200:02:462
29715050,0irq/26-eth0-rx-8399-21ssh21:03:543
29715050,0irq/26-eth0-rx-27456-21yum23:36:093
29715050,0irq/26-eth0-rx-24685-21sensors_fan19:16:103
29715050,0irq/26-eth0-rx-14284-21ssh21:26:003
29715050,0irq/26-eth0-rx-0-21swapper/323:41:013
29715050,0irq/26-eth0-rx-0-21swapper/323:24:163
29715050,0irq/26-eth0-rx-0-21swapper/323:05:553
29715050,0irq/26-eth0-rx-0-21swapper/322:58:393
29715050,0irq/26-eth0-rx-0-21swapper/322:51:023
29715050,0irq/26-eth0-rx-0-21swapper/322:05:583
29715050,0irq/26-eth0-rx-0-21swapper/322:00:563
29715050,0irq/26-eth0-rx-0-21swapper/321:53:543
29715050,0irq/26-eth0-rx-0-21swapper/321:41:243
29715050,0irq/26-eth0-rx-0-21swapper/300:12:083
29715050,0irq/26-eth0-rx-0-21swapper/300:05:433
29715050,0irq/26-eth0-rx-0-21swapper/300:00:013
27150,0ktimersoftd/20-21swapper/223:42:402
259950,0migration/27722-21aten2.4-expect19:55:542
259950,0migration/218182-21timerwakeupswit19:11:042
249950,0watchdog/27533-21interrupts22:45:552
179950,0migration/15801-21fschecks_count23:25:551
179950,0migration/130988-21taskset21:36:491
179950,0migration/123203-21idleruntime22:55:551
179950,0migration/121633-21munin-node21:51:001
179950,0migration/12096-21munin-node21:00:561
179950,0migration/118474-21kthreadcore19:35:571
179950,0migration/118209-21kthreadcore23:30:581
179950,0migration/115897-21hwlatdetect22:30:551
169950,0watchdog/19366-21diskmemload21:13:241
169950,0watchdog/132656-21ssh20:58:441
169950,0watchdog/11288-21grep00:00:591
149950,0migration/020959-21kthreadcore21:50:590
149950,0migration/011767-21cpuspeed_turbos18:45:530
112449955,0cyclictest0-21swapper/223:47:522
112449955,0cyclictest0-21swapper/222:19:262
112449955,0cyclictest0-21swapper/221:46:292
112449955,0cyclictest0-21swapper/221:38:452
112449955,0cyclictest0-21swapper/221:21:432
112449955,0cyclictest0-21swapper/221:02:482
112449955,0cyclictest0-21swapper/220:41:122
112449950,0cyclictest0-21swapper/223:30:222
112449950,0cyclictest0-21swapper/223:24:042
112449950,0cyclictest0-21swapper/222:58:222
112449950,0cyclictest0-21swapper/222:26:032
112439950,0cyclictest0-21swapper/123:42:221
112429955,0cyclictest9366-21diskmemload21:58:010
112429954,0cyclictest0-21swapper/023:33:520
112429953,0cyclictest7605-21awk19:30:580
112429953,0cyclictest7605-21awk19:30:580
112429953,0cyclictest3241-21irqbalance23:17:290
112429950,0cyclictest0-21swapper/023:24:440
112429950,0cyclictest0-21swapper/021:02:500
339940,0migration/33367-21fschecks_count20:40:563
339940,0migration/331783-21taskset19:22:283
339940,0migration/331344-21latency_hist19:00:423
339940,0migration/319933-21latency_hist19:40:423
339940,0migration/319081-21kthreadcore20:25:593
339940,0migration/314042-21diskmemload23:10:523
29725040,0irq/27-eth0-tx-27993-21ssh23:16:062
29725040,0irq/27-eth0-tx-27128-21ssh23:56:442
29725040,0irq/27-eth0-tx-18479-21ssh00:11:122
29725040,0irq/27-eth0-tx-15695-21munin-node22:11:032
29725040,0irq/27-eth0-tx-14481-21munin-node22:50:552
29725040,0irq/27-eth0-tx-0-21swapper/223:30:582
29725040,0irq/27-eth0-tx-0-21swapper/222:35:482
29725040,0irq/27-eth0-tx-0-21swapper/222:06:462
29725040,0irq/27-eth0-tx-0-21swapper/222:00:522
29725040,0irq/27-eth0-tx-0-21swapper/221:50:502
29725040,0irq/27-eth0-tx-0-21swapper/221:06:002
29725040,0irq/27-eth0-tx-0-21swapper/020:21:030
29715040,0irq/26-eth0-rx-4224-21ps22:41:023
29715040,0irq/26-eth0-rx-1804-21munin-node20:36:023
29715040,0irq/26-eth0-rx-0-21swapper/323:52:433
29715040,0irq/26-eth0-rx-0-21swapper/323:25:553
29715040,0irq/26-eth0-rx-0-21swapper/323:15:533
29715040,0irq/26-eth0-rx-0-21swapper/322:45:533
29715040,0irq/26-eth0-rx-0-21swapper/322:25:563
29715040,0irq/26-eth0-rx-0-21swapper/322:20:423
29715040,0irq/26-eth0-rx-0-21swapper/321:56:083
29715040,0irq/26-eth0-rx-0-21swapper/321:36:343
29715040,0irq/26-eth0-rx-0-21swapper/321:31:523
29715040,0irq/26-eth0-rx-0-21swapper/321:07:113
29715040,0irq/26-eth0-rx-0-21swapper/320:47:003
29715040,0irq/26-eth0-rx-0-21swapper/320:10:533
259940,0migration/25228-21latency19:26:002
259940,0migration/231553-21kthreadcore20:35:582
259940,0migration/231513-21aten2.4-expect19:00:532
259940,0migration/231181-21cpuspeed_turbos22:40:532
259940,0migration/223326-21kthreadcore23:35:582
259940,0migration/220194-21aten2.4-expect19:40:542
259940,0migration/217676-21munin-node21:10:522
259940,0migration/217183-21munin-node21:30:552
259940,0migration/215135-21munin-node20:45:592
259940,0migration/214073-21aten_r1power_vo23:10:522
249940,0watchdog/226399-21netstat20:06:002
249940,0watchdog/220504-21sed22:30:592
249940,0watchdog/2190082sleep219:15:442
249940,0watchdog/212147-21yum23:06:072
249940,0watchdog/20-21swapper/222:20:552
249940,0watchdog/20-21swapper/221:55:592
249940,0watchdog/20-21swapper/219:05:562
19140,0ktimersoftd/10-21swapper/120:47:421
179940,0migration/19736-21kthreadcore23:06:001
179940,0migration/131322-21munin-node21:16:041
179940,0migration/130675-21munin-node23:20:561
179940,0migration/129306-21kthreadcore20:10:591
179940,0migration/12714-21latency_hist20:40:411
179940,0migration/12680-21kthreadcore21:20:581
179940,0migration/123919-21kthreadcore23:15:571
179940,0migration/122443-21kthreadcore20:05:581
179940,0migration/122006-21gltestperf23:35:541
179940,0migration/121744-21munin-node23:55:581
179940,0migration/120604-21kthreadcore20:50:581
179940,0migration/120421-21df_abs19:40:551
179940,0migration/120290-21kthreadcore21:30:591
179940,0migration/119397-21taskset23:52:351
179940,0migration/118682-21taskset19:14:371
179940,0migration/11680-21kthreadcore19:00:581
179940,0migration/114751-21idleruntime23:10:551
169940,0watchdog/18544-21cut22:25:591
169940,0watchdog/16325-21cut19:05:561
169940,0watchdog/14775-21ssh22:41:191
169940,0watchdog/124732-21munin-node18:55:561
169940,0watchdog/124218-21kthreadcore20:31:001
169940,0watchdog/120983-21smartctl20:26:041
169940,0watchdog/115695-21munin-node22:10:551
169940,0watchdog/114116-21ntp_states20:21:001
169940,0watchdog/112524-21ssh23:50:391
169940,0watchdog/10-21swapper/122:36:031
169940,0watchdog/10-21swapper/122:15:551
169940,0watchdog/10-21swapper/120:16:001
169940,0watchdog/10-21swapper/100:10:551
169940,0watchdog/10-21swapper/100:05:511
149940,0migration/05319-21kthreadcore21:20:580
112449944,0cyclictest0-21swapper/220:56:022
112449944,0cyclictest0-21swapper/220:12:592
112449944,0cyclictest0-21swapper/219:46:582
112449944,0cyclictest0-21swapper/219:31:122
112449944,0cyclictest0-21swapper/219:31:122
112449943,0cyclictest9616-21awk21:25:552
112449943,0cyclictest3556-21lldpd19:20:482
112449943,0cyclictest3514-21snmpd21:40:512
112449943,0cyclictest3514-21snmpd20:26:202
112449943,0cyclictest3514-21snmpd18:50:052
112449943,0cyclictest3241-21irqbalance20:50:502
112449943,0cyclictest31266-21awk23:00:562
112449943,0cyclictest2829-21awk20:15:552
112449943,0cyclictest15684-21kthreadcore20:00:582
112449943,0cyclictest14179-21awk19:35:542
112449940,0cyclictest24926-21latency_hist21:15:412
112439943,0cyclictest9745-21awk21:25:561
112439943,0cyclictest8609-21awk19:55:581
112439943,0cyclictest7410-21awk22:45:551
112439943,0cyclictest3514-21snmpd22:50:511
112439943,0cyclictest3514-21snmpd22:00:511
112439943,0cyclictest3514-21snmpd21:58:511
112439943,0cyclictest3514-21snmpd20:42:211
112439943,0cyclictest3514-21snmpd19:46:351
112439943,0cyclictest3514-21snmpd19:26:501
112439943,0cyclictest31290-21awk23:00:561
112439943,0cyclictest10350-21awk21:06:011
112439940,0cyclictest0-21swapper/122:21:481
112429944,0cyclictest0-21swapper/023:47:350
112429944,0cyclictest0-21swapper/023:43:010
112429944,0cyclictest0-21swapper/023:38:270
112429944,0cyclictest0-21swapper/023:29:180
112429944,0cyclictest0-21swapper/021:46:210
112429944,0cyclictest0-21swapper/021:31:320
112429943,0cyclictest552-21awk22:00:560
112429943,0cyclictest3797-21awk20:40:570
112429943,0cyclictest3556-21lldpd00:10:490
112429943,0cyclictest3514-21snmpd20:18:350
112429943,0cyclictest3514-21snmpd20:11:350
112429943,0cyclictest3514-21snmpd19:09:050
112429943,0cyclictest32497-21awk22:20:540
112429943,0cyclictest3241-21irqbalance23:03:190
112429943,0cyclictest3241-21irqbalance22:43:490
112429943,0cyclictest3241-21irqbalance22:11:400
112429943,0cyclictest3241-21irqbalance22:05:500
112429943,0cyclictest3241-21irqbalance21:26:000
112429943,0cyclictest3241-21irqbalance19:40:500
112429943,0cyclictest3241-21irqbalance19:36:000
112429943,0cyclictest3241-21irqbalance19:27:400
112429943,0cyclictest3241-21irqbalance19:16:000
112429943,0cyclictest3241-21irqbalance00:00:490
112429943,0cyclictest27558-21awk19:45:560
112429943,0cyclictest21573-21awk23:55:570
112429943,0cyclictest1876-21awk19:50:560
112429943,0cyclictest13755-21awk23:50:540
112429943,0cyclictest1028-21awk21:40:550
112429942,0cyclictest26079-21awk21:15:560
112429941,0cyclictest9941-21perl21:05:550
112429941,0cyclictest25672-21munin-node19:21:000
112429941,0cyclictest18503-21diskstats18:50:540
112429941,0cyclictest14885-21diskstats22:50:530
112429941,0cyclictest13572-21sh23:10:290
112429941,0cyclictest12975-21perl19:10:550
112429940,0cyclictest17772-21kthreadcore23:10:580
112429940,0cyclictest0-21swapper/022:48:280
112429940,0cyclictest0-21swapper/022:36:430
112429940,0cyclictest0-21swapper/021:36:430
339930,0migration/31950-21aten_r1power_fr20:15:523
29725030,0irq/27-eth0-tx-0-21swapper/020:00:540
29715030,0irq/26-eth0-rx-6407-21expr19:05:563
29715030,0irq/26-eth0-rx-0-21swapper/321:10:463
29715030,0irq/26-eth0-rx-0-21swapper/319:57:543
29715030,0irq/26-eth0-rx-0-21swapper/319:25:533
29715030,0irq/26-eth0-rx-0-21swapper/318:55:513
249930,0watchdog/20-21swapper/218:55:562
169930,0watchdog/19366-21diskmemload21:40:561
169930,0watchdog/10-21swapper/119:50:521
112449933,0cyclictest3556-21lldpd20:30:482
112449932,0cyclictest9543-21awk20:20:552
112449932,0cyclictest3514-21snmpd19:51:202
112449932,0cyclictest18994-21awk18:50:562
112439933,0cyclictest18979-21awk18:50:561
112439932,0cyclictest8632-21awk22:05:561
112439932,0cyclictest8249-21basename21:45:521
112439932,0cyclictest3514-21snmpd18:46:351
112439932,0cyclictest15233-21awk20:00:561
112439932,0cyclictest12311-21find19:31:041
112439932,0cyclictest12311-21find19:31:041
112439930,0cyclictest0-21swapper/119:16:141
112429932,0cyclictest3241-21irqbalance19:55:500
112429932,0cyclictest28882-21tr20:35:540
112429932,0cyclictest25492-21sh20:55:420
112429932,0cyclictest22526-21ssh22:55:530
112429932,0cyclictest21993-21aten_r1power_cu20:30:540
112429932,0cyclictest18124-21tune2fs20:50:540
112429932,0cyclictest0-21swapper/020:05:530
112429931,0cyclictest7454-21df22:25:550
112429931,0cyclictest4618-21df00:05:500
112429931,0cyclictest41ktimersoftd/022:15:510
112429931,0cyclictest31475-21aten2.4-expect19:00:530
112429931,0cyclictest24789-21aten2.4-expect18:55:510
112429931,0cyclictest17609-21rm21:10:520
112429931,0cyclictest15436-21aten2.4-expect20:25:530
112429931,0cyclictest15080-21sh22:30:480
112429931,0cyclictest10245-21perl20:45:530
29715020,0irq/26-eth0-rx-0-21swapper/319:40:543
169920,0watchdog/10-21swapper/119:20:441
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional