You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-12 - 00:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Dec 11, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350790,0irq/25-eth10-21swapper/306:36:493
2715723424,0sleep20-21swapper/206:37:132
2696823430,0sleep10-21swapper/106:35:001
2716023321,0sleep00-21swapper/006:37:150
339960,0migration/36408-21fschecks_count08:24:113
28415060,0irq/27-eth0-tx-23985-21ssh08:55:452
28415060,0irq/27-eth0-tx-16511-21ssh08:52:112
28415060,0irq/27-eth0-tx-0-21swapper/212:08:012
28415060,0irq/27-eth0-tx-0-21swapper/212:02:322
28415060,0irq/27-eth0-tx-0-21swapper/211:54:312
28415060,0irq/27-eth0-tx-0-21swapper/211:51:182
28415060,0irq/27-eth0-tx-0-21swapper/211:43:192
28415060,0irq/27-eth0-tx-0-21swapper/211:36:582
28415060,0irq/27-eth0-tx-0-21swapper/211:06:392
28415060,0irq/27-eth0-tx-0-21swapper/211:02:172
28415060,0irq/27-eth0-tx-0-21swapper/210:34:062
28415060,0irq/27-eth0-tx-0-21swapper/210:17:222
28415060,0irq/27-eth0-tx-0-21swapper/210:04:152
28415060,0irq/27-eth0-tx-0-21swapper/209:59:422
28415060,0irq/27-eth0-tx-0-21swapper/209:27:282
28415060,0irq/27-eth0-tx-0-21swapper/209:16:392
28415060,0irq/27-eth0-tx-0-21swapper/209:12:542
28415060,0irq/27-eth0-tx-0-21swapper/209:04:352
28415060,0irq/27-eth0-tx-0-21swapper/208:43:542
28415060,0irq/27-eth0-tx-0-21swapper/008:38:570
28405060,0irq/26-eth0-rx-31551-40yum-updatesd-he09:23:023
28405060,0irq/26-eth0-rx-29345-21diskstats10:24:073
273779966,0cyclictest0-21swapper/111:43:031
249960,0watchdog/20-21swapper/211:24:132
169960,0watchdog/12872-21ssh10:24:531
169960,0watchdog/10-21swapper/108:49:171
169960,0watchdog/10-21swapper/106:45:051
8225050,0irq/24-0000:00:25651-21munin-run08:38:542
8225050,0irq/24-0000:00:13121-21turbostat10:13:542
8225050,0irq/24-0000:00:0-21swapper/208:33:542
35150,0ktimersoftd/30-21swapper/309:58:583
339950,0migration/39174-21kthreadcore10:29:123
339950,0migration/38737-21kthreadcore07:59:113
339950,0migration/38312-21taskset06:48:203
339950,0migration/37355-21kthreadcore07:34:103
339950,0migration/37234-21kthreadcore10:09:113
339950,0migration/33715-21fschecks_time10:49:083
339950,0migration/331363-21munin-node08:39:163
339950,0migration/330756-21switchtime11:44:163
339950,0migration/329034-21kthreadcore08:59:133
339950,0migration/328459-21kthreadcore07:49:113
339950,0migration/32805-21munin-node07:29:153
339950,0migration/32805-21munin-node07:29:153
339950,0migration/323115-21sendmail_mailqu08:54:163
339950,0migration/322050-21kthreadcore10:39:123
339950,0migration/321237-21runrttasks10:19:053
339950,0migration/319373-21kthreadcore07:44:123
339950,0migration/31878-21kthreadcore08:19:123
339950,0migration/317780-21munin-node11:39:103
339950,0migration/314755-21kthreadcore07:14:133
339950,0migration/313519-21irqcore10:34:113
339950,0migration/31236-21taskset11:28:573
28415050,0irq/27-eth0-tx-8904-21ssh11:33:492
28415050,0irq/27-eth0-tx-19242-21munin-node08:34:150
28415050,0irq/27-eth0-tx-19191-21kthreadcore11:19:162
28415050,0irq/27-eth0-tx-15463-21ssh09:34:052
28415050,0irq/27-eth0-tx-10824-21ssh11:14:112
28415050,0irq/27-eth0-tx-0-21swapper/210:39:302
28415050,0irq/27-eth0-tx-0-21swapper/210:32:542
28415050,0irq/27-eth0-tx-0-21swapper/210:24:582
28415050,0irq/27-eth0-tx-0-21swapper/210:23:412
28415050,0irq/27-eth0-tx-0-21swapper/209:54:202
28415050,0irq/27-eth0-tx-0-21swapper/209:51:132
28415050,0irq/27-eth0-tx-0-21swapper/209:45:212
28415050,0irq/27-eth0-tx-0-21swapper/209:29:132
28415050,0irq/27-eth0-tx-0-21swapper/208:59:542
28415050,0irq/27-eth0-tx-0-21swapper/208:42:562
28405050,0irq/26-eth0-rx-9370-21sh11:54:103
28405050,0irq/26-eth0-rx-8017-21entropy09:29:073
28405050,0irq/26-eth0-rx-7211-21ssh11:50:493
28405050,0irq/26-eth0-rx-30781-21kthreadcore10:44:143
28405050,0irq/26-eth0-rx-23546-21cpu09:39:063
28405050,0irq/26-eth0-rx-17894-21aten_r1power_en11:19:083
28405050,0irq/26-eth0-rx-17518-21kthreadcore09:54:143
28405050,0irq/26-eth0-rx-11822-21kthreadcore09:49:133
28405050,0irq/26-eth0-rx-0-21swapper/312:08:123
28405050,0irq/26-eth0-rx-0-21swapper/311:59:103
28405050,0irq/26-eth0-rx-0-21swapper/311:27:503
28405050,0irq/26-eth0-rx-0-21swapper/311:04:113
28405050,0irq/26-eth0-rx-0-21swapper/310:58:393
28405050,0irq/26-eth0-rx-0-21swapper/310:06:453
28405050,0irq/26-eth0-rx-0-21swapper/309:44:123
28405050,0irq/26-eth0-rx-0-21swapper/309:36:503
28405050,0irq/26-eth0-rx-0-21swapper/309:07:483
28405050,0irq/26-eth0-rx-0-21swapper/308:51:093
28405050,0irq/26-eth0-rx-0-21swapper/308:46:203
28405050,0irq/26-eth0-rx-0-21swapper/308:04:133
273779955,0cyclictest0-21swapper/110:53:051
273779955,0cyclictest0-21swapper/109:57:441
273779955,0cyclictest0-21swapper/109:25:351
273779955,0cyclictest0-21swapper/108:42:321
273779950,0cyclictest0-21swapper/111:05:231
273779950,0cyclictest0-21swapper/109:30:561
273779950,0cyclictest0-21swapper/109:15:101
273779950,0cyclictest0-21swapper/108:09:071
273769955,0cyclictest0-21swapper/008:14:150
273769955,0cyclictest0-21swapper/007:43:590
273769950,0cyclictest0-21swapper/011:42:220
273769950,0cyclictest0-21swapper/008:52:430
259950,0migration/24705-21idleruntime07:34:072
259950,0migration/24073-21if_eth110:49:092
259950,0migration/23459-21latency07:54:142
259950,0migration/231466-21sh09:22:462
259950,0migration/227338-21kthreadcore09:39:122
259950,0migration/225155-21cpuspeed_turbos11:44:062
259950,0migration/22357-21fschecks_count06:44:092
259950,0migration/216879-21switchtime10:54:182
179950,0migration/19617-21switchtime07:34:151
179950,0migration/131452-21pmu-power11:24:161
179950,0migration/129370-21ntp_offset07:49:141
179950,0migration/118176-21kthreadcore11:59:111
179950,0migration/112474-21munin-node10:34:061
179950,0migration/112380-21kthreadcore11:14:141
179950,0migration/110649-21kthreadcore11:34:131
169950,0watchdog/10-21swapper/110:05:251
169950,0watchdog/10-21swapper/109:37:331
169950,0watchdog/10-21swapper/107:05:331
149950,0migration/09044-21latency_hist11:33:540
149950,0migration/031563-21sh09:23:040
149950,0migration/02963-21kthreadcore09:44:120
149950,0migration/017265-21gltestperf09:14:110
149950,0migration/015594-21sh11:14:160
149950,0migration/01553-21cpuspeed_turbos08:44:070
149950,0migration/015346-21sh09:10:560
149950,0migration/014669-21sh09:32:180
149950,0migration/013820-21kthreadcore10:34:120
149950,0migration/011539-21sendmail_mailst08:24:190
149950,0migration/010106-21kthreadcore10:09:120
8225040,0irq/24-0000:00:0-21swapper/207:19:172
339940,0migration/39123-21diskmemload09:09:093
339940,0migration/325245-21runrttasks08:34:243
339940,0migration/325099-21kthreadcore06:59:113
339940,0migration/324404-21kthreadcore07:24:123
339940,0migration/32340-21kthreadcore07:54:143
339940,0migration/321179-21kthreadcore09:14:153
339940,0migration/31942-21latency_hist11:08:543
28415040,0irq/27-eth0-tx-0-21swapper/008:09:190
28415040,0irq/27-eth0-tx-0-21swapper/007:59:060
28415040,0irq/27-eth0-tx-0-21swapper/007:14:140
28415040,0irq/27-eth0-tx-0-21swapper/006:54:060
28405040,0irq/26-eth0-rx-22433-21kthreadcore10:59:133
28405040,0irq/26-eth0-rx-0-21swapper/311:36:133
28405040,0irq/26-eth0-rx-0-21swapper/311:14:253
28405040,0irq/26-eth0-rx-0-21swapper/310:14:273
28405040,0irq/26-eth0-rx-0-21swapper/309:25:153
28405040,0irq/26-eth0-rx-0-21swapper/307:04:133
273789943,0cyclictest9458-21awk06:49:092
273789943,0cyclictest6445-21snmpd08:06:522
273789943,0cyclictest6445-21snmpd07:33:522
273789943,0cyclictest6445-21snmpd07:33:522
273789943,0cyclictest6445-21snmpd07:17:522
273789943,0cyclictest6445-21snmpd06:59:212
273789943,0cyclictest3425-21lldpd08:09:182
273789943,0cyclictest0-21swapper/210:44:102
273779943,0cyclictest9442-21awk06:49:091
273779943,0cyclictest7467-21awk09:49:101
273779943,0cyclictest6626-21tail08:24:121
273779943,0cyclictest6445-21snmpd10:59:231
273779943,0cyclictest6445-21snmpd09:04:221
273779943,0cyclictest6445-21snmpd08:45:071
273779943,0cyclictest6445-21snmpd08:19:071
273779943,0cyclictest6445-21snmpd08:15:221
273779943,0cyclictest6445-21snmpd07:47:521
273779943,0cyclictest5716-21awk07:59:091
273779943,0cyclictest31682-21kworker/1:209:21:121
273779943,0cyclictest26898-21kworker/1:011:54:501
273779943,0cyclictest26898-21kworker/1:010:44:121
273779943,0cyclictest23300-21kworker/1:110:09:221
273779943,0cyclictest2129-21awk11:29:101
273779943,0cyclictest18236-21awk08:54:101
273769944,0cyclictest0-21swapper/011:00:350
273769943,0cyclictest7492-21awk09:49:100
273769943,0cyclictest6445-21snmpd10:56:380
273769943,0cyclictest6445-21snmpd10:08:530
273769943,0cyclictest4794-21awk07:34:070
273769943,0cyclictest3425-21lldpd08:18:580
273769943,0cyclictest3425-21lldpd08:04:180
273769943,0cyclictest3425-21lldpd06:47:140
273769943,0cyclictest3110-21irqbalance11:57:280
273769943,0cyclictest3110-21irqbalance11:03:580
273769943,0cyclictest3110-21irqbalance09:40:490
273769943,0cyclictest30428-21awk07:29:090
273769943,0cyclictest30428-21awk07:29:090
273769943,0cyclictest26697-21awk11:24:100
273769943,0cyclictest25674-21awk12:04:120
273769943,0cyclictest23095-21awk09:59:100
273769943,0cyclictest16398-21awk09:34:110
273769943,0cyclictest14393-21awk10:14:100
273769943,0cyclictest1337-21awk11:49:090
273769943,0cyclictest11438-21awk07:39:140
259940,0migration/24225-21kthreadcore08:19:122
259940,0migration/230321-21kthreadcore07:04:122
259940,0migration/227814-21kthreadcore08:14:132
259940,0migration/225804-21kthreadcore07:24:132
259940,0migration/217219-21latency_hist07:43:552
249940,0watchdog/20-21swapper/211:08:572
249940,0watchdog/20-21swapper/207:39:132
249940,0watchdog/20-21swapper/207:09:092
249940,0watchdog/20-21swapper/206:39:172
22421240,0sleep00-21swapper/007:20:420
179940,0migration/19461-21latency_hist07:13:551
179940,0migration/14361-21kthreadcore11:09:151
179940,0migration/13652-21kthreadcore09:44:121
179940,0migration/131241-21kthreadcore07:29:111
179940,0migration/131241-21kthreadcore07:29:111
179940,0migration/129708-21taskset09:39:521
179940,0migration/124634-21sh12:04:011
179940,0migration/124307-21kthreadcore10:19:111
179940,0migration/124040-21kthreadcore08:34:121
179940,0migration/122688-21kthreadcore10:39:131
179940,0migration/118744-21taskset08:30:531
179940,0migration/115855-21kthreadcore10:14:121
179940,0migration/113927-21kthreadcore07:14:121
179940,0migration/111482-21kthreadcore09:09:131
179940,0migration/11065-21kthreadcore07:54:131
169940,0watchdog/130394-21netstat11:44:131
169940,0watchdog/125649-21diskmemload10:30:131
169940,0watchdog/125360-21munin-node08:59:091
169940,0watchdog/122181-21ssh09:59:011
169940,0watchdog/121764-21grep11:19:171
169940,0watchdog/120552-21mdstat06:54:131
169940,0watchdog/10-21swapper/111:49:051
169940,0watchdog/10-21swapper/110:54:131
169940,0watchdog/10-21swapper/107:39:131
169940,0watchdog/10-21swapper/106:59:091
169940,0watchdog/10-21swapper/106:39:131
149940,0migration/05885-21kthreadcore09:04:120
149940,0migration/05227-21kthreadcore11:09:150
149940,0migration/04831-21kthreadcore07:09:110
149940,0migration/030868-21kthreadcore07:04:120
149940,0migration/030632-21aten_r1power_en07:54:070
149940,0migration/029550-21kthreadcore11:44:110
149940,0migration/029111-21taskset07:26:190
149940,0migration/026979-21runrttasks10:41:310
149940,0migration/022577-21fschecks_time06:59:080
149940,0migration/019952-21kthreadcore11:19:160
149940,0migration/019599-21kthreadcore08:54:120
149940,0migration/01942-21sh10:46:140
149940,0migration/017748-21kthreadcore09:54:130
28415030,0irq/27-eth0-tx-0-21swapper/008:29:150
28415030,0irq/27-eth0-tx-0-21swapper/006:49:150
28415030,0irq/27-eth0-tx-0-21swapper/006:38:570
28405030,0irq/26-eth0-rx-0-21swapper/308:29:053
28405030,0irq/26-eth0-rx-0-21swapper/308:14:083
28405030,0irq/26-eth0-rx-0-21swapper/308:09:123
28405030,0irq/26-eth0-rx-0-21swapper/307:39:093
28405030,0irq/26-eth0-rx-0-21swapper/307:21:273
28405030,0irq/26-eth0-rx-0-21swapper/307:09:053
28405030,0irq/26-eth0-rx-0-21swapper/306:54:063
28405030,0irq/26-eth0-rx-0-21swapper/306:49:163
28405030,0irq/26-eth0-rx-0-21swapper/306:40:093
273789932,0cyclictest6445-21snmpd08:00:072
273789932,0cyclictest24867-21awk07:49:092
273789932,0cyclictest15992-21grep06:54:092
273779933,0cyclictest16448-21kworker/1:108:04:121
273779933,0cyclictest16448-21kworker/1:107:19:061
273769933,0cyclictest3425-21lldpd10:27:210
273769932,0cyclictest7869-21kthreadcore10:49:120
273769932,0cyclictest6445-21snmpd10:19:230
273769932,0cyclictest3425-21lldpd11:38:140
273769932,0cyclictest31592-21ssh08:59:400
273769932,0cyclictest26635-21kthreadcore07:49:110
273769931,0cyclictest0-21swapper/010:29:080
273769930,0cyclictest22428-21sshd11:59:170
249930,0watchdog/211322-21nfs4_client08:24:172
169930,0watchdog/10-21swapper/107:23:571
149930,0migration/01544-21kthreadcore09:24:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional