You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 19:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Mon Nov 24, 2025 12:46:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24466210995sleep100-21swapper/1007:05:362
24603210793sleep60-21swapper/607:07:218
12250980irq/42-ahci0-21swapper/107:08:331
12250970irq/42-ahci0-21swapper/207:05:284
3066350960irq/52-eth0-rx-0-21swapper/407:05:186
3066350910irq/52-eth0-rx-0-21swapper/507:05:227
3066350900irq/52-eth0-rx-0-21swapper/807:05:1410
12250860irq/42-ahci0-21swapper/307:09:435
2457628476sleep90-21swapper/907:06:5911
12250690irq/42-ahci0-21swapper/707:08:269
3066350680irq/52-eth0-rx-0-21swapper/1107:06:503
2458126755sleep00-21swapper/007:07:030
2495499330cyclictest0-21swapper/008:13:500
24966992726cyclictest3066350irq/52-eth0-rx-08:13:504
24956992726cyclictest12250irq/42-ahci08:14:071
24987992422cyclictest3066450irq/53-eth0-tx-08:13:517
25002992316cyclictest13090-21python10:30:302
25002992316cyclictest0-21swapper/1008:25:332
25002992214cyclictest0-21swapper/1011:08:512
2500399190cyclictest0-21swapper/1109:14:273
24954991917cyclictest0-21swapper/009:20:030
25002991816cyclictest811rcuc/1011:15:522
24993991816cyclictest8800-21df11:55:1510
24993991816cyclictest8346-21snmp_rack3slot910:25:1510
24993991816cyclictest6789-21snmp_easybox.os12:35:2510
24993991816cyclictest5466-21snmp_rack3slot908:51:4710
24993991816cyclictest4036-21snmp_rack3slot912:31:4910
24993991816cyclictest30333-21df_inode11:40:1410
24993991816cyclictest29857-21df10:55:1610
24993991816cyclictest26993-21ntp_states10:05:2810
24993991816cyclictest23398-21irqstats10:45:2110
24993991816cyclictest22070-21df10:00:1410
24993991816cyclictest22070-21df10:00:1310
24993991816cyclictest20154-21iostat_ios11:25:1910
24993991816cyclictest14567-21snmp_rack3slot912:01:4810
24993991816cyclictest12664-21snmp_rack3slot909:00:4710
24993991816cyclictest1142-21snmp_rack3slot908:00:4910
24991991816cyclictest27998-21qemu-kvm10:26:108
24991991816cyclictest27998-21qemu-kvm10:15:118
24991991816cyclictest27997-21qemu-kvm10:10:348
24954991814cyclictest0-21swapper/009:19:570
2500399170cyclictest0-21swapper/1109:17:223
25002991715cyclictest11169-21python07:30:302
2500299171cyclictest37252sleep1011:47:562
2500299170cyclictest95962chrt09:41:402
2500299170cyclictest86362sleep1008:11:072
2500299170cyclictest82902sleep1007:28:142
2500299170cyclictest71702chrt11:50:312
2500299170cyclictest44822sleep1007:20:492
2500299170cyclictest4262sleep1008:45:222
2500299170cyclictest40632sleep1012:32:112
2500299170cyclictest3572sleep1012:27:482
2500299170cyclictest321202sleep1010:58:512
2500299170cyclictest311092sleep1009:27:382
2500299170cyclictest3066350irq/52-eth0-rx-12:05:132
2500299170cyclictest3066350irq/52-eth0-rx-11:10:222
2500299170cyclictest3066350irq/52-eth0-rx-08:30:142
2500299170cyclictest304572sleep1008:40:472
2500299170cyclictest299562chrt07:55:532
2500299170cyclictest290642sleep1012:21:122
2500299170cyclictest287112sleep1011:36:412
2500299170cyclictest284312sleep1007:13:122
2500299170cyclictest27872sleep1010:16:212
2500299170cyclictest277952sleep1010:06:072
2500299170cyclictest273212sleep1009:20:392
2500299170cyclictest250232sleep1011:30:542
2500299170cyclictest239982sleep1008:35:092
2500299170cyclictest238422sleep1010:45:232
2500299170cyclictest228202sleep1007:47:522
2500299170cyclictest22422sleep1009:31:292
2500299170cyclictest212932sleep1010:00:072
2500299170cyclictest212932sleep1010:00:072
2500299170cyclictest212562sleep1010:45:002
2500299170cyclictest200652sleep1009:11:152
2500299170cyclictest174292chrt10:37:232
2500299170cyclictest164502sleep1009:07:242
2500299170cyclictest162062sleep1007:40:022
2500299170cyclictest159612sleep1008:21:442
2500299170cyclictest154092chrt07:36:032
2500299170cyclictest144482sleep1012:00:452
2500299170cyclictest128312sleep1009:02:372
2500299170cyclictest125772chrt09:45:302
2500299170cyclictest125772chrt09:45:302
2500299170cyclictest12382sleep1008:01:432
2500299170cyclictest12250irq/42-ahci10:50:072
2500299170cyclictest12250irq/42-ahci10:21:022
2500299170cyclictest12250irq/42-ahci08:19:252
2500299170cyclictest12250irq/42-ahci08:09:122
2500299170cyclictest109422chrt11:57:202
24995991716cyclictest0-21swapper/908:05:1311
24993991715cyclictest9808-21snmp_rack3slot907:30:1510
24993991715cyclictest9047-21snmp_rack3slot908:56:1310
24993991715cyclictest614-21df10:15:1010
24993991715cyclictest30567-21snmp_rack3slot907:15:1610
24993991715cyclictest28724-21snmp_rack3slot911:36:4910
24993991715cyclictest28324-21snmp_rack3slot910:51:4910
24993991715cyclictest26350-21snmp_rack3slot907:51:1310
24993991715cyclictest26228-21df07:10:1410
24993991715cyclictest26167-21snmp_easybox.os08:35:2510
24993991715cyclictest25878-21snmp_rack3slot909:20:1510
24993991715cyclictest23446-21diskstats12:15:1210
24993991715cyclictest22003-21snmp_easybox.os07:45:2310
24993991715cyclictest21840-21snmp_rack3slot912:11:5110
24993991715cyclictest19231-21snmp_easybox.os10:40:2610
24993991715cyclictest18318-21iostat08:25:2010
24993991715cyclictest17691-21users11:20:3010
24993991715cyclictest17237-21snmp_easybox.os09:10:1210
24993991715cyclictest10552-21snmp_rack3slot908:15:1410
2499199175cyclictest0-21swapper/609:14:488
24991991716cyclictest27997-21qemu-kvm10:35:158
24991991715cyclictest5139-21qemu-kvm08:08:408
24991991715cyclictest27998-21qemu-kvm10:30:148
24991991715cyclictest24713-21qemu-kvm07:25:148
24966991715cyclictest2406-21snmpd09:07:194
24966991715cyclictest2406-21snmpd08:24:034
2495499170cyclictest0-21swapper/007:19:080
2500399160cyclictest0-21swapper/1107:27:183
24995991615cyclictest12250irq/42-ahci08:10:3111
24995991615cyclictest0-21swapper/912:35:1611
24995991615cyclictest0-21swapper/911:20:0011
24995991615cyclictest0-21swapper/907:58:1311
2499599161cyclictest0-21swapper/908:25:2711
24993991615cyclictest29016-21snmp_rack3slot908:40:1810
24993991615cyclictest17320-21snmp_rack3slot912:05:2410
24993991615cyclictest14115-21snmp_rack3slot911:15:5910
24993991614cyclictest9685-21ntp_states11:10:2510
24993991614cyclictest9517-21yum09:40:3210
24993991614cyclictest6562-21snmp_rack3slot907:25:1710
24993991614cyclictest4899-21diskstats11:05:1610
24993991614cyclictest4456-21df_inode10:20:1310
24993991614cyclictest3566-21snmp_rack3slot911:45:5010
24993991614cyclictest3321-21snmp_rack3slot911:02:1310
24993991614cyclictest28008-21diskstats07:55:1510
24993991614cyclictest27372-21perl12:20:1510
24993991614cyclictest13351-21df07:35:1210
24993991614cyclictest11741-21snmp_rack3slot910:30:1410
24991991614cyclictest27998-21qemu-kvm10:41:188
2498099160cyclictest0-21swapper/408:30:246
2497499160cyclictest0-21swapper/308:30:255
2495499160cyclictest65332sleep010:22:360
25002991514cyclictest20775-21python11:25:272
2500299150cyclictest0-21swapper/1010:25:152
2499599154cyclictest0-21swapper/909:23:0911
24995991514cyclictest19088-21missed_timers09:10:2211
24995991514cyclictest0-21swapper/911:25:0611
24995991514cyclictest0-21swapper/907:20:1711
2499599151cyclictest0-21swapper/908:20:0111
2499599150cyclictest0-21swapper/908:00:5311
24993991513cyclictest7950-21snmp_easybox.os08:10:2210
24993991513cyclictest29280-21snmp_rack3slot909:25:1610
24993991513cyclictest25125-21snmp_rack3slot911:32:1310
24993991513cyclictest18242-21ntp_states07:40:2210
24993991513cyclictest15583-21snmp_easybox.os09:05:2510
24991991514cyclictest27998-21qemu-kvm10:45:368
24991991513cyclictest27998-21qemu-kvm10:08:598
24980991513cyclictest16710-21sensors10:35:266
24980991513cyclictest1032-21sensors08:45:316
2497499150cyclictest0-21swapper/307:45:125
2495499150cyclictest0-21swapper/008:45:250
8999140migration/1113009-21turbostat09:09:543
31140ksoftirqd/00-21swapper/011:05:260
31140ksoftirqd/00-21swapper/009:30:240
25002991413cyclictest30885-21python10:10:282
25002991412cyclictest24794-21python12:15:262
24995991413cyclictest26611-21sshd07:54:4611
24995991413cyclictest2539-1kworker/9:2H09:45:0211
24995991413cyclictest2539-1kworker/9:2H09:45:0211
24995991413cyclictest0-21swapper/912:20:1211
2499599141cyclictest0-21swapper/907:30:3211
24993991412cyclictest965-21irqstats09:30:1910
24993991412cyclictest7242-21snmp_rack3slot911:51:2410
24993991412cyclictest3158-21snmp_easybox.os09:35:1310
24993991412cyclictest30286-21irqstats10:10:2310
24993991412cyclictest29930-21snmp_easybox.os12:25:1210
24993991412cyclictest20505-21snmp_rack3slot909:56:3710
24993991412cyclictest1585-21snmp_easybox.os07:20:0910
24993991412cyclictest13496-21munin-run09:50:0110
24993991412cyclictest13496-21munin-run09:50:0110
2499199143cyclictest0-21swapper/611:30:008
2496699144cyclictest0-21swapper/210:30:194
24966991412cyclictest2406-21snmpd10:42:034
24966991412cyclictest2406-21snmpd09:45:494
24966991412cyclictest2406-21snmpd09:45:484
24954991413cyclictest7155-21turbostat11:14:540
25002991312cyclictest23086-21python09:15:282
2500299130cyclictest90312sleep1008:56:002
2500299130cyclictest77632sleep1012:38:132
2500299130cyclictest263072sleep1007:50:412
2500299130cyclictest169312chrt09:52:182
2499599133cyclictest0-21swapper/911:07:0011
2499599133cyclictest0-21swapper/910:26:3811
2499599133cyclictest0-21swapper/910:06:3711
24995991312cyclictest0-21swapper/910:30:2411
24995991312cyclictest0-21swapper/908:40:2411
24995991312cyclictest0-21swapper/907:50:0211
24995991312cyclictest0-21swapper/907:40:0111
2499599131cyclictest0-21swapper/910:50:1911
2499399133cyclictest0-21swapper/810:39:5010
24993991311cyclictest2872-21df08:05:1510
2499199135cyclictest0-21swapper/608:50:348
2499199133cyclictest0-21swapper/611:41:548
2499199133cyclictest0-21swapper/607:43:338
24991991312cyclictest24714-21qemu-kvm07:47:198
24980991312cyclictest13844-21sensors12:00:246
24966991311cyclictest2406-21snmpd09:19:184
24966991311cyclictest2406-21snmpd08:06:184
14850130irq/18-uhci_hcd0-21swapper/011:15:270
13450130irq/18-ehci_hcd0-21swapper/007:20:230
2500299123cyclictest0-21swapper/1011:05:002
2500299123cyclictest0-21swapper/1008:55:012
2499599125cyclictest0-21swapper/908:35:0311
2499599122cyclictest0-21swapper/911:55:1711
2499599122cyclictest0-21swapper/909:58:0611
2499599122cyclictest0-21swapper/909:15:4411
2499399122cyclictest0-21swapper/809:19:1910
2499399122cyclictest0-21swapper/808:21:4910
24993991210cyclictest20424-21snmp_easybox.os08:30:1210
2499199126cyclictest12250irq/42-ahci07:30:018
2499199123cyclictest0-21swapper/609:00:198
2499199122cyclictest0-21swapper/612:15:048
2499199122cyclictest0-21swapper/611:10:108
2499199122cyclictest0-21swapper/609:46:048
2499199122cyclictest0-21swapper/609:46:048
2499199122cyclictest0-21swapper/608:45:348
2499199122cyclictest0-21swapper/608:13:488
24980991210cyclictest3883-21turbostat11:50:006
2496699124cyclictest0-21swapper/207:15:234
24966991210cyclictest2406-21snmpd08:29:194
2495499123cyclictest0-21swapper/010:30:010
24954991211cyclictest3066350irq/52-eth0-rx-08:28:450
24954991211cyclictest2600-21nmbd08:24:530
24954991210cyclictest0-21swapper/010:15:220
2495499120cyclictest0-21swapper/010:00:250
2495499120cyclictest0-21swapper/010:00:240
2495499120cyclictest0-21swapper/009:00:250
8999110migration/1111978-21munin-node09:45:223
8999110migration/1111978-21munin-node09:45:213
5499110migration/613856-21munin-node12:00:238
4799110migration/531293-21turbostat09:30:007
2500399110cyclictest0-21swapper/1111:20:233
2500299113cyclictest0-21swapper/1010:00:002
2500299113cyclictest0-21swapper/1007:16:082
2499599115cyclictest0-21swapper/911:36:4911
2499599113cyclictest0-21swapper/911:53:3511
2499599113cyclictest0-21swapper/910:55:3911
2499599113cyclictest0-21swapper/910:01:4411
2499599113cyclictest0-21swapper/910:01:4411
2499599113cyclictest0-21swapper/909:27:3311
2499599113cyclictest0-21swapper/907:28:5911
24995991110cyclictest2539-1kworker/9:2H11:20:1911
24995991110cyclictest0-21swapper/911:30:1811
24995991110cyclictest0-21swapper/908:15:2411
24995991110cyclictest0-21swapper/907:15:1911
2499599111cyclictest29072-21cpuspeed_turbos10:10:1511
2499599111cyclictest0-21swapper/911:02:0111
2499599111cyclictest0-21swapper/909:40:3711
2499599111cyclictest0-21swapper/909:35:0611
2499599111cyclictest0-21swapper/908:56:1911
2499599111cyclictest0-21swapper/908:47:3311
2499599111cyclictest0-21swapper/907:11:0011
2499399119cyclictest14238-21snmp_rack3slot909:50:1210
2499299110cyclictest6364-21cpuspeed08:55:109
2499299110cyclictest0-21swapper/708:35:009
2499199119cyclictest3886-21munin-run11:49:598
2499199113cyclictest0-21swapper/612:20:298
2499199113cyclictest0-21swapper/611:16:248
2499199113cyclictest0-21swapper/610:58:048
2499199113cyclictest0-21swapper/609:37:048
2499199112cyclictest0-21swapper/612:29:058
2499199112cyclictest0-21swapper/608:39:598
2498099119cyclictest21153-21sensors12:10:266
2498099113cyclictest0-21swapper/407:30:016
2497499113cyclictest0-21swapper/311:30:175
2497499110cyclictest0-21swapper/310:35:435
2497499110cyclictest0-21swapper/307:15:345
2496699119cyclictest2406-21snmpd10:35:054
2496699119cyclictest2406-21snmpd09:36:334
2496699119cyclictest2406-21snmpd08:31:494
2496699113cyclictest0-21swapper/209:25:204
2496699112cyclictest0-21swapper/208:35:174
2495499113cyclictest0-21swapper/007:50:170
2495499110cyclictest0-21swapper/009:50:280
1699110watchdog/00-21swapper/009:44:590
13450110irq/18-ehci_hcd0-21swapper/007:25:280
901100ksoftirqd/110-21swapper/1111:50:123
901100ksoftirqd/110-21swapper/1109:00:253
8999100migration/1114388-21taskset11:19:453
8999100migration/1110276-21timerwakeupswit11:10:313
8799100watchdog/110-21swapper/1110:52:583
8799100watchdog/110-21swapper/1107:56:553
831100ksoftirqd/100-21swapper/1012:10:022
761100ksoftirqd/90-21swapper/910:20:1511
7599100migration/92877-21taskset10:17:3611
621100ksoftirqd/70-21swapper/712:10:029
621100ksoftirqd/70-21swapper/710:40:159
621100ksoftirqd/70-21swapper/710:20:169
621100ksoftirqd/70-21swapper/708:00:139
6199100migration/730023-21munin-node10:10:189
6199100migration/729719-21cpuspeed_turbos11:40:139
6199100migration/727051-21munin-node07:10:219
6199100migration/726610-21sshd07:54:459
6199100migration/724871-21cpuspeed_turbos09:20:129
551100ksoftirqd/65139-21qemu-kvm08:20:088
5499100migration/69597-21sshd09:41:428
5499100migration/626760-21latency_hist07:55:018
5499100migration/616790-21munin-node11:20:218
481100ksoftirqd/524713-21qemu-kvm07:30:317
481100ksoftirqd/50-21swapper/512:10:027
481100ksoftirqd/50-21swapper/510:40:167
481100ksoftirqd/50-21swapper/510:27:167
4599100watchdog/50-21swapper/512:39:107
4599100watchdog/50-21swapper/512:02:477
4599100watchdog/50-21swapper/511:52:307
411100ksoftirqd/40-21swapper/411:50:126
411100ksoftirqd/40-21swapper/410:20:156
411100ksoftirqd/40-21swapper/409:00:246
411100ksoftirqd/40-21swapper/408:00:136
411100ksoftirqd/40-21swapper/407:30:316
3899100watchdog/40-21swapper/410:28:356
341100ksoftirqd/30-21swapper/311:50:115
341100ksoftirqd/30-21swapper/310:20:165
341100ksoftirqd/30-21swapper/307:35:145
3199100watchdog/30-21swapper/312:02:315
3199100watchdog/30-21swapper/308:00:595
3066350100irq/52-eth0-rx-0-21swapper/112:11:491
3066350100irq/52-eth0-rx-0-21swapper/012:15:170
2500399108cyclictest0-21swapper/1107:40:143
2500399100cyclictest0-21swapper/1112:25:133
2500399100cyclictest0-21swapper/1108:25:033
2500399100cyclictest0-21swapper/1108:08:343
2500399100cyclictest0-21swapper/1107:13:053
2500299103cyclictest0-21swapper/1011:22:242
2499599109cyclictest0-21swapper/912:30:1211
2499599103cyclictest0-21swapper/912:04:2011
2499599103cyclictest0-21swapper/910:40:0911
2499599103cyclictest0-21swapper/909:51:0911
2499599102cyclictest0-21swapper/911:48:1911
2499599102cyclictest0-21swapper/911:43:3011
2499599101cyclictest0-21swapper/909:05:1811
2499599100cyclictest0-21swapper/910:45:1111
2499599100cyclictest0-21swapper/909:30:2411
2499599100cyclictest0-21swapper/908:30:2111
2499399102cyclictest0-21swapper/808:46:3310
2499299100cyclictest0-21swapper/709:05:299
2499199108cyclictest29011-21snmp_rack3slot908:40:188
2499199104cyclictest0-21swapper/611:50:198
2499199103cyclictest0-21swapper/611:30:358
2499199103cyclictest0-21swapper/610:50:018
2499199103cyclictest0-21swapper/610:03:078
2499199103cyclictest0-21swapper/610:03:078
2499199103cyclictest0-21swapper/609:31:038
2499199103cyclictest0-21swapper/608:15:178
2499199102cyclictest0-21swapper/612:06:118
2499199102cyclictest0-21swapper/609:57:298
2499199102cyclictest0-21swapper/607:12:478
2499199101cyclictest0-21swapper/611:57:058
2499199100cyclictest16555-21qemu-kvm09:29:568
2498799108cyclictest24714-21qemu-kvm07:47:267
2498799108cyclictest10231-21python11:55:297
2498799108cyclictest0-21swapper/508:50:027
2498799100cyclictest0-21swapper/510:58:117
2498799100cyclictest0-21swapper/510:13:197
2498799100cyclictest0-21swapper/508:22:157
2498099107cyclictest0-21swapper/410:53:466
2498099103cyclictest0-21swapper/410:40:296
2498099102cyclictest0-21swapper/410:50:006
2498099102cyclictest0-21swapper/410:10:196
2497499108cyclictest0-21swapper/308:15:305
2497499107cyclictest2340-40kipmi009:05:215
2497499102cyclictest0-21swapper/311:20:185
2497499102cyclictest0-21swapper/310:50:175
2497499100cyclictest0-21swapper/308:05:185
2497499100cyclictest0-21swapper/307:10:155
2496699108cyclictest2406-21snmpd11:28:344
2496699108cyclictest2406-21snmpd11:02:354
2496699108cyclictest2406-21snmpd09:23:494
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional