You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 12:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Thu Nov 27, 2025 00:46:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16520210892sleep00-21swapper/019:05:510
16693210795sleep30-21swapper/319:08:065
16608210694sleep70-21swapper/719:06:589
16611210390sleep100-21swapper/1019:07:012
3066350980irq/52-eth0-rx-0-21swapper/519:05:237
3066450960irq/53-eth0-tx-0-21swapper/919:05:1311
3066350960irq/52-eth0-rx-0-21swapper/119:05:161
12250960irq/42-ahci0-21swapper/219:09:224
3066350870irq/52-eth0-rx-0-21swapper/419:05:306
1663727364sleep80-21swapper/819:07:2310
124050650irq/18-i801_smb0-21swapper/1119:05:263
1662125955sleep60-21swapper/619:07:098
1698999320cyclictest0-21swapper/021:14:450
17031993014cyclictest0-21swapper/1119:15:473
1698999290cyclictest0-21swapper/022:26:040
1698999290cyclictest0-21swapper/019:13:120
12250280irq/42-ahci0-21swapper/1022:26:022
17029992214cyclictest0-21swapper/923:40:0111
17029992214cyclictest0-21swapper/923:40:0011
17029992114cyclictest0-21swapper/922:43:4411
17029992114cyclictest0-21swapper/922:10:1811
17029992114cyclictest0-21swapper/922:07:5711
17029992114cyclictest0-21swapper/922:02:1311
17029992114cyclictest0-21swapper/921:59:4911
17029992114cyclictest0-21swapper/920:02:3411
17029992114cyclictest0-21swapper/900:38:1211
17029992114cyclictest0-21swapper/900:32:0211
17029992114cyclictest0-21swapper/900:23:3611
17029992113cyclictest0-21swapper/921:22:1011
17029992014cyclictest0-21swapper/922:30:5711
17029992014cyclictest0-21swapper/922:22:2711
17029992014cyclictest0-21swapper/921:34:1511
17029992014cyclictest0-21swapper/920:45:1011
17029992014cyclictest0-21swapper/920:23:2411
17029992014cyclictest0-21swapper/920:16:2111
17029992014cyclictest0-21swapper/919:51:4811
17029992013cyclictest0-21swapper/923:00:0611
17029992013cyclictest0-21swapper/920:43:5511
17029992013cyclictest0-21swapper/920:31:5211
17029992013cyclictest0-21swapper/920:27:2711
17029992013cyclictest0-21swapper/920:14:2711
17029992013cyclictest0-21swapper/919:19:2711
1703099192cyclictest87522chrt20:25:092
17029991913cyclictest0-21swapper/922:46:5411
17029991913cyclictest0-21swapper/922:38:1211
17029991913cyclictest0-21swapper/919:28:5611
1703099181cyclictest3066350irq/52-eth0-rx-20:40:142
17029991812cyclictest0-21swapper/923:32:4811
1702999181cyclictest0-21swapper/923:56:2911
1702999181cyclictest0-21swapper/923:56:2811
1702999181cyclictest0-21swapper/923:05:2311
17028991816cyclictest404-21snmp_easybox.os21:50:1110
17028991816cyclictest30605-21snmp_rack3slot922:30:1610
17028991816cyclictest29569-21df_inode21:00:1310
17028991816cyclictest26946-21snmp_rack3slot919:20:1710
17028991816cyclictest24102-21df_inode23:50:1610
17028991816cyclictest2259-21snmp_rack3slot920:15:1510
17028991816cyclictest2065-21snmp_rack3slot919:30:1410
17028991816cyclictest19958-21snmp_easybox.os21:30:2310
17028991816cyclictest18518-21snmp_rack3slot900:25:2510
17028991816cyclictest13023-21snmp_easybox.os21:20:2410
17028991816cyclictest11649-21snmp_easybox.os20:30:1910
17028991816cyclictest10279-21ntp_states19:40:2310
1702699183cyclictest0-21swapper/620:24:028
1703199170cyclictest0-21swapper/1120:30:083
17030991716cyclictest65352sleep1021:10:512
17030991716cyclictest242282sleep1020:50:372
17030991716cyclictest0-21swapper/1023:45:562
17030991716cyclictest0-21swapper/1021:35:172
17030991715cyclictest25456-21python23:50:292
1703099170cyclictest9672chrt00:01:462
1703099170cyclictest9672chrt00:01:452
1703099170cyclictest83062chrt00:11:222
1703099170cyclictest83062chrt00:11:222
1703099170cyclictest831ksoftirqd/1019:30:142
1703099170cyclictest80772chrt20:22:572
1703099170cyclictest79772sleep1023:27:192
1703099170cyclictest7822sleep1020:14:072
1703099170cyclictest69792sleep1021:56:192
1703099170cyclictest68522sleep1022:40:242
1703099170cyclictest4902chrt23:15:522
1703099170cyclictest45152chrt23:24:402
1703099170cyclictest42932sleep1020:15:552
1703099170cyclictest34252sleep1021:52:192
1703099170cyclictest322962chrt21:49:312
1703099170cyclictest3066350irq/52-eth0-rx-23:37:032
1703099170cyclictest3066350irq/52-eth0-rx-23:37:022
1703099170cyclictest3066350irq/52-eth0-rx-22:30:202
1703099170cyclictest3066350irq/52-eth0-rx-22:10:132
1703099170cyclictest3066350irq/52-eth0-rx-19:55:222
1703099170cyclictest3066350irq/52-eth0-rx-19:14:242
1703099170cyclictest297702chrt23:56:582
1703099170cyclictest297702chrt23:56:572
1703099170cyclictest294732chrt23:12:512
1703099170cyclictest289902chrt19:22:382
1703099170cyclictest286932sleep1020:05:312
1703099170cyclictest283872sleep1021:40:542
1703099170cyclictest282682sleep1021:00:012
1703099170cyclictest280362sleep1020:58:002
1703099170cyclictest27932sleep1021:05:372
1703099170cyclictest265392sleep1000:37:012
1703099170cyclictest256932sleep1023:05:522
1703099170cyclictest253392sleep1022:21:592
1703099170cyclictest252642chrt19:16:252
1703099170cyclictest250462sleep1020:00:302
1703099170cyclictest211202sleep1021:30:522
1703099170cyclictest206142sleep1020:45:592
1703099170cyclictest202602chrt23:00:152
1703099170cyclictest192972chrt00:27:242
1703099170cyclictest187532chrt23:40:312
1703099170cyclictest184982sleep1022:57:252
1703099170cyclictest184222sleep1019:51:422
1703099170cyclictest17472sleep1000:05:092
1703099170cyclictest160102chrt21:25:152
1703099170cyclictest156962sleep1000:23:002
1703099170cyclictest148982chrt19:48:282
1703099170cyclictest143202sleep1022:06:462
1703099170cyclictest138762sleep1021:21:392
1703099170cyclictest134922sleep1020:38:222
1703099170cyclictest12250irq/42-ahci22:15:362
1703099170cyclictest12250irq/42-ahci19:36:382
1703099170cyclictest12250irq/42-ahci19:25:312
1703099170cyclictest119102sleep1000:15:582
1703099170cyclictest116262chrt23:32:302
1703099170cyclictest113392sleep1022:49:132
1703099170cyclictest110612sleep1019:40:522
1703099170cyclictest107212sleep1022:02:082
1703099170cyclictest104892sleep1020:30:092
1703099170cyclictest101772sleep1021:15:512
1702999173cyclictest0-21swapper/922:50:0111
17029991713cyclictest0-21swapper/919:45:3911
1702999171cyclictest0-21swapper/923:20:1111
1702999170cyclictest0-21swapper/921:40:0111
17028991716cyclictest27660-21df23:55:1310
17028991716cyclictest27660-21df23:55:1310
17028991715cyclictest9494-21snmp_easybox.os22:00:2610
17028991715cyclictest8227-21diskstats21:15:1310
17028991715cyclictest7469-21snmp_rack3slot900:10:2410
17028991715cyclictest7469-21snmp_rack3slot900:10:2310
17028991715cyclictest6368-21perl20:20:1510
17028991715cyclictest5754-21perl19:35:1610
17028991715cyclictest5000-21diskstats21:55:1210
17028991715cyclictest4729-21snmp_rack3slot900:07:0010
17028991715cyclictest31297-21df00:00:1510
17028991715cyclictest31297-21df00:00:1410
17028991715cyclictest31010-21df_inode23:15:1210
17028991715cyclictest30742-21snmp_rack3slot919:25:1510
17028991715cyclictest30016-21snmp_rack3slot921:45:1310
17028991715cyclictest29205-21munin-run22:30:0010
17028991715cyclictest24478-21iostat_ios23:05:1710
17028991715cyclictest24445-21irqstats20:00:2310
17028991715cyclictest23309-21diskstats22:20:1410
17028991715cyclictest23117-21snmp_rack3slot921:35:1510
17028991715cyclictest2290-21snmp_rack3slot922:35:1710
17028991715cyclictest22036-21snmp_rack3slot919:56:2710
17028991715cyclictest21136-21snmp_easybox.os19:15:0210
17028991715cyclictest20386-21perl23:00:1610
17028991715cyclictest1825-21snmp_rack3slot923:20:1210
17028991715cyclictest17589-21ntp_states22:55:2310
17028991715cyclictest17579-21irqstats23:40:1810
17028991715cyclictest1735-21memory21:05:2410
17028991715cyclictest13808-21perl00:20:1610
17028991715cyclictest13186-21diskstats23:35:1310
17028991715cyclictest13186-21diskstats23:35:1310
17028991715cyclictest13093-21perl22:50:1510
17028991715cyclictest11516-21snmp_rack3slot922:05:1210
1702699172cyclictest0-21swapper/622:24:068
1702699172cyclictest0-21swapper/620:12:318
17026991715cyclictest29524-21qemu-kvm20:08:398
1701099172cyclictest2340-40kipmi019:24:445
17005991715cyclictest2406-21snmpd21:27:154
17031991615cyclictest881rcuc/1121:10:393
17031991615cyclictest4342-1kworker/11:2H20:26:583
17029991615cyclictest29032-21chrt19:23:1311
17029991615cyclictest22802-21diskstats21:35:1311
1702999160cyclictest0-21swapper/921:10:1811
17028991615cyclictest9872-21df00:15:1310
17028991614cyclictest4602-21diskstats21:10:1710
17028991614cyclictest29382-21snmp_rack3slot923:11:3710
17028991614cyclictest26768-21perl21:40:1510
17028991614cyclictest22863-21snmp_rack3slot900:31:2710
17028991614cyclictest21398-21memory23:45:2010
17028991614cyclictest18370-21snmp_rack3slot919:51:0110
17028991614cyclictest17432-21users21:25:3110
17028991614cyclictest17195-21ntp_states22:10:2610
17028991614cyclictest12741-21df_inode19:45:1210
1702799165cyclictest0-21swapper/720:15:009
1702699163cyclictest0-21swapper/623:46:408
1702699163cyclictest0-21swapper/623:16:098
1702699163cyclictest0-21swapper/622:57:098
1702699163cyclictest0-21swapper/622:27:378
1702699163cyclictest0-21swapper/622:07:068
1702699163cyclictest0-21swapper/621:47:308
1702699163cyclictest0-21swapper/621:29:048
1702699162cyclictest0-21swapper/619:18:518
17026991614cyclictest29524-21qemu-kvm20:28:168
17023991614cyclictest3194-21python19:30:287
17010991614cyclictest18532-21cpuspeed_turbos21:30:115
16989991615cyclictest0-21swapper/021:09:460
16989991614cyclictest29852-21munin-node19:25:150
1698999160cyclictest0-21swapper/021:24:050
3066350150irq/52-eth0-rx-1703199cyclictest22:26:113
1703099154cyclictest0-21swapper/1022:35:012
17029991514cyclictest0-21swapper/921:15:1211
17029991514cyclictest0-21swapper/920:55:1311
17028991514cyclictest18346-21snmp_rack3slot919:10:1410
17028991513cyclictest25235-21iostat_ios00:35:1910
17028991513cyclictest24204-21users20:50:3310
17028991513cyclictest19154-21snmp_rack3slot920:45:1710
1702699151cyclictest0-21swapper/623:20:018
1702699151cyclictest0-21swapper/621:43:058
17023991514cyclictest28194-21latency_hist21:00:027
17010991513cyclictest549-21timerwakeupswit00:00:305
17010991513cyclictest549-21timerwakeupswit00:00:295
17010991513cyclictest30156-21cstates19:25:125
1699799153cyclictest0-21swapper/120:00:001
3066450140irq/53-eth0-tx-0-21swapper/919:13:1111
3066450140irq/53-eth0-tx-0-21swapper/322:26:025
3066350140irq/52-eth0-rx-0-21swapper/522:26:007
3066350140irq/52-eth0-rx-0-21swapper/521:10:237
3066350140irq/52-eth0-rx-0-21swapper/321:24:525
3066350140irq/52-eth0-rx-0-21swapper/219:13:044
31140ksoftirqd/00-21swapper/021:35:270
31140ksoftirqd/00-21swapper/019:40:250
1703199145cyclictest0-21swapper/1122:06:273
17031991413cyclictest12250irq/42-ahci20:19:233
17029991413cyclictest29081-21latency_hist21:45:0211
17029991413cyclictest21662-1kworker/9:0H21:25:2611
17029991413cyclictest19365-21kworker/9:121:05:0111
17029991413cyclictest0-21swapper/923:25:1811
1702899144cyclictest0-21swapper/820:25:0110
17028991412cyclictest28090-21iostat_ios20:05:2210
1702799144cyclictest0-21swapper/721:50:019
1702799144cyclictest0-21swapper/721:45:019
1702799143cyclictest0-21swapper/721:40:029
1702699142cyclictest0-21swapper/623:43:418
1702699141cyclictest0-21swapper/620:15:018
1702699141cyclictest0-21swapper/620:00:318
17010991413cyclictest25073-21munin-node20:55:285
17010991412cyclictest2406-21snmpd21:56:305
17005991412cyclictest2406-21snmpd21:53:154
17005991412cyclictest2406-21snmpd21:18:154
17005991412cyclictest2406-21snmpd20:21:154
16989991413cyclictest4340-21taskset20:16:350
16989991412cyclictest80102sleep020:21:590
1698999141cyclictest0-21swapper/022:11:070
12250140irq/42-ahci0-21swapper/422:26:036
12250140irq/42-ahci0-21swapper/419:13:086
1703199133cyclictest0-21swapper/1119:45:003
17031991312cyclictest12250irq/42-ahci19:23:173
17031991311cyclictest20280-21taskset19:10:373
17029991312cyclictest0-21swapper/923:54:2611
1702899133cyclictest0-21swapper/822:15:0110
17028991311cyclictest7060-21snmp_easybox.os23:25:2610
17028991311cyclictest31054-21df_inode20:10:1610
1702799133cyclictest0-21swapper/722:25:019
1702799133cyclictest0-21swapper/719:50:019
1702699133cyclictest0-21swapper/623:25:408
1702699133cyclictest0-21swapper/622:50:388
1702699133cyclictest0-21swapper/621:16:348
1702699132cyclictest0-21swapper/621:12:358
1702699132cyclictest0-21swapper/619:52:028
1702699131cyclictest0-21swapper/622:36:088
1702699131cyclictest0-21swapper/620:46:038
1702699131cyclictest0-21swapper/619:57:018
17023991312cyclictest27352-21cstates23:55:137
17023991312cyclictest27352-21cstates23:55:127
17023991312cyclictest25451-21python23:50:287
17023991312cyclictest21556-21timerandwakeup19:55:337
1701099131cyclictest2340-40kipmi019:55:225
1700599133cyclictest0-21swapper/223:40:004
1700599133cyclictest0-21swapper/223:40:004
1698999133cyclictest0-21swapper/023:20:160
1698999130cyclictest0-21swapper/023:00:250
13450130irq/18-ehci_hcd0-21swapper/022:50:260
31120ksoftirqd/00-21swapper/023:55:240
31120ksoftirqd/00-21swapper/023:55:240
1703199129cyclictest0-21swapper/1121:08:393
1703199123cyclictest0-21swapper/1121:40:023
1703199123cyclictest0-21swapper/1120:58:003
1703199122cyclictest0-21swapper/1123:40:473
1703199122cyclictest0-21swapper/1119:55:223
1702999122cyclictest0-21swapper/920:05:0111
17029991211cyclictest22073-21chrt19:56:5311
17029991211cyclictest21662-1kworker/9:0H23:49:0711
1702899122cyclictest0-21swapper/822:41:0110
1702799124cyclictest0-21swapper/720:00:009
1702799123cyclictest0-21swapper/719:15:029
1702699121cyclictest0-21swapper/622:02:058
1702399123cyclictest0-21swapper/523:10:017
1702399123cyclictest0-21swapper/522:10:197
17023991211cyclictest6830-21python19:35:287
17023991211cyclictest15999-21cstates19:50:127
1702399120cyclictest0-21swapper/523:25:157
1701099128cyclictest0-21swapper/323:55:185
1701099128cyclictest0-21swapper/323:55:185
1701099128cyclictest0-21swapper/323:15:145
17005991210cyclictest2406-21snmpd23:12:164
16997991210cyclictest5950-21python21:10:331
1699799121cyclictest0-21swapper/123:46:231
1698999121cyclictest0-21swapper/021:30:210
1698999121cyclictest0-21swapper/020:08:280
1698999120cyclictest0-21swapper/000:35:260
14850120irq/18-uhci_hcd16744-21qemu-kvm19:30:268
14850120irq/18-uhci_hcd0-21swapper/023:10:290
14850120irq/18-uhci_hcd0-21swapper/022:20:280
14850120irq/18-uhci_hcd0-21swapper/019:50:260
13450120irq/18-ehci_hcd0-21swapper/000:20:290
341110ksoftirqd/30-21swapper/319:30:125
31110ksoftirqd/00-21swapper/019:35:270
271110ksoftirqd/20-21swapper/219:30:114
1703199119cyclictest0-21swapper/1120:23:183
1703199119cyclictest0-21swapper/1119:30:133
1703199115cyclictest0-21swapper/1121:32:313
1703199114cyclictest0-21swapper/1121:05:003
1703199113cyclictest0-21swapper/1123:55:013
1703199113cyclictest0-21swapper/1123:55:013
1703199113cyclictest0-21swapper/1123:53:023
1703199113cyclictest0-21swapper/1123:00:013
1703199113cyclictest0-21swapper/1122:55:003
1703199113cyclictest0-21swapper/1122:44:573
1703199113cyclictest0-21swapper/1122:35:463
1703199113cyclictest0-21swapper/1100:28:103
1703199113cyclictest0-21swapper/1100:19:113
1703199112cyclictest0-21swapper/1120:40:593
1703199112cyclictest0-21swapper/1119:36:293
1703199112cyclictest0-21swapper/1100:20:023
1703199112cyclictest0-21swapper/1100:13:463
1703199112cyclictest0-21swapper/1100:13:463
1703199111cyclictest0-21swapper/1100:32:173
1702999112cyclictest0-21swapper/920:55:0111
17029991110cyclictest0-21swapper/923:13:1511
17029991110cyclictest0-21swapper/922:55:1811
1702999111cyclictest0-21swapper/923:43:2211
1702899114cyclictest0-21swapper/820:40:0010
1702899113cyclictest0-21swapper/820:40:0010
17028991110cyclictest11529-21snmp_rack3slot923:31:1410
1702799119cyclictest0-21swapper/722:30:219
1702799114cyclictest0-21swapper/723:50:019
1702799113cyclictest0-21swapper/720:28:019
1702699114cyclictest0-21swapper/600:00:028
1702699114cyclictest0-21swapper/600:00:018
1702699113cyclictest3047-21qemu-kvm21:21:348
1702699113cyclictest0-21swapper/619:27:308
1702699110cyclictest0-21swapper/619:40:008
1702399119cyclictest0-21swapper/522:20:137
1702399113cyclictest0-21swapper/520:40:007
17023991110cyclictest27794-21python21:40:287
17023991110cyclictest23304-21cstates20:00:147
1702399111cyclictest0-21swapper/520:55:107
1702399110cyclictest0-21swapper/523:00:127
1701799113cyclictest0-21swapper/422:20:006
17017991110cyclictest14114-21sensors22:50:276
1701099117cyclictest0-21swapper/323:45:145
1701099117cyclictest0-21swapper/321:40:425
1701099117cyclictest0-21swapper/320:41:215
1701099117cyclictest0-21swapper/319:50:125
1701099117cyclictest0-21swapper/319:15:235
1701099113cyclictest0-21swapper/323:50:195
1701099113cyclictest0-21swapper/322:10:195
1701099113cyclictest0-21swapper/300:05:175
1701099111cyclictest2340-40kipmi022:20:005
1700599119cyclictest2406-21snmpd21:30:164
1700599119cyclictest2406-21snmpd20:06:594
1700599113cyclictest0-21swapper/221:40:174
1700599113cyclictest0-21swapper/220:10:204
1700599110cyclictest0-21swapper/219:25:144
1699799119cyclictest0-21swapper/123:35:151
1699799119cyclictest0-21swapper/123:35:141
1699799119cyclictest0-21swapper/122:50:001
1699799113cyclictest0-21swapper/119:15:311
1699799113cyclictest0-21swapper/100:15:171
16997991110cyclictest3066350irq/52-eth0-rx-22:00:301
16997991110cyclictest3066350irq/52-eth0-rx-21:03:121
16997991110cyclictest3066350irq/52-eth0-rx-00:33:051
1698999119cyclictest0-21swapper/021:25:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional