You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 09:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Fri Nov 21, 2025 00:46:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19515210894sleep20-21swapper/219:07:274
19505210893sleep80-21swapper/819:07:2210
19392210895sleep30-21swapper/319:05:535
16589210894sleep100-21swapper/1019:05:112
19539210390sleep110-21swapper/1119:07:483
1964729382sleep40-21swapper/419:08:306
12250920irq/42-ahci0-21swapper/119:08:161
3066350870irq/52-eth0-rx-0-21swapper/519:05:287
3066350850irq/52-eth0-rx-0-21swapper/719:08:419
1942628476sleep90-21swapper/919:06:2211
1955826556sleep00-21swapper/019:08:010
1946026454sleep60-21swapper/619:06:438
1990299340cyclictest0-21swapper/1120:09:163
1990299310cyclictest0-21swapper/1121:09:083
19855993129cyclictest30902sleep019:26:510
19902992423cyclictest0-21swapper/1120:15:153
1989299220cyclictest0-21swapper/623:00:018
1990299210cyclictest0-21swapper/1122:22:483
19896992013cyclictest31244-1kworker/10:2H21:44:532
19902991918cyclictest0-21swapper/1121:11:463
1989699192cyclictest3066350irq/52-eth0-rx-22:15:342
19896991913cyclictest19492-21cstates20:35:132
19896991912cyclictest0-21swapper/1022:57:262
1989699183cyclictest0-21swapper/1022:30:192
19896991812cyclictest25288-21smartctl21:25:302
19896991812cyclictest1042-21python23:05:302
1989699181cyclictest12250irq/42-ahci21:08:312
1989699181cyclictest12250irq/42-ahci20:10:342
1989599181cyclictest4412-1kworker/9:2H23:56:3511
1989599181cyclictest0-21swapper/920:17:3111
1989599181cyclictest0-21swapper/900:05:0111
1989399185cyclictest0-21swapper/720:00:019
19888991816cyclictest32386-21qemu-kvm20:08:237
19888991816cyclictest22893-21qemu-kvm22:46:047
19855991813cyclictest9722-21turbostat00:05:000
1989699173cyclictest0-21swapper/1019:45:312
1989699173cyclictest0-21swapper/1000:07:542
1989699173cyclictest0-21swapper/1000:07:542
1989699172cyclictest0-21swapper/1023:46:582
19896991716cyclictest9119-21cpuspeed_turbos21:50:142
19896991716cyclictest31244-1kworker/10:2H22:10:172
19896991716cyclictest31244-1kworker/10:2H22:10:172
1989699170cyclictest92172sleep1023:18:382
1989699170cyclictest77032chrt21:02:112
1989699170cyclictest74722sleep1021:45:282
1989699170cyclictest74722sleep1021:45:282
1989699170cyclictest72452sleep1020:17:082
1989699170cyclictest47762sleep1023:10:272
1989699170cyclictest39612sleep1020:56:232
1989699170cyclictest38332chrt19:30:012
1989699170cyclictest322742sleep1020:07:182
1989699170cyclictest316472sleep1023:50:062
1989699170cyclictest31582sleep1019:27:512
1989699170cyclictest312312sleep1000:31:002
1989699170cyclictest312312sleep1000:31:002
1989699170cyclictest312132sleep1021:35:152
1989699170cyclictest3066350irq/52-eth0-rx-23:55:302
1989699170cyclictest3066350irq/52-eth0-rx-23:35:232
1989699170cyclictest3066350irq/52-eth0-rx-23:21:032
1989699170cyclictest3066350irq/52-eth0-rx-22:25:142
1989699170cyclictest3066350irq/52-eth0-rx-20:47:092
1989699170cyclictest3066350irq/52-eth0-rx-20:00:202
1989699170cyclictest3066350irq/52-eth0-rx-00:20:222
1989699170cyclictest296922sleep1019:20:122
1989699170cyclictest296172sleep1021:32:142
1989699170cyclictest283982sleep1023:00:142
1989699170cyclictest281112chrt19:16:112
1989699170cyclictest258292chrt23:40:182
1989699170cyclictest255132sleep1020:42:572
1989699170cyclictest250902sleep1019:58:522
1989699170cyclictest24812chrt00:36:112
1989699170cyclictest233252sleep1022:53:112
1989699170cyclictest231602sleep1019:10:482
1989699170cyclictest213582sleep1019:52:422
1989699170cyclictest213582sleep1019:52:422
1989699170cyclictest199532chrt23:31:162
1989699170cyclictest190992sleep1000:15:212
1989699170cyclictest190992sleep1000:15:212
1989699170cyclictest182302sleep1020:32:562
1989699170cyclictest167792chrt00:12:092
1989699170cyclictest164262sleep1023:27:532
1989699170cyclictest159262chrt22:41:482
1989699170cyclictest157552sleep1021:15:012
1989699170cyclictest144512sleep1020:26:082
1989699170cyclictest141392sleep1021:10:252
1989699170cyclictest138922sleep1021:55:172
1989699170cyclictest125052sleep1019:40:152
1989699170cyclictest125052sleep1019:40:142
1989699170cyclictest12250irq/42-ahci22:45:482
1989699170cyclictest12250irq/42-ahci22:35:062
1989699170cyclictest12250irq/42-ahci22:08:572
1989699170cyclictest12250irq/42-ahci00:27:022
1989699170cyclictest12250irq/42-ahci00:27:012
1989699170cyclictest108102sleep1020:21:072
19895991716cyclictest0-21swapper/920:08:4111
19895991715cyclictest22521-21chrt21:24:4911
1989599170cyclictest31560-21fschecks_time21:35:1611
1989599170cyclictest0-21swapper/920:20:2111
1989399175cyclictest0-21swapper/723:00:029
19888991715cyclictest32387-21qemu-kvm20:47:237
19888991715cyclictest22893-21qemu-kvm22:38:587
19881991715cyclictest1027-21sensors23:05:306
1988199170cyclictest0-21swapper/420:30:436
1985599170cyclictest0-21swapper/020:26:140
1989699163cyclictest0-21swapper/1019:38:452
1989699163cyclictest0-21swapper/1019:38:452
1989699162cyclictest0-21swapper/1022:21:182
1989599162cyclictest0-21swapper/922:40:1911
19895991615cyclictest4412-1kworker/9:2H19:19:3111
19895991615cyclictest0-21swapper/922:28:0211
19895991615cyclictest0-21swapper/919:20:2311
19888991615cyclictest3404-21cstates23:55:137
19888991615cyclictest32387-21qemu-kvm20:22:517
19888991615cyclictest22893-21qemu-kvm22:21:357
19888991614cyclictest32386-21qemu-kvm20:10:187
19881991614cyclictest30201-21sensors23:45:246
19859991615cyclictest3066350irq/52-eth0-rx-22:05:321
1990299152cyclictest0-21swapper/1119:22:453
1990299150cyclictest0-21swapper/1121:40:463
1989599154cyclictest0-21swapper/920:35:0111
19895991514cyclictest4412-1kworker/9:2H20:43:0511
19895991514cyclictest4412-1kworker/9:2H20:10:5311
19895991514cyclictest20564-21taskset00:19:0711
19895991514cyclictest20564-21taskset00:19:0711
19895991514cyclictest19066-21taskset22:01:2111
19895991514cyclictest0-21swapper/923:25:3011
19895991514cyclictest0-21swapper/920:50:1711
1989399154cyclictest0-21swapper/723:25:009
19888991514cyclictest14201-21cstates_1123:25:137
19888991513cyclictest22893-21qemu-kvm22:14:447
19888991513cyclictest22893-21qemu-kvm22:14:447
19859991514cyclictest3066350irq/52-eth0-rx-23:51:021
19859991514cyclictest10640-21python21:05:311
19855991514cyclictest13022chrt22:22:110
1985599150cyclictest7405-21chrt20:19:220
1990299144cyclictest0-21swapper/1100:10:013
19896991413cyclictest31244-1kworker/10:2H20:51:572
1989599144cyclictest0-21swapper/900:35:0111
1989599144cyclictest0-21swapper/900:35:0111
19895991413cyclictest0-21swapper/923:40:1611
19894991412cyclictest9361-21snmp_rack3slot900:00:3710
19894991412cyclictest8853-21snmp_rack3slot920:20:1610
19894991412cyclictest8016-21snmp_rack3slot919:35:1510
19894991412cyclictest8016-21snmp_rack3slot919:35:1410
19894991412cyclictest7466-21sensors_fan21:45:2910
19894991412cyclictest7466-21sensors_fan21:45:2810
19894991412cyclictest6740-21snmp_rack3slot922:30:1410
19894991412cyclictest6387-21snmp_rack3slot923:15:1110
19894991412cyclictest5903-21perl21:00:1510
19894991412cyclictest5123-21snmp_rack3slot923:55:2710
19894991412cyclictest4511-21snmp_rack3slot920:15:1310
19894991412cyclictest443-21diskstats00:35:1510
19894991412cyclictest4027-21snmp_rack3slot919:30:1310
19894991412cyclictest319-21perl23:50:1610
19894991412cyclictest31651-21snmp_rack3slot923:05:1610
19894991412cyclictest30862-21snmp_easybox.os22:20:1410
19894991412cyclictest30858-21snmp_rack3slot923:46:2310
19894991412cyclictest30431-21perl20:05:1510
19894991412cyclictest30398-21snmp_easybox.os21:35:1310
19894991412cyclictest30019-21irqstats00:30:2210
19894991412cyclictest30019-21irqstats00:30:2110
19894991412cyclictest29897-21snmp_easybox.os20:50:1310
19894991412cyclictest29799-21snmp_rack3slot919:20:1410
19894991412cyclictest2917-21snmp_easybox.os20:55:3010
19894991412cyclictest28439-21perl22:15:1610
19894991412cyclictest28025-21snmp_rack3slot921:30:1810
19894991412cyclictest27686-21snmp_easybox.os23:00:1310
19894991412cyclictest27501-21munin-run23:45:0110
19894991412cyclictest26802-21perl20:00:1710
19894991412cyclictest2601-21snmp_easybox.os23:10:1110
19894991412cyclictest24787-21snmp_easybox.os00:25:1110
19894991412cyclictest24787-21snmp_easybox.os00:25:1110
19894991412cyclictest24233-21perl21:25:1510
19894991412cyclictest24048-21snmp_easybox.os22:55:1410
19894991412cyclictest23902-21snmp_easybox.os19:15:0010
19894991412cyclictest23573-21snmp_easybox.os22:10:1810
19894991412cyclictest23573-21snmp_easybox.os22:10:1710
19894991412cyclictest22761-21snmp_rack3slot922:07:0210
19894991412cyclictest22379-21snmp_easybox.os19:10:2410
19894991412cyclictest22340-21perl00:20:1710
19894991412cyclictest22339-21iostat_ios23:35:2210
19894991412cyclictest21573-21perl22:50:1510
19894991412cyclictest2117-21snmp_easybox.os22:25:1410
19894991412cyclictest20771-21snmp_easybox.os20:35:2410
19894991412cyclictest20290-21diskstats21:20:1710
19894991412cyclictest19871-21users23:30:3110
19894991412cyclictest18408-21snmp_easybox.os19:50:1210
19894991412cyclictest18408-21snmp_easybox.os19:50:1110
19894991412cyclictest17692-21snmp_rack3slot900:15:1510
19894991412cyclictest17692-21snmp_rack3slot900:15:1410
19894991412cyclictest17488-21sensors_volt20:30:3410
19894991412cyclictest17441-21perl22:00:1610
19894991412cyclictest17291-21snmp_rack3slot922:45:1310
19894991412cyclictest16725-21snmp_rack3slot900:11:2710
19894991412cyclictest16627-21snmp_rack3slot921:15:1410
19894991412cyclictest15524-21df19:45:1210
19894991412cyclictest1469-21diskstats20:10:1610
19894991412cyclictest14673-21perl23:25:1410
19894991412cyclictest13899-21df_inode22:40:1510
19894991412cyclictest12654-21snmp_rack3slot923:21:0410
19894991412cyclictest12457-21snmp_rack3slot920:25:1710
19894991412cyclictest1236-21perl19:25:1610
19894991412cyclictest12187-21snmp_easybox.os21:10:1010
19894991412cyclictest11840-21irqstats00:05:2010
19894991412cyclictest11840-21irqstats00:05:1910
19894991412cyclictest10772-21snmp_rack3slot922:35:1710
19894991412cyclictest10221-21memory21:05:2410
1989399144cyclictest0-21swapper/720:40:019
1989399144cyclictest0-21swapper/700:15:019
1989399144cyclictest0-21swapper/700:15:009
1989399143cyclictest0-21swapper/700:25:029
1989399143cyclictest0-21swapper/700:25:019
19888991413cyclictest9317-21latency_hist23:20:007
19888991413cyclictest6940-21cstates_1123:15:127
19888991413cyclictest31695-21missed_timers20:50:267
19888991413cyclictest22892-21qemu-kvm22:25:467
1985599140cyclictest0-21swapper/019:22:470
13450140irq/18-ehci_hcd0-21swapper/020:20:290
1990299135cyclictest0-21swapper/1123:41:433
1990299133cyclictest0-21swapper/1120:51:173
1990299133cyclictest0-21swapper/1119:58:023
1990299132cyclictest0-21swapper/1121:50:013
1990299132cyclictest0-21swapper/1121:50:003
1989599134cyclictest0-21swapper/922:20:0111
1989599133cyclictest0-21swapper/919:45:0011
19895991312cyclictest3066350irq/52-eth0-rx-22:35:3811
19895991312cyclictest0-21swapper/922:55:2411
19894991311cyclictest22821-21df19:55:1410
19894991311cyclictest10908-21snmp_rack3slot921:50:2610
1989399134cyclictest0-21swapper/723:50:019
1989399134cyclictest0-21swapper/719:15:009
1989299133cyclictest0-21swapper/623:59:248
1989299133cyclictest0-21swapper/621:43:328
1989299133cyclictest0-21swapper/619:40:478
1989299133cyclictest0-21swapper/619:40:468
19888991312cyclictest32386-21qemu-kvm20:36:577
19888991312cyclictest22893-21qemu-kvm22:44:327
19888991312cyclictest22893-21qemu-kvm22:32:557
19888991312cyclictest20632-21cpuspeed_turbos19:10:147
1988199133cyclictest0-21swapper/421:35:196
1987599134cyclictest0-21swapper/319:20:005
19859991312cyclictest22569-21python19:10:271
19855991311cyclictest23585-21munin-node20:30:170
1990299124cyclictest0-21swapper/1123:16:593
1990299124cyclictest0-21swapper/1121:31:583
1990299124cyclictest0-21swapper/1119:50:023
1990299124cyclictest0-21swapper/1119:50:013
1990299122cyclictest0-21swapper/1123:58:333
1990299122cyclictest0-21swapper/1123:06:243
1989599124cyclictest0-21swapper/921:45:0111
1989599124cyclictest0-21swapper/900:38:1311
1989599123cyclictest0-21swapper/920:59:3311
1989599123cyclictest0-21swapper/900:05:0011
1989599123cyclictest0-21swapper/900:05:0011
19895991211cyclictest741rcuc/922:10:1811
19895991211cyclictest741rcuc/922:10:1811
19895991211cyclictest18866-21taskset21:19:4511
1989399123cyclictest0-21swapper/719:25:029
1989399123cyclictest0-21swapper/719:25:019
1989399122cyclictest0-21swapper/721:12:189
1989299122cyclictest0-21swapper/622:17:188
1989299121cyclictest23067-21snmp_rack3slot920:40:168
19888991211cyclictest17288-21cpuspeed_turbos23:30:137
1988199124cyclictest0-21swapper/422:05:196
1988199121cyclictest0-21swapper/420:40:276
1987599122cyclictest0-21swapper/321:30:005
1987599122cyclictest0-21swapper/300:25:185
1987599122cyclictest0-21swapper/300:25:175
19859991211cyclictest18398-21python22:00:271
19855991211cyclictest2443-21lldpd20:13:460
19855991210cyclictest0-21swapper/023:10:250
13450120irq/18-ehci_hcd0-21swapper/000:10:260
31110ksoftirqd/00-21swapper/022:30:260
2499110watchdog/20-21swapper/222:42:304
1990299115cyclictest0-21swapper/1123:35:083
1990299115cyclictest0-21swapper/1123:13:243
1990299115cyclictest0-21swapper/1100:05:193
1990299115cyclictest0-21swapper/1100:05:183
1990299113cyclictest0-21swapper/1123:28:513
1990299113cyclictest0-21swapper/1123:24:183
1990299113cyclictest0-21swapper/1122:42:483
1990299113cyclictest0-21swapper/1122:02:483
1990299113cyclictest0-21swapper/1121:28:323
1990299113cyclictest0-21swapper/1121:00:523
1990299113cyclictest0-21swapper/1120:56:233
1990299113cyclictest0-21swapper/1119:48:013
1990299113cyclictest0-21swapper/1119:19:173
1990299113cyclictest0-21swapper/1100:17:543
1990299113cyclictest0-21swapper/1100:17:533
1990299113cyclictest0-21swapper/1100:01:543
1990299112cyclictest0-21swapper/1120:30:183
19902991110cyclictest2691-21runrttasks20:12:343
1990299111cyclictest0-21swapper/1122:58:183
1990299111cyclictest0-21swapper/1121:54:523
1990299110cyclictest0-21swapper/1123:52:013
1989599114cyclictest0-21swapper/923:10:0111
1989599113cyclictest0-21swapper/923:51:3211
1989599113cyclictest0-21swapper/923:22:3211
1989599113cyclictest0-21swapper/921:30:0111
1989599113cyclictest0-21swapper/900:20:0211
1989599112cyclictest0-21swapper/922:30:4811
1989599112cyclictest0-21swapper/921:51:1811
19895991110cyclictest4412-1kworker/9:2H22:08:3311
19895991110cyclictest0-21swapper/921:46:1511
19895991110cyclictest0-21swapper/921:46:1511
19895991110cyclictest0-21swapper/919:40:2811
19895991110cyclictest0-21swapper/919:40:2711
1989599111cyclictest0-21swapper/920:30:2811
1989499113cyclictest0-21swapper/821:56:4710
1989399114cyclictest0-21swapper/723:35:019
1989399113cyclictest0-21swapper/722:35:209
1989399113cyclictest0-21swapper/722:15:189
1989399113cyclictest0-21swapper/721:30:429
1989399113cyclictest0-21swapper/720:26:179
1989399113cyclictest0-21swapper/720:14:019
1989399110cyclictest0-21swapper/722:00:009
1989399110cyclictest0-21swapper/719:39:569
1989399110cyclictest0-21swapper/719:39:569
1989299119cyclictest32387-21qemu-kvm20:15:158
1989299113cyclictest0-21swapper/621:32:338
1989299113cyclictest0-21swapper/620:46:178
1989299113cyclictest0-21swapper/620:39:178
1989299113cyclictest0-21swapper/620:01:168
1989299113cyclictest0-21swapper/619:39:478
1989299113cyclictest0-21swapper/619:39:468
1989299113cyclictest0-21swapper/600:06:558
1989299113cyclictest0-21swapper/600:06:548
1989299112cyclictest0-21swapper/623:20:178
1989299111cyclictest0-21swapper/620:32:468
1989299110cyclictest0-21swapper/623:10:188
1988899113cyclictest0-21swapper/523:00:187
19888991110cyclictest22893-21qemu-kvm22:15:507
19888991110cyclictest12980-21cpuspeed_turbos21:55:147
1988199113cyclictest0-21swapper/423:25:186
1988199113cyclictest0-21swapper/423:10:186
1988199113cyclictest0-21swapper/419:30:016
1987599113cyclictest0-21swapper/323:40:185
1987599113cyclictest0-21swapper/321:20:215
1987599113cyclictest0-21swapper/321:00:005
19875991110cyclictest341ksoftirqd/322:50:165
1985999113cyclictest0-21swapper/121:00:181
1985999113cyclictest0-21swapper/120:00:011
19859991110cyclictest3066350irq/52-eth0-rx-22:34:151
1985599119cyclictest0-21swapper/022:40:270
1985599119cyclictest0-21swapper/020:35:280
1985599113cyclictest0-21swapper/022:00:180
19855991110cyclictest11400-21nscd21:30:160
19855991110cyclictest0-21swapper/022:05:260
14850110irq/18-uhci_hcd0-21swapper/020:50:290
12250110irq/42-ahci0-21swapper/700:35:019
12250110irq/42-ahci0-21swapper/700:35:009
901100ksoftirqd/110-21swapper/1120:40:403
8999100migration/114188-21munin-node22:25:253
8999100migration/1130723-21switchtime00:30:303
8999100migration/1130723-21switchtime00:30:293
761100ksoftirqd/90-21swapper/922:50:1611
6199100migration/75462-21ipmi_temp19:30:219
5999100watchdog/70-21swapper/721:01:279
5499100migration/67756-21runrttasks21:02:558
5499100migration/6415-21munin-node23:05:228
5499100migration/627602-21latency_hist23:45:008
5499100migration/61847-21unixbench-2d23:50:308
531100rcuc/60-21swapper/600:31:258
531100rcuc/60-21swapper/600:31:258
481100ksoftirqd/50-21swapper/519:35:167
481100ksoftirqd/50-21swapper/519:35:167
4599100watchdog/50-21swapper/523:13:037
411100ksoftirqd/40-21swapper/422:50:156
411100ksoftirqd/40-21swapper/419:35:176
411100ksoftirqd/40-21swapper/419:35:166
3899100watchdog/40-21swapper/421:55:466
3899100watchdog/40-21swapper/421:45:226
3899100watchdog/40-21swapper/421:45:226
3199100watchdog/30-21swapper/320:41:145
31100ksoftirqd/00-21swapper/020:05:120
31100ksoftirqd/00-21swapper/019:30:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional