You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 06:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sun Nov 23, 2025 00:46:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501130irq/42-ahci0-21swapper/119:09:431
122501130irq/42-ahci0-21swapper/119:09:431
20511210892sleep70-21swapper/719:07:429
20511210892sleep70-21swapper/719:07:429
20508210895sleep40-21swapper/419:07:396
20508210895sleep40-21swapper/419:07:396
20402210290sleep50-21swapper/519:06:167
20402210290sleep50-21swapper/519:06:167
3066350960irq/52-eth0-rx-0-21swapper/219:09:274
3066350960irq/52-eth0-rx-0-21swapper/219:09:274
12250920irq/42-ahci0-21swapper/319:08:265
12250920irq/42-ahci0-21swapper/319:08:265
2071829186sleep100-21swapper/1019:09:452
2071829186sleep100-21swapper/1019:09:452
1977227263sleep90-21swapper/919:05:3211
1977227263sleep90-21swapper/919:05:3211
2051726763sleep00-21swapper/019:07:470
2051726763sleep00-21swapper/019:07:460
3066350630irq/52-eth0-rx-0-21swapper/1119:06:123
3066350630irq/52-eth0-rx-0-21swapper/1119:06:123
2049625955sleep80-21swapper/819:07:3110
2049625955sleep80-21swapper/819:07:3110
2043625955sleep60-21swapper/619:06:418
2043625955sleep60-21swapper/619:06:418
2084199470cyclictest0-21swapper/022:24:410
20883994229cyclictest901ksoftirqd/1122:24:043
2088399360cyclictest0-21swapper/1122:26:383
2088399350cyclictest0-21swapper/1122:09:473
2088399350cyclictest0-21swapper/1122:09:473
2088399320cyclictest0-21swapper/1122:10:293
2088399320cyclictest0-21swapper/1122:10:293
2084199310cyclictest0-21swapper/022:25:590
2084199310cyclictest0-21swapper/022:10:400
2084199310cyclictest0-21swapper/022:10:390
20883992813cyclictest901ksoftirqd/1121:10:003
20883992726cyclictest0-21swapper/1122:16:053
20883992726cyclictest0-21swapper/1122:16:053
2084199234cyclictest0-21swapper/020:05:010
20883992019cyclictest901ksoftirqd/1119:20:463
2084199206cyclictest0-21swapper/021:22:510
20883991817cyclictest13573-21kworker/11:119:17:163
2088299181cyclictest831ksoftirqd/1019:30:172
2088299181cyclictest12250irq/42-ahci22:47:412
2088199181cyclictest0-21swapper/919:15:1911
20880991816cyclictest9040-21yum21:45:3810
20880991816cyclictest7720-21snmp_rack3slot922:30:1410
20880991816cyclictest7720-21snmp_rack3slot922:30:1410
20880991816cyclictest628-21snmp_rack3slot922:20:1710
20880991816cyclictest4953-21snmp_rack3slot920:56:2710
20880991816cyclictest3202-21snmp_easybox.os19:25:2610
20880991816cyclictest23757-21ntp_states23:35:2410
20880991816cyclictest18534-21diskstats22:45:1410
20880991816cyclictest17512-21snmp_rack3slot920:30:1610
20880991816cyclictest17244-21snmp_easybox.os22:00:1210
20880991816cyclictest14088-21snmp_rack3slot900:06:3710
20869991816cyclictest25301-21sensors19:15:146
2084199180cyclictest0-21swapper/019:16:280
2088399170cyclictest0-21swapper/1121:20:323
2088399170cyclictest0-21swapper/1119:25:563
20882991716cyclictest10544-1kworker/10:2H22:30:012
20882991716cyclictest10544-1kworker/10:2H22:30:012
2088299170cyclictest92792sleep1021:48:512
2088299170cyclictest87982sleep1021:03:452
2088299170cyclictest831ksoftirqd/1022:04:162
2088299170cyclictest67472sleep1023:56:062
2088299170cyclictest61072sleep1020:15:162
2088299170cyclictest59762sleep1022:26:302
2088299170cyclictest55772sleep1021:43:032
2088299170cyclictest49282sleep1020:56:082
2088299170cyclictest44852sleep1020:10:392
2088299170cyclictest40512sleep1019:26:342
2088299170cyclictest36192sleep1000:37:332
2088299170cyclictest35662sleep1023:10:132
2088299170cyclictest323442sleep1023:05:112
2088299170cyclictest318562sleep1023:46:272
2088299170cyclictest315222sleep1019:20:212
2088299170cyclictest314942sleep1023:01:232
2088299170cyclictest31062sleep1023:51:042
2088299170cyclictest310152chrt00:30:202
2088299170cyclictest3066450irq/53-eth0-tx-23:40:322
2088299170cyclictest3066350irq/52-eth0-rx-22:23:182
2088299170cyclictest3066350irq/52-eth0-rx-21:20:272
2088299170cyclictest3066350irq/52-eth0-rx-21:15:292
2088299170cyclictest3066350irq/52-eth0-rx-20:08:522
2088299170cyclictest304132sleep1022:15:292
2088299170cyclictest304132sleep1022:15:292
2088299170cyclictest302442sleep1020:49:202
2088299170cyclictest296962sleep1021:30:242
2088299170cyclictest294812sleep1020:00:502
2088299170cyclictest291202sleep1019:16:332
2088299170cyclictest277822sleep1022:55:572
2088299170cyclictest274392sleep1022:11:532
2088299170cyclictest274392sleep1022:11:532
2088299170cyclictest273712chrt00:25:202
2088299170cyclictest259912sleep1019:57:502
2088299170cyclictest250312sleep1020:40:182
2088299170cyclictest250002sleep1000:21:322
2088299170cyclictest246562sleep1023:37:152
2088299170cyclictest236952sleep1022:06:172
2088299170cyclictest236952sleep1022:06:172
2088299170cyclictest229312chrt22:50:212
2088299170cyclictest228922sleep1019:10:202
2088299170cyclictest227772chrt20:36:422
2088299170cyclictest222352sleep1019:51:142
2088299170cyclictest222352sleep1019:51:142
2088299170cyclictest213842sleep1000:16:422
2088299170cyclictest202322sleep1023:30:262
2088299170cyclictest191262sleep1020:31:422
2088299170cyclictest175242sleep1000:10:302
2088299170cyclictest16992sleep1021:35:482
2088299170cyclictest164652sleep1023:25:252
2088299170cyclictest161802sleep1019:45:122
2088299170cyclictest159002sleep1021:10:362
2088299170cyclictest154932sleep1020:26:532
2088299170cyclictest149912sleep1019:42:002
2088299170cyclictest149912sleep1019:42:002
2088299170cyclictest143972chrt21:55:152
2088299170cyclictest140312sleep1000:05:542
2088299170cyclictest128092sleep1021:52:152
2088299170cyclictest12250irq/42-ahci21:27:092
2088299170cyclictest11942chrt20:51:082
2088299170cyclictest118382sleep1020:21:412
2088299170cyclictest115972sleep1023:20:122
2088299170cyclictest113222sleep1019:36:352
2088299170cyclictest110112sleep1021:05:212
2088299170cyclictest103882chrt22:35:082
2088299170cyclictest103662sleep1000:00:542
2088299170cyclictest101412sleep1023:17:482
20881991715cyclictest8052-21diskstats23:15:1511
20881991715cyclictest3601-21sshd00:37:1611
20881991715cyclictest11256-21diskstats22:35:1611
2088199170cyclictest0-21swapper/922:50:1711
20880991715cyclictest953-21df23:50:1210
20880991715cyclictest8627-21snmp_rack3slot921:01:2610
20880991715cyclictest7344-21snmp_easybox.os20:15:2610
20880991715cyclictest677-21df_inode23:05:1310
20880991715cyclictest4644-21ntp_states21:40:2610
20880991715cyclictest32410-21iostat20:50:2010
20880991715cyclictest31886-21snmp_rack3slot923:46:5010
20880991715cyclictest31466-21snmp_easybox.os00:30:2410
20880991715cyclictest3125-21snmp_easybox.os22:25:1210
20880991715cyclictest30988-21perl19:20:1510
20880991715cyclictest30200-21iostat23:00:1910
20880991715cyclictest28620-21snmp_rack3slot900:26:2710
20880991715cyclictest25348-21memory20:40:2310
20880991715cyclictest25030-21snmp_rack3slot900:21:5410
20880991715cyclictest25009-21snmp_rack3slot922:10:1310
20880991715cyclictest25009-21snmp_rack3slot922:10:1310
20880991715cyclictest23025-21munin-run20:40:0010
20880991715cyclictest20887-21yum23:30:3410
20880991715cyclictest11990-21iostat22:35:2110
20879991715cyclictest18812-21perl22:45:159
2087899175cyclictest0-21swapper/623:05:018
20841991715cyclictest3066350irq/52-eth0-rx-20:10:280
2088399164cyclictest0-21swapper/1120:50:013
2088199165cyclictest0-21swapper/923:55:0111
20881991615cyclictest0-21swapper/922:40:2611
20881991615cyclictest0-21swapper/920:25:1611
20881991615cyclictest0-21swapper/900:20:3411
20881991614cyclictest13765-21taskset23:22:3811
20880991615cyclictest27488-21perl19:15:2010
20880991614cyclictest9565-21perl19:35:1610
20880991614cyclictest8637-21perl00:00:1810
20880991614cyclictest5723-21memory23:55:2510
20880991614cyclictest5687-21snmp_rack3slot919:30:1610
20880991614cyclictest31125-21df20:05:1410
20880991614cyclictest28555-21snmp_rack3slot921:30:1310
20880991614cyclictest2843-21snmp_rack3slot920:10:1610
20880991614cyclictest25869-21snmp_easybox.os21:25:2610
20880991614cyclictest25378-21snmp_easybox.os23:40:1110
20880991614cyclictest24752-21snmp_easybox.os19:15:0110
20880991614cyclictest23726-21snmp_rack3slot922:06:3910
20880991614cyclictest23726-21snmp_rack3slot922:06:3810
20880991614cyclictest21392-21snmp_rack3slot900:16:4910
20880991614cyclictest14329-21perl21:10:1410
20878991615cyclictest915-21qemu-kvm20:19:168
20878991614cyclictest915-21qemu-kvm20:08:378
20877991615cyclictest30823-21python23:00:277
20877991615cyclictest21830-21timerandwakeup19:50:327
20877991615cyclictest21830-21timerandwakeup19:50:327
20877991615cyclictest20289-21missed_timers00:15:217
20877991615cyclictest12440-21qemu-kvm21:19:567
20865991614cyclictest0-21swapper/322:25:015
20857991614cyclictest2406-21snmpd19:34:594
20849991615cyclictest11619-21python21:05:301
20841991614cyclictest23915-21smbd22:30:420
20841991614cyclictest23915-21smbd22:30:410
20841991614cyclictest20623-21smbd19:30:400
20881991514cyclictest0-21swapper/923:35:1211
20881991514cyclictest0-21swapper/921:50:3211
20881991514cyclictest0-21swapper/921:20:2511
20881991513cyclictest7600-21snmp_rack3slot921:45:1811
20880991513cyclictest28193-21snmp_easybox.os22:15:1210
20880991513cyclictest28193-21snmp_easybox.os22:15:1210
20880991513cyclictest27459-21df20:00:1510
20880991513cyclictest25902-21snmp_rack3slot919:56:4010
20880991513cyclictest17775-21snmp_rack3slot900:12:1410
20880991513cyclictest14414-21memory20:25:2710
20880991513cyclictest13205-21perl19:40:1710
20880991513cyclictest13205-21perl19:40:1710
2087999154cyclictest0-21swapper/720:35:019
20878991513cyclictest915-21qemu-kvm20:45:368
20877991514cyclictest5769-21python23:10:317
20877991514cyclictest12440-21qemu-kvm21:20:077
2085799150cyclictest0-21swapper/219:40:244
2085799150cyclictest0-21swapper/219:40:244
20849991514cyclictest29916-21python21:30:301
3899140watchdog/40-21swapper/419:35:436
3066350140irq/52-eth0-rx-0-21swapper/922:10:3311
3066350140irq/52-eth0-rx-0-21swapper/922:10:3311
2088199144cyclictest0-21swapper/921:15:0111
2088199144cyclictest0-21swapper/919:40:0111
2088199143cyclictest0-21swapper/920:15:0111
20881991413cyclictest0-21swapper/923:00:1811
20881991413cyclictest0-21swapper/922:26:2211
20881991413cyclictest0-21swapper/900:30:2011
20880991413cyclictest6362-21snmp_rack3slot923:10:4910
2087999144cyclictest0-21swapper/720:45:019
20877991413cyclictest6133-21python23:55:297
20877991413cyclictest23947-21python23:35:277
20857991412cyclictest2406-21snmpd00:09:304
20849991413cyclictest3066350irq/52-eth0-rx-19:25:211
20849991413cyclictest30406-21python22:15:271
20849991413cyclictest30406-21python22:15:271
12250140irq/42-ahci0-21swapper/722:24:419
12250140irq/42-ahci0-21swapper/122:24:441
2088399133cyclictest0-21swapper/1122:47:453
2088399133cyclictest0-21swapper/1121:32:203
2088399133cyclictest0-21swapper/1119:56:393
20883991312cyclictest901ksoftirqd/1121:18:423
20883991311cyclictest12012-21chrt20:24:063
2088399131cyclictest0-21swapper/1121:35:143
2088199135cyclictest0-21swapper/921:40:2511
2088199134cyclictest0-21swapper/922:00:0111
2088199134cyclictest0-21swapper/921:30:0111
2088199134cyclictest0-21swapper/920:00:0111
2088199133cyclictest0-21swapper/923:45:0011
2088199133cyclictest0-21swapper/922:45:0111
2088199133cyclictest0-21swapper/922:08:5211
2088199133cyclictest0-21swapper/922:08:5211
2088199133cyclictest0-21swapper/921:10:1911
2088199133cyclictest0-21swapper/920:20:0111
20880991311cyclictest29969-21yum20:45:3710
20880991311cyclictest25752-21df22:55:1310
20880991311cyclictest21799-21snmp_rack3slot922:50:1410
20880991311cyclictest16235-21sensors_temp22:40:2710
20880991311cyclictest10164-21perl20:20:1810
20880991310cyclictest0-21swapper/819:45:2010
2087999135cyclictest0-21swapper/722:32:559
2087999135cyclictest0-21swapper/722:32:559
2087999135cyclictest0-21swapper/720:56:449
2087999135cyclictest0-21swapper/720:18:199
2087999134cyclictest0-21swapper/721:15:019
2087999133cyclictest0-21swapper/722:27:599
2087999133cyclictest0-21swapper/721:41:459
2087999133cyclictest0-21swapper/721:21:299
2087999133cyclictest0-21swapper/721:10:019
2087999133cyclictest0-21swapper/720:27:499
2087999133cyclictest0-21swapper/720:10:019
2087999133cyclictest0-21swapper/719:57:349
2087999133cyclictest0-21swapper/719:30:019
2087999133cyclictest0-21swapper/719:18:439
2087999133cyclictest0-21swapper/700:26:259
2087899134cyclictest0-21swapper/622:40:018
2087899133cyclictest0-21swapper/622:15:018
2087899133cyclictest0-21swapper/622:15:018
2087899133cyclictest0-21swapper/621:42:008
2087899133cyclictest0-21swapper/619:30:018
20877991312cyclictest8464-21python21:45:337
20877991312cyclictest7747-21cpuspeed_turbos00:00:157
20877991312cyclictest4829-21python21:40:297
20877991312cyclictest4312-21python20:55:317
20877991312cyclictest28913-21cstates_1122:15:137
20877991312cyclictest28913-21cstates_1122:15:137
20877991312cyclictest12440-21qemu-kvm21:08:447
2086599135cyclictest0-21swapper/323:20:265
2086599135cyclictest0-21swapper/322:46:085
2086599135cyclictest0-21swapper/300:16:595
2086599132cyclictest0-21swapper/319:15:155
20865991311cyclictest0-21swapper/322:01:255
2086599130cyclictest0-21swapper/320:35:145
2085799133cyclictest0-21swapper/220:05:184
20857991311cyclictest2406-21snmpd22:36:304
2084999133cyclictest0-21swapper/122:10:011
2084999133cyclictest0-21swapper/122:10:011
2084999133cyclictest0-21swapper/119:55:201
20849991312cyclictest3066350irq/52-eth0-rx-20:17:191
2084999131cyclictest0-21swapper/120:05:201
2084199134cyclictest0-21swapper/020:30:010
2084199133cyclictest0-21swapper/023:55:210
20841991312cyclictest12250irq/42-ahci19:10:010
1699130watchdog/00-21swapper/023:08:030
1699130watchdog/00-21swapper/000:06:550
5299120watchdog/60-21swapper/623:54:358
411120ksoftirqd/40-21swapper/422:25:016
2088399123cyclictest0-21swapper/1123:45:003
2088399122cyclictest0-21swapper/1119:31:173
2088199124cyclictest0-21swapper/923:15:0111
2088199124cyclictest0-21swapper/919:25:0111
2088199124cyclictest0-21swapper/900:06:2511
2088199123cyclictest0-21swapper/922:19:4411
2088199123cyclictest0-21swapper/922:19:4411
2088199123cyclictest0-21swapper/900:10:2011
2088199122cyclictest0-21swapper/923:41:5311
2088199122cyclictest0-21swapper/923:05:2211
2088199122cyclictest0-21swapper/922:23:2211
2088199122cyclictest0-21swapper/921:25:5011
2088199122cyclictest0-21swapper/920:45:2111
2088199122cyclictest0-21swapper/920:08:2811
2088199122cyclictest0-21swapper/919:40:4711
2088199122cyclictest0-21swapper/919:40:4611
2088199122cyclictest0-21swapper/919:29:1711
2088199122cyclictest0-21swapper/900:03:5211
20881991211cyclictest0-21swapper/920:55:2211
2088199121cyclictest0-21swapper/921:35:0011
2087999124cyclictest0-21swapper/719:25:549
2087999123cyclictest0-21swapper/723:20:029
2087999123cyclictest0-21swapper/723:00:029
2087999123cyclictest0-21swapper/722:20:019
2087999123cyclictest0-21swapper/722:20:019
2087999123cyclictest0-21swapper/700:00:019
2087999122cyclictest0-21swapper/722:14:469
2087999122cyclictest0-21swapper/722:14:459
2087999122cyclictest0-21swapper/720:43:549
2087999122cyclictest0-21swapper/719:39:139
2087899124cyclictest0-21swapper/623:26:528
2087899124cyclictest0-21swapper/600:29:558
2087899123cyclictest0-21swapper/621:30:008
20878991211cyclictest915-21qemu-kvm20:43:218
2087899121cyclictest0-21swapper/622:35:018
2087899121cyclictest0-21swapper/622:35:018
2086999122cyclictest0-21swapper/400:00:006
2086599124cyclictest0-21swapper/323:44:485
2086599124cyclictest0-21swapper/321:41:515
2086599124cyclictest0-21swapper/300:30:465
20865991211cyclictest0-21swapper/323:36:525
20865991211cyclictest0-21swapper/323:10:545
20865991211cyclictest0-21swapper/300:20:355
20865991210cyclictest0-21swapper/323:51:125
20865991210cyclictest0-21swapper/323:00:245
20865991210cyclictest0-21swapper/322:35:295
20865991210cyclictest0-21swapper/322:09:475
20865991210cyclictest0-21swapper/322:09:475
20865991210cyclictest0-21swapper/321:49:525
20865991210cyclictest0-21swapper/321:28:255
20865991210cyclictest0-21swapper/320:00:325
20865991210cyclictest0-21swapper/319:33:385
20865991210cyclictest0-21swapper/300:29:105
2086599120cyclictest0-21swapper/320:51:025
2085799122cyclictest0-21swapper/222:55:184
20857991210cyclictest2406-21snmpd20:18:294
20857991210cyclictest0-21swapper/221:20:134
20849991211cyclictest16664-21python23:25:291
2084199124cyclictest0-21swapper/021:35:210
2084199124cyclictest0-21swapper/019:50:200
2084199124cyclictest0-21swapper/019:50:200
551110ksoftirqd/60-21swapper/622:25:018
2088399119cyclictest0-21swapper/1120:08:223
2088399115cyclictest0-21swapper/1120:42:083
2088399114cyclictest0-21swapper/1121:45:013
2088399113cyclictest0-21swapper/1123:10:013
2088399113cyclictest0-21swapper/1122:57:243
2088399113cyclictest0-21swapper/1122:50:403
2088399113cyclictest0-21swapper/1121:27:193
2088399113cyclictest0-21swapper/1120:28:403
2088399113cyclictest0-21swapper/1120:16:293
2088399113cyclictest0-21swapper/1100:08:503
20883991110cyclictest3066350irq/52-eth0-rx-19:13:183
2088299113cyclictest0-21swapper/1022:40:522
2088199115cyclictest0-21swapper/921:07:1911
2088199115cyclictest0-21swapper/921:01:1911
2088199113cyclictest0-21swapper/920:43:0311
2088199113cyclictest0-21swapper/919:45:1311
2088199112cyclictest0-21swapper/919:50:5911
2088199112cyclictest0-21swapper/919:50:5911
20881991110cyclictest741rcuc/922:04:2711
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional