You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 06:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Mon Nov 17, 2025 00:46:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20430210995sleep30-21swapper/319:07:475
20430210995sleep30-21swapper/319:07:465
20376210894sleep80-21swapper/819:07:0310
20376210894sleep80-21swapper/819:07:0310
20272210390sleep110-21swapper/1119:05:413
20272210390sleep110-21swapper/1119:05:413
122501020irq/42-ahci0-21swapper/119:05:131
122501020irq/42-ahci0-21swapper/119:05:131
3066350950irq/52-eth0-rx-0-21swapper/219:07:034
3066350950irq/52-eth0-rx-0-21swapper/219:07:034
3066450860irq/53-eth0-tx-0-21swapper/719:08:409
3066450860irq/53-eth0-tx-0-21swapper/719:08:399
3066350850irq/52-eth0-rx-0-21swapper/519:07:417
3066350850irq/52-eth0-rx-0-21swapper/519:07:417
2060227876sleep40-21swapper/419:09:246
2060227876sleep40-21swapper/419:09:236
2033526556sleep100-21swapper/1019:06:292
2033526556sleep100-21swapper/1019:06:282
2036826055sleep00-21swapper/019:06:560
2036826055sleep00-21swapper/019:06:550
2040525955sleep60-21swapper/619:07:258
2040525955sleep60-21swapper/619:07:258
2039525955sleep90-21swapper/919:07:1611
2039525955sleep90-21swapper/919:07:1611
20790993819cyclictest13752-1kworker/11:2H21:12:343
2074899320cyclictest0-21swapper/021:15:210
20766993029cyclictest3066450irq/53-eth0-tx-21:15:285
2079099290cyclictest0-21swapper/1122:26:373
2074899280cyclictest0-21swapper/022:11:110
20774992625cyclictest12250irq/42-ahci21:15:157
20764992625cyclictest12250irq/42-ahci21:15:254
20786992524cyclictest12250irq/42-ahci21:15:189
20788992423cyclictest12250irq/42-ahci21:15:2111
2078999225cyclictest831ksoftirqd/1021:00:492
14850200irq/18-uhci_hcd0-21swapper/421:45:306
2078999192cyclictest50392sleep1020:58:412
2078999192cyclictest3066350irq/52-eth0-rx-19:55:292
2078899192cyclictest0-21swapper/923:40:2111
20748991916cyclictest2388-21nscd22:16:180
20789991817cyclictest0-21swapper/1022:00:212
20789991817cyclictest0-21swapper/1021:29:442
20789991817cyclictest0-21swapper/1000:02:522
2078999181cyclictest831ksoftirqd/1021:20:292
2078999181cyclictest831ksoftirqd/1020:05:012
2078999181cyclictest831ksoftirqd/1019:25:122
2078999181cyclictest831ksoftirqd/1019:25:122
2078999181cyclictest3066350irq/52-eth0-rx-22:17:092
2078999181cyclictest279742sleep1022:59:432
2078999181cyclictest12250irq/42-ahci21:10:502
2078999181cyclictest12250irq/42-ahci00:05:212
2078899181cyclictest29499-21kworker/9:223:25:1511
2078899181cyclictest12250irq/42-ahci21:40:4911
2078899181cyclictest0-21swapper/920:20:2211
20787991816cyclictest9745-21snmp_rack3slot919:35:2010
20787991816cyclictest8886-21snmp_easybox.os20:20:1210
20787991816cyclictest3384-21snmp_rack3slot921:40:1610
20787991816cyclictest31271-21snmp_easybox.os21:35:1410
20787991816cyclictest30926-21perl19:20:1410
20787991816cyclictest29507-21snmp_rack3slot920:01:5310
20787991816cyclictest26746-21users21:25:4110
20787991816cyclictest2634-21snmp_rack3slot923:06:1710
20787991816cyclictest25831-21snmp_rack3slot919:56:5310
20787991816cyclictest24062-21snmp_rack3slot922:51:0210
20787991816cyclictest22126-21snmp_easybox.os21:20:2810
20787991816cyclictest21713-21snmp_easybox.os23:35:1210
20787991816cyclictest21704-21snmp_rack3slot919:10:1610
20787991816cyclictest16293-21snmp_rack3slot921:56:2910
20787991816cyclictest16007-21snmp_easybox.os22:40:2510
20787991816cyclictest1584-21perl00:35:1810
20787991816cyclictest12352-21perl00:05:1610
20787991816cyclictest10919-21iostat_ios21:05:2210
20787991816cyclictest10661-21df_inode21:50:1410
20787991816cyclictest1063-21yum20:50:3810
20748991817cyclictest0-21swapper/019:15:570
20748991817cyclictest0-21swapper/019:15:560
20790991716cyclictest22452sleep1122:22:273
20789991716cyclictest831ksoftirqd/1020:35:142
20789991716cyclictest3825-21python20:10:322
20789991716cyclictest0-21swapper/1022:40:152
2078999170cyclictest831ksoftirqd/1023:30:332
2078999170cyclictest831ksoftirqd/1023:25:122
2078999170cyclictest831ksoftirqd/1020:20:342
2078999170cyclictest831ksoftirqd/1019:50:342
2078999170cyclictest831ksoftirqd/1019:40:162
2078999170cyclictest831ksoftirqd/1019:15:142
2078999170cyclictest831ksoftirqd/1019:15:142
2078999170cyclictest80582sleep1020:15:592
2078999170cyclictest75562sleep1019:30:552
2078999170cyclictest75562sleep1019:30:542
2078999170cyclictest61522sleep1021:45:112
2078999170cyclictest53852sleep1021:41:332
2078999170cyclictest3532sleep1019:23:312
2078999170cyclictest34312sleep1000:36:162
2078999170cyclictest317112chrt23:45:472
2078999170cyclictest315972sleep1020:50:162
2078999170cyclictest314452chrt23:01:552
2078999170cyclictest3066350irq/52-eth0-rx-23:55:152
2078999170cyclictest3066350irq/52-eth0-rx-23:10:142
2078999170cyclictest3066350irq/52-eth0-rx-22:50:382
2078999170cyclictest3066350irq/52-eth0-rx-21:30:342
2078999170cyclictest3066350irq/52-eth0-rx-21:15:232
2078999170cyclictest3066350irq/52-eth0-rx-21:05:132
2078999170cyclictest3066350irq/52-eth0-rx-00:15:172
2078999170cyclictest30242chrt23:51:122
2078999170cyclictest30192sleep1022:25:022
2078999170cyclictest299052chrt20:46:042
2078999170cyclictest296102sleep1000:30:172
2078999170cyclictest287662sleep1000:29:412
2078999170cyclictest273112sleep1022:11:252
2078999170cyclictest252822sleep1023:40:102
2078999170cyclictest239002sleep1020:40:142
2078999170cyclictest237162sleep1022:07:492
2078999170cyclictest220422chrt00:20:032
2078999170cyclictest217052sleep1019:10:162
2078999170cyclictest216482sleep1023:35:112
2078999170cyclictest21362sleep1022:21:012
2078999170cyclictest203822sleep1022:45:402
2078999170cyclictest190482sleep1020:31:502
2078999170cyclictest176542sleep1000:11:502
2078999170cyclictest171442chrt19:45:202
2078999170cyclictest17022sleep1021:37:092
2078999170cyclictest162382sleep1021:55:462
2078999170cyclictest123332chrt23:20:202
2078999170cyclictest12250irq/42-ahci23:15:532
2078999170cyclictest12250irq/42-ahci23:05:312
2078999170cyclictest12250irq/42-ahci22:30:022
2078999170cyclictest12250irq/42-ahci21:50:562
2078999170cyclictest12250irq/42-ahci20:26:092
2078999170cyclictest12250irq/42-ahci20:08:542
2078999170cyclictest113782sleep1019:38:312
2078899170cyclictest0-21swapper/923:10:1411
2078899170cyclictest0-21swapper/919:32:3011
2078899170cyclictest0-21swapper/919:32:3011
2078899170cyclictest0-21swapper/900:10:2211
20787991715cyclictest9967-21snmp_rack3slot923:16:3910
20787991715cyclictest8326-21diskstats00:00:1610
20787991715cyclictest6983-21snmp_rack3slot922:30:1810
20787991715cyclictest6839-21iostat_ios20:15:2010
20787991715cyclictest5080-21irqstats23:10:2610
20787991715cyclictest4974-21perl23:55:1610
20787991715cyclictest4908-21snmp_rack3slot920:57:0310
20787991715cyclictest30927-21irqstats00:30:2410
20787991715cyclictest28143-21snmp_rack3slot923:41:3710
20787991715cyclictest26184-21snmp_rack3slot919:15:1710
20787991715cyclictest26184-21snmp_rack3slot919:15:1610
20787991715cyclictest25602-21snmp_rack3slot922:10:1710
20787991715cyclictest24858-21snmp_rack3slot900:20:5410
20787991715cyclictest23942-21munin-run22:10:0110
20787991715cyclictest22165-21snmp_rack3slot919:51:2810
20787991715cyclictest19619-21perl00:15:1610
20787991715cyclictest1877-21snmp_rack3slot919:25:1610
20787991715cyclictest1877-21snmp_rack3slot919:25:1510
20787991715cyclictest17782-21irqstats20:30:2310
20787991715cyclictest17101-21snmp_easybox.os22:00:1510
20787991715cyclictest15318-21yum20:25:3910
20787991715cyclictest14590-21snmp_rack3slot923:25:1210
20787991715cyclictest13492-21snmp_rack3slot919:40:2210
20787991715cyclictest1255-21perl23:50:1510
20780991716cyclictest20511-21qemu-kvm19:40:188
20780991716cyclictest20510-21qemu-kvm19:35:158
20780991716cyclictest20510-21qemu-kvm19:30:578
20780991716cyclictest20510-21qemu-kvm19:30:578
20748991716cyclictest58452sleep022:25:540
20788991615cyclictest31558-21sshd23:03:2911
20788991615cyclictest0-21swapper/922:25:1311
20788991615cyclictest0-21swapper/922:25:0111
20788991615cyclictest0-21swapper/920:45:2711
20787991614cyclictest8188-21ntp_states21:45:2810
20787991614cyclictest5620-21snmp_rack3slot919:30:1710
20787991614cyclictest5620-21snmp_rack3slot919:30:1710
20787991614cyclictest30934-21ntp_states23:45:2810
20787991614cyclictest29867-21hddtemp_smartct23:00:1710
20787991614cyclictest27942-21diskstats20:45:1510
20787991614cyclictest27694-21snmp_rack3slot922:56:0310
20787991614cyclictest24645-21perl20:40:1810
20787991614cyclictest20898-21perl20:35:1710
20787991614cyclictest19595-21iostat_ios23:30:2310
20787991614cyclictest17806-21snmp_rack3slot922:45:1210
20787991614cyclictest17559-21yum00:10:3510
20787991614cyclictest17505-21snmp_rack3slot921:15:1810
20787991614cyclictest13092-21snmp_easybox.os21:10:1410
2078099163cyclictest0-21swapper/621:28:038
20780991615cyclictest20510-21qemu-kvm19:45:168
20780991615cyclictest20510-21qemu-kvm19:11:138
20774991615cyclictest25160-21python19:55:327
20774991614cyclictest23793-21qemu-kvm22:08:287
20769991615cyclictest3066350irq/52-eth0-rx-21:15:316
20769991614cyclictest12511-21sensors22:35:276
2076499163cyclictest0-21swapper/221:55:234
2076499163cyclictest0-21swapper/200:30:004
20790991514cyclictest3066450irq/53-eth0-tx-21:15:143
2078899154cyclictest0-21swapper/921:25:0111
2078899154cyclictest0-21swapper/900:25:0211
20788991514cyclictest0-21swapper/923:30:1511
20788991513cyclictest9389-21snmp_easybox.os21:05:1311
2078899150cyclictest0-21swapper/919:15:2711
2078899150cyclictest0-21swapper/919:15:2711
20787991513cyclictest26761-21perl00:25:1510
20787991513cyclictest12531-21memory23:20:2210
2078699150cyclictest0-21swapper/722:00:009
2078099154cyclictest0-21swapper/622:45:018
20780991514cyclictest20510-21qemu-kvm19:25:148
20780991514cyclictest20510-21qemu-kvm19:25:148
20780991513cyclictest23794-21qemu-kvm22:15:138
2076699150cyclictest0-21swapper/322:40:135
2076499153cyclictest0-21swapper/221:08:104
2076499152cyclictest0-21swapper/221:45:214
20764991514cyclictest2406-21snmpd22:11:334
20748991514cyclictest14774-21nscd20:10:350
20748991514cyclictest116932sleep020:20:510
3199140watchdog/30-21swapper/319:45:185
31140ksoftirqd/00-21swapper/022:30:360
2079099145cyclictest0-21swapper/1122:58:173
20790991413cyclictest6662-21taskset22:30:033
20789991413cyclictest12645-21cat22:35:312
2078899145cyclictest0-21swapper/922:10:0211
2078899144cyclictest0-21swapper/922:40:0111
2078899144cyclictest0-21swapper/921:35:0011
2078899144cyclictest0-21swapper/921:00:0011
2078899144cyclictest0-21swapper/900:05:0111
20788991413cyclictest0-21swapper/922:30:3411
20788991413cyclictest0-21swapper/921:46:1811
20788991413cyclictest0-21swapper/920:25:0011
20788991413cyclictest0-21swapper/919:15:0011
20788991412cyclictest14832-21snmp_easybox.os21:10:2711
20787991412cyclictest28777-21perl21:30:1710
20787991412cyclictest17868-21sensors_volt19:45:2910
20780991413cyclictest20510-21qemu-kvm19:15:058
20780991413cyclictest20510-21qemu-kvm19:15:058
20774991413cyclictest9795-21cstates21:05:147
20774991413cyclictest20510-21qemu-kvm19:28:097
20774991413cyclictest20510-21qemu-kvm19:28:097
20769991412cyclictest29810-21sensors21:30:316
2076499143cyclictest0-21swapper/221:40:214
20764991413cyclictest2406-21snmpd23:23:174
2076499140cyclictest0-21swapper/221:00:364
2079099132cyclictest0-21swapper/1100:35:003
2079099130cyclictest6955-21munin-run00:00:003
2078899134cyclictest0-21swapper/923:45:0011
2078899134cyclictest0-21swapper/923:15:0211
2078899133cyclictest0-21swapper/920:43:0111
2078899133cyclictest0-21swapper/920:15:5211
20788991312cyclictest0-21swapper/921:20:1611
20788991312cyclictest0-21swapper/900:30:1711
2078899131cyclictest0-21swapper/920:10:1311
2078799133cyclictest0-21swapper/822:15:5910
20787991311cyclictest7684-21ntp_states21:00:2910
20787991311cyclictest3998-21snmp_rack3slot922:25:1510
2078699130cyclictest0-21swapper/720:35:009
2078099137cyclictest0-21swapper/623:40:008
2078099133cyclictest0-21swapper/623:10:398
2078099133cyclictest0-21swapper/621:41:588
2078099133cyclictest0-21swapper/620:35:018
20774991312cyclictest8091-21timerandwakeup21:00:367
2076499131cyclictest0-21swapper/219:35:204
2074899133cyclictest0-21swapper/020:50:200
31120ksoftirqd/00-21swapper/020:55:290
20790991211cyclictest3066350irq/52-eth0-rx-20:10:303
2078899123cyclictest0-21swapper/923:09:0011
2078899123cyclictest0-21swapper/920:30:0111
2078899123cyclictest0-21swapper/919:44:4611
2078899122cyclictest0-21swapper/921:54:2811
20788991211cyclictest5513-1kworker/9:0H22:09:3011
2078799124cyclictest0-21swapper/820:05:2810
2078799122cyclictest0-21swapper/822:36:4810
2078799122cyclictest0-21swapper/822:24:2310
2078699122cyclictest0-21swapper/722:01:089
2078099123cyclictest0-21swapper/623:45:018
2078099122cyclictest0-21swapper/622:37:008
2078099122cyclictest0-21swapper/621:57:548
2078099122cyclictest0-21swapper/619:57:538
2078099122cyclictest0-21swapper/600:24:188
2078099122cyclictest0-21swapper/600:02:328
20780991210cyclictest23794-21qemu-kvm22:32:218
20774991211cyclictest25759-21gltestperf22:10:177
20774991211cyclictest19297-21python22:00:297
2076999122cyclictest0-21swapper/421:00:226
20756991210cyclictest3066350irq/52-eth0-rx-20:05:241
20748991210cyclictest317372sleep022:20:030
20748991210cyclictest0-21swapper/019:40:330
2074899120cyclictest0-21swapper/000:35:280
2074899120cyclictest0-21swapper/000:30:300
1699120watchdog/00-21swapper/022:02:420
5499110migration/69783-21munin-node21:05:158
5499110migration/629159-21munin-node20:00:378
481110ksoftirqd/516628-21cstates20:30:147
481110ksoftirqd/50-21swapper/520:55:247
271110ksoftirqd/20-21swapper/222:25:134
271110ksoftirqd/20-21swapper/220:30:134
2079099113cyclictest0-21swapper/1123:10:013
2079099113cyclictest0-21swapper/1121:46:393
2079099113cyclictest0-21swapper/1120:58:023
2079099113cyclictest0-21swapper/1120:50:233
2079099112cyclictest0-21swapper/1123:20:003
20790991110cyclictest27400-21taskset22:12:383
20790991110cyclictest23904-21chrt22:09:373
2079099111cyclictest0-21swapper/1123:20:213
2079099110cyclictest0-21swapper/1122:55:003
2078899114cyclictest0-21swapper/920:50:2311
2078899113cyclictest0-21swapper/920:00:0111
2078899113cyclictest0-21swapper/900:35:3311
2078899112cyclictest0-21swapper/920:06:4611
20788991110cyclictest0-21swapper/923:55:1611
2078799115cyclictest0-21swapper/820:13:1810
2078699113cyclictest0-21swapper/723:53:329
2078699113cyclictest0-21swapper/723:00:149
2078699113cyclictest0-21swapper/720:11:579
2078699113cyclictest0-21swapper/720:01:389
2078699110cyclictest0-21swapper/723:35:009
2078699110cyclictest0-21swapper/722:20:309
2078699110cyclictest0-21swapper/719:50:019
2078099113cyclictest23793-21qemu-kvm22:14:408
2078099113cyclictest12352-21qemu-kvm21:13:548
2078099113cyclictest0-21swapper/623:21:108
2078099113cyclictest0-21swapper/623:04:448
2078099113cyclictest0-21swapper/622:25:588
2078099113cyclictest0-21swapper/620:54:348
2078099113cyclictest0-21swapper/620:47:178
2078099113cyclictest0-21swapper/620:19:478
2078099113cyclictest0-21swapper/620:09:228
2078099113cyclictest0-21swapper/600:30:348
2078099113cyclictest0-21swapper/600:29:348
2078099113cyclictest0-21swapper/600:17:038
2078099113cyclictest0-21swapper/600:13:258
2078099113cyclictest0-21swapper/600:07:498
2078099111cyclictest0-21swapper/623:54:328
2078099111cyclictest0-21swapper/622:04:038
2078099111cyclictest0-21swapper/621:03:588
2077499113cyclictest0-21swapper/520:40:007
2077499113cyclictest0-21swapper/500:35:207
20774991110cyclictest20510-21qemu-kvm19:47:337
2076999119cyclictest9296-21sensors23:15:296
2076699113cyclictest0-21swapper/322:30:265
2076699113cyclictest0-21swapper/322:00:005
2076699113cyclictest0-21swapper/320:55:205
2076699113cyclictest0-21swapper/300:10:215
2076499113cyclictest0-21swapper/220:20:214
2076499113cyclictest0-21swapper/220:10:014
20756991110cyclictest3066350irq/52-eth0-rx-23:51:141
2074899113cyclictest0-21swapper/020:09:050
2074899112cyclictest0-21swapper/021:05:210
20748991110cyclictest161222sleep021:14:460
20748991110cyclictest0-21swapper/021:25:370
2074899110cyclictest0-21swapper/023:30:180
2074899110cyclictest0-21swapper/022:35:170
201110ksoftirqd/10-21swapper/119:30:191
201110ksoftirqd/10-21swapper/119:30:191
90892100sleep60-21swapper/621:47:318
901100ksoftirqd/110-21swapper/1120:20:403
901100ksoftirqd/110-21swapper/1119:40:293
901100ksoftirqd/110-21swapper/1100:05:243
8999100migration/1128978-21munin-node23:00:143
8999100migration/1110389-21munin-node00:01:513
881100rcuc/110-21swapper/1120:45:143
621100ksoftirqd/70-21swapper/722:25:149
621100ksoftirqd/70-21swapper/719:30:209
621100ksoftirqd/70-21swapper/719:30:199
6199100migration/79409-21cstates20:20:159
6199100migration/79312-21munin-node23:15:309
6199100migration/726234-21munin-node22:55:189
6199100migration/718359-21munin-node22:45:139
5499100migration/632000-21cstates_1121:35:168
5499100migration/631857-21taskset23:47:418
5499100migration/62118-21munin-node20:55:128
5299100watchdog/60-21swapper/621:50:308
481100ksoftirqd/520511-21qemu-kvm19:30:207
481100ksoftirqd/520511-21qemu-kvm19:30:197
481100ksoftirqd/50-21swapper/519:40:297
4599100watchdog/50-21swapper/522:40:357
411100ksoftirqd/40-21swapper/422:25:136
411100ksoftirqd/40-21swapper/422:15:256
411100ksoftirqd/40-21swapper/420:05:186
411100ksoftirqd/40-21swapper/419:40:286
341100ksoftirqd/30-21swapper/322:25:145
341100ksoftirqd/30-21swapper/319:30:195
341100ksoftirqd/30-21swapper/319:30:195
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional