You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 04:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sat Nov 22, 2025 00:46:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30663501230irq/52-eth0-rx-0-21swapper/119:09:201
30664501190irq/53-eth0-tx-0-21swapper/419:05:226
122501140irq/42-ahci0-21swapper/319:09:365
20008210794sleep90-21swapper/919:07:1911
122501020irq/42-ahci0-21swapper/219:09:184
3066350900irq/52-eth0-rx-0-21swapper/519:07:467
3066350890irq/52-eth0-rx-0-21swapper/1119:06:093
3066450860irq/53-eth0-tx-0-21swapper/1019:05:122
2020128074sleep00-21swapper/019:09:100
12250750irq/42-ahci0-21swapper/819:05:5310
3066450650irq/53-eth0-tx-0-21swapper/719:05:209
1844026353sleep60-21swapper/619:05:168
20415992827cyclictest0-21swapper/1121:27:323
2041599190cyclictest0-21swapper/1119:21:253
2041599190cyclictest0-21swapper/1119:18:293
20414991817cyclictest31244-1kworker/10:2H22:21:252
20414991817cyclictest31244-1kworker/10:2H00:34:532
20414991817cyclictest0-21swapper/1022:58:132
2041499181cyclictest84842sleep1020:20:052
2041499181cyclictest3066350irq/52-eth0-rx-23:40:202
2041499181cyclictest3066350irq/52-eth0-rx-23:40:202
2041499181cyclictest3066350irq/52-eth0-rx-21:46:432
2041499181cyclictest3066350irq/52-eth0-rx-20:37:562
2041499181cyclictest3066350irq/52-eth0-rx-19:28:552
2041499181cyclictest12250irq/42-ahci22:15:242
2041499181cyclictest12250irq/42-ahci22:06:302
2041499181cyclictest12250irq/42-ahci20:12:422
2041499181cyclictest12250irq/42-ahci20:09:302
2041499181cyclictest12250irq/42-ahci19:40:452
2041499181cyclictest12250irq/42-ahci19:16:022
2041499181cyclictest12250irq/42-ahci00:15:422
20413991816cyclictest25353-21diskstats22:55:1411
2041399181cyclictest761ksoftirqd/923:45:2411
2041399181cyclictest761ksoftirqd/920:20:1911
2041399181cyclictest6010-21tr21:45:1211
2041399181cyclictest0-21swapper/921:55:3111
2041399181cyclictest0-21swapper/921:30:1611
2041399181cyclictest0-21swapper/919:45:3211
2041299184cyclictest0-21swapper/822:50:0010
20412991816cyclictest9626-21snmp_rack3slot923:17:1210
20412991816cyclictest8272-21snmp_rack3slot919:35:1110
20412991816cyclictest6659-21memory20:15:2410
20412991816cyclictest3503-21snmp_rack3slot922:25:1310
20412991816cyclictest27341-21perl20:00:1810
20412991816cyclictest20924-21snmp_easybox.os19:10:1010
20412991816cyclictest17547-21snmp_rack3slot921:15:1710
20412991816cyclictest16666-21snmp_rack3slot920:30:1410
20412991816cyclictest16182-21memory00:10:2110
20412991816cyclictest12243-21snmp_rack3slot921:51:0210
20406991816cyclictest11970-21qemu-kvm21:08:587
12250180irq/42-ahci2041499cyclictest21:31:552
2041599170cyclictest0-21swapper/1121:08:393
20414991716cyclictest31244-1kworker/10:2H21:15:192
20414991716cyclictest31244-1kworker/10:2H00:35:022
20414991716cyclictest0-21swapper/1023:38:322
2041499170cyclictest89382chrt21:05:072
2041499170cyclictest83552sleep1021:04:072
2041499170cyclictest811rcuc/1000:22:432
2041499170cyclictest49362sleep1021:40:362
2041499170cyclictest34422sleep1023:55:032
2041499170cyclictest318032sleep1023:05:102
2041499170cyclictest313592sleep1023:46:142
2041499170cyclictest3066350irq/52-eth0-rx-23:10:292
2041499170cyclictest3066350irq/52-eth0-rx-22:30:202
2041499170cyclictest3066350irq/52-eth0-rx-22:25:142
2041499170cyclictest3066350irq/52-eth0-rx-20:40:092
2041499170cyclictest3066350irq/52-eth0-rx-20:00:162
2041499170cyclictest3066350irq/52-eth0-rx-19:51:012
2041499170cyclictest3066350irq/52-eth0-rx-19:30:272
2041499170cyclictest3066350irq/52-eth0-rx-00:25:132
2041499170cyclictest3066350irq/52-eth0-rx-00:00:202
2041499170cyclictest303412sleep1020:50:062
2041499170cyclictest294202chrt19:20:082
2041499170cyclictest258942sleep1021:25:352
2041499170cyclictest255252sleep1019:58:002
2041499170cyclictest237222sleep1019:11:312
2041499170cyclictest215462sleep1021:20:222
2041499170cyclictest185802sleep1020:30:402
2041499170cyclictest180842sleep1023:30:132
2041499170cyclictest169502sleep1023:28:002
2041499170cyclictest167022sleep1022:00:032
2041499170cyclictest158492sleep1021:55:372
2041499170cyclictest154532sleep1021:10:462
2041499170cyclictest154532sleep1021:10:452
2041499170cyclictest154222sleep1019:45:102
2041499170cyclictest132592sleep1023:22:112
2041499170cyclictest128532sleep1022:37:312
2041499170cyclictest122562sleep1021:51:132
2041499170cyclictest12250irq/42-ahci23:50:122
2041499170cyclictest12250irq/42-ahci23:15:122
2041499170cyclictest12250irq/42-ahci23:00:182
2041499170cyclictest12250irq/42-ahci22:50:372
2041499170cyclictest12250irq/42-ahci22:45:062
2041499170cyclictest12250irq/42-ahci22:40:482
2041499170cyclictest12250irq/42-ahci22:10:552
2041499170cyclictest12250irq/42-ahci21:36:482
2041499170cyclictest12250irq/42-ahci20:55:222
2041499170cyclictest12250irq/42-ahci20:45:152
2041499170cyclictest12250irq/42-ahci20:25:372
2041499170cyclictest12250irq/42-ahci20:16:032
2041499170cyclictest12250irq/42-ahci00:10:012
2041499170cyclictest12250irq/42-ahci00:05:012
2041499170cyclictest109402sleep1019:37:462
2041399173cyclictest0-21swapper/923:55:1711
2041399170cyclictest18839-1kworker/9:0H22:08:5611
2041399170cyclictest12250irq/42-ahci21:38:4511
2041399170cyclictest0-21swapper/900:10:1411
20412991715cyclictest9670-21snmp_rack3slot920:20:1810
20412991715cyclictest7943-21diskstats00:00:1510
20412991715cyclictest7365-21iostat21:45:1810
20412991715cyclictest680-21yum20:50:3910
20412991715cyclictest6251-21snmp_rack3slot923:55:4610
20412991715cyclictest5343-21snmp_easybox.os21:00:1210
20412991715cyclictest3938-21memory21:40:2210
20412991715cyclictest3237-21ntp_states20:10:2410
20412991715cyclictest31946-21snmp_rack3slot922:20:1110
20412991715cyclictest3055-21snmp_rack3slot900:36:2510
20412991715cyclictest29159-21snmp_easybox.os21:30:2510
20412991715cyclictest28063-21users00:25:3710
20412991715cyclictest26702-21munin-run21:30:0010
20412991715cyclictest26654-21snmp_rack3slot919:15:1810
20412991715cyclictest2635-21snmp_rack3slot923:51:3610
20412991715cyclictest25671-21iostat22:10:1910
20412991715cyclictest24548-21snmp_rack3slot900:21:4910
20412991715cyclictest22878-21ntp_states22:50:2510
20412991715cyclictest22115-21df_inode23:35:1310
20412991715cyclictest20794-21users00:15:3210
20412991715cyclictest18783-21perl23:30:1510
20412991715cyclictest1847-21perl19:25:1710
20412991715cyclictest17261-21snmp_easybox.os19:45:2410
20412991715cyclictest15943-21snmp_rack3slot921:56:5110
20412991715cyclictest12868-21perl19:40:2010
20412991715cyclictest12825-21snmp_rack3slot921:10:1310
20412991715cyclictest12825-21snmp_rack3slot921:10:1210
20412991715cyclictest11092-21perl22:35:1610
20412991715cyclictest10132-21perl21:05:1610
20410991716cyclictest428-21qemu-kvm20:25:008
20410991716cyclictest23409-21qemu-kvm22:25:058
20410991716cyclictest13042-21sadc22:40:018
20401991715cyclictest9274-21sensors00:00:266
20415991615cyclictest12250irq/42-ahci22:20:483
20413991615cyclictest28913-21df23:00:1511
20413991615cyclictest22457-21taskset20:38:5611
20413991615cyclictest0-21swapper/920:45:2911
20413991615cyclictest0-21swapper/920:30:1611
20412991614cyclictest7148-21diskstats22:30:1410
20412991614cyclictest5986-21snmp_rack3slot923:12:0710
20412991614cyclictest4585-21snmp_rack3slot919:30:1510
20412991614cyclictest31037-21snmp_rack3slot923:01:3910
20412991614cyclictest30727-21diskstats20:05:1410
20412991614cyclictest29629-21perl23:45:1510
20412991614cyclictest25402-21snmp_rack3slot919:56:2410
20412991614cyclictest24320-21snmp_rack3slot920:40:1310
20412991614cyclictest20994-21snmp_rack3slot921:20:1710
20412991614cyclictest12467-21snmp_rack3slot923:20:2310
20410991615cyclictest428-21qemu-kvm20:25:028
20406991614cyclictest11969-21qemu-kvm21:25:477
2040099160cyclictest0-21swapper/323:30:335
20393991614cyclictest2406-21snmpd19:12:444
20388991615cyclictest3066350irq/52-eth0-rx-22:16:431
20388991615cyclictest3066350irq/52-eth0-rx-20:19:581
20388991615cyclictest17998-21python20:30:301
20388991614cyclictest23465-21python23:35:271
20388991614cyclictest11628-21python21:50:291
20413991514cyclictest20024-1kworker/9:2H22:47:0611
20413991514cyclictest0-21swapper/922:10:1811
20413991514cyclictest0-21swapper/921:40:1211
20413991514cyclictest0-21swapper/920:17:2511
20413991514cyclictest0-21swapper/920:12:4911
20412991513cyclictest32088-21perl21:35:1710
20412991513cyclictest26912-21ntp_states23:40:2710
20412991513cyclictest26912-21ntp_states23:40:2610
20412991513cyclictest21461-21ntp_states20:35:2910
20412991513cyclictest13372-21snmp_rack3slot920:25:1510
20411991514cyclictest26543-21perl00:25:189
20411991514cyclictest0-21swapper/721:39:519
2041099152cyclictest0-21swapper/622:42:448
2041099152cyclictest0-21swapper/621:31:438
20406991514cyclictest27622-21turbostat23:04:567
20406991514cyclictest23552-21switchtime23:35:317
2039399152cyclictest0-21swapper/222:30:454
2038899150cyclictest0-21swapper/121:25:021
271140ksoftirqd/20-21swapper/200:26:094
20413991413cyclictest0-21swapper/923:05:3311
20413991413cyclictest0-21swapper/919:40:2911
20413991413cyclictest0-21swapper/919:30:2611
2041299143cyclictest0-21swapper/823:05:0110
20412991412cyclictest31775-21snmp_rack3slot900:31:2610
2041199141cyclictest0-21swapper/721:22:279
20401991412cyclictest26276-21sensors22:10:276
2038899144cyclictest0-21swapper/123:20:001
761130ksoftirqd/90-21swapper/922:36:3911
551130ksoftirqd/60-21swapper/623:20:478
481130ksoftirqd/50-21swapper/523:20:467
481130ksoftirqd/50-21swapper/521:48:267
481130ksoftirqd/50-21swapper/500:19:577
411130ksoftirqd/40-21swapper/423:20:466
411130ksoftirqd/40-21swapper/421:48:276
341130ksoftirqd/30-21swapper/321:48:265
31130ksoftirqd/00-21swapper/023:20:460
31130ksoftirqd/00-21swapper/022:36:390
271130ksoftirqd/20-21swapper/223:10:594
271130ksoftirqd/20-21swapper/222:35:064
2041599134cyclictest0-21swapper/1123:30:013
2041599133cyclictest0-21swapper/1120:55:303
2041599133cyclictest0-21swapper/1119:40:003
20415991312cyclictest8485-21chrt20:20:043
20415991312cyclictest5122sleep1120:09:143
20415991310cyclictest0-21swapper/1122:45:243
20413991312cyclictest761ksoftirqd/921:20:3611
20413991312cyclictest14111-21snmp_rack3slot922:40:1311
20413991312cyclictest0-21swapper/900:05:1411
2041299133cyclictest0-21swapper/822:41:4710
2041299133cyclictest0-21swapper/822:16:0110
20412991311cyclictest27322-21snmp_rack3slot922:56:1310
20412991311cyclictest20143-21perl19:50:1710
2041299131cyclictest0-21swapper/820:45:2110
2041199133cyclictest0-21swapper/723:41:079
2041199133cyclictest0-21swapper/723:41:069
2041199133cyclictest0-21swapper/700:35:019
2041199132cyclictest0-21swapper/722:27:199
20411991312cyclictest0-21swapper/723:17:409
20411991311cyclictest0-21swapper/722:52:419
20411991311cyclictest0-21swapper/722:12:499
2041199131cyclictest0-21swapper/722:30:099
2041199131cyclictest0-21swapper/720:41:079
2041099130cyclictest0-21swapper/600:15:128
2040699133cyclictest0-21swapper/519:35:187
2040699133cyclictest0-21swapper/519:10:167
20406991311cyclictest20120-21qemu-kvm19:47:207
2039399134cyclictest0-21swapper/200:10:304
2039399132cyclictest0-21swapper/219:41:204
20393991312cyclictest2406-21snmpd23:41:474
20393991312cyclictest2406-21snmpd23:41:464
2039399130cyclictest0-21swapper/223:06:374
2039399130cyclictest0-21swapper/221:35:184
2039399130cyclictest0-21swapper/200:40:004
2038899133cyclictest0-21swapper/121:05:201
20388991312cyclictest23865-21python00:20:261
2038899130cyclictest0-21swapper/120:35:361
20377991311cyclictest325282sleep019:24:480
20377991311cyclictest114422sleep020:22:410
20377991311cyclictest1101-21munin-node20:10:240
201130ksoftirqd/10-21swapper/121:48:261
14850130irq/18-uhci_hcd0-21swapper/021:30:290
901120ksoftirqd/110-21swapper/1123:35:583
901120ksoftirqd/110-21swapper/1100:19:573
761120ksoftirqd/90-21swapper/923:20:4611
691120ksoftirqd/80-21swapper/822:05:3110
551120ksoftirqd/623409-21qemu-kvm22:30:098
551120ksoftirqd/60-21swapper/622:05:328
551120ksoftirqd/60-21swapper/621:48:268
551120ksoftirqd/60-21swapper/620:41:078
551120ksoftirqd/60-21swapper/600:12:458
481120ksoftirqd/511969-21qemu-kvm21:22:267
481120ksoftirqd/50-21swapper/522:30:097
481120ksoftirqd/50-21swapper/521:59:487
481120ksoftirqd/50-21swapper/520:41:087
411120ksoftirqd/40-21swapper/422:36:396
411120ksoftirqd/40-21swapper/422:30:106
411120ksoftirqd/40-21swapper/421:59:486
341120ksoftirqd/30-21swapper/323:20:465
341120ksoftirqd/30-21swapper/322:36:395
341120ksoftirqd/30-21swapper/321:22:275
31120ksoftirqd/00-21swapper/022:46:290
31120ksoftirqd/00-21swapper/022:30:090
31120ksoftirqd/00-21swapper/022:05:310
31120ksoftirqd/00-21swapper/020:41:070
31120ksoftirqd/00-21swapper/020:15:270
31120ksoftirqd/00-21swapper/019:15:310
31120ksoftirqd/00-21swapper/000:00:070
271120ksoftirqd/20-21swapper/222:05:314
2041599124cyclictest0-21swapper/1123:55:013
2041599124cyclictest0-21swapper/1100:10:013
2041599123cyclictest0-21swapper/1123:50:013
2041599123cyclictest0-21swapper/1123:10:563
2041599123cyclictest0-21swapper/1123:05:013
2041599123cyclictest0-21swapper/1100:30:013
20415991211cyclictest13500-21taskset20:25:173
2041399124cyclictest0-21swapper/919:15:2611
20413991211cyclictest0-21swapper/923:19:0311
2041199125cyclictest0-21swapper/720:25:009
2041199124cyclictest0-21swapper/723:39:449
2041199124cyclictest0-21swapper/720:52:229
2041199124cyclictest0-21swapper/720:01:349
2041199124cyclictest0-21swapper/700:20:029
2041199123cyclictest0-21swapper/721:55:019
20411991211cyclictest0-21swapper/700:04:469
20411991210cyclictest0-21swapper/720:46:379
2041199121cyclictest0-21swapper/721:18:309
2041199121cyclictest0-21swapper/720:39:179
2041099122cyclictest0-21swapper/623:15:018
2041099121cyclictest23410-21qemu-kvm22:20:118
2040699122cyclictest0-21swapper/522:00:207
20400991211cyclictest0-21swapper/322:30:095
20400991211cyclictest0-21swapper/320:41:075
2040099121cyclictest0-21swapper/319:45:155
2039399123cyclictest0-21swapper/221:47:324
2039399123cyclictest0-21swapper/220:55:224
2039399123cyclictest0-21swapper/220:10:204
20393991210cyclictest0-21swapper/222:22:194
20393991210cyclictest0-21swapper/221:50:294
2039399120cyclictest0-21swapper/222:45:194
2039399120cyclictest0-21swapper/200:30:174
2038899123cyclictest0-21swapper/123:20:001
2038899122cyclictest0-21swapper/122:20:161
20377991211cyclictest37782sleep019:29:250
20377991210cyclictest23740-21taskset19:11:450
201120ksoftirqd/10-21swapper/122:05:311
201120ksoftirqd/10-21swapper/120:59:561
201120ksoftirqd/10-21swapper/100:19:581
13450120irq/18-ehci_hcd0-21swapper/000:30:270
901110ksoftirqd/110-21swapper/1123:17:403
901110ksoftirqd/110-21swapper/1122:33:483
901110ksoftirqd/110-21swapper/1121:48:263
901110ksoftirqd/110-21swapper/1120:41:073
761110ksoftirqd/90-21swapper/923:35:5711
761110ksoftirqd/90-21swapper/920:41:0811
761110ksoftirqd/90-21swapper/920:28:5111
761110ksoftirqd/90-21swapper/900:00:0811
551110ksoftirqd/611970-21qemu-kvm21:25:508
551110ksoftirqd/60-21swapper/621:22:268
551110ksoftirqd/60-21swapper/600:29:298
5499110migration/68335-21cpuspeed_turbos19:35:118
481110ksoftirqd/50-21swapper/523:17:417
481110ksoftirqd/50-21swapper/522:08:087
481110ksoftirqd/50-21swapper/520:59:567
481110ksoftirqd/50-21swapper/520:39:177
481110ksoftirqd/50-21swapper/500:29:287
481110ksoftirqd/50-21swapper/500:00:077
411110ksoftirqd/40-21swapper/423:17:406
411110ksoftirqd/40-21swapper/422:46:296
411110ksoftirqd/40-21swapper/421:37:006
411110ksoftirqd/40-21swapper/421:12:476
411110ksoftirqd/40-21swapper/421:12:466
411110ksoftirqd/40-21swapper/420:59:576
411110ksoftirqd/40-21swapper/420:43:296
411110ksoftirqd/40-21swapper/420:28:516
341110ksoftirqd/30-21swapper/323:35:575
341110ksoftirqd/30-21swapper/322:52:405
341110ksoftirqd/30-21swapper/322:46:295
341110ksoftirqd/30-21swapper/321:25:505
341110ksoftirqd/30-21swapper/320:28:515
341110ksoftirqd/30-21swapper/300:26:085
341110ksoftirqd/30-21swapper/300:12:455
3066350110irq/52-eth0-rx-0-21swapper/021:40:540
31110ksoftirqd/00-21swapper/023:15:380
31110ksoftirqd/00-21swapper/000:29:290
31110ksoftirqd/00-21swapper/000:12:460
271110ksoftirqd/20-21swapper/223:35:574
271110ksoftirqd/20-21swapper/223:17:404
271110ksoftirqd/20-21swapper/221:15:084
271110ksoftirqd/20-21swapper/200:00:074
2041599119cyclictest15772-21chrt20:30:053
2041599114cyclictest0-21swapper/1122:25:183
2041599113cyclictest0-21swapper/1123:50:013
2041599113cyclictest0-21swapper/1122:58:453
2041599113cyclictest0-21swapper/1122:11:453
2041599113cyclictest0-21swapper/1121:05:003
2041599113cyclictest0-21swapper/1119:45:303
2041599113cyclictest0-21swapper/1119:36:093
20415991110cyclictest42562sleep1120:14:033
20415991110cyclictest3066350irq/52-eth0-rx-19:26:353
2041599111cyclictest0-21swapper/1122:40:163
2041599110cyclictest0-21swapper/1100:39:593
2041399114cyclictest0-21swapper/922:20:0111
2041399114cyclictest0-21swapper/919:50:4111
20413991110cyclictest0-21swapper/920:57:2111
20413991110cyclictest0-21swapper/920:08:5211
20412991110cyclictest0-21swapper/823:26:0110
20412991110cyclictest0-21swapper/800:06:2810
2041299110cyclictest0-21swapper/822:00:0610
2041299110cyclictest0-21swapper/820:57:0010
2041199119cyclictest0-21swapper/723:26:019
2041199119cyclictest0-21swapper/723:24:379
2041199119cyclictest0-21swapper/723:07:029
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional