You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 02:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Wed Nov 19, 2025 00:46:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501130irq/42-ahci0-21swapper/119:08:281
16710210593sleep20-21swapper/219:06:584
30663501020irq/52-eth0-rx-0-21swapper/719:05:189
12250980irq/42-ahci0-21swapper/319:09:025
3066350970irq/52-eth0-rx-0-21swapper/419:05:256
12250940irq/42-ahci0-21swapper/919:09:2811
3066350890irq/52-eth0-rx-0-21swapper/519:07:457
12250840irq/42-ahci0-21swapper/819:05:2310
1695227270sleep100-21swapper/1019:09:302
1667626553sleep00-21swapper/019:06:320
1357126353sleep60-21swapper/619:05:028
1665925955sleep110-21swapper/1119:06:193
1709199330cyclictest0-21swapper/020:29:380
1709199330cyclictest0-21swapper/020:29:370
1709199300cyclictest0-21swapper/021:16:330
1714799192cyclictest3066350irq/52-eth0-rx-21:30:212
1714799192cyclictest177742sleep1000:18:412
17147991911cyclictest7879-21python21:05:312
1714399192cyclictest761ksoftirqd/923:25:5111
1714399192cyclictest761ksoftirqd/923:17:4011
1714399192cyclictest761ksoftirqd/922:50:0011
1714399192cyclictest761ksoftirqd/922:13:5811
1714399192cyclictest761ksoftirqd/922:00:1711
1714399192cyclictest761ksoftirqd/921:19:0311
1714399192cyclictest761ksoftirqd/920:50:2711
1714399192cyclictest761ksoftirqd/919:52:3811
1714399192cyclictest761ksoftirqd/919:35:3611
1714399192cyclictest761ksoftirqd/919:24:1111
1714399192cyclictest761ksoftirqd/919:10:2511
1714399192cyclictest761ksoftirqd/900:10:2511
1714399192cyclictest0-21swapper/922:50:3711
1714399192cyclictest0-21swapper/921:42:1511
1714399192cyclictest0-21swapper/921:31:0511
1714399192cyclictest0-21swapper/920:30:2011
1714399192cyclictest0-21swapper/920:30:2011
1714399192cyclictest0-21swapper/920:20:0911
1711999192cyclictest0-21swapper/420:20:206
1709199190cyclictest0-21swapper/019:30:390
1714799183cyclictest0-21swapper/1022:12:102
1714799183cyclictest0-21swapper/1020:16:362
17147991817cyclictest15389-21timerandwakeup21:15:322
17147991817cyclictest0-21swapper/1021:55:232
17147991811cyclictest6938-21python19:35:302
17147991811cyclictest0-21swapper/1019:45:002
1714799181cyclictest811rcuc/1020:28:142
1714799181cyclictest811rcuc/1020:28:142
1714799181cyclictest12250irq/42-ahci23:12:172
1714799181cyclictest12250irq/42-ahci22:47:312
1714399181cyclictest761ksoftirqd/923:55:3211
1714399181cyclictest761ksoftirqd/923:10:2311
1714399181cyclictest761ksoftirqd/921:45:2211
1714399181cyclictest761ksoftirqd/920:25:3311
1714399181cyclictest761ksoftirqd/920:25:3311
1714399181cyclictest761ksoftirqd/920:05:1411
1714399181cyclictest761ksoftirqd/920:00:1911
1714399181cyclictest761ksoftirqd/919:55:2311
1714399181cyclictest761ksoftirqd/919:15:1911
1714399181cyclictest761ksoftirqd/900:25:2411
1714399181cyclictest0-21swapper/923:50:0811
1714399181cyclictest0-21swapper/923:45:0611
1714399181cyclictest0-21swapper/923:40:1711
1714399181cyclictest0-21swapper/923:35:1611
1714399181cyclictest0-21swapper/923:30:2211
1714399181cyclictest0-21swapper/923:20:0411
1714399181cyclictest0-21swapper/923:20:0411
1714399181cyclictest0-21swapper/923:05:1311
1714399181cyclictest0-21swapper/923:00:1011
1714399181cyclictest0-21swapper/922:55:3211
1714399181cyclictest0-21swapper/922:55:3211
1714399181cyclictest0-21swapper/922:40:2511
1714399181cyclictest0-21swapper/922:35:2811
1714399181cyclictest0-21swapper/922:30:1511
1714399181cyclictest0-21swapper/922:25:1011
1714399181cyclictest0-21swapper/922:20:1511
1714399181cyclictest0-21swapper/922:15:1811
1714399181cyclictest0-21swapper/922:05:1311
1714399181cyclictest0-21swapper/921:55:2811
1714399181cyclictest0-21swapper/921:50:0311
1714399181cyclictest0-21swapper/921:35:2711
1714399181cyclictest0-21swapper/921:25:2211
1714399181cyclictest0-21swapper/921:20:2111
1714399181cyclictest0-21swapper/921:10:1011
1714399181cyclictest0-21swapper/921:05:0211
1714399181cyclictest0-21swapper/921:00:1311
1714399181cyclictest0-21swapper/920:56:0411
1714399181cyclictest0-21swapper/920:45:1911
1714399181cyclictest0-21swapper/920:40:3111
1714399181cyclictest0-21swapper/920:35:1411
1714399181cyclictest0-21swapper/920:15:1211
1714399181cyclictest0-21swapper/920:10:0211
1714399181cyclictest0-21swapper/919:45:0911
1714399181cyclictest0-21swapper/919:40:0311
1714399181cyclictest0-21swapper/919:30:0411
1714399181cyclictest0-21swapper/919:25:0111
1714399181cyclictest0-21swapper/900:35:0711
1714399181cyclictest0-21swapper/900:30:4011
1714399181cyclictest0-21swapper/900:20:3311
1714399181cyclictest0-21swapper/900:15:2611
1714399181cyclictest0-21swapper/900:05:3811
1714399181cyclictest0-21swapper/900:00:2111
17136991816cyclictest9960-21snmp_rack3slot923:21:5010
17136991816cyclictest9960-21snmp_rack3slot923:21:5010
17136991816cyclictest9056-21snmp_rack3slot920:25:1310
17136991816cyclictest9056-21snmp_rack3slot920:25:1310
17136991816cyclictest8644-21perl00:05:1510
17136991816cyclictest8541-21snmp_rack3slot921:06:4210
17136991816cyclictest5398-21iostat00:00:2010
17136991816cyclictest4104-21iostat_ios21:45:2310
17136991816cyclictest31063-21snmp_rack3slot919:25:1810
17136991816cyclictest3001-21snmp_rack3slot923:56:0410
17136991816cyclictest28465-21snmp_rack3slot921:35:1610
17136991816cyclictest27012-21snmp_rack3slot919:20:1510
17136991816cyclictest24016-21perl20:00:1810
17136991816cyclictest22819-21perl23:40:1810
17136991816cyclictest21600-21snmp_rack3slot919:15:1410
17136991816cyclictest20860-21snmp_rack3slot923:36:4010
17136991816cyclictest17652-21snmp_easybox.os19:10:1210
17136991816cyclictest17610-21snmp_rack3slot900:16:2710
17136991816cyclictest14571-21snmp_rack3slot923:30:1410
17136991816cyclictest13074-21perl19:45:1810
17136991816cyclictest10233-21diskstats21:10:1410
1710799181cyclictest0-21swapper/220:30:214
1710799181cyclictest0-21swapper/220:30:214
1714799175cyclictest0-21swapper/1019:10:202
17147991716cyclictest27249-1kworker/10:0H19:46:282
17147991711cyclictest31098-21python23:50:262
17147991711cyclictest28808-21python20:05:312
17147991711cyclictest11648-21smartctl22:40:172
1714799170cyclictest99032chrt23:21:092
1714799170cyclictest99032chrt23:21:092
1714799170cyclictest89782sleep1021:51:122
1714799170cyclictest81662sleep1020:22:372
1714799170cyclictest66412sleep1022:35:052
1714799170cyclictest6392sleep1020:10:482
1714799170cyclictest63022chrt23:16:332
1714799170cyclictest58172chrt22:30:402
1714799170cyclictest48632chrt21:01:052
1714799170cyclictest45432chrt00:00:142
1714799170cyclictest41092chrt21:45:232
1714799170cyclictest321912chrt00:37:062
1714799170cyclictest309692sleep1022:21:522
1714799170cyclictest30912sleep1023:57:142
1714799170cyclictest3066350irq/52-eth0-rx-23:36:352
1714799170cyclictest3066350irq/52-eth0-rx-23:26:202
1714799170cyclictest3066350irq/52-eth0-rx-20:50:152
1714799170cyclictest3066350irq/52-eth0-rx-19:25:222
1714799170cyclictest285252chrt23:05:082
1714799170cyclictest277522chrt23:01:322
1714799170cyclictest275772sleep1021:35:092
1714799170cyclictest262842sleep1020:46:282
1714799170cyclictest261472sleep1019:20:062
1714799170cyclictest259452sleep1020:03:122
1714799170cyclictest254442sleep1019:17:302
1714799170cyclictest252582sleep1023:45:112
1714799170cyclictest248762sleep1000:26:282
1714799170cyclictest245312sleep1023:42:122
1714799170cyclictest241372sleep1022:57:192
1714799170cyclictest241372sleep1022:57:192
1714799170cyclictest233542sleep1021:29:092
1714799170cyclictest228052sleep1020:43:402
1714799170cyclictest221342sleep1019:56:232
1714799170cyclictest21832sleep1022:26:512
1714799170cyclictest197992sleep1022:50:302
1714799170cyclictest195892sleep1021:22:192
1714799170cyclictest190412sleep1020:36:512
1714799170cyclictest184602sleep1019:50:462
1714799170cyclictest183872sleep1000:20:052
1714799170cyclictest170882sleep1022:05:022
1714799170cyclictest163542sleep1022:02:002
1714799170cyclictest15972sleep1021:40:592
1714799170cyclictest139422sleep1000:11:152
1714799170cyclictest124952sleep1020:30:032
1714799170cyclictest124952sleep1020:30:032
1714799170cyclictest122032sleep1021:11:082
1714799170cyclictest12250irq/42-ahci22:15:412
1714799170cyclictest12250irq/42-ahci19:30:542
1714799170cyclictest10762sleep1020:56:052
1714799170cyclictest105182sleep1000:09:032
1714799170cyclictest0-21swapper/1023:30:312
17136991715cyclictest6308-21perl20:20:1610
17136991715cyclictest3305-21sensors_temp19:30:3610
17136991715cyclictest31162-21df_inode20:10:1810
17136991715cyclictest30538-21perl00:35:1810
17136991715cyclictest29947-21snmp_rack3slot920:51:4010
17136991715cyclictest27811-21perl20:05:1710
17136991715cyclictest25718-21snmp_easybox.os00:30:1310
17136991715cyclictest24134-21snmp_rack3slot921:30:1510
17136991715cyclictest2400-21diskstats20:15:1710
17136991715cyclictest23163-21perl00:25:1710
17136991715cyclictest22814-21snmp_easybox.os22:10:2510
17136991715cyclictest21485-21snmp_rack3slot921:25:1810
17136991715cyclictest20657-21df_inode20:40:1710
17136991715cyclictest18545-21snmp_rack3slot919:51:5210
17136991715cyclictest17765-21iostat_ios20:35:2210
17136991715cyclictest14265-21snmp_rack3slot921:15:1610
17136991715cyclictest11885-21iostat22:40:2110
17129991715cyclictest20134-21qemu-kvm22:24:278
17129991715cyclictest20134-21qemu-kvm22:08:528
17107991715cyclictest2406-21snmpd21:09:064
1709199173cyclictest0-21swapper/022:23:570
17091991716cyclictest0-21swapper/021:28:020
1714799160cyclictest287542sleep1000:34:552
17136991614cyclictest4772-21users21:00:3410
17136991614cyclictest4185-21snmp_rack3slot923:15:1510
17136991614cyclictest3080-21snmp_easybox.os22:30:1310
17136991614cyclictest28898-21snmp_easybox.os23:50:1110
17136991614cyclictest22121-21snmp_rack3slot919:56:1310
17136991614cyclictest19249-21df_inode00:20:1510
17136991614cyclictest14517-21snmp_easybox.os20:30:2510
17136991614cyclictest14517-21snmp_easybox.os20:30:2410
17136991614cyclictest13574-21snmp_rack3slot923:26:3910
17136991614cyclictest13494-21snmp_easybox.os22:00:1510
17136991614cyclictest1086-21snmp_rack3slot920:56:1410
17129991615cyclictest20134-21qemu-kvm22:19:268
17129991614cyclictest20133-21qemu-kvm22:35:158
17119991614cyclictest23414-21sensors22:55:276
17119991614cyclictest23414-21sensors22:55:276
17107991610cyclictest0-21swapper/223:56:474
17098991615cyclictest3066350irq/52-eth0-rx-21:05:141
17136991513cyclictest7513-21diskstats22:35:1610
17136991513cyclictest29760-21perl23:05:1610
17136991513cyclictest18840-21fw_packets22:50:1610
17136991513cyclictest10640-21df_inode21:55:1410
17129991513cyclictest20134-21qemu-kvm22:29:308
17129991513cyclictest20133-21qemu-kvm22:10:168
17119991513cyclictest13306-21sensors00:10:286
1710799153cyclictest0-21swapper/200:14:194
17098991514cyclictest3066350irq/52-eth0-rx-22:35:331
17098991513cyclictest29281-21python20:50:301
17091991513cyclictest0-21swapper/022:41:000
17091991513cyclictest0-21swapper/021:08:220
1713699144cyclictest0-21swapper/823:45:0110
1713699144cyclictest0-21swapper/820:50:0210
17136991412cyclictest29684-21iostat_ios22:20:2110
17136991412cyclictest13901-21snmp_rack3slot900:10:4010
1713499144cyclictest0-21swapper/700:15:019
1713499143cyclictest0-21swapper/721:20:029
1712999144cyclictest0-21swapper/621:23:078
1712999142cyclictest0-21swapper/621:50:228
1710799145cyclictest0-21swapper/220:47:374
17107991413cyclictest2406-21snmpd19:28:514
17107991413cyclictest1948-21python23:10:374
17107991412cyclictest2406-21snmpd20:41:064
4599130watchdog/50-21swapper/519:41:197
3199130watchdog/30-21swapper/320:43:145
31130ksoftirqd/00-21swapper/020:10:360
1713699133cyclictest0-21swapper/822:57:0710
1713699133cyclictest0-21swapper/822:57:0710
1713699133cyclictest0-21swapper/822:29:1410
1713699133cyclictest0-21swapper/821:20:3510
17136991311cyclictest5500-21df19:35:1510
1713499135cyclictest0-21swapper/719:10:019
1713499133cyclictest0-21swapper/722:55:019
1713499133cyclictest0-21swapper/722:55:019
1713499133cyclictest0-21swapper/721:55:229
1713499133cyclictest0-21swapper/720:05:209
1713499133cyclictest0-21swapper/700:30:239
1712999134cyclictest0-21swapper/623:00:238
17129991312cyclictest16849-21qemu-kvm19:27:168
1710799135cyclictest0-21swapper/221:53:434
1710799133cyclictest0-21swapper/223:49:454
1710799133cyclictest0-21swapper/223:29:154
1710799133cyclictest0-21swapper/223:04:434
1710799133cyclictest0-21swapper/221:36:584
1710799133cyclictest0-21swapper/221:20:094
1710799133cyclictest0-21swapper/220:04:574
1710799133cyclictest0-21swapper/219:43:454
17107991312cyclictest2406-21snmpd00:16:084
1710799131cyclictest0-21swapper/223:43:594
1710799131cyclictest0-21swapper/221:18:414
17091991311cyclictest2233-21taskset22:27:290
31120ksoftirqd/00-21swapper/023:00:310
31120ksoftirqd/00-21swapper/000:00:270
1713699122cyclictest0-21swapper/823:00:2110
1713499124cyclictest0-21swapper/721:10:019
1713499123cyclictest0-21swapper/722:15:019
1713499122cyclictest0-21swapper/722:03:099
1713499122cyclictest0-21swapper/719:34:049
1712999124cyclictest0-21swapper/623:10:018
1712999123cyclictest0-21swapper/623:40:008
17129991210cyclictest29623-21qemu-kvm20:08:248
17129991210cyclictest0-21swapper/621:45:198
1711999122cyclictest0-21swapper/423:25:186
17119991210cyclictest24217-21sensors00:25:276
1710799125cyclictest0-21swapper/200:05:214
1710799122cyclictest0-21swapper/222:00:434
1710799122cyclictest0-21swapper/221:43:494
1710799122cyclictest0-21swapper/220:35:204
1710799122cyclictest0-21swapper/219:55:534
1710799120cyclictest0-21swapper/223:18:454
1710799120cyclictest0-21swapper/222:42:094
1710799120cyclictest0-21swapper/222:16:334
1710799120cyclictest0-21swapper/220:59:174
1710799120cyclictest0-21swapper/219:19:244
1710799120cyclictest0-21swapper/219:13:034
1710799120cyclictest0-21swapper/200:31:534
1709899122cyclictest0-21swapper/122:55:191
1709899122cyclictest0-21swapper/122:55:191
17098991211cyclictest3066350irq/52-eth0-rx-00:02:201
1709899120cyclictest0-21swapper/123:05:141
1709199123cyclictest0-21swapper/020:09:050
1709199122cyclictest0-21swapper/022:05:200
17091991211cyclictest238782chrt22:14:280
1709199120cyclictest0-21swapper/019:15:380
481110ksoftirqd/50-21swapper/520:30:277
481110ksoftirqd/50-21swapper/520:30:267
411110ksoftirqd/40-21swapper/423:35:336
411110ksoftirqd/40-21swapper/422:30:166
31110ksoftirqd/00-21swapper/023:35:330
271110ksoftirqd/20-21swapper/223:35:334
1713699119cyclictest32332-21perl21:40:1710
1713699119cyclictest16694-21users22:45:3410
1713699114cyclictest0-21swapper/821:54:5910
1713499119cyclictest30367-21perl00:35:179
1713499115cyclictest0-21swapper/720:22:459
1713499113cyclictest31126-21chrt22:24:009
1713499113cyclictest0-21swapper/723:09:129
1713499113cyclictest0-21swapper/723:00:389
1713499113cyclictest0-21swapper/722:39:079
1713499113cyclictest0-21swapper/722:30:409
1713499113cyclictest0-21swapper/722:05:309
1713499113cyclictest0-21swapper/720:46:529
1713499113cyclictest0-21swapper/719:50:019
1713499113cyclictest0-21swapper/719:41:049
1713499113cyclictest0-21swapper/719:35:349
1713499112cyclictest0-21swapper/721:00:519
1713499112cyclictest0-21swapper/719:20:359
1713499111cyclictest0-21swapper/723:50:019
1713499111cyclictest0-21swapper/723:19:129
1712999115cyclictest0-21swapper/623:05:218
1712999114cyclictest0-21swapper/621:39:228
1712999113cyclictest0-21swapper/623:24:428
1712999113cyclictest0-21swapper/623:24:428
1712999113cyclictest0-21swapper/621:58:098
1712999113cyclictest0-21swapper/621:35:008
1712999113cyclictest0-21swapper/621:10:068
1712999112cyclictest0-21swapper/621:15:028
1712999112cyclictest0-21swapper/621:00:368
1712999110cyclictest24357-21idleruntime-cro23:00:018
1712999110cyclictest24357-21idleruntime-cro23:00:018
1712599110cyclictest0-21swapper/520:15:357
1712599110cyclictest0-21swapper/519:10:137
1712599110cyclictest0-21swapper/500:35:267
1711999113cyclictest0-21swapper/419:45:216
1711999113cyclictest0-21swapper/419:15:306
17119991110cyclictest27849-21sensors00:30:276
1711299113cyclictest0-21swapper/321:10:185
1711299110cyclictest0-21swapper/322:25:315
1710799114cyclictest0-21swapper/222:35:524
1710799113cyclictest0-21swapper/223:34:434
1710799113cyclictest0-21swapper/223:05:494
1710799113cyclictest0-21swapper/222:27:054
1710799113cyclictest0-21swapper/222:06:114
1710799113cyclictest0-21swapper/221:01:114
1710799113cyclictest0-21swapper/220:50:064
1710799113cyclictest0-21swapper/220:26:534
1710799113cyclictest0-21swapper/220:26:534
1710799113cyclictest0-21swapper/219:49:294
1710799113cyclictest0-21swapper/219:35:234
1710799113cyclictest0-21swapper/200:36:394
1710799113cyclictest0-21swapper/200:25:374
1710799113cyclictest0-21swapper/200:22:074
1710799112cyclictest0-21swapper/200:01:214
1710799111cyclictest0-21swapper/223:53:394
1710799111cyclictest0-21swapper/222:47:484
1710799111cyclictest0-21swapper/222:34:474
1710799111cyclictest0-21swapper/222:20:134
1710799111cyclictest0-21swapper/222:11:154
1710799111cyclictest0-21swapper/221:55:534
1710799111cyclictest0-21swapper/221:46:284
1710799111cyclictest0-21swapper/221:30:114
1710799111cyclictest0-21swapper/221:28:254
1710799111cyclictest0-21swapper/221:12:064
1710799111cyclictest0-21swapper/220:17:034
1710799111cyclictest0-21swapper/220:10:244
1710799111cyclictest0-21swapper/220:09:374
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional