You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 13:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Thu Nov 20, 2025 00:46:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501140irq/42-ahci0-21swapper/219:08:264
122501140irq/42-ahci0-21swapper/219:08:254
14915210995sleep80-21swapper/819:05:3810
14915210995sleep80-21swapper/819:05:3710
122501050irq/42-ahci0-21swapper/119:08:171
122501050irq/42-ahci0-21swapper/119:08:161
12250960irq/42-ahci0-21swapper/419:05:126
12250960irq/42-ahci0-21swapper/419:05:126
1526029489sleep30-21swapper/319:09:345
1526029489sleep30-21swapper/319:09:335
1526428985sleep70-21swapper/719:09:379
1526428985sleep70-21swapper/719:09:379
15407998727cyclictest0-21swapper/019:15:240
3066350850irq/52-eth0-rx-0-21swapper/1119:05:233
3066350850irq/52-eth0-rx-0-21swapper/1119:05:223
182050740irq/55-eth1-rx-0-21swapper/919:09:1811
182050740irq/55-eth1-rx-0-21swapper/919:09:1811
15407997228cyclictest0-21swapper/019:14:320
1499926653sleep00-21swapper/019:06:410
1499926653sleep00-21swapper/019:06:410
13450650irq/18-ehci_hcd0-21swapper/519:05:307
13450650irq/18-ehci_hcd0-21swapper/519:05:307
1502426354sleep100-21swapper/1019:07:042
1502426354sleep100-21swapper/1019:07:042
1495626353sleep60-21swapper/619:06:128
1495626353sleep60-21swapper/619:06:118
15445995028cyclictest0-21swapper/1119:15:463
15445995028cyclictest0-21swapper/1119:11:443
1544599340cyclictest0-21swapper/1122:11:163
1540799290cyclictest2691-21runrttasks20:22:360
15407992612cyclictest0-21swapper/021:12:020
15407992019cyclictest0-21swapper/020:10:310
1540799200cyclictest0-21swapper/022:10:290
15445991918cyclictest0-21swapper/1120:10:363
1544399181cyclictest4412-1kworker/9:2H23:30:3011
1544399181cyclictest0-21swapper/923:00:2611
1544399181cyclictest0-21swapper/921:40:3311
1544399181cyclictest0-21swapper/919:56:0711
15442991816cyclictest7660-21snmp_rack3slot920:20:1710
15442991816cyclictest5282-21snmp_rack3slot921:45:1310
15442991816cyclictest4151-21users23:10:3010
15442991816cyclictest4151-21users23:10:3010
15442991816cyclictest31559-21snmp_rack3slot920:52:0410
15442991816cyclictest24456-21df_inode00:25:1410
15442991816cyclictest23499-21snmp_rack3slot922:10:1210
15442991816cyclictest19978-21snmp_easybox.os00:20:1210
15442991816cyclictest18767-21snmp_rack3slot923:31:1310
15442991816cyclictest15103-21snmp_rack3slot923:26:0410
15442991816cyclictest1220-21snmp_rack3slot900:35:4910
15442991816cyclictest11529-21snmp_rack3slot921:55:1110
15442991816cyclictest10170-21perl00:05:1610
15423991816cyclictest2406-21snmpd21:48:084
15418991816cyclictest27738-21python21:30:271
1544599170cyclictest0-21swapper/1122:18:323
15444991716cyclictest811rcuc/1000:37:512
15444991716cyclictest27249-1kworker/10:0H21:55:212
15444991716cyclictest0-21swapper/1023:49:212
15444991716cyclictest0-21swapper/1021:15:132
15444991715cyclictest24593-21python22:10:232
15443991716cyclictest10392-21chrt21:09:3911
1544399170cyclictest761ksoftirqd/923:05:2711
1544399170cyclictest761ksoftirqd/923:05:2711
1544399170cyclictest761ksoftirqd/920:01:4411
1544399170cyclictest761ksoftirqd/920:01:4411
1544399170cyclictest761ksoftirqd/919:42:0611
1544399170cyclictest761ksoftirqd/900:15:1511
1544399170cyclictest4412-1kworker/9:2H21:23:0311
15442991715cyclictest3624-21snmp_easybox.os21:00:1210
15442991715cyclictest31278-21irqstats22:20:1910
15442991715cyclictest31213-21perl23:05:1410
15442991715cyclictest31213-21perl23:05:1310
15442991715cyclictest30218-21snmp_easybox.os20:05:2410
15442991715cyclictest28813-21yum22:15:3410
15442991715cyclictest2814-21snmp_rack3slot920:57:0010
15442991715cyclictest27659-21munin-run20:05:0110
15442991715cyclictest27659-21munin-run20:05:0010
15442991715cyclictest27304-21snmp_easybox.os00:30:1510
15442991715cyclictest26348-21snmp_rack3slot920:45:1510
15442991715cyclictest25282-21snmp_easybox.os23:40:2110
15442991715cyclictest2502-21iostat22:25:1710
15442991715cyclictest23958-21perl22:55:1710
15442991715cyclictest23958-21perl22:55:1610
15442991715cyclictest23918-21ntp_states21:25:2810
15442991715cyclictest20424-21diskstats23:35:1210
15442991715cyclictest1959-21iostat_ios21:40:1710
15442991715cyclictest19353-21snmp_rack3slot921:20:1310
15442991715cyclictest17929-21iostat_ios00:15:1910
15442991715cyclictest16967-21snmp_rack3slot919:15:1310
15442991715cyclictest16865-21snmp_rack3slot922:45:1510
15442991715cyclictest15828-21munin-run00:15:0010
15442991715cyclictest15474-21snmp_rack3slot921:15:1410
15442991715cyclictest15221-21perl20:30:1710
15442991715cyclictest1466-21snmp_easybox.os20:10:2610
15442991715cyclictest12062-21perl21:10:1610
15442991715cyclictest11584-21perl20:25:1710
15442991715cyclictest11504-21snmp_rack3slot923:21:2310
15442991715cyclictest11504-21snmp_rack3slot923:21:2210
15442991715cyclictest10533-21snmp_rack3slot921:50:4910
15442991715cyclictest10033-21yum21:05:3310
15432991715cyclictest21710-21qemu-kvm22:11:467
15432991715cyclictest21709-21qemu-kvm22:08:427
15423991715cyclictest2406-21snmpd23:35:094
15423991715cyclictest2406-21snmpd22:23:084
15423991715cyclictest2406-21snmpd19:36:524
15423991715cyclictest2406-21snmpd00:29:234
15445991614cyclictest0-21swapper/1119:20:283
15444991615cyclictest60832sleep1020:17:252
15444991615cyclictest29348-21kworker/10:120:10:172
15444991615cyclictest28237-21python22:15:282
15444991615cyclictest27249-1kworker/10:0H20:47:342
15443991615cyclictest8519-21taskset00:04:4511
15443991615cyclictest4412-1kworker/9:2H23:28:5911
15443991615cyclictest24231-21chrt20:41:3711
15443991615cyclictest0-21swapper/922:04:5611
15443991615cyclictest0-21swapper/921:30:1411
15443991615cyclictest0-21swapper/919:15:1611
1544299164cyclictest0-21swapper/822:30:0010
15442991614cyclictest8289-21snmp_easybox.os22:35:1210
15442991614cyclictest820-21snmp_rack3slot923:50:5010
15442991614cyclictest7884-21snmp_rack3slot923:16:3710
15442991614cyclictest3341-21snmp_rack3slot920:15:1310
15442991614cyclictest30019-21diskstats21:35:1310
15442991614cyclictest25579-21snmp_rack3slot919:56:0810
15442991614cyclictest24224-21snmp_rack3slot919:50:4610
15442991614cyclictest24224-21snmp_rack3slot919:50:4510
15442991614cyclictest21962-21snmp_rack3slot922:50:4710
15442991614cyclictest20998-21munin-run19:35:0010
15442991614cyclictest16214-21perl22:00:1710
1544199167cyclictest0-21swapper/723:18:599
15432991615cyclictest21710-21qemu-kvm22:28:407
15432991615cyclictest2033-21cpuspeed_turbos23:55:127
15423991614cyclictest2406-21snmpd23:25:074
15423991614cyclictest2406-21snmpd20:21:524
15407991614cyclictest0-21swapper/020:47:310
1540799160cyclictest0-21swapper/021:30:000
15444991514cyclictest27249-1kworker/10:0H23:52:542
15444991514cyclictest27249-1kworker/10:0H22:53:462
15444991514cyclictest27249-1kworker/10:0H21:50:342
15444991513cyclictest24099-21python21:25:302
15444991513cyclictest17237-21python22:00:292
1544499151cyclictest12250irq/42-ahci19:30:392
1544499150cyclictest83942sleep1000:03:022
1544499150cyclictest325222sleep1022:21:262
1544499150cyclictest262762sleep1023:44:352
1544499150cyclictest163312sleep1000:15:022
1544499150cyclictest12250irq/42-ahci23:01:022
1544499150cyclictest12250irq/42-ahci22:44:012
1544499150cyclictest12250irq/42-ahci22:08:542
1544499150cyclictest12250irq/42-ahci20:34:262
1544499150cyclictest12250irq/42-ahci20:08:362
1544499150cyclictest12250irq/42-ahci19:47:282
1544499150cyclictest12250irq/42-ahci19:47:282
1544499150cyclictest12250irq/42-ahci00:11:562
15443991514cyclictest492-21cat23:50:3111
15442991514cyclictest22724-21snmp_rack3slot920:40:1810
15442991513cyclictest6979-21iostat00:00:1910
15442991513cyclictest27692-21diskstats23:45:1810
15442991513cyclictest2582-21df_inode23:55:1510
1544199155cyclictest0-21swapper/723:30:019
1544199155cyclictest0-21swapper/720:15:019
15432991514cyclictest30940-21cstates23:50:157
15432991513cyclictest21710-21qemu-kvm22:47:327
15431991514cyclictest11267-21sensors00:05:276
15423991513cyclictest2406-21snmpd22:42:074
15407991513cyclictest0-21swapper/019:28:550
13450150irq/18-ehci_hcd0-21swapper/020:55:250
1544499140cyclictest91082sleep1022:35:142
1544499140cyclictest80762sleep1023:19:192
1544499140cyclictest64702sleep1021:01:282
1544499140cyclictest5292sleep1023:06:542
1544499140cyclictest5292sleep1023:06:542
1544499140cyclictest45822chrt22:30:022
1544499140cyclictest40742sleep1021:45:082
1544499140cyclictest37912sleep1022:26:132
1544499140cyclictest319662sleep1021:35:452
1544499140cyclictest314792sleep1020:51:022
1544499140cyclictest3066350irq/52-eth0-rx-23:27:252
1544499140cyclictest3066350irq/52-eth0-rx-23:10:242
1544499140cyclictest3066350irq/52-eth0-rx-23:10:232
1544499140cyclictest3066350irq/52-eth0-rx-20:20:182
1544499140cyclictest3066350irq/52-eth0-rx-00:05:122
1544499140cyclictest300922sleep1000:31:282
1544499140cyclictest284082sleep1021:31:552
1544499140cyclictest275862chrt20:04:102
1544499140cyclictest275862chrt20:04:102
1544499140cyclictest27242chrt20:55:522
1544499140cyclictest255532sleep1019:55:442
1544499140cyclictest250032sleep1022:55:292
1544499140cyclictest250032sleep1022:55:292
1544499140cyclictest243242sleep1019:52:092
1544499140cyclictest243242sleep1019:52:092
1544499140cyclictest241592sleep1020:40:372
1544499140cyclictest235912sleep1000:25:032
1544499140cyclictest224232sleep1023:36:332
1544499140cyclictest224122sleep1019:40:432
1544499140cyclictest221782sleep1000:20:272
1544499140cyclictest216862sleep1019:37:442
1544499140cyclictest21642sleep1021:40:212
1544499140cyclictest212622chrt21:23:422
1544499140cyclictest206532sleep1020:37:242
1544499140cyclictest197862sleep1019:25:192
1544499140cyclictest191632sleep1019:23:432
1544499140cyclictest187332chrt23:30:452
1544499140cyclictest177352chrt22:45:282
1544499140cyclictest158982sleep1019:10:152
1544499140cyclictest134782sleep1020:29:012
1544499140cyclictest12250irq/42-ahci23:20:212
1544499140cyclictest12250irq/42-ahci23:20:202
1544499140cyclictest121482sleep1021:10:182
1544499140cyclictest100692chrt21:06:032
1544399145cyclictest0-21swapper/900:07:0911
15443991413cyclictest4412-1kworker/9:2H22:08:4011
15443991413cyclictest0-21swapper/923:40:2211
15443991413cyclictest0-21swapper/921:45:2811
15443991413cyclictest0-21swapper/919:14:0811
1544399141cyclictest0-21swapper/920:45:1511
1544399141cyclictest0-21swapper/920:15:1311
15442991412cyclictest26754-21perl21:30:1410
15442991412cyclictest20271-21iostat_ios22:05:1910
15442991412cyclictest20135-21munin-run19:30:0010
15442991412cyclictest13030-21perl22:40:1810
1544199143cyclictest0-21swapper/723:50:009
15432991413cyclictest30570-21cstates23:05:127
15432991413cyclictest30570-21cstates23:05:127
15423991412cyclictest2406-21snmpd00:33:244
15423991412cyclictest2406-21snmpd00:08:234
15418991413cyclictest3066350irq/52-eth0-rx-22:45:141
15418991413cyclictest3066350irq/52-eth0-rx-22:43:521
15418991413cyclictest3066350irq/52-eth0-rx-22:36:491
15407991413cyclictest290732sleep022:19:150
13450140irq/18-ehci_hcd0-21swapper/000:15:250
31130ksoftirqd/00-21swapper/022:00:270
31130ksoftirqd/00-21swapper/021:20:250
31130ksoftirqd/00-21swapper/020:15:270
1544599134cyclictest0-21swapper/1123:25:013
1544599134cyclictest0-21swapper/1123:15:013
1544599134cyclictest0-21swapper/1123:15:003
1544599134cyclictest0-21swapper/1123:00:023
1544599134cyclictest0-21swapper/1122:50:003
1544599134cyclictest0-21swapper/1122:45:003
1544599134cyclictest0-21swapper/1120:05:013
1544599133cyclictest0-21swapper/1120:00:023
1544599133cyclictest0-21swapper/1120:00:023
15445991311cyclictest24855-21chrt21:28:073
1544499133cyclictest0-21swapper/1019:18:212
1544499130cyclictest47012sleep1023:57:352
15443991312cyclictest4412-1kworker/9:2H20:12:4111
15443991312cyclictest29381-21ls20:50:1511
15443991312cyclictest0-21swapper/921:55:3111
15443991312cyclictest0-21swapper/921:29:4911
1544299133cyclictest0-21swapper/819:22:0610
15442991311cyclictest28527-21snmp_easybox.os23:00:2510
1544199134cyclictest0-21swapper/719:45:019
1544199134cyclictest0-21swapper/719:45:019
1544199134cyclictest0-21swapper/719:10:019
1544199133cyclictest0-21swapper/722:21:379
1544199133cyclictest0-21swapper/722:16:349
1544199133cyclictest0-21swapper/719:26:219
1544199133cyclictest0-21swapper/700:18:049
1543899133cyclictest0-21swapper/623:56:548
1543899133cyclictest0-21swapper/623:39:358
1543899133cyclictest0-21swapper/623:34:058
1543899133cyclictest0-21swapper/623:16:358
1543899133cyclictest0-21swapper/623:11:158
1543899133cyclictest0-21swapper/623:11:148
1543899133cyclictest0-21swapper/622:51:088
1543899133cyclictest0-21swapper/622:06:258
1543899133cyclictest0-21swapper/622:00:368
1543899133cyclictest0-21swapper/620:50:538
1543899133cyclictest0-21swapper/620:45:008
1543899133cyclictest0-21swapper/620:06:078
1543199133cyclictest0-21swapper/423:20:016
15431991312cyclictest5366-21sensors20:15:266
15423991311cyclictest2406-21snmpd19:42:354
1541899133cyclictest0-21swapper/123:50:201
1541899133cyclictest0-21swapper/123:05:171
1541899133cyclictest0-21swapper/123:05:161
15418991312cyclictest3066350irq/52-eth0-rx-23:55:301
15418991312cyclictest3066350irq/52-eth0-rx-23:32:201
15407991312cyclictest2377-21nscd21:19:200
15407991312cyclictest19137-21chrt19:23:190
14850130irq/18-uhci_hcd0-21swapper/000:00:250
5499120migration/67449-21munin-node00:00:238
1799120watchdog/10-21swapper/119:23:061
1544599123cyclictest0-21swapper/1121:05:013
1544599122cyclictest0-21swapper/1123:16:533
1544599122cyclictest0-21swapper/1119:56:183
1544599122cyclictest0-21swapper/1119:46:513
1544599122cyclictest0-21swapper/1119:46:503
1544599122cyclictest0-21swapper/1119:42:293
1544599122cyclictest0-21swapper/1100:39:353
15445991211cyclictest0-21swapper/1119:25:323
1544399123cyclictest0-21swapper/922:25:0111
1544399123cyclictest0-21swapper/920:25:0111
1544399122cyclictest0-21swapper/922:57:5311
1544399122cyclictest0-21swapper/922:57:5311
15443991211cyclictest741rcuc/922:50:3811
15443991211cyclictest0-21swapper/922:35:1211
1544299122cyclictest0-21swapper/819:42:3510
1544299122cyclictest0-21swapper/819:10:3610
1544199123cyclictest0-21swapper/700:30:019
1544199122cyclictest0-21swapper/722:27:349
1544199122cyclictest0-21swapper/720:21:229
1544199122cyclictest0-21swapper/719:17:369
15441991210cyclictest0-21swapper/721:35:119
1543899124cyclictest0-21swapper/623:45:018
1543899123cyclictest0-21swapper/600:20:018
1543899122cyclictest0-21swapper/623:00:158
1543899122cyclictest0-21swapper/622:55:388
1543899122cyclictest0-21swapper/622:55:378
1543899122cyclictest0-21swapper/620:34:008
1543899122cyclictest0-21swapper/620:02:348
1543899122cyclictest0-21swapper/620:02:338
1542399122cyclictest0-21swapper/200:00:014
1541899122cyclictest0-21swapper/122:10:151
15418991211cyclictest3066350irq/52-eth0-rx-22:29:551
15418991211cyclictest3066350irq/52-eth0-rx-20:30:051
7599110migration/910561-21munin-node21:51:1411
4599110watchdog/50-21swapper/500:19:137
182050110irq/55-eth1-rx-0-21swapper/323:55:265
182050110irq/55-eth1-rx-0-21swapper/322:30:275
182050110irq/55-eth1-rx-0-21swapper/321:55:265
182050110irq/55-eth1-rx-0-21swapper/300:25:255
1699110watchdog/00-21swapper/022:08:380
1699110watchdog/00-21swapper/000:30:350
1699110watchdog/00-21swapper/000:23:230
1699110watchdog/00-21swapper/000:12:550
1544599113cyclictest0-21swapper/1123:37:053
1544599113cyclictest0-21swapper/1122:30:593
1544599113cyclictest0-21swapper/1122:28:133
1544599113cyclictest0-21swapper/1121:39:503
1544599113cyclictest0-21swapper/1120:42:023
1544599113cyclictest0-21swapper/1120:36:193
1544599113cyclictest0-21swapper/1119:35:523
1544599113cyclictest0-21swapper/1100:20:303
1544599113cyclictest0-21swapper/1100:12:053
1544599112cyclictest0-21swapper/1122:45:203
1544599112cyclictest0-21swapper/1120:31:523
1544599112cyclictest0-21swapper/1100:32:243
1544599111cyclictest0-21swapper/1100:15:013
1544599110cyclictest0-21swapper/1122:08:413
1544399113cyclictest0-21swapper/923:48:4011
1544399113cyclictest0-21swapper/922:34:0511
1544399113cyclictest0-21swapper/921:00:0111
1544399113cyclictest0-21swapper/920:25:0211
1544399113cyclictest0-21swapper/920:09:5911
1544399113cyclictest0-21swapper/919:25:5911
1544399112cyclictest0-21swapper/920:30:2411
15443991110cyclictest3984-21taskset22:28:5211
15443991110cyclictest0-21swapper/922:10:3411
1544399111cyclictest0-21swapper/919:48:2011
1544399111cyclictest0-21swapper/919:48:2011
1544399110cyclictest0-21swapper/900:35:0011
1544299113cyclictest0-21swapper/820:35:3010
1544299113cyclictest0-21swapper/819:45:0610
1544299113cyclictest0-21swapper/819:45:0510
1544199115cyclictest0-21swapper/722:32:049
1544199113cyclictest0-21swapper/721:47:329
1544199113cyclictest0-21swapper/721:40:019
1544199113cyclictest0-21swapper/721:20:389
1544199113cyclictest0-21swapper/721:10:539
1544199113cyclictest0-21swapper/720:32:219
1544199113cyclictest0-21swapper/720:13:519
1544199113cyclictest0-21swapper/700:35:259
1543899119cyclictest0-21swapper/621:45:108
1543899113cyclictest0-21swapper/623:50:538
1543899113cyclictest0-21swapper/621:58:228
1543899113cyclictest0-21swapper/621:42:458
1543899113cyclictest0-21swapper/621:05:008
1543899112cyclictest0-21swapper/600:14:238
1543899111cyclictest0-21swapper/600:18:258
1543299113cyclictest0-21swapper/522:35:177
15432991110cyclictest4054-21cstates21:00:147
1543199113cyclictest0-21swapper/420:20:216
1543199113cyclictest0-21swapper/420:08:106
1543099113cyclictest0-21swapper/323:00:185
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional