You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 21:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Wed Nov 26, 2025 12:46:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30664501150irq/53-eth0-tx-0-21swapper/1007:06:372
23470210895sleep90-21swapper/907:06:2711
23529210692sleep60-21swapper/607:07:128
122501040irq/42-ahci0-21swapper/107:09:091
3066450950irq/53-eth0-tx-0-21swapper/207:08:264
2222029485sleep30-21swapper/307:05:215
3066350920irq/52-eth0-rx-0-21swapper/507:07:407
3066450820irq/53-eth0-tx-0-21swapper/407:05:306
2344526755sleep00-21swapper/007:06:060
2359826658sleep110-21swapper/1107:08:063
2354426354sleep70-21swapper/707:07:259
2350125853sleep80-21swapper/807:06:5010
2390599320cyclictest0-21swapper/007:25:160
23945992322cyclictest0-21swapper/1107:28:023
2393099192cyclictest0-21swapper/312:10:175
2393099192cyclictest0-21swapper/311:51:575
2393099192cyclictest0-21swapper/307:48:265
23945991817cyclictest7686-21taskset08:12:363
23944991817cyclictest0-21swapper/1011:29:502
23944991816cyclictest22438-21python10:00:262
2394499181cyclictest31372chrt12:34:042
2394499181cyclictest12250irq/42-ahci10:35:162
2394499181cyclictest12250irq/42-ahci08:46:362
2394499181cyclictest12250irq/42-ahci08:22:162
2394399181cyclictest0-21swapper/911:15:2211
2394399181cyclictest0-21swapper/908:26:3611
2394399181cyclictest0-21swapper/907:45:1511
23942991816cyclictest9498-21perl08:15:1310
23942991816cyclictest796-21iostat11:00:1910
23942991816cyclictest7950-21snmp_easybox.os07:30:1210
23942991816cyclictest5886-21snmp_rack3slot912:35:2410
23942991816cyclictest5197-21snmp_rack3slot908:10:1110
23942991816cyclictest31216-21snmp_rack3slot909:30:1210
23942991816cyclictest28407-21snmp_rack3slot909:25:1810
23942991816cyclictest24452-21snmp_easybox.os10:50:1110
23942991816cyclictest24257-21memory07:50:2110
23942991816cyclictest22314-21df12:15:1310
23942991816cyclictest15799-21snmp_rack3slot909:51:2510
23942991816cyclictest13453-21snmp_easybox.os10:35:1210
23942991816cyclictest13290-21snmp_rack3slot908:20:1810
23942991816cyclictest12627-21ntp_states12:00:2410
23942991816cyclictest12627-21ntp_states12:00:2410
2393399180cyclictest0-21swapper/607:28:028
2393099182cyclictest0-21swapper/312:36:225
2393099182cyclictest0-21swapper/312:30:085
2393099182cyclictest0-21swapper/312:26:145
2393099182cyclictest0-21swapper/312:20:535
2393099182cyclictest0-21swapper/312:15:455
2393099182cyclictest0-21swapper/312:07:005
2393099182cyclictest0-21swapper/312:01:315
2393099182cyclictest0-21swapper/312:01:305
2393099182cyclictest0-21swapper/311:55:535
2393099182cyclictest0-21swapper/311:55:525
2393099182cyclictest0-21swapper/311:46:475
2393099182cyclictest0-21swapper/311:40:465
2393099182cyclictest0-21swapper/311:38:025
2393099182cyclictest0-21swapper/311:32:125
2393099182cyclictest0-21swapper/311:21:035
2393099182cyclictest0-21swapper/311:15:455
2393099182cyclictest0-21swapper/311:10:075
2393099182cyclictest0-21swapper/311:05:525
2393099182cyclictest0-21swapper/311:01:205
2393099182cyclictest0-21swapper/310:57:515
2393099182cyclictest0-21swapper/310:50:455
2393099182cyclictest0-21swapper/310:45:055
2393099182cyclictest0-21swapper/310:42:175
2393099182cyclictest0-21swapper/310:36:585
2393099182cyclictest0-21swapper/310:33:025
2393099182cyclictest0-21swapper/310:29:385
2393099182cyclictest0-21swapper/310:20:015
2393099182cyclictest0-21swapper/310:18:025
2393099182cyclictest0-21swapper/310:06:345
2393099182cyclictest0-21swapper/310:00:315
2393099182cyclictest0-21swapper/309:55:085
2393099182cyclictest0-21swapper/309:54:255
2393099182cyclictest0-21swapper/309:46:265
2393099182cyclictest0-21swapper/309:35:345
2393099182cyclictest0-21swapper/309:30:535
2393099182cyclictest0-21swapper/309:28:135
2393099182cyclictest0-21swapper/309:20:435
2393099182cyclictest0-21swapper/309:20:425
2393099182cyclictest0-21swapper/309:16:565
2393099182cyclictest0-21swapper/309:16:565
2393099182cyclictest0-21swapper/309:11:395
2393099182cyclictest0-21swapper/309:05:095
2393099182cyclictest0-21swapper/309:02:015
2393099182cyclictest0-21swapper/308:55:585
2393099182cyclictest0-21swapper/308:50:095
2393099182cyclictest0-21swapper/308:46:565
2393099182cyclictest0-21swapper/308:40:575
2393099182cyclictest0-21swapper/308:37:225
2393099182cyclictest0-21swapper/308:31:255
2393099182cyclictest0-21swapper/308:27:105
2393099182cyclictest0-21swapper/308:20:445
2393099182cyclictest0-21swapper/308:19:185
2393099182cyclictest0-21swapper/308:12:115
2393099182cyclictest0-21swapper/307:57:065
2393099182cyclictest0-21swapper/307:52:105
2393099182cyclictest0-21swapper/307:44:435
2393099182cyclictest0-21swapper/307:36:245
2393099182cyclictest0-21swapper/307:32:085
2393099182cyclictest0-21swapper/307:27:565
2393099182cyclictest0-21swapper/307:23:105
2393099182cyclictest0-21swapper/307:15:435
2393099180cyclictest0-21swapper/311:25:105
2393099180cyclictest0-21swapper/309:42:305
23905991817cyclictest0-21swapper/009:14:040
23944991716cyclictest0-21swapper/1012:10:232
2394499170cyclictest97592sleep1011:55:372
2394499170cyclictest97592sleep1011:55:362
2394499170cyclictest94742sleep1011:11:312
2394499170cyclictest84352chrt09:40:342
2394499170cyclictest831ksoftirqd/1009:20:162
2394499170cyclictest831ksoftirqd/1009:20:152
2394499170cyclictest81182sleep1008:57:542
2394499170cyclictest75682sleep1008:10:582
2394499170cyclictest71302sleep1007:26:542
2394499170cyclictest62762sleep1011:52:362
2394499170cyclictest48222sleep1009:35:312
2394499170cyclictest37372sleep1008:50:272
2394499170cyclictest327652chrt08:03:212
2394499170cyclictest317442sleep1012:27:282
2394499170cyclictest31702sleep1010:20:142
2394499170cyclictest313042sleep1011:41:462
2394499170cyclictest3066350irq/52-eth0-rx-12:15:122
2394499170cyclictest3066350irq/52-eth0-rx-09:30:112
2394499170cyclictest3066350irq/52-eth0-rx-09:25:182
2394499170cyclictest3066350irq/52-eth0-rx-09:15:562
2394499170cyclictest3066350irq/52-eth0-rx-09:15:552
2394499170cyclictest3066350irq/52-eth0-rx-08:05:212
2394499170cyclictest3066350irq/52-eth0-rx-07:30:252
2394499170cyclictest3066350irq/52-eth0-rx-07:10:152
2394499170cyclictest303982sleep1010:10:362
2394499170cyclictest294342sleep1008:41:142
2394499170cyclictest281972sleep1012:23:392
2394499170cyclictest280162sleep1010:55:052
2394499170cyclictest273062sleep1010:52:162
2394499170cyclictest260932sleep1007:55:092
2394499170cyclictest259962sleep1008:39:012
2394499170cyclictest254672sleep1007:53:342
2394499170cyclictest247942sleep1011:35:092
2394499170cyclictest24382sleep1011:45:352
2394499170cyclictest239072sleep1010:05:122
2394499170cyclictest229452chrt10:45:282
2394499170cyclictest221082sleep1008:30:372
2394499170cyclictest22042chrt11:02:302
2394499170cyclictest203922sleep1007:45:192
2394499170cyclictest194892sleep1009:57:102
2394499170cyclictest193052sleep1010:40:282
2394499170cyclictest189852sleep1009:10:542
2394499170cyclictest181902sleep1007:43:552
2394499170cyclictest178632chrt08:25:242
2394499170cyclictest17282sleep1010:17:262
2394499170cyclictest171342sleep1012:06:382
2394499170cyclictest168602sleep1011:23:072
2394499170cyclictest157482sleep1009:50:462
2394499170cyclictest155442chrt09:08:402
2394499170cyclictest135462sleep1012:02:122
2394499170cyclictest135462sleep1012:02:122
2394499170cyclictest130722sleep1011:16:082
2394499170cyclictest126122sleep1010:30:392
2394499170cyclictest12250irq/42-ahci12:38:322
2394499170cyclictest12250irq/42-ahci11:06:352
2394499170cyclictest12250irq/42-ahci10:26:072
2394499170cyclictest12250irq/42-ahci09:47:302
2394499170cyclictest12250irq/42-ahci07:35:252
2394499170cyclictest116012sleep1009:00:402
2394499170cyclictest112102sleep1008:15:582
23943991716cyclictest30987-21taskset10:57:5611
23943991716cyclictest0-21swapper/911:00:1311
2394399170cyclictest0-21swapper/911:39:2211
2394399170cyclictest0-21swapper/909:10:2311
23942991716cyclictest2875-21snmp_rack3slot912:30:4710
23942991715cyclictest8810-21sensors_temp11:10:2510
23942991715cyclictest7284-21irqstats09:40:2310
23942991715cyclictest6173-21snmp_easybox.os10:25:1110
23942991715cyclictest6021-21diskstats08:55:1410
23942991715cyclictest5498-21snmp_rack3slot907:25:1310
23942991715cyclictest31426-21snmp_rack3slot910:15:1310
23942991715cyclictest31184-21users11:40:3110
23942991715cyclictest30921-21snmp_rack3slot910:57:0110
23942991715cyclictest28900-21snmp_easybox.os07:15:1610
23942991715cyclictest28325-21snmp_rack3slot910:10:1310
23942991715cyclictest28047-21snmp_rack3slot912:21:3810
23942991715cyclictest26876-21snmp_easybox.os11:35:2410
23942991715cyclictest26818-21snmp_rack3slot910:07:1310
23942991715cyclictest2545-21snmp_rack3slot911:46:4910
23942991715cyclictest24847-21snmp_rack3slot907:10:1410
23942991715cyclictest24326-21df_inode09:20:1410
23942991715cyclictest24326-21df_inode09:20:1310
23942991715cyclictest21151-21snmp_easybox.os11:30:1010
23942991715cyclictest19647-21snmp_rack3slot907:45:1510
23942991715cyclictest19122-21irqstats11:25:2310
23942991715cyclictest18177-21iostat_ios09:55:2010
23942991715cyclictest17913-21snmp_easybox.os12:10:1110
23942991715cyclictest16763-21snmp_rack3slot911:21:4810
23942991715cyclictest1500-21snmp_easybox.os08:50:1110
23942991715cyclictest12682-21snmp_rack3slot907:35:1410
23942991715cyclictest11609-21snmp_rack3slot909:00:4610
23933991716cyclictest23652-21qemu-kvm07:17:488
23926991715cyclictest2406-21snmpd12:14:244
2390599170cyclictest0-21swapper/008:15:180
23944991615cyclictest0-21swapper/1007:20:282
2394399167cyclictest0-21swapper/912:11:4411
2394399167cyclictest0-21swapper/911:23:4911
2394399167cyclictest0-21swapper/908:55:5711
2394399167cyclictest0-21swapper/907:40:1811
23943991615cyclictest3066350irq/52-eth0-rx-07:17:0911
23943991615cyclictest30606-21taskset10:13:2711
23943991615cyclictest0-21swapper/912:15:2211
23943991615cyclictest0-21swapper/908:50:1411
23943991614cyclictest2760-21chrt11:49:3411
23943991614cyclictest12839-21chrt10:33:5111
23942991614cyclictest9855-21snmp_rack3slot911:56:5110
23942991614cyclictest9855-21snmp_rack3slot911:56:5010
23942991614cyclictest6139-21snmp_rack3slot911:50:4910
23942991614cyclictest3646-21perl10:20:1510
23942991614cyclictest3631-21iostat_ios09:35:1710
23942991614cyclictest30953-21snmp_rack3slot912:25:2410
23942991614cyclictest29968-21snmp_rack3slot908:00:1210
23942991614cyclictest23418-21snmp_rack3slot908:35:1210
23942991614cyclictest23049-21snmp_rack3slot910:00:5910
23942991614cyclictest20728-21snmp_easybox.os10:45:1110
23942991614cyclictest20169-21snmp_rack3slot908:30:1410
23942991614cyclictest17765-21snmp_easybox.os08:25:2310
23942991614cyclictest17088-21snmp_rack3slot912:06:0110
23942991614cyclictest16538-21snmp_rack3slot907:40:1610
23942991614cyclictest11964-21snmp_easybox.os10:30:2510
23942991614cyclictest10381-21snmp_rack3slot909:45:1610
2393399168cyclictest23863-21munin-run10:50:008
23933991615cyclictest15501-21qemu-kvm09:40:018
23932991615cyclictest30206-21cstates08:00:137
23932991615cyclictest17325-21cstates_1109:55:137
23932991615cyclictest11231-21timerandwakeup09:00:287
2391699161cyclictest0-21swapper/109:20:141
2391699161cyclictest0-21swapper/109:20:141
2390599160cyclictest0-21swapper/007:16:340
13450160irq/18-ehci_hcd0-21swapper/011:50:270
2394599153cyclictest0-21swapper/1110:28:533
23945991513cyclictest226442sleep1109:16:083
23945991513cyclictest226442sleep1109:16:073
23944991514cyclictest0-21swapper/1011:30:062
2394399157cyclictest0-21swapper/912:35:0311
2394399157cyclictest0-21swapper/912:31:0211
2394399157cyclictest0-21swapper/911:58:3311
2394399157cyclictest0-21swapper/911:58:3211
2394399157cyclictest0-21swapper/910:25:0311
2394399157cyclictest0-21swapper/909:42:2111
2394399157cyclictest0-21swapper/909:32:0111
2394399157cyclictest0-21swapper/909:17:1111
2394399157cyclictest0-21swapper/909:17:1011
2394399157cyclictest0-21swapper/909:05:1511
2394399157cyclictest0-21swapper/908:30:3011
2394399157cyclictest0-21swapper/908:20:1611
2394399157cyclictest0-21swapper/908:01:1411
2394399157cyclictest0-21swapper/907:55:5411
2394399157cyclictest0-21swapper/907:50:5711
2394399157cyclictest0-21swapper/907:11:2111
2394399156cyclictest0-21swapper/908:05:1611
2394399153cyclictest0-21swapper/907:25:0111
23943991514cyclictest0-21swapper/908:40:1811
23943991513cyclictest17673-21chrt10:40:1411
2394399150cyclictest0-21swapper/909:25:1811
23942991514cyclictest26625-21snmp_rack3slot907:55:1210
23942991513cyclictest5065-21snmp_rack3slot911:05:2310
23942991513cyclictest496-21snmp_easybox.os07:20:1310
23942991513cyclictest16235-21snmp_easybox.os09:10:1310
23942991513cyclictest11104-21df_inode11:15:1310
2393399158cyclictest20635-21munin-run11:30:018
23932991514cyclictest13914-21cstates10:35:127
2393199150cyclictest0-21swapper/409:40:176
2393099152cyclictest0-21swapper/310:14:275
2393099152cyclictest0-21swapper/308:09:175
2393099152cyclictest0-21swapper/308:00:025
2393099152cyclictest0-21swapper/307:12:195
23926991513cyclictest2406-21snmpd11:36:404
23926991513cyclictest2406-21snmpd10:48:534
23926991513cyclictest2406-21snmpd09:03:374
2392699150cyclictest0-21swapper/209:10:124
13450150irq/18-ehci_hcd0-21swapper/008:40:270
13450150irq/18-ehci_hcd0-21swapper/008:35:230
2394599145cyclictest0-21swapper/1112:35:013
2394599142cyclictest0-21swapper/1109:22:083
2394599142cyclictest0-21swapper/1109:22:083
23945991413cyclictest3608-21chrt07:23:313
2394399147cyclictest0-21swapper/912:26:1711
2394399147cyclictest0-21swapper/912:24:5611
2394399147cyclictest0-21swapper/912:02:0411
2394399147cyclictest0-21swapper/912:02:0411
2394399147cyclictest0-21swapper/911:43:4911
2394399147cyclictest0-21swapper/910:45:5711
2394399147cyclictest0-21swapper/910:36:0011
2394399147cyclictest0-21swapper/910:21:1511
2394399147cyclictest0-21swapper/909:54:5711
2394399147cyclictest0-21swapper/909:47:5711
2394399147cyclictest0-21swapper/909:20:0211
2394399147cyclictest0-21swapper/909:20:0211
2394399147cyclictest0-21swapper/909:00:1011
2394399141cyclictest0-21swapper/907:30:2611
2394399140cyclictest0-21swapper/911:32:1511
2394399140cyclictest0-21swapper/911:25:0811
2394399140cyclictest0-21swapper/911:13:2411
2394399140cyclictest0-21swapper/911:05:2711
2394399140cyclictest0-21swapper/910:50:2011
2394399140cyclictest0-21swapper/910:18:0411
2394399140cyclictest0-21swapper/910:07:3211
2394399140cyclictest0-21swapper/908:45:5511
2394399140cyclictest0-21swapper/908:20:0011
2394399140cyclictest0-21swapper/907:35:2011
23942991412cyclictest3931-21snmp_rack3slot908:06:5110
2394199144cyclictest0-21swapper/712:20:019
2393399143cyclictest0-21swapper/607:35:008
23932991413cyclictest31242-21timerandwakeup12:25:307
2391699143cyclictest0-21swapper/111:10:171
23905991413cyclictest76262sleep008:11:480
23905991413cyclictest2377-21nscd09:15:150
23905991413cyclictest2377-21nscd09:15:150
23905991413cyclictest15617-21taskset09:09:430
5299130watchdog/60-21swapper/612:21:358
2394599134cyclictest0-21swapper/1112:15:153
2394599134cyclictest0-21swapper/1110:45:003
2394599133cyclictest0-21swapper/1112:01:253
2394599133cyclictest0-21swapper/1112:01:243
2394599133cyclictest0-21swapper/1111:55:543
2394599133cyclictest0-21swapper/1111:55:543
2394599133cyclictest0-21swapper/1111:15:003
2394599133cyclictest0-21swapper/1109:41:363
2394599131cyclictest0-21swapper/1107:58:343
2394399137cyclictest0-21swapper/910:00:1411
2394399137cyclictest0-21swapper/907:22:5311
2394399136cyclictest0-21swapper/909:35:3611
2394399134cyclictest0-21swapper/910:00:0011
2394399131cyclictest0-21swapper/911:50:2111
2394299134cyclictest0-21swapper/808:45:5510
2394199134cyclictest0-21swapper/712:05:209
2394199134cyclictest0-21swapper/708:10:009
2394199134cyclictest0-21swapper/707:10:059
2394199133cyclictest0-21swapper/711:21:309
2394199133cyclictest0-21swapper/711:00:019
2394199133cyclictest0-21swapper/710:53:109
2394199133cyclictest0-21swapper/709:52:389
2394199133cyclictest0-21swapper/709:43:069
2394199133cyclictest0-21swapper/709:28:389
2394199133cyclictest0-21swapper/708:04:389
2394199133cyclictest0-21swapper/707:57:379
2393399134cyclictest0-21swapper/612:05:028
2393399134cyclictest0-21swapper/612:05:018
2393399134cyclictest0-21swapper/609:00:028
2393399133cyclictest0-21swapper/610:35:028
2393399133cyclictest0-21swapper/609:35:018
2393399133cyclictest0-21swapper/609:30:018
2393399133cyclictest0-21swapper/607:50:018
23932991312cyclictest19309-21idleruntime12:10:177
2393199133cyclictest0-21swapper/407:25:166
2391699133cyclictest0-21swapper/109:45:191
2391699131cyclictest0-21swapper/108:10:271
14850130irq/18-uhci_hcd0-21swapper/011:30:240
13450130irq/18-ehci_hcd0-21swapper/008:20:300
5999120watchdog/70-21swapper/707:20:469
481120ksoftirqd/50-21swapper/507:36:407
4599120watchdog/54080-21qemu-kvm08:12:557
2394599124cyclictest0-21swapper/1112:35:013
2394599123cyclictest0-21swapper/1108:00:343
2394599123cyclictest0-21swapper/1107:33:073
2394599122cyclictest0-21swapper/1110:16:533
2394599122cyclictest0-21swapper/1109:33:363
2394599122cyclictest0-21swapper/1108:46:083
2394599122cyclictest0-21swapper/1108:09:223
2394599122cyclictest0-21swapper/1107:41:083
23944991211cyclictest0-21swapper/1007:16:422
2394399127cyclictest0-21swapper/908:36:2511
23943991211cyclictest0-21swapper/912:05:1511
2394399121cyclictest0-21swapper/908:10:1811
23942991210cyclictest20707-21snmp_rack3slot909:15:1510
23942991210cyclictest20707-21snmp_rack3slot909:15:1510
2394199124cyclictest0-21swapper/709:49:359
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional