You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 04:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Tue Dec 02, 2025 00:46:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
201142121108sleep100-21swapper/1019:07:062
18825210995sleep30-21swapper/319:05:225
20126210594sleep70-21swapper/719:07:159
30663501040irq/52-eth0-rx-0-21swapper/219:05:144
2015829182sleep90-21swapper/919:07:4111
12250910irq/42-ahci0-21swapper/119:08:351
2032428280sleep40-21swapper/419:09:126
2000227363sleep50-21swapper/519:05:367
2031526763sleep80-21swapper/819:09:0410
124050670irq/18-i801_smb0-21swapper/1119:05:273
2002525955sleep00-21swapper/019:05:550
2000325955sleep60-21swapper/619:05:378
2048599330cyclictest0-21swapper/021:30:130
2048799301cyclictest0-21swapper/221:35:304
2052799210cyclictest0-21swapper/1119:26:033
2052799210cyclictest0-21swapper/1119:26:033
2048599210cyclictest0-21swapper/019:30:080
2052799184cyclictest0-21swapper/1119:30:163
2052799180cyclictest0-21swapper/1121:27:193
20526991817cyclictest27318-1kworker/10:1H22:47:292
20526991817cyclictest0-21swapper/1022:15:362
2052699181cyclictest831ksoftirqd/1022:50:152
2052699181cyclictest831ksoftirqd/1022:25:142
2052699181cyclictest831ksoftirqd/1021:45:152
2052699181cyclictest831ksoftirqd/1020:25:322
2052699181cyclictest831ksoftirqd/1000:25:262
2052699181cyclictest811rcuc/1023:15:132
2052699181cyclictest36712chrt19:26:072
2052699181cyclictest36712chrt19:26:072
2052699181cyclictest3066350irq/52-eth0-rx-22:20:282
2052699181cyclictest124050irq/18-i801_smb19:30:312
2052699181cyclictest12250irq/42-ahci23:01:222
2052699181cyclictest120422chrt21:08:062
2052599181cyclictest3066450irq/53-eth0-tx-22:15:1511
2052599181cyclictest0-21swapper/921:40:2311
2052599181cyclictest0-21swapper/920:00:1811
20524991816cyclictest9971-21snmp_rack3slot920:20:1910
20524991816cyclictest9643-21snmp_rack3slot923:15:4910
20524991816cyclictest4371-21perl23:10:1410
20524991816cyclictest32573-21iostat21:35:1910
20524991816cyclictest31932-21snmp_easybox.os20:05:2910
20524991816cyclictest27923-21iostat_ios20:00:2210
20524991816cyclictest25545-21snmp_easybox.os21:25:2610
20524991816cyclictest25080-21snmp_easybox.os23:40:1410
20524991816cyclictest21006-21snmp_easybox.os19:50:2510
20524991816cyclictest20982-21snmp_easybox.os22:50:1110
20524991816cyclictest1702-21snmp_rack3slot920:10:1310
20524991816cyclictest14117-21snmp_rack3slot921:10:1510
20524991816cyclictest1373-21perl00:35:1510
20524991816cyclictest11281-21perl22:35:1710
20524991816cyclictest11065-21snmp_rack3slot921:05:2510
20524991816cyclictest1059-21iostat_ios23:05:2210
20524991816cyclictest10156-21ntp_states19:35:2610
20526991716cyclictest831ksoftirqd/1000:05:132
20526991716cyclictest3066350irq/52-eth0-rx-21:30:162
20526991716cyclictest0-21swapper/1021:55:302
2052699170cyclictest831ksoftirqd/1023:59:002
2052699170cyclictest831ksoftirqd/1022:40:142
2052699170cyclictest831ksoftirqd/1022:10:132
2052699170cyclictest831ksoftirqd/1021:50:312
2052699170cyclictest831ksoftirqd/1021:40:252
2052699170cyclictest831ksoftirqd/1020:55:182
2052699170cyclictest831ksoftirqd/1020:45:152
2052699170cyclictest831ksoftirqd/1020:20:172
2052699170cyclictest831ksoftirqd/1019:35:132
2052699170cyclictest831ksoftirqd/1019:15:162
2052699170cyclictest831ksoftirqd/1000:15:182
2052699170cyclictest831ksoftirqd/1000:00:252
2052699170cyclictest8262sleep1023:05:212
2052699170cyclictest77742sleep1020:15:362
2052699170cyclictest60202sleep1023:10:562
2052699170cyclictest4432sleep1023:50:122
2052699170cyclictest326142sleep1019:24:192
2052699170cyclictest318732sleep1000:31:062
2052699170cyclictest309752sleep1021:35:092
2052699170cyclictest3066350irq/52-eth0-rx-23:40:332
2052699170cyclictest3066350irq/52-eth0-rx-23:35:292
2052699170cyclictest3066350irq/52-eth0-rx-21:10:172
2052699170cyclictest3066350irq/52-eth0-rx-20:50:152
2052699170cyclictest3066350irq/52-eth0-rx-20:10:272
2052699170cyclictest3066350irq/52-eth0-rx-19:10:272
2052699170cyclictest3066350irq/52-eth0-rx-00:12:172
2052699170cyclictest29642sleep1000:35:292
2052699170cyclictest271152chrt22:55:302
2052699170cyclictest265932sleep1021:26:562
2052699170cyclictest252462sleep1019:55:342
2052699170cyclictest246402sleep1000:21:282
2052699170cyclictest233222sleep1022:06:012
2052699170cyclictest205652sleep1023:30:582
2052699170cyclictest195272sleep1020:35:032
2052699170cyclictest188682sleep1022:00:242
2052699170cyclictest178982chrt19:45:332
2052699170cyclictest170102sleep1023:27:102
2052699170cyclictest163992sleep1021:15:072
2052699170cyclictest133912sleep1023:22:222
2052699170cyclictest130282sleep1022:38:182
2052699170cyclictest12250irq/42-ahci22:31:442
2052699170cyclictest12250irq/42-ahci22:31:442
2052699170cyclictest12250irq/42-ahci20:31:192
2052699170cyclictest12250irq/42-ahci20:08:212
20525991715cyclictest5168-21iostat_ios23:55:2111
20525991715cyclictest26715-21taskset21:28:3511
2052599170cyclictest5772-1kworker/9:0H22:37:0111
2052599170cyclictest0-21swapper/923:30:0111
2052599170cyclictest0-21swapper/923:00:2411
2052599170cyclictest0-21swapper/922:05:2011
2052599170cyclictest0-21swapper/921:05:2211
2052599170cyclictest0-21swapper/919:15:0111
20524991715cyclictest9970-21users00:00:3010
20524991715cyclictest6833-21snmp_rack3slot921:45:1610
20524991715cyclictest5707-21perl19:30:1710
20524991715cyclictest4022-21perl22:25:1610
20524991715cyclictest3480-21perl21:40:1810
20524991715cyclictest32326-21users19:20:3210
20524991715cyclictest31127-21snmp_rack3slot923:01:1610
20524991715cyclictest30240-21perl00:30:1410
20524991715cyclictest28698-21diskstats22:15:1410
20524991715cyclictest27405-21snmp_rack3slot922:55:3710
20524991715cyclictest2611-21snmp_easybox.os19:25:2610
20524991715cyclictest2611-21snmp_easybox.os19:25:2610
20524991715cyclictest25598-21snmp_rack3slot919:57:1610
20524991715cyclictest25282-21snmp_rack3slot919:15:1410
20524991715cyclictest24329-21perl20:40:1610
20524991715cyclictest23412-21ntp_states23:35:2810
20524991715cyclictest21952-21snmp_rack3slot919:10:1510
20524991715cyclictest20282-21snmp_rack3slot900:15:2410
20524991715cyclictest19971-21snmp_rack3slot920:35:1310
20524991715cyclictest19766-21snmp_rack3slot922:02:1310
20524991715cyclictest18285-21snmp_rack3slot922:45:1710
20524991715cyclictest18027-21snmp_rack3slot923:30:1410
20524991715cyclictest16762-21df_inode20:30:1610
20524991715cyclictest16048-21snmp_rack3slot921:56:3910
20524991715cyclictest13336-21snmp_rack3slot923:21:3610
20524991715cyclictest12696-21snmp_easybox.os00:05:2410
20505991715cyclictest4974-21sensors22:25:276
2049899171cyclictest0-21swapper/321:15:145
2049899170cyclictest0-21swapper/319:55:475
20487991715cyclictest2406-21snmpd21:02:034
20487991715cyclictest2406-21snmpd20:12:484
20487991715cyclictest2406-21snmpd19:57:044
2048599170cyclictest0-21swapper/019:30:010
2048599170cyclictest0-21swapper/019:30:010
2052799160cyclictest28327-21latency_hist23:45:013
2052799160cyclictest0-21swapper/1122:20:163
20525991615cyclictest0-21swapper/922:20:1911
20525991614cyclictest8866-21taskset21:47:5011
20525991614cyclictest30229-21chrt21:31:5611
20524991614cyclictest6841-21snmp_rack3slot922:30:1310
20524991614cyclictest6841-21snmp_rack3slot922:30:1310
20524991614cyclictest4563-21snmp_rack3slot920:55:5210
20524991614cyclictest32255-21df22:20:1410
20524991614cyclictest31336-21diskstats20:50:1510
20524991614cyclictest28812-21snmp_rack3slot923:45:1110
20524991614cyclictest18161-21users19:45:3410
20524991614cyclictest16868-21users23:25:3010
20524991614cyclictest13125-21diskstats20:25:1710
20511991615cyclictest12093-21qemu-kvm21:08:447
2050599160cyclictest0-21swapper/422:20:246
2050599160cyclictest0-21swapper/421:20:296
2049899160cyclictest0-21swapper/320:56:065
20487991614cyclictest2406-21snmpd22:55:364
1699160watchdog/00-21swapper/021:50:270
31150ksoftirqd/00-21swapper/000:10:270
2052799150cyclictest0-21swapper/1123:32:153
2052799150cyclictest0-21swapper/1123:15:223
2052799150cyclictest0-21swapper/1123:05:153
2052799150cyclictest0-21swapper/1122:35:163
2052799150cyclictest0-21swapper/1121:51:383
2052799150cyclictest0-21swapper/1121:00:153
2052799150cyclictest0-21swapper/1119:45:183
2052799150cyclictest0-21swapper/1100:15:293
2052799150cyclictest0-21swapper/1100:10:303
2052599154cyclictest0-21swapper/919:35:0111
20525991514cyclictest0-21swapper/923:20:1511
20525991514cyclictest0-21swapper/923:12:5211
20525991514cyclictest0-21swapper/922:50:2511
20525991514cyclictest0-21swapper/921:00:3111
20525991514cyclictest0-21swapper/920:10:1411
20525991514cyclictest0-21swapper/900:05:1911
20525991513cyclictest14366-21iostat_ios21:10:1811
20524991513cyclictest28265-21snmp_rack3slot900:26:2410
20524991513cyclictest27685-21diskstats20:45:1610
20524991513cyclictest22091-21iostat22:05:1910
20524991513cyclictest16099-21iostat_ios00:10:2110
20524991513cyclictest1071-21perl23:50:1410
20511991514cyclictest555-21qemu-kvm20:27:597
20511991514cyclictest555-21qemu-kvm20:19:127
20511991514cyclictest27232-21python23:40:327
20511991514cyclictest21259-21python19:50:307
2051199150cyclictest0-21swapper/521:00:177
2050599152cyclictest0-21swapper/422:30:166
2050599152cyclictest0-21swapper/422:30:166
2050599151cyclictest0-21swapper/423:05:156
2050599151cyclictest0-21swapper/421:51:386
2050599151cyclictest0-21swapper/420:15:106
2050599151cyclictest0-21swapper/419:35:276
2050599150cyclictest0-21swapper/421:00:366
2049899156cyclictest0-21swapper/322:25:225
2049899150cyclictest0-21swapper/323:38:275
2049899150cyclictest0-21swapper/322:35:295
2049899150cyclictest0-21swapper/321:40:075
2049899150cyclictest0-21swapper/320:05:305
2049899150cyclictest0-21swapper/319:35:005
20487991513cyclictest2406-21snmpd23:26:064
2048799150cyclictest0-21swapper/221:40:274
2048799150cyclictest0-21swapper/221:05:244
2048799150cyclictest0-21swapper/200:25:174
2048599150cyclictest0-21swapper/022:15:250
2048599150cyclictest0-21swapper/020:45:170
2048599150cyclictest0-21swapper/000:30:380
31140ksoftirqd/00-21swapper/022:35:280
31140ksoftirqd/00-21swapper/000:25:250
2052799140cyclictest9507-21snmp_rack3slot920:20:173
2052799140cyclictest0-21swapper/1123:36:563
2052799140cyclictest0-21swapper/1123:20:323
2052799140cyclictest0-21swapper/1123:10:223
2052799140cyclictest0-21swapper/1122:50:173
2052799140cyclictest0-21swapper/1122:26:013
2052799140cyclictest0-21swapper/1122:14:513
2052799140cyclictest0-21swapper/1121:55:403
2052799140cyclictest0-21swapper/1121:32:443
2052799140cyclictest0-21swapper/1121:20:243
2052799140cyclictest0-21swapper/1121:08:033
2052799140cyclictest0-21swapper/1120:50:243
2052799140cyclictest0-21swapper/1120:45:033
2052799140cyclictest0-21swapper/1120:31:263
2052799140cyclictest0-21swapper/1120:15:103
2052799140cyclictest0-21swapper/1120:00:343
2052799140cyclictest0-21swapper/1119:57:263
2052799140cyclictest0-21swapper/1119:50:153
2052799140cyclictest0-21swapper/1119:40:123
2052799140cyclictest0-21swapper/1119:40:123
2052799140cyclictest0-21swapper/1119:36:183
2052799140cyclictest0-21swapper/1100:38:003
2052799140cyclictest0-21swapper/1100:20:193
2052799140cyclictest0-21swapper/1100:04:273
2052599144cyclictest0-21swapper/920:20:0111
20525991412cyclictest12424-21snmp_rack3slot920:25:1411
2052499144cyclictest0-21swapper/821:05:0010
20524991412cyclictest22994-21perl00:20:1410
20524991412cyclictest21663-21iostat_ios21:20:2010
20524991412cyclictest12401-21snmp_rack3slot921:51:2510
20518991412cyclictest23532-21qemu-kvm22:08:408
20511991413cyclictest555-21qemu-kvm20:47:337
20511991413cyclictest32665-21cstates_1123:05:177
2050599143cyclictest0-21swapper/419:20:006
2050599142cyclictest0-21swapper/423:32:156
2050599142cyclictest0-21swapper/422:35:166
2050599142cyclictest0-21swapper/400:10:306
2050599141cyclictest0-21swapper/423:35:286
2050599141cyclictest0-21swapper/423:15:196
2050599141cyclictest0-21swapper/422:54:296
2050599141cyclictest0-21swapper/422:45:106
2050599141cyclictest0-21swapper/422:40:336
2050599141cyclictest0-21swapper/422:10:116
2050599141cyclictest0-21swapper/422:03:146
2050599141cyclictest0-21swapper/421:46:176
2050599141cyclictest0-21swapper/421:32:336
2050599141cyclictest0-21swapper/421:15:256
2050599141cyclictest0-21swapper/421:10:546
2050599141cyclictest0-21swapper/420:50:246
2050599141cyclictest0-21swapper/420:45:036
2050599141cyclictest0-21swapper/420:37:426
2050599141cyclictest0-21swapper/420:30:436
2050599141cyclictest0-21swapper/420:27:296
2050599141cyclictest0-21swapper/420:20:106
2050599141cyclictest0-21swapper/420:06:426
2050599141cyclictest0-21swapper/419:45:186
2050599141cyclictest0-21swapper/419:25:236
2050599141cyclictest0-21swapper/419:25:236
2050599141cyclictest0-21swapper/419:20:126
2049899148cyclictest0-21swapper/320:25:175
2049899147cyclictest0-21swapper/321:30:285
2049899146cyclictest0-21swapper/320:04:165
2049899141cyclictest0-21swapper/319:40:245
2049899141cyclictest0-21swapper/319:40:245
2049899140cyclictest0-21swapper/323:45:595
2049899140cyclictest0-21swapper/323:42:245
2049899140cyclictest0-21swapper/323:34:315
2049899140cyclictest0-21swapper/323:15:005
2049899140cyclictest0-21swapper/322:45:265
2049899140cyclictest0-21swapper/322:44:075
2049899140cyclictest0-21swapper/321:53:395
2049899140cyclictest0-21swapper/321:37:165
2049899140cyclictest0-21swapper/321:11:105
2049899140cyclictest0-21swapper/321:06:325
2049899140cyclictest0-21swapper/320:51:435
2049899140cyclictest0-21swapper/320:24:105
2049899140cyclictest0-21swapper/319:20:305
2049899140cyclictest0-21swapper/300:04:455
31130ksoftirqd/00-21swapper/019:10:270
2052799130cyclictest0-21swapper/1123:57:403
2052799130cyclictest0-21swapper/1123:46:173
2052799130cyclictest0-21swapper/1123:26:473
2052799130cyclictest0-21swapper/1122:58:423
2052799130cyclictest0-21swapper/1122:40:523
2052799130cyclictest0-21swapper/1122:17:013
2052799130cyclictest0-21swapper/1121:45:433
2052799130cyclictest0-21swapper/1121:37:443
2052799130cyclictest0-21swapper/1121:15:423
2052799130cyclictest0-21swapper/1121:10:143
2052799130cyclictest0-21swapper/1120:55:183
2052799130cyclictest0-21swapper/1120:42:103
2052799130cyclictest0-21swapper/1120:36:223
2052799130cyclictest0-21swapper/1120:25:063
2052799130cyclictest0-21swapper/1120:13:073
2052799130cyclictest0-21swapper/1119:15:243
2052799130cyclictest0-21swapper/1119:11:173
2052799130cyclictest0-21swapper/1100:30:143
2052799130cyclictest0-21swapper/1100:27:233
2052799130cyclictest0-21swapper/1100:07:353
2052599134cyclictest0-21swapper/919:30:0111
2052599133cyclictest0-21swapper/920:55:0011
20525991312cyclictest0-21swapper/922:10:0111
20525991311cyclictest3066450irq/53-eth0-tx-21:15:1211
2052499133cyclictest0-21swapper/822:42:1910
20524991311cyclictest5043-21snmp_easybox.os20:15:1310
20524991311cyclictest16569-21snmp_rack3slot921:15:1210
20511991312cyclictest25057-21cpuspeed_turbos22:55:137
20511991312cyclictest12086-21qemu-kvm21:30:057
2050599133cyclictest0-21swapper/423:50:176
2050599131cyclictest0-21swapper/423:45:186
2050599131cyclictest0-21swapper/423:40:066
2050599131cyclictest0-21swapper/423:26:186
2050599131cyclictest0-21swapper/423:22:386
2050599131cyclictest0-21swapper/423:12:156
2050599131cyclictest0-21swapper/423:01:556
2050599131cyclictest0-21swapper/422:58:426
2050599131cyclictest0-21swapper/422:17:006
2050599131cyclictest0-21swapper/422:06:176
2050599131cyclictest0-21swapper/421:55:246
2050599131cyclictest0-21swapper/421:40:056
2050599131cyclictest0-21swapper/421:36:066
2050599131cyclictest0-21swapper/421:25:056
2050599131cyclictest0-21swapper/421:05:066
2050599131cyclictest0-21swapper/420:55:396
2050599131cyclictest0-21swapper/420:41:426
2050599131cyclictest0-21swapper/420:10:276
2050599131cyclictest0-21swapper/420:00:186
2050599131cyclictest0-21swapper/419:55:276
2050599131cyclictest0-21swapper/419:50:066
2050599131cyclictest0-21swapper/419:40:126
2050599131cyclictest0-21swapper/419:40:126
2050599131cyclictest0-21swapper/419:30:136
2050599131cyclictest0-21swapper/419:10:196
2050599131cyclictest0-21swapper/400:37:556
2050599131cyclictest0-21swapper/400:30:156
2050599131cyclictest0-21swapper/400:25:026
2050599131cyclictest0-21swapper/400:20:026
2050599131cyclictest0-21swapper/400:15:046
2050599131cyclictest0-21swapper/400:00:536
2049899136cyclictest0-21swapper/323:08:535
2049899136cyclictest0-21swapper/323:00:485
2049899136cyclictest0-21swapper/322:51:575
2049899136cyclictest0-21swapper/322:06:515
2049899136cyclictest0-21swapper/321:46:215
2049899136cyclictest0-21swapper/321:26:535
2049899136cyclictest0-21swapper/321:24:595
2049899136cyclictest0-21swapper/320:45:265
2049899136cyclictest0-21swapper/320:16:575
2049899136cyclictest0-21swapper/319:50:025
2049899136cyclictest0-21swapper/319:28:095
2049899136cyclictest0-21swapper/319:28:095
2049899130cyclictest0-21swapper/322:11:215
2049899130cyclictest0-21swapper/322:01:395
2049899130cyclictest0-21swapper/320:30:165
2049899130cyclictest0-21swapper/319:46:415
2049899130cyclictest0-21swapper/300:33:545
20486991312cyclictest12250irq/42-ahci21:30:121
2499120watchdog/20-21swapper/220:07:384
2052799120cyclictest0-21swapper/1122:49:553
2052799120cyclictest0-21swapper/1122:31:493
2052799120cyclictest0-21swapper/1122:31:483
2052799120cyclictest0-21swapper/1122:00:573
2052799120cyclictest0-21swapper/1121:43:243
2052799120cyclictest0-21swapper/1120:06:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional