You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 22:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Tue Nov 25, 2025 12:46:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20937210693sleep80-21swapper/807:07:1810
20851210492sleep10-21swapper/107:06:111
20960210392sleep50-21swapper/507:07:407
21066210297sleep100-21swapper/1007:08:202
3066350920irq/52-eth0-rx-0-21swapper/207:05:184
2086828778sleep40-21swapper/407:06:266
3066350810irq/52-eth0-rx-0-21swapper/307:05:325
2099427666sleep60-21swapper/607:08:058
14850710irq/18-uhci_hcd0-21swapper/1107:05:283
13450710irq/18-ehci_hcd0-21swapper/707:05:279
2090826354sleep90-21swapper/907:06:5511
2092825955sleep00-21swapper/007:07:110
21340993617cyclictest0-21swapper/1108:15:113
2134099310cyclictest0-21swapper/1109:09:023
2134099301cyclictest0-21swapper/1108:12:283
21291992516cyclictest0-21swapper/008:08:180
2134099190cyclictest0-21swapper/1110:24:503
2130799190cyclictest0-21swapper/210:00:194
21333991817cyclictest0-21swapper/1007:35:142
2133399181cyclictest12250irq/42-ahci10:46:172
2133399181cyclictest12250irq/42-ahci10:18:412
2133399181cyclictest12250irq/42-ahci08:08:542
2133399180cyclictest138012sleep1010:37:562
21330991816cyclictest6835-21snmp_rack3slot911:11:1210
21330991816cyclictest683-21diskstats10:20:1410
21330991816cyclictest5941-21snmp_rack3slot908:15:1110
21330991816cyclictest5344-21snmp_easybox.os07:30:1310
21330991816cyclictest5335-21snmp_rack3slot908:55:3710
21330991816cyclictest32501-21snmp_easybox.os07:20:2210
21330991816cyclictest29103-21snmp_rack3slot909:30:1410
21330991816cyclictest29103-21snmp_rack3slot909:30:1310
21330991816cyclictest27747-21snmp_rack3slot907:15:2010
21330991816cyclictest27409-21iostat_ios11:40:2010
21330991816cyclictest22456-21iostat09:20:1910
21330991816cyclictest22456-21iostat09:20:1810
21330991816cyclictest21703-21users12:15:3310
21330991816cyclictest21703-21users12:15:3310
21330991816cyclictest20855-21snmp_rack3slot908:35:1410
21330991816cyclictest20517-21snmp_rack3slot910:01:3710
21330991816cyclictest2010-21snmp_rack3slot907:25:1310
21330991816cyclictest19353-21snmp_easybox.os09:15:2510
21330991816cyclictest19353-21snmp_easybox.os09:15:2410
21330991816cyclictest19281-21snmp_rack3slot910:45:1510
21330991816cyclictest18713-21snmp_easybox.os08:30:2310
21330991816cyclictest16872-21snmp_rack3slot909:57:0010
21330991816cyclictest12511-21df_inode12:05:1310
21330991816cyclictest12511-21df_inode12:05:1210
21330991816cyclictest11878-21ntp_states09:05:2310
21330991816cyclictest11703-21diskstats10:35:1510
21330991816cyclictest10890-21snmp_rack3slot912:01:3910
21330991816cyclictest10508-21iostat07:35:1910
21328991816cyclictest24334-21qemu-kvm10:30:148
21328991816cyclictest24333-21qemu-kvm10:43:388
21328991816cyclictest24333-21qemu-kvm10:36:338
21328991816cyclictest24333-21qemu-kvm10:29:168
21328991816cyclictest24333-21qemu-kvm10:15:158
21328991816cyclictest24333-21qemu-kvm10:11:408
21328991816cyclictest24333-21qemu-kvm10:08:268
2129199184cyclictest0-21swapper/009:25:170
21333991716cyclictest7495-1kworker/10:0H07:32:012
21333991716cyclictest20741-21smartctl11:30:262
21333991716cyclictest0-21swapper/1012:03:232
21333991716cyclictest0-21swapper/1009:50:142
2133399170cyclictest96212chrt09:47:252
2133399170cyclictest92192sleep1009:03:212
2133399170cyclictest72482chrt11:56:522
2133399170cyclictest72022chrt10:30:052
2133399170cyclictest6992sleep1007:20:442
2133399170cyclictest67882sleep1011:10:352
2133399170cyclictest63692sleep1010:25:422
2133399170cyclictest54702sleep1008:57:212
2133399170cyclictest35212chrt11:50:392
2133399170cyclictest33402chrt11:07:592
2133399170cyclictest322382sleep1011:45:392
2133399170cyclictest309942sleep1009:31:232
2133399170cyclictest309942sleep1009:31:232
2133399170cyclictest3066350irq/52-eth0-rx-12:30:232
2133399170cyclictest3066350irq/52-eth0-rx-12:05:122
2133399170cyclictest3066350irq/52-eth0-rx-12:05:122
2133399170cyclictest3066350irq/52-eth0-rx-11:00:262
2133399170cyclictest3066350irq/52-eth0-rx-10:42:412
2133399170cyclictest3066350irq/52-eth0-rx-10:10:032
2133399170cyclictest3066350irq/52-eth0-rx-09:55:172
2133399170cyclictest3066350irq/52-eth0-rx-09:40:162
2133399170cyclictest3066350irq/52-eth0-rx-09:20:152
2133399170cyclictest3066350irq/52-eth0-rx-09:20:152
2133399170cyclictest3066350irq/52-eth0-rx-09:15:212
2133399170cyclictest3066350irq/52-eth0-rx-09:15:202
2133399170cyclictest3066350irq/52-eth0-rx-09:10:362
2133399170cyclictest3066350irq/52-eth0-rx-08:25:262
2133399170cyclictest3066350irq/52-eth0-rx-08:20:182
2133399170cyclictest3066350irq/52-eth0-rx-08:15:132
2133399170cyclictest3066350irq/52-eth0-rx-08:00:252
2133399170cyclictest3066350irq/52-eth0-rx-07:25:032
2133399170cyclictest291652sleep1012:27:562
2133399170cyclictest286342sleep1011:41:022
2133399170cyclictest270182sleep1010:55:202
2133399170cyclictest269452sleep1008:42:542
2133399170cyclictest263342chrt07:56:252
2133399170cyclictest247882chrt12:20:302
2133399170cyclictest247882chrt12:20:302
2133399170cyclictest247202sleep1007:12:312
2133399170cyclictest247172sleep1010:52:332
2133399170cyclictest242452sleep1010:07:412
2133399170cyclictest23082sleep1009:37:492
2133399170cyclictest228212chrt07:53:012
2133399170cyclictest221752chrt11:35:012
2133399170cyclictest218112chrt12:16:422
2133399170cyclictest218112chrt12:16:412
2133399170cyclictest206412sleep1010:03:142
2133399170cyclictest180632chrt08:30:162
2133399170cyclictest177012sleep1011:25:492
2133399170cyclictest169582sleep1007:45:122
2133399170cyclictest16552sleep1008:51:072
2133399170cyclictest152862sleep1012:10:052
2133399170cyclictest145522sleep1007:40:232
2133399170cyclictest127332sleep1009:06:442
2133399170cyclictest12250irq/42-ahci10:21:242
2133399170cyclictest12250irq/42-ahci08:47:212
2133399170cyclictest12250irq/42-ahci08:35:422
2133399170cyclictest12250irq/42-ahci08:10:102
2133399170cyclictest12250irq/42-ahci07:15:162
2133399170cyclictest107412sleep1011:19:592
21330991715cyclictest9228-21iostat_ios11:15:1910
21330991715cyclictest915-21iostat09:35:1810
21330991715cyclictest8811-21iostat_ios10:30:2110
21330991715cyclictest7553-21diskstats09:45:1410
21330991715cyclictest6693-21munin-run10:29:5910
21330991715cyclictest526-21sensors_temp08:05:2610
21330991715cyclictest4268-21perl09:40:1410
21330991715cyclictest3288-21snmp_rack3slot911:07:1510
21330991715cyclictest32266-21snmp_rack3slot911:46:0110
21330991715cyclictest31139-21snmp_easybox.os11:00:2510
21330991715cyclictest30946-21perl12:30:1410
21330991715cyclictest29502-21snmp_rack3slot910:15:1510
21330991715cyclictest29274-21snmp_easybox.os08:00:2610
21330991715cyclictest28297-21snmp_rack3slot910:56:4910
21330991715cyclictest27209-21sensors_volt10:10:2610
21330991715cyclictest24848-21diskstats08:40:1610
21330991715cyclictest24215-21snmp_rack3slot910:07:1510
21330991715cyclictest23435-21df_inode12:20:1710
21330991715cyclictest23435-21df_inode12:20:1610
21330991715cyclictest23028-21fw_packets10:50:1410
21330991715cyclictest22716-21snmp_rack3slot907:51:3610
21330991715cyclictest2179-21perl12:35:1410
21330991715cyclictest20124-21iostat_ios11:30:1910
21330991715cyclictest1896-21perl11:50:1510
21330991715cyclictest16429-21perl12:10:1510
21330991715cyclictest14421-21diskstats09:10:1410
21330991715cyclictest14257-21perl08:25:1510
21330991715cyclictest13409-21snmp_rack3slot907:40:1410
21330991715cyclictest12042-21df11:20:1410
21330991715cyclictest11446-21ntp_states08:20:2510
21330991715cyclictest11199-21diskstats09:50:1610
21328991716cyclictest24334-21qemu-kvm10:45:128
21319991715cyclictest9412-21sensors10:30:276
21307991715cyclictest2406-21snmpd09:48:124
21307991715cyclictest2406-21snmpd08:08:104
2129199176cyclictest0-21swapper/011:15:000
2133399160cyclictest143052sleep1011:24:002
21330991614cyclictest7228-21snmp_rack3slot911:56:3810
21330991614cyclictest3393-21snmp_rack3slot908:10:1410
21330991614cyclictest25305-21munin-run11:40:0010
21330991614cyclictest1677-21snmp_rack3slot908:51:2410
21328991614cyclictest24334-21qemu-kvm10:22:098
21307991615cyclictest2406-21snmpd10:34:424
21307991614cyclictest2406-21snmpd11:32:114
2129999161cyclictest0-21swapper/108:10:271
21340991513cyclictest20846-21chrt09:20:123
21340991513cyclictest20846-21chrt09:20:113
21340991513cyclictest20314-21chrt09:20:003
21340991513cyclictest20314-21chrt09:20:003
2133099155cyclictest0-21swapper/811:25:0110
21330991513cyclictest7993-21memory09:00:2210
21330991513cyclictest25360-21diskstats09:25:1410
2132999154cyclictest0-21swapper/712:05:029
2132999154cyclictest0-21swapper/712:05:019
2132999154cyclictest0-21swapper/708:45:019
2132999153cyclictest0-21swapper/708:10:009
21325991514cyclictest10226-21switchtime12:00:307
21291991513cyclictest12276-21chrt08:21:300
2129199150cyclictest0-21swapper/008:35:140
2134099144cyclictest0-21swapper/1111:45:003
2133399140cyclictest0-21swapper/1009:28:272
21330991412cyclictest16998-21snmp_rack3slot907:45:1210
2132999145cyclictest0-21swapper/710:30:229
2132999144cyclictest0-21swapper/710:40:019
2132999143cyclictest0-21swapper/707:55:009
2132999142cyclictest0-21swapper/707:47:219
2130799145cyclictest0-21swapper/209:55:184
2130799143cyclictest0-21swapper/210:05:194
2130799142cyclictest0-21swapper/211:57:044
21299991413cyclictest3066350irq/52-eth0-rx-12:11:411
2134099133cyclictest0-21swapper/1110:52:243
2134099133cyclictest0-21swapper/1109:01:173
2134099133cyclictest0-21swapper/1108:57:413
21340991312cyclictest12250irq/42-ahci07:28:123
21340991311cyclictest27843-21chrt10:11:173
21340991310cyclictest0-21swapper/1107:50:183
2133099133cyclictest0-21swapper/812:28:4810
21330991311cyclictest24344-21df_inode07:55:1410
2132999136cyclictest0-21swapper/707:40:209
2132999134cyclictest0-21swapper/708:55:009
2132999134cyclictest0-21swapper/707:20:009
2132999133cyclictest0-21swapper/711:53:439
2132999133cyclictest0-21swapper/711:18:239
2132999133cyclictest0-21swapper/710:38:579
2132999133cyclictest0-21swapper/710:15:019
2132999133cyclictest0-21swapper/709:30:009
2132999133cyclictest0-21swapper/709:20:029
2132999133cyclictest0-21swapper/709:20:029
2132999133cyclictest0-21swapper/709:18:429
2132999133cyclictest0-21swapper/709:18:429
2132999133cyclictest0-21swapper/707:31:469
2132899132cyclictest0-21swapper/607:45:508
21325991312cyclictest24333-21qemu-kvm10:19:017
21319991311cyclictest31255-21sensors11:00:276
2130799134cyclictest0-21swapper/209:40:204
2130799133cyclictest0-21swapper/209:00:264
2130799133cyclictest0-21swapper/207:40:194
2130799130cyclictest0-21swapper/207:15:334
2129999130cyclictest201ksoftirqd/108:20:261
14850130irq/18-uhci_hcd0-21swapper/008:25:260
7399120watchdog/90-21swapper/909:22:1011
7399120watchdog/90-21swapper/909:22:1011
481120ksoftirqd/521050-21qemu-kvm07:20:317
31120ksoftirqd/00-21swapper/009:35:240
2134099122cyclictest0-21swapper/1110:05:473
2134099120cyclictest0-21swapper/1112:30:243
2133399125cyclictest0-21swapper/1012:35:062
2132999125cyclictest0-21swapper/709:45:029
2132999123cyclictest0-21swapper/708:30:009
2132999122cyclictest0-21swapper/711:47:239
2132999122cyclictest0-21swapper/709:04:369
2132999122cyclictest0-21swapper/708:52:229
2132999122cyclictest0-21swapper/707:14:479
2132899123cyclictest0-21swapper/608:10:018
2132899123cyclictest0-21swapper/607:37:208
2132899122cyclictest0-21swapper/611:42:258
2132899122cyclictest0-21swapper/609:44:238
21325991211cyclictest27366-21cpuspeed_turbos08:00:157
21325991211cyclictest23825-21latency11:35:217
21325991211cyclictest15297-21switchtime08:25:307
21325991211cyclictest14205-21cpuspeed_turbos09:55:127
2131999120cyclictest0-21swapper/409:30:126
2131999120cyclictest0-21swapper/409:30:116
2130799123cyclictest0-21swapper/210:27:044
2130799120cyclictest0-21swapper/209:30:154
2130799120cyclictest0-21swapper/209:30:144
2129199124cyclictest0-21swapper/010:26:390
2129199123cyclictest0-21swapper/007:10:220
13450120irq/18-ehci_hcd0-21swapper/011:05:270
13450120irq/18-ehci_hcd0-21swapper/009:05:250
761110ksoftirqd/90-21swapper/909:08:3911
761110ksoftirqd/90-21swapper/907:50:1511
761110ksoftirqd/90-21swapper/907:20:3111
621110ksoftirqd/70-21swapper/712:25:149
551110ksoftirqd/60-21swapper/612:25:138
531110rcuc/60-21swapper/611:45:128
411110ksoftirqd/40-21swapper/407:20:306
341110ksoftirqd/30-21swapper/307:20:305
31110ksoftirqd/00-21swapper/007:20:300
2134099115cyclictest0-21swapper/1109:57:533
2134099113cyclictest0-21swapper/1112:25:433
2134099113cyclictest0-21swapper/1111:21:233
2134099113cyclictest0-21swapper/1110:57:323
2134099113cyclictest0-21swapper/1110:44:543
2134099113cyclictest0-21swapper/1110:00:273
2134099113cyclictest0-21swapper/1107:46:553
2134099113cyclictest0-21swapper/1107:20:503
2133099119cyclictest28477-21diskstats08:45:1310
2133099119cyclictest22868-21perl07:10:1410
2133099112cyclictest0-21swapper/810:40:4110
2132999113cyclictest0-21swapper/712:34:429
2132999113cyclictest0-21swapper/711:42:429
2132999113cyclictest0-21swapper/711:35:259
2132999113cyclictest0-21swapper/710:17:239
2132999113cyclictest0-21swapper/709:55:019
2132999113cyclictest0-21swapper/709:32:239
2132999113cyclictest0-21swapper/709:32:239
2132999113cyclictest0-21swapper/709:09:429
2132999113cyclictest0-21swapper/708:30:529
2132999113cyclictest0-21swapper/708:15:509
2132999113cyclictest0-21swapper/708:08:559
2132999111cyclictest0-21swapper/711:13:529
2132999111cyclictest0-21swapper/707:18:259
2132999110cyclictest0-21swapper/710:50:199
2132899115cyclictest0-21swapper/607:24:568
2132899113cyclictest25308-21idleruntime-cro11:40:018
2132899113cyclictest21042-21qemu-kvm07:25:208
2132899113cyclictest0-21swapper/612:15:418
2132899113cyclictest0-21swapper/612:15:418
2132899113cyclictest0-21swapper/610:55:028
2132899113cyclictest0-21swapper/609:27:418
2132899113cyclictest0-21swapper/609:18:538
2132899113cyclictest0-21swapper/609:18:528
2132899113cyclictest0-21swapper/608:50:468
2132899113cyclictest0-21swapper/607:44:518
2132899112cyclictest0-21swapper/612:05:008
2132899112cyclictest0-21swapper/611:15:018
21325991110cyclictest22063-21python07:50:267
2131999113cyclictest0-21swapper/409:35:186
21319991110cyclictest29327-21sensors08:00:276
2131999110cyclictest0-21swapper/409:55:126
2131999110cyclictest0-21swapper/409:25:236
2130799119cyclictest2406-21snmpd10:20:114
2130799115cyclictest0-21swapper/211:35:194
2130799115cyclictest0-21swapper/211:11:184
2130799115cyclictest0-21swapper/208:45:204
2130799113cyclictest0-21swapper/210:55:214
2130799113cyclictest0-21swapper/208:35:364
2130799113cyclictest0-21swapper/207:51:074
2130799113cyclictest0-21swapper/207:10:184
2129999112cyclictest0-21swapper/110:15:191
2129199119cyclictest0-21swapper/010:45:250
2129199118cyclictest0-21swapper/007:25:280
2129199113cyclictest0-21swapper/010:17:110
2129199113cyclictest0-21swapper/010:00:170
2129199113cyclictest0-21swapper/008:00:210
2129199112cyclictest0-21swapper/008:12:260
2129199110cyclictest0-21swapper/011:35:250
2129199110cyclictest0-21swapper/009:20:300
2129199110cyclictest0-21swapper/009:20:300
2129199110cyclictest0-21swapper/009:10:250
201110ksoftirqd/10-21swapper/112:25:141
1699110watchdog/00-21swapper/008:54:070
13450110irq/18-ehci_hcd0-21swapper/011:45:240
13450110irq/18-ehci_hcd0-21swapper/007:15:140
124050110irq/18-i801_smb0-21swapper/009:00:260
901100ksoftirqd/110-21swapper/1112:20:033
901100ksoftirqd/110-21swapper/1112:20:023
901100ksoftirqd/110-21swapper/1108:35:223
901100ksoftirqd/110-21swapper/1108:30:163
8999100migration/118612-21latency_hist07:35:003
8999100migration/1131866-21marklog11:00:503
761100ksoftirqd/90-21swapper/910:40:1211
761100ksoftirqd/90-21swapper/908:35:2211
7599100migration/99523-21runrttasks09:46:0511
7599100migration/97789-21munin-node09:00:2111
7599100migration/95558-21runrttasks08:58:2411
7599100migration/917619-21cpuspeed_turbos09:15:1311
7599100migration/917619-21cpuspeed_turbos09:15:1311
7599100migration/913090-21cstates07:40:1311
621100ksoftirqd/70-21swapper/711:26:469
551100ksoftirqd/60-21swapper/609:22:578
551100ksoftirqd/60-21swapper/609:22:568
551100ksoftirqd/60-21swapper/608:55:008
551100ksoftirqd/60-21swapper/608:35:228
551100ksoftirqd/60-21swapper/607:50:148
481100ksoftirqd/521050-21qemu-kvm07:25:037
481100ksoftirqd/512887-21qemu-kvm09:22:567
481100ksoftirqd/512887-21qemu-kvm09:22:567
481100ksoftirqd/50-21swapper/508:35:227
481100ksoftirqd/50-21swapper/507:55:227
411100ksoftirqd/40-21swapper/412:20:026
411100ksoftirqd/40-21swapper/412:20:026
411100ksoftirqd/40-21swapper/409:22:576
411100ksoftirqd/40-21swapper/409:22:566
411100ksoftirqd/40-21swapper/408:55:016
341100ksoftirqd/30-21swapper/309:15:245
341100ksoftirqd/30-21swapper/309:15:245
3199100watchdog/30-21swapper/308:55:395
3199100watchdog/30-21swapper/308:49:425
3199100watchdog/30-21swapper/308:30:425
31100ksoftirqd/00-21swapper/012:20:030
31100ksoftirqd/00-21swapper/012:20:030
31100ksoftirqd/00-21swapper/009:50:260
31100ksoftirqd/00-21swapper/007:50:140
271100ksoftirqd/20-21swapper/209:22:574
271100ksoftirqd/20-21swapper/209:22:564
271100ksoftirqd/20-21swapper/208:30:164
271100ksoftirqd/20-21swapper/207:55:224
271100ksoftirqd/20-21swapper/207:20:314
2134099109cyclictest16073-21taskset08:28:423
2134099105cyclictest0-21swapper/1109:11:523
2134099103cyclictest0-21swapper/1110:30:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional