You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 19:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sun Nov 30, 2025 12:46:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30663501140irq/52-eth0-rx-0-21swapper/107:09:201
30540210893sleep80-21swapper/807:06:0010
30093210894sleep30-21swapper/307:05:315
30657210793sleep20-21swapper/207:07:314
30569210290sleep50-21swapper/507:06:227
3066350970irq/52-eth0-rx-0-21swapper/407:05:146
3066350870irq/52-eth0-rx-0-21swapper/1107:05:243
3066450730irq/53-eth0-tx-0-21swapper/907:08:4311
2773226857sleep70-21swapper/707:05:119
3064826653sleep60-21swapper/607:07:228
3062526553sleep00-21swapper/007:07:040
3059126354sleep100-21swapper/1007:06:382
3104899340cyclictest0-21swapper/1108:10:263
30999993314cyclictest0-21swapper/010:10:030
30999993014cyclictest0-21swapper/010:09:470
3099999290cyclictest0-21swapper/008:10:320
3104399192cyclictest12250irq/42-ahci09:10:2211
3104399192cyclictest12250irq/42-ahci09:10:2211
3104899184cyclictest0-21swapper/1110:16:333
3104799181cyclictest12250irq/42-ahci10:43:022
3104799181cyclictest12250irq/42-ahci10:01:392
3104799181cyclictest12250irq/42-ahci08:08:282
3104799181cyclictest12250irq/42-ahci07:41:042
3104399181cyclictest0-21swapper/909:50:2111
31038991816cyclictest8070-21snmp_easybox.os10:15:2410
31038991816cyclictest31418-21munin-run11:35:0110
31038991816cyclictest31162-21perl08:35:1610
31038991816cyclictest28176-21snmp_rack3slot909:15:1710
31038991816cyclictest25362-21snmp_easybox.os09:10:2410
31038991816cyclictest25362-21snmp_easybox.os09:10:2310
31038991816cyclictest23472-21snmp_rack3slot908:25:1510
31038991816cyclictest21181-21iostat_ios09:05:1810
31038991816cyclictest18296-21snmp_rack3slot910:30:1710
31038991816cyclictest17795-21snmp_rack3slot909:45:1710
31038991816cyclictest10120-21snmp_rack3slot910:20:1410
31036991816cyclictest0-21swapper/609:55:018
31180ksoftirqd/00-21swapper/007:40:260
3104899170cyclictest18561-21chrt08:19:243
31047991716cyclictest15961-21python11:10:292
3104799170cyclictest97202sleep1011:46:462
3104799170cyclictest86652sleep1011:00:292
3104799170cyclictest81672sleep1009:30:302
3104799170cyclictest73672sleep1008:01:202
3104799170cyclictest71702sleep1008:45:262
3104799170cyclictest71702sleep1008:45:252
3104799170cyclictest69832sleep1007:17:142
3104799170cyclictest65142sleep1012:27:152
3104799170cyclictest4842chrt08:36:472
3104799170cyclictest42402sleep1008:41:492
3104799170cyclictest39482chrt07:59:442
3104799170cyclictest326592sleep1007:54:442
3104799170cyclictest316682sleep1012:18:132
3104799170cyclictest316682sleep1012:18:132
3104799170cyclictest315162chrt10:50:022
3104799170cyclictest312482sleep1011:32:452
3104799170cyclictest310332sleep1010:05:112
3104799170cyclictest3066350irq/52-eth0-rx-11:40:512
3104799170cyclictest3066350irq/52-eth0-rx-11:15:252
3104799170cyclictest3066350irq/52-eth0-rx-10:55:192
3104799170cyclictest3066350irq/52-eth0-rx-09:50:202
3104799170cyclictest3066350irq/52-eth0-rx-09:46:202
3104799170cyclictest3066350irq/52-eth0-rx-09:01:402
3104799170cyclictest3066350irq/52-eth0-rx-07:20:192
3104799170cyclictest297342sleep1009:15:402
3104799170cyclictest28012sleep1012:21:252
3104799170cyclictest279372sleep1012:11:592
3104799170cyclictest275262sleep1011:26:442
3104799170cyclictest261982sleep1009:12:052
3104799170cyclictest261982sleep1009:12:052
3104799170cyclictest258962sleep1007:45:052
3104799170cyclictest238122sleep1011:20:432
3104799170cyclictest235222sleep1010:37:372
3104799170cyclictest23242sleep1011:35:562
3104799170cyclictest207402sleep1012:03:002
3104799170cyclictest18972chrt07:11:142
3104799170cyclictest189622sleep1010:30:252
3104799170cyclictest183702sleep1008:16:462
3104799170cyclictest178092chrt07:30:522
3104799170cyclictest152362sleep1008:57:512
3104799170cyclictest144652sleep1009:40:202
3104799170cyclictest141612sleep1007:25:532
3104799170cyclictest133532sleep1011:51:462
3104799170cyclictest129512sleep1011:06:172
3104799170cyclictest12412sleep1009:24:542
3104799170cyclictest12250irq/42-ahci10:45:432
3104799170cyclictest12250irq/42-ahci10:28:202
3104799170cyclictest12250irq/42-ahci09:55:242
3104799170cyclictest12250irq/42-ahci09:08:542
3104799170cyclictest12250irq/42-ahci08:33:262
3104799170cyclictest12250irq/42-ahci08:25:022
3104799170cyclictest119782sleep1009:35:542
3104799170cyclictest117002sleep1008:54:152
3104799170cyclictest100682sleep1012:31:152
3104799170cyclictest100682sleep1012:31:142
3104399170cyclictest0-21swapper/909:46:4911
31038991716cyclictest6752-21snmp_easybox.os11:00:1310
31038991716cyclictest30655-21yum10:45:4010
31038991716cyclictest26354-21snmp_rack3slot907:45:1210
31038991715cyclictest8577-21diskstats07:20:1910
31038991715cyclictest6290-21memory08:00:2110
31038991715cyclictest4079-21snmp_easybox.os07:15:1710
31038991715cyclictest3911-21ntp_states09:25:2310
31038991715cyclictest3622-21snmp_easybox.os12:25:1310
31038991715cyclictest32752-21perl07:10:1510
31038991715cyclictest32356-21snmp_rack3slot907:50:3810
31038991715cyclictest31224-21snmp_rack3slot910:05:1410
31038991715cyclictest28025-21iostat_ios08:30:2210
31038991715cyclictest27518-21snmp_rack3slot911:26:3910
31038991715cyclictest24281-21snmp_easybox.os10:40:1310
31038991715cyclictest2375-21iostat07:55:1910
31038991715cyclictest23626-21snmp_rack3slot907:40:1710
31038991715cyclictest22186-21df12:05:1410
31038991715cyclictest22186-21df12:05:1410
31038991715cyclictest17114-21irqstats08:15:2210
31038991715cyclictest15948-21snmp_easybox.os09:00:1110
31038991715cyclictest15145-21snmp_rack3slot908:56:3710
31038991715cyclictest14329-21snmp_rack3slot910:25:1510
31038991715cyclictest14005-21snmp_easybox.os08:10:2510
31038991715cyclictest13596-21munin-run11:55:0010
31038991715cyclictest13025-21snmp_rack3slot911:07:1310
31038991715cyclictest12125-21df07:25:1410
31038991715cyclictest10959-21snmp_rack3slot908:05:5010
31036991715cyclictest30755-21qemu-kvm07:45:368
31047991615cyclictest4601-21python10:10:282
3104799160cyclictest138642sleep1012:38:272
3104799160cyclictest138642sleep1012:38:262
31043991615cyclictest7850-1kworker/9:2H10:35:0011
31043991615cyclictest0-21swapper/911:45:4411
31043991615cyclictest0-21swapper/911:40:2111
31043991615cyclictest0-21swapper/909:00:2311
31043991615cyclictest0-21swapper/908:30:1311
31043991614cyclictest32039-21snmp_rack3slot911:35:1411
31043991614cyclictest25154-21chrt07:41:5311
31038991615cyclictest28609-21perl10:00:1710
31038991614cyclictest9834-21perl08:50:1610
31038991614cyclictest8313-21perl12:30:1410
31038991614cyclictest8313-21perl12:30:1410
31038991614cyclictest6741-21perl09:30:1410
31038991614cyclictest5177-21yum10:10:3410
31038991614cyclictest31372-21df09:20:1410
31038991614cyclictest22256-21snmp_rack3slot909:50:2410
31038991614cyclictest21900-21users08:20:3010
31038991614cyclictest19399-21ntp_states11:15:2510
31038991614cyclictest15705-21iostat_ios11:55:2010
31038991614cyclictest12663-21memory12:35:2310
31038991614cyclictest12663-21memory12:35:2310
3103799165cyclictest0-21swapper/710:05:019
3103799164cyclictest0-21swapper/710:55:019
31032991615cyclictest8664-21python11:00:297
31032991615cyclictest28751-21cstates11:30:147
31011991614cyclictest2406-21snmpd11:32:464
31000991615cyclictest13568-21python07:25:281
30999991614cyclictest8850-21chrt10:16:020
3099999160cyclictest0-21swapper/008:28:520
3104399154cyclictest0-21swapper/907:20:0111
3104399153cyclictest0-21swapper/907:40:0011
31043991514cyclictest7850-1kworker/9:2H10:13:2811
31043991514cyclictest31681-21ipmitool08:35:2111
31043991514cyclictest0-21swapper/907:55:1311
31038991513cyclictest8454-21irqstats11:45:2110
31038991513cyclictest27927-21snmp_rack3slot912:11:5210
31038991513cyclictest23882-21snmp_rack3slot911:21:3810
31038991513cyclictest2139-21diskstats08:40:1510
31038991513cyclictest15916-21munin-run09:45:0210
31038991513cyclictest10763-21iostat_ios09:35:1710
31038991513cyclictest1029-21perl12:20:1410
3103799154cyclictest0-21swapper/712:05:029
3103799154cyclictest0-21swapper/712:05:019
3103799154cyclictest0-21swapper/709:25:019
31032991514cyclictest21798-21python09:05:267
31032991514cyclictest18424-21timerwakeupswit09:00:297
3103299150cyclictest0-21swapper/509:55:137
3101799150cyclictest0-21swapper/309:45:155
31011991513cyclictest2406-21snmpd11:49:464
30999991514cyclictest12472chrt07:10:290
3099999150cyclictest0-21swapper/009:15:190
12250150irq/42-ahci0-21swapper/907:16:1011
5299140watchdog/60-21swapper/612:32:078
5299140watchdog/60-21swapper/612:32:068
3104899144cyclictest0-21swapper/1107:35:003
3104899143cyclictest0-21swapper/1111:30:013
31048991413cyclictest281962sleep1109:15:183
31047991413cyclictest20488-21smartctl08:20:162
31047991413cyclictest13212-21smartctl08:10:172
31043991413cyclictest0-21swapper/911:29:0611
31043991413cyclictest0-21swapper/908:51:4711
31043991413cyclictest0-21swapper/908:00:1211
3103899143cyclictest0-21swapper/810:50:0010
3103899143cyclictest0-21swapper/807:30:0010
31038991412cyclictest25740-21ntp_states09:55:2610
31038991412cyclictest14887-21perl11:10:1610
3103799144cyclictest0-21swapper/712:20:019
3103799144cyclictest0-21swapper/712:20:019
3103799144cyclictest0-21swapper/710:40:019
3103799144cyclictest0-21swapper/709:55:029
3103799144cyclictest0-21swapper/708:25:019
3103799143cyclictest0-21swapper/708:55:019
3103799143cyclictest0-21swapper/707:10:029
3103699144cyclictest0-21swapper/608:25:018
3103699143cyclictest0-21swapper/610:55:008
31024991413cyclictest2489-21sendmail12:26:466
31011991412cyclictest2406-21snmpd09:41:464
31000991413cyclictest3066350irq/52-eth0-rx-10:40:341
31000991413cyclictest3066350irq/52-eth0-rx-07:38:221
3066350140irq/52-eth0-rx-0-21swapper/708:10:329
3066350140irq/52-eth0-rx-0-21swapper/308:10:395
3066350140irq/52-eth0-rx-0-21swapper/108:10:361
12250140irq/42-ahci0-21swapper/208:10:354
5299130watchdog/60-21swapper/609:36:478
5299130watchdog/60-21swapper/608:31:388
3104899133cyclictest0-21swapper/1112:38:223
3104899133cyclictest0-21swapper/1112:38:213
3104899133cyclictest0-21swapper/1110:00:013
3104899133cyclictest0-21swapper/1109:50:023
3104899133cyclictest0-21swapper/1109:20:013
3104899133cyclictest0-21swapper/1107:25:003
3104399134cyclictest0-21swapper/909:30:0111
3104399134cyclictest0-21swapper/908:45:0211
3104399134cyclictest0-21swapper/908:45:0111
3104399134cyclictest0-21swapper/908:45:0111
3104399133cyclictest0-21swapper/912:38:3211
3104399133cyclictest0-21swapper/912:38:3111
3104399133cyclictest0-21swapper/911:59:0111
3104399133cyclictest0-21swapper/909:20:0111
3104399133cyclictest0-21swapper/908:26:1611
3104399133cyclictest0-21swapper/907:10:1511
31043991311cyclictest2954-21taskset12:23:1611
31038991311cyclictest31449-21users12:15:3110
31038991311cyclictest31449-21users12:15:3010
31038991311cyclictest20640-21snmp_rack3slot912:01:4010
3103799134cyclictest0-21swapper/710:10:189
3103799134cyclictest0-21swapper/709:05:009
3103799133cyclictest0-21swapper/712:25:009
3103799133cyclictest0-21swapper/711:05:029
3103799133cyclictest0-21swapper/710:26:319
3103799133cyclictest0-21swapper/709:33:159
3103799133cyclictest0-21swapper/709:17:019
3103799133cyclictest0-21swapper/707:57:459
3103799133cyclictest0-21swapper/707:53:029
3103799133cyclictest0-21swapper/707:29:309
3103699133cyclictest0-21swapper/611:54:308
3103699133cyclictest0-21swapper/611:15:008
3103699133cyclictest0-21swapper/608:54:578
3103699133cyclictest0-21swapper/608:24:588
3103699131cyclictest0-21swapper/612:18:148
3103699131cyclictest0-21swapper/612:18:138
31032991312cyclictest22287-21python09:50:277
31032991312cyclictest12323-21python11:05:287
31011991311cyclictest2406-21snmpd12:09:174
31011991311cyclictest2406-21snmpd12:09:164
31000991312cyclictest3066350irq/52-eth0-rx-07:10:241
3099999133cyclictest0-21swapper/010:00:190
341120ksoftirqd/30-21swapper/308:55:255
3104899123cyclictest0-21swapper/1112:00:013
3104899122cyclictest0-21swapper/1108:31:083
31047991211cyclictest16322-21smartctl11:55:312
3104399125cyclictest0-21swapper/910:51:2811
3104399124cyclictest0-21swapper/912:05:0111
3104399123cyclictest0-21swapper/911:05:0111
3104399122cyclictest0-21swapper/909:32:4611
31043991211cyclictest0-21swapper/912:28:1411
31043991211cyclictest0-21swapper/912:11:3111
31043991211cyclictest0-21swapper/910:15:2211
31043991211cyclictest0-21swapper/909:35:1311
31038991210cyclictest2444-21snmp_rack3slot911:37:1410
3103799124cyclictest0-21swapper/709:45:029
3103799123cyclictest0-21swapper/710:40:009
3103799123cyclictest0-21swapper/708:35:019
3103799122cyclictest0-21swapper/710:45:319
3103799122cyclictest0-21swapper/709:48:169
3103799122cyclictest0-21swapper/708:26:009
3103799122cyclictest0-21swapper/708:18:019
3103799122cyclictest0-21swapper/707:44:309
3103699124cyclictest11187-21qemu-kvm08:40:228
3103699122cyclictest0-21swapper/609:44:318
3103699122cyclictest0-21swapper/609:27:468
31032991211cyclictest26863-21python11:25:307
3102499122cyclictest0-21swapper/409:25:176
3102499121cyclictest0-21swapper/408:30:356
3101799122cyclictest2340-40kipmi007:30:205
31011991210cyclictest2406-21snmpd09:49:154
3100099129cyclictest0-21swapper/108:30:131
31000991211cyclictest6699-21python08:00:251
30999991210cyclictest1024-21chrt09:21:560
3099999120cyclictest0-21swapper/012:10:280
3099999120cyclictest0-21swapper/009:40:240
901110ksoftirqd/110-21swapper/1108:55:263
761110ksoftirqd/90-21swapper/908:55:2611
6199110migration/727978-21munin-node08:30:219
5499110migration/64542-21munin-node10:10:288
5499110migration/624164-21latency_hist10:40:008
5499110migration/619170-21latency_hist09:05:018
5499110migration/612806-21needreboot12:35:228
5499110migration/612806-21needreboot12:35:228
411110ksoftirqd/40-21swapper/412:00:336
3104899119cyclictest901ksoftirqd/1108:40:133
3104899113cyclictest0-21swapper/1111:15:593
3104899113cyclictest0-21swapper/1109:02:433
3104899113cyclictest0-21swapper/1107:58:583
3104899113cyclictest0-21swapper/1107:45:013
31048991110cyclictest901ksoftirqd/1108:45:263
31048991110cyclictest901ksoftirqd/1108:45:263
31048991110cyclictest901ksoftirqd/1107:25:123
3104899111cyclictest0-21swapper/1111:24:203
3104899111cyclictest0-21swapper/1110:49:493
3104899111cyclictest0-21swapper/1108:08:403
3104799113cyclictest0-21swapper/1012:07:042
3104799113cyclictest0-21swapper/1012:07:032
3104799113cyclictest0-21swapper/1010:21:192
3104799113cyclictest0-21swapper/1009:25:282
3104399113cyclictest0-21swapper/912:19:5011
3104399113cyclictest0-21swapper/912:19:5011
3104399113cyclictest0-21swapper/910:42:4811
3104399113cyclictest0-21swapper/910:33:4711
3104399113cyclictest0-21swapper/909:41:5311
3104399113cyclictest0-21swapper/908:15:0111
3104399113cyclictest0-21swapper/908:11:1611
3103799113cyclictest0-21swapper/710:15:479
3103799111cyclictest0-21swapper/709:54:529
3103699113cyclictest0-21swapper/612:28:168
3103699113cyclictest0-21swapper/611:45:168
3103699113cyclictest0-21swapper/611:33:238
3103699113cyclictest0-21swapper/611:27:588
3103699113cyclictest0-21swapper/611:10:168
3103699113cyclictest0-21swapper/610:56:078
3103699113cyclictest0-21swapper/610:38:028
3103699113cyclictest0-21swapper/610:02:178
3103699113cyclictest0-21swapper/609:34:318
3103699113cyclictest0-21swapper/608:35:138
3103699113cyclictest0-21swapper/608:13:148
3103699113cyclictest0-21swapper/607:40:288
3103699113cyclictest0-21swapper/607:32:438
3103699112cyclictest0-21swapper/610:45:008
31036991110cyclictest30755-21qemu-kvm07:39:258
3103699111cyclictest0-21swapper/607:55:028
3103299119cyclictest0-21swapper/510:40:267
31032991110cyclictest5024-21python10:55:317
31032991110cyclictest29569-21python10:00:287
31032991110cyclictest19950-21python12:00:287
31032991110cyclictest13071-21python12:35:277
31032991110cyclictest13071-21python12:35:267
3103299110cyclictest0-21swapper/509:40:107
3102499119cyclictest0-21swapper/410:15:136
3102499113cyclictest0-21swapper/411:20:016
3102499113cyclictest0-21swapper/411:00:196
3102499113cyclictest0-21swapper/409:40:176
3102499111cyclictest0-21swapper/408:00:016
3101799119cyclictest0-21swapper/310:25:165
3101799113cyclictest0-21swapper/310:30:195
3101799113cyclictest0-21swapper/309:30:165
3101799113cyclictest0-21swapper/309:00:175
3101799110cyclictest0-21swapper/310:15:155
3101799110cyclictest0-21swapper/307:20:185
3101199113cyclictest0-21swapper/212:25:194
3101199113cyclictest0-21swapper/211:50:174
3101199113cyclictest0-21swapper/210:05:204
3101199113cyclictest0-21swapper/208:50:014
3101199113cyclictest0-21swapper/208:50:004
3100099113cyclictest0-21swapper/111:40:211
3100099113cyclictest0-21swapper/111:20:201
3100099113cyclictest0-21swapper/108:30:011
31000991110cyclictest9053-21python11:45:271
3099999117cyclictest0-21swapper/007:25:260
3099999113cyclictest0-21swapper/012:20:010
3099999113cyclictest0-21swapper/012:20:010
3099999113cyclictest0-21swapper/007:19:520
30999991110cyclictest105572sleep007:21:170
3099999110cyclictest0-21swapper/009:00:250
3099999110cyclictest0-21swapper/008:45:270
3099999110cyclictest0-21swapper/008:45:260
31110ksoftirqd/00-21swapper/011:25:300
271110ksoftirqd/20-21swapper/208:55:254
1799110watchdog/10-21swapper/108:49:021
1799110watchdog/10-21swapper/108:49:021
1699110watchdog/00-21swapper/008:53:470
761100ksoftirqd/90-21swapper/908:05:1711
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional