You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 00:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sat Nov 29, 2025 12:46:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501130irq/42-ahci0-21swapper/307:05:145
670211293sleep00-21swapper/007:06:160
628211096sleep100-21swapper/1007:05:492
122501010irq/42-ahci0-21swapper/207:09:334
12250940irq/42-ahci0-21swapper/407:09:046
12250930irq/42-ahci0-21swapper/107:06:581
3066350890irq/52-eth0-rx-0-21swapper/507:05:297
12250890irq/42-ahci0-21swapper/907:08:4711
3066350880irq/52-eth0-rx-0-21swapper/707:06:159
93528781sleep80-21swapper/807:09:0010
3066350870irq/52-eth0-rx-0-21swapper/1107:05:313
111899600cyclictest0-21swapper/007:20:340
80125955sleep60-21swapper/607:07:588
1160993617cyclictest0-21swapper/1109:20:383
116099310cyclictest0-21swapper/1108:16:323
116099310cyclictest0-21swapper/1108:16:323
111899310cyclictest0-21swapper/009:28:000
1118993014cyclictest0-21swapper/010:18:500
12250280irq/42-ahci0-21swapper/907:20:3511
12250280irq/42-ahci0-21swapper/807:20:3910
12250280irq/42-ahci0-21swapper/407:20:406
12250280irq/42-ahci0-21swapper/307:20:415
12250280irq/42-ahci0-21swapper/1107:20:413
1160992827cyclictest8653-1kworker/11:3H10:11:553
1160992827cyclictest0-21swapper/1107:13:463
1160992726cyclictest0-21swapper/1108:24:563
1160992726cyclictest0-21swapper/1108:24:563
1118992713cyclictest0-21swapper/007:10:280
116099240cyclictest0-21swapper/1107:25:203
1118992414cyclictest0-21swapper/008:10:290
111899240cyclictest0-21swapper/007:27:050
116099237cyclictest5191-21chrt10:10:013
116099237cyclictest11178-21taskset10:15:243
111899230cyclictest0-21swapper/007:16:300
116099220cyclictest0-21swapper/1107:17:573
116099210cyclictest0-21swapper/1107:30:193
1160992019cyclictest0-21swapper/1108:10:373
115999192cyclictest12250irq/42-ahci12:20:012
115799190cyclictest0-21swapper/810:39:5510
1159991817cyclictest24644-1kworker/10:2H11:27:382
1159991817cyclictest13228-1kworker/10:1H07:47:432
1159991817cyclictest0-21swapper/1009:26:372
1159991817cyclictest0-21swapper/1008:37:112
1159991817cyclictest0-21swapper/1008:07:252
1159991817cyclictest0-21swapper/1007:39:552
115999181cyclictest3066350irq/52-eth0-rx-11:45:202
115999181cyclictest12250irq/42-ahci12:35:262
115999181cyclictest12250irq/42-ahci12:31:002
115999181cyclictest12250irq/42-ahci12:20:432
115999181cyclictest12250irq/42-ahci12:06:092
115999181cyclictest12250irq/42-ahci12:00:312
115999181cyclictest12250irq/42-ahci11:51:142
115999181cyclictest12250irq/42-ahci11:35:422
115999181cyclictest12250irq/42-ahci11:30:562
115999181cyclictest12250irq/42-ahci11:21:282
115999181cyclictest12250irq/42-ahci11:21:272
115999181cyclictest12250irq/42-ahci11:15:552
115999181cyclictest12250irq/42-ahci11:10:352
115999181cyclictest12250irq/42-ahci11:07:022
115999181cyclictest12250irq/42-ahci11:00:032
115999181cyclictest12250irq/42-ahci10:56:512
115999181cyclictest12250irq/42-ahci10:51:082
115999181cyclictest12250irq/42-ahci10:45:532
115999181cyclictest12250irq/42-ahci10:37:012
115999181cyclictest12250irq/42-ahci10:31:402
115999181cyclictest12250irq/42-ahci10:25:212
115999181cyclictest12250irq/42-ahci10:20:462
115999181cyclictest12250irq/42-ahci10:15:032
115999181cyclictest12250irq/42-ahci10:05:242
115999181cyclictest12250irq/42-ahci10:00:082
115999181cyclictest12250irq/42-ahci09:56:042
115999181cyclictest12250irq/42-ahci09:40:122
115999181cyclictest12250irq/42-ahci09:35:452
115999181cyclictest12250irq/42-ahci09:30:532
115999181cyclictest12250irq/42-ahci09:21:232
115999181cyclictest12250irq/42-ahci09:17:532
115999181cyclictest12250irq/42-ahci09:10:182
115999181cyclictest12250irq/42-ahci09:09:002
115999181cyclictest12250irq/42-ahci09:03:122
115999181cyclictest12250irq/42-ahci08:55:442
115999181cyclictest12250irq/42-ahci08:52:592
115999181cyclictest12250irq/42-ahci08:46:002
115999181cyclictest12250irq/42-ahci08:40:452
115999181cyclictest12250irq/42-ahci08:31:582
115999181cyclictest12250irq/42-ahci08:22:232
115999181cyclictest12250irq/42-ahci08:22:222
115999181cyclictest12250irq/42-ahci08:15:442
115999181cyclictest12250irq/42-ahci08:15:442
115999181cyclictest12250irq/42-ahci08:12:482
115999181cyclictest12250irq/42-ahci08:00:442
115999181cyclictest12250irq/42-ahci07:50:312
115999181cyclictest12250irq/42-ahci07:40:292
115999181cyclictest12250irq/42-ahci07:30:342
115999181cyclictest12250irq/42-ahci07:17:492
115999181cyclictest12250irq/42-ahci07:10:092
1158991816cyclictest26589-21iostat_ios07:40:2111
115899181cyclictest761ksoftirqd/912:02:4711
115899181cyclictest761ksoftirqd/911:55:1011
115899181cyclictest761ksoftirqd/911:25:3211
115899181cyclictest761ksoftirqd/910:25:0011
115899181cyclictest21662-1kworker/9:0H09:29:2611
115899181cyclictest21662-1kworker/9:0H09:16:0611
115899181cyclictest1634-1kworker/9:2H08:41:2011
115899181cyclictest12250irq/42-ahci11:40:3511
115899181cyclictest12250irq/42-ahci11:33:4211
115899181cyclictest12250irq/42-ahci11:14:2311
115899181cyclictest12250irq/42-ahci10:39:4011
115899181cyclictest12250irq/42-ahci09:07:0011
115899181cyclictest0-21swapper/912:34:4811
115899181cyclictest0-21swapper/912:20:2811
115899181cyclictest0-21swapper/912:16:2211
115899181cyclictest0-21swapper/911:50:5511
115899181cyclictest0-21swapper/911:48:0411
115899181cyclictest0-21swapper/911:22:5311
115899181cyclictest0-21swapper/911:22:5211
115899181cyclictest0-21swapper/911:02:5711
115899181cyclictest0-21swapper/910:30:3211
115899181cyclictest0-21swapper/910:29:1811
115899181cyclictest0-21swapper/910:12:0711
115899181cyclictest0-21swapper/910:07:4611
115899181cyclictest0-21swapper/910:00:1811
115899181cyclictest0-21swapper/909:54:5811
115899181cyclictest0-21swapper/909:42:3611
115899181cyclictest0-21swapper/909:20:3811
115899181cyclictest0-21swapper/908:39:2111
115899181cyclictest0-21swapper/908:30:1011
115899181cyclictest0-21swapper/908:28:1711
115899181cyclictest0-21swapper/908:20:5111
115899181cyclictest0-21swapper/908:20:5011
115899181cyclictest0-21swapper/908:01:0911
115899181cyclictest0-21swapper/907:58:4311
115899181cyclictest0-21swapper/907:51:5411
115899181cyclictest0-21swapper/907:36:1111
115899181cyclictest0-21swapper/907:15:2511
115899181cyclictest0-21swapper/907:10:4511
1157991816cyclictest8746-21df08:45:1510
1157991816cyclictest7813-21users09:25:4210
1157991816cyclictest6847-21snmp_rack3slot907:15:1610
1157991816cyclictest5543-21snmp_rack3slot910:51:2510
1157991816cyclictest5235-21irqstats07:55:2110
1157991816cyclictest4836-21snmp_rack3slot908:40:1310
1157991816cyclictest3887-21snmp_easybox.os12:20:1410
1157991816cyclictest31792-21snmp_rack3slot912:11:5010
1157991816cyclictest31664-21irqstats09:15:2010
1157991816cyclictest31434-21snmp_easybox.os08:30:2410
1157991816cyclictest28137-21snmp_rack3slot912:06:5210
1157991816cyclictest26546-21diskstats08:25:1510
1157991816cyclictest24789-21users08:20:3110
1157991816cyclictest24789-21users08:20:3110
1157991816cyclictest2442-21snmp_rack3slot911:31:2610
1157991816cyclictest23789-21snmp_rack3slot911:15:3810
1157991816cyclictest23518-21snmp_rack3slot909:50:1310
1157991816cyclictest22426-21diskstats12:00:1610
1157991816cyclictest21989-21irqstats10:30:2010
1157991816cyclictest21844-21snmp_rack3slot907:35:1610
1157991816cyclictest2080-21munin-run10:50:0110
1157991816cyclictest2068-21snmp_rack3slot909:20:1410
1157991816cyclictest19377-21ntp_states11:10:2610
1157991816cyclictest18597-21snmp_rack3slot907:30:1510
1157991816cyclictest14756-21snmp_easybox.os08:10:1410
1157991816cyclictest10350-21snmp_rack3slot910:15:1710
1127991816cyclictest2406-21snmpd09:03:044
1159991716cyclictest0-21swapper/1010:10:132
115999170cyclictest139872sleep1007:25:042
115999170cyclictest12250irq/42-ahci12:25:192
115999170cyclictest12250irq/42-ahci12:10:302
115999170cyclictest12250irq/42-ahci11:55:282
115999170cyclictest12250irq/42-ahci11:40:352
115999170cyclictest12250irq/42-ahci10:40:422
115999170cyclictest12250irq/42-ahci09:50:172
115999170cyclictest12250irq/42-ahci09:45:232
115999170cyclictest12250irq/42-ahci08:25:202
115999170cyclictest12250irq/42-ahci07:55:072
115999170cyclictest12250irq/42-ahci07:20:052
1158991716cyclictest28180-21snmp_easybox.os09:10:2411
115899170cyclictest761ksoftirqd/910:55:1511
115899170cyclictest741rcuc/912:08:2711
115899170cyclictest21662-1kworker/9:0H08:55:1411
115899170cyclictest1634-1kworker/9:2H08:47:3411
115899170cyclictest12250irq/42-ahci11:35:4311
115899170cyclictest12250irq/42-ahci09:45:1911
115899170cyclictest0-21swapper/912:39:2411
115899170cyclictest0-21swapper/912:10:5611
115899170cyclictest0-21swapper/910:43:3311
115899170cyclictest0-21swapper/908:50:5311
115899170cyclictest0-21swapper/908:19:0711
115899170cyclictest0-21swapper/908:19:0711
115899170cyclictest0-21swapper/908:06:5511
115899170cyclictest0-21swapper/907:26:2411
1157991715cyclictest712-21diskstats07:50:1510
1157991715cyclictest6966-21snmp_easybox.os11:40:1410
1157991715cyclictest6450-21snmp_rack3slot910:10:1510
1157991715cyclictest6106-21snmp_rack3slot911:36:0110
1157991715cyclictest4811-21snmp_rack3slot910:05:5010
1157991715cyclictest31223-21snmp_rack3slot911:27:0510
1157991715cyclictest27684-21snmp_rack3slot909:10:1510
1157991715cyclictest25047-21snmp_easybox.os07:40:1410
1157991715cyclictest20829-21snmp_rack3slot911:57:1610
1157991715cyclictest19228-21diskstats08:15:1510
1157991715cyclictest19228-21diskstats08:15:1410
1157991715cyclictest1811-21snmp_rack3slot907:10:1210
1157991715cyclictest17266-21snmp_rack3slot909:40:1610
1157991715cyclictest17142-21snmp_rack3slot911:51:4810
1157991715cyclictest15086-21perl07:25:1610
1157991715cyclictest14563-21snmp_rack3slot908:52:1510
1157991715cyclictest14191-21snmp_rack3slot910:20:1610
1157991715cyclictest14052-21snmp_rack3slot912:31:3910
1157991715cyclictest12567-21iostat_ios08:05:2010
1157991715cyclictest1255-21perl12:15:1810
1157991715cyclictest1091-21snmp_rack3slot910:01:2510
1157991715cyclictest10082-21snmp_rack3slot908:00:3810
1140991715cyclictest12797-21sensors11:45:286
1118991715cyclictest0-21swapper/008:16:400
1118991715cyclictest0-21swapper/008:16:390
12250160irq/42-ahci0-21swapper/507:20:407
1158991615cyclictest761ksoftirqd/910:45:1811
1158991615cyclictest5580-1kworker/9:0H11:17:5811
1158991615cyclictest12250irq/42-ahci12:29:2211
1158991615cyclictest12250irq/42-ahci10:15:2811
1158991615cyclictest12250irq/42-ahci08:11:1911
1158991615cyclictest12250irq/42-ahci07:34:5611
1158991615cyclictest0-21swapper/910:50:1711
1158991615cyclictest0-21swapper/909:32:1811
1158991615cyclictest0-21swapper/909:00:2211
1157991614cyclictest6404-21snmp_easybox.os10:55:1310
1157991614cyclictest29894-21snmp_rack3slot909:57:0410
1157991614cyclictest27581-21snmp_rack3slot911:22:1310
1157991614cyclictest27581-21snmp_rack3slot911:22:1310
1157991614cyclictest25419-21users09:05:3510
1157991614cyclictest21026-21snmp_rack3slot909:45:1710
1157991614cyclictest16498-21snmp_rack3slot911:05:4910
1157991614cyclictest15149-21yum09:35:3510
1157991614cyclictest12873-21snmp_rack3slot911:01:1410
1157991614cyclictest11456-21diskstats11:45:1610
1147991615cyclictest17057-21python12:35:297
1127991614cyclictest2406-21snmpd07:33:484
111899163cyclictest0-21swapper/010:30:180
116099155cyclictest0-21swapper/1109:50:023
116099154cyclictest0-21swapper/1111:35:023
1158991514cyclictest0-21swapper/909:58:5911
1158991514cyclictest0-21swapper/907:49:4511
115799154cyclictest0-21swapper/810:40:0110
1157991513cyclictest9892-21perl09:30:1510
1157991513cyclictest21002-21ntp_states09:00:2910
1157991513cyclictest18205-21snmp_rack3slot908:57:0110
1157991513cyclictest17629-21yum12:35:3410
1157991513cyclictest16776-21snmp_easybox.os10:25:1410
1157991513cyclictest1352-21df08:35:1610
1156991514cyclictest12250irq/42-ahci07:20:349
115399154cyclictest0-21swapper/612:35:018
115399153cyclictest0-21swapper/611:55:368
114799150cyclictest0-21swapper/512:30:137
1127991514cyclictest12250irq/42-ahci07:20:484
1127991513cyclictest2406-21snmpd09:18:504
1118991513cyclictest0-21swapper/010:35:300
111899150cyclictest0-21swapper/010:11:550
111899150cyclictest0-21swapper/008:47:300
12250140irq/42-ahci0-21swapper/312:13:545
12250140irq/42-ahci0-21swapper/107:20:411
1158991413cyclictest0-21swapper/911:05:2811
1158991413cyclictest0-21swapper/909:38:2911
1157991412cyclictest8740-21perl12:25:1710
115399143cyclictest11830-21awk09:35:008
1147991413cyclictest4642-21cpuspeed_turbos08:40:137
1147991413cyclictest29089-21cpuspeed_turbos12:10:147
1140991413cyclictest207602sleep407:34:356
111899140cyclictest0-21swapper/012:25:280
31130ksoftirqd/00-21swapper/008:38:110
14850130irq/18-uhci_hcd0-21swapper/011:55:270
116099134cyclictest0-21swapper/1111:20:013
116099134cyclictest0-21swapper/1111:20:013
116099133cyclictest0-21swapper/1111:59:463
116099133cyclictest0-21swapper/1111:54:223
116099133cyclictest0-21swapper/1111:43:043
116099133cyclictest0-21swapper/1110:02:203
116099133cyclictest0-21swapper/1109:45:023
116099133cyclictest0-21swapper/1108:54:033
116099133cyclictest0-21swapper/1108:45:513
116099133cyclictest0-21swapper/1108:26:333
116099133cyclictest0-21swapper/1107:43:293
1160991311cyclictest29517-21awk08:30:113
1157991311cyclictest29983-21hddtemp_smartct07:45:1610
115399134cyclictest0-21swapper/612:35:208
115399133cyclictest0-21swapper/610:58:378
115399133cyclictest0-21swapper/610:51:068
115399133cyclictest0-21swapper/609:24:058
115399133cyclictest0-21swapper/608:37:188
115399133cyclictest0-21swapper/608:34:048
115399133cyclictest0-21swapper/608:02:018
112799134cyclictest0-21swapper/211:26:244
112799133cyclictest0-21swapper/211:35:194
1127991311cyclictest2406-21snmpd08:29:044
1118991312cyclictest0-21swapper/011:00:400
1118991312cyclictest0-21swapper/010:21:440
481120ksoftirqd/50-21swapper/508:00:567
4599120watchdog/50-21swapper/512:27:317
31120ksoftirqd/00-21swapper/008:54:150
31120ksoftirqd/00-21swapper/008:05:280
2499120watchdog/20-21swapper/210:25:514
14850120irq/18-uhci_hcd0-21swapper/010:00:240
14850120irq/18-uhci_hcd0-21swapper/009:55:270
116099124cyclictest0-21swapper/1112:14:053
116099123cyclictest0-21swapper/1109:40:013
116099123cyclictest0-21swapper/1107:37:313
116099122cyclictest0-21swapper/1111:49:353
116099122cyclictest0-21swapper/1111:14:563
115699129cyclictest30955-21ipmi_temp08:30:209
1156991211cyclictest621ksoftirqd/711:52:369
1156991211cyclictest621ksoftirqd/707:10:289
115399125cyclictest0-21swapper/611:12:368
115399124cyclictest0-21swapper/611:30:028
115399123cyclictest0-21swapper/610:35:018
115399123cyclictest0-21swapper/607:45:008
115399122cyclictest0-21swapper/611:36:518
112799123cyclictest0-21swapper/209:53:034
112799122cyclictest0-21swapper/211:05:194
112799120cyclictest0-21swapper/212:00:054
1118991211cyclictest623-21md0_raid110:41:110
1118991211cyclictest2690-21marklog08:28:100
1118991210cyclictest0-21swapper/012:30:310
111899120cyclictest0-21swapper/009:18:560
901110ksoftirqd/110-21swapper/1108:55:143
901110ksoftirqd/110-21swapper/1108:41:243
901110ksoftirqd/110-21swapper/1108:00:563
551110ksoftirqd/60-21swapper/612:21:268
551110ksoftirqd/60-21swapper/608:41:248
481110ksoftirqd/50-21swapper/512:21:277
481110ksoftirqd/50-21swapper/509:57:517
481110ksoftirqd/50-21swapper/509:09:287
481110ksoftirqd/50-21swapper/508:55:147
481110ksoftirqd/50-21swapper/507:10:297
411110ksoftirqd/40-21swapper/410:45:196
411110ksoftirqd/40-21swapper/408:41:246
411110ksoftirqd/40-21swapper/408:00:566
341110ksoftirqd/30-21swapper/310:10:315
341110ksoftirqd/30-21swapper/309:45:315
341110ksoftirqd/30-21swapper/308:41:235
341110ksoftirqd/30-21swapper/307:50:095
3066350110irq/52-eth0-rx-0-21swapper/411:30:356
31110ksoftirqd/00-21swapper/011:30:250
31110ksoftirqd/00-21swapper/011:24:040
31110ksoftirqd/00-21swapper/011:24:030
31110ksoftirqd/00-21swapper/008:40:260
31110ksoftirqd/00-21swapper/007:44:020
271110ksoftirqd/20-21swapper/208:20:044
271110ksoftirqd/20-21swapper/208:20:034
271110ksoftirqd/20-21swapper/208:00:554
271110ksoftirqd/20-21swapper/207:50:094
201110ksoftirqd/10-21swapper/110:45:191
201110ksoftirqd/10-21swapper/110:10:311
201110ksoftirqd/10-21swapper/108:41:231
181110rcuc/10-21swapper/109:45:121
1699110watchdog/00-21swapper/007:52:510
12250110irq/42-ahci0-21swapper/410:37:186
12250110irq/42-ahci0-21swapper/410:05:346
12250110irq/42-ahci0-21swapper/409:45:216
12250110irq/42-ahci0-21swapper/311:31:405
12250110irq/42-ahci0-21swapper/310:25:155
12250110irq/42-ahci0-21swapper/309:10:145
116099119cyclictest0-21swapper/1111:25:143
116099113cyclictest0-21swapper/1112:20:373
116099113cyclictest0-21swapper/1112:07:513
116099113cyclictest0-21swapper/1111:10:013
116099113cyclictest0-21swapper/1111:03:053
116099113cyclictest0-21swapper/1110:47:063
116099113cyclictest0-21swapper/1110:34:403
116099113cyclictest0-21swapper/1109:59:353
116099113cyclictest0-21swapper/1109:32:053
116099113cyclictest0-21swapper/1107:52:153
116099110cyclictest0-21swapper/1110:25:183
115699119cyclictest0-21swapper/709:10:119
115699119cyclictest0-21swapper/707:50:099
115699112cyclictest621ksoftirqd/710:18:249
115699112cyclictest621ksoftirqd/708:12:109
115699112cyclictest621ksoftirqd/707:55:239
115699112cyclictest0-21swapper/709:49:259
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional