You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 15:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Mon Dec 01, 2025 12:46:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501110irq/42-ahci0-21swapper/207:08:254
30664501090irq/53-eth0-tx-0-21swapper/307:05:145
122501000irq/42-ahci0-21swapper/107:09:001
3066350970irq/52-eth0-rx-0-21swapper/407:09:116
2950029690sleep90-21swapper/907:09:3211
2948029085sleep70-21swapper/707:09:189
2952528782sleep80-21swapper/807:09:5510
3066350850irq/52-eth0-rx-0-21swapper/507:05:297
3066350680irq/52-eth0-rx-0-21swapper/1107:05:193
2918626755sleep60-21swapper/607:06:058
2916226556sleep100-21swapper/1007:05:452
2916525955sleep00-21swapper/007:05:480
29686992813cyclictest1480-21jbd2/md0-809:14:033
2964899280cyclictest0-21swapper/008:10:230
29648992524cyclictest0-21swapper/010:14:280
29648992524cyclictest0-21swapper/010:14:280
29648992221cyclictest0-21swapper/007:10:260
2964899210cyclictest0-21swapper/007:30:280
2964899190cyclictest0-21swapper/007:20:230
2968499181cyclictest0-21swapper/911:05:1711
2968499181cyclictest0-21swapper/910:36:0311
2968499181cyclictest0-21swapper/909:26:3811
2968499181cyclictest0-21swapper/907:40:0111
29683991816cyclictest9706-21snmp_rack3slot908:07:1710
29683991816cyclictest576-21perl07:55:1610
29683991816cyclictest5354-21snmp_rack3slot909:30:1410
29683991816cyclictest31710-21iostat_ios10:50:1810
29683991816cyclictest30063-21diskstats09:20:1510
29683991816cyclictest29542-21df_inode08:35:1610
29683991816cyclictest28771-21snmp_rack3slot910:00:4810
29683991816cyclictest2876-21snmp_rack3slot907:15:1710
29683991816cyclictest26895-21snmp_rack3slot909:15:1610
29683991816cyclictest26303-21snmp_rack3slot908:30:1510
29683991816cyclictest20029-21users07:35:2810
29683991816cyclictest13501-21snmp_easybox.os09:40:2410
29683991816cyclictest12866-21snmp_rack3slot911:55:1110
29674991816cyclictest32678-21qemu-kvm10:20:148
29674991816cyclictest32678-21qemu-kvm10:10:158
29674991816cyclictest32678-21qemu-kvm10:10:148
29674991816cyclictest32678-21qemu-kvm10:08:248
29674991816cyclictest32677-21qemu-kvm10:45:288
29673991816cyclictest5336-21python08:00:257
29664991816cyclictest2406-21snmpd08:51:114
2964899180cyclictest0-21swapper/010:00:260
2964899180cyclictest0-21swapper/007:55:260
2968499170cyclictest5772-1kworker/9:0H09:39:4711
2968499170cyclictest0-21swapper/908:08:5811
2968499170cyclictest0-21swapper/907:10:2511
29683991715cyclictest9138-21snmp_rack3slot909:35:1610
29683991715cyclictest7214-21snmp_rack3slot907:20:1310
29683991715cyclictest6607-21perl11:45:1710
29683991715cyclictest439-21snmp_easybox.os09:25:1210
29683991715cyclictest3549-21snmp_easybox.os10:55:2610
29683991715cyclictest3403-21perl12:25:1510
29683991715cyclictest31040-21snmp_rack3slot907:10:1610
29683991715cyclictest29787-21snmp_rack3slot911:32:0310
29683991715cyclictest27332-21snmp_easybox.os12:15:1210
29683991715cyclictest27320-21diskstats10:45:1510
29683991715cyclictest26095-21snmp_rack3slot911:26:2710
29683991715cyclictest25711-21snmp_easybox.os12:10:2310
29683991715cyclictest23181-21df_inode09:55:1410
29683991715cyclictest23181-21df_inode09:55:1310
29683991715cyclictest22770-21snmp_rack3slot909:10:1310
29683991715cyclictest21602-21snmp_rack3slot909:52:1310
29683991715cyclictest21104-21snmp_rack3slot909:07:0310
29683991715cyclictest20777-21perl11:20:1310
29683991715cyclictest20047-21df_inode10:35:1410
29683991715cyclictest18067-21snmp_easybox.os11:15:2410
29683991715cyclictest17782-21snmp_easybox.os08:20:1310
29683991715cyclictest17331-21snmp_rack3slot909:00:3510
29683991715cyclictest16919-21snmp_rack3slot910:30:1810
29683991715cyclictest13952-21irqstats11:10:2110
29683991715cyclictest1333-21snmp_rack3slot912:21:3810
29683991715cyclictest12338-21snmp_rack3slot912:36:4910
29683991715cyclictest11497-21yum11:05:3210
29683991715cyclictest10529-21snmp_rack3slot907:25:1610
29683991715cyclictest10069-21snmp_rack3slot908:50:4810
2968099175cyclictest0-21swapper/712:10:009
29674991716cyclictest9818-21qemu-kvm08:08:178
29674991716cyclictest32678-21qemu-kvm10:29:528
29674991716cyclictest32678-21qemu-kvm10:16:288
29674991716cyclictest32677-21qemu-kvm10:41:398
29674991716cyclictest32677-21qemu-kvm10:38:288
29673991715cyclictest21233-21qemu-kvm09:12:557
2967299170cyclictest0-21swapper/411:55:216
29664991715cyclictest2406-21snmpd12:25:424
29664991715cyclictest2406-21snmpd11:10:574
29664991715cyclictest2406-21snmpd09:12:564
2964899170cyclictest0-21swapper/009:15:260
29684991615cyclictest741rcuc/912:31:4311
29684991615cyclictest0-21swapper/910:40:1811
29684991615cyclictest0-21swapper/907:25:2311
2968499161cyclictest0-21swapper/908:45:1511
29683991614cyclictest7858-21snmp_rack3slot911:00:5010
29683991614cyclictest31297-21df11:35:1210
29683991614cyclictest25304-21df07:45:1410
29683991614cyclictest22265-21diskstats08:25:1410
29683991614cyclictest21588-21iostat_ios12:05:2010
29683991614cyclictest16918-21yum08:15:3210
29683991614cyclictest16399-21snmp_easybox.os12:00:1110
29683991614cyclictest1508-21irqstats08:40:2010
29683991614cyclictest14487-21diskstats07:30:1410
29683991614cyclictest11341-21df_inode08:10:1210
2968099165cyclictest0-21swapper/708:35:009
29673991614cyclictest21234-21qemu-kvm09:08:597
2967299160cyclictest0-21swapper/411:36:176
2967299160cyclictest0-21swapper/409:10:016
29664991614cyclictest2406-21snmpd11:44:284
2964899160cyclictest0-21swapper/011:30:280
2964899160cyclictest0-21swapper/010:35:260
2964899160cyclictest0-21swapper/009:35:240
2964899160cyclictest0-21swapper/008:50:250
2964899160cyclictest0-21swapper/008:00:230
29686991514cyclictest881rcuc/1108:24:063
2968699150cyclictest560-21chrt07:11:573
29684991514cyclictest5772-1kworker/9:0H08:20:0011
29684991514cyclictest0-21swapper/909:20:2711
2968399154cyclictest0-21swapper/807:40:0110
29683991513cyclictest2190-21perl10:10:2510
29683991513cyclictest2190-21perl10:10:2510
2968099154cyclictest0-21swapper/711:05:029
2967499153cyclictest0-21swapper/609:00:018
29674991514cyclictest32677-21qemu-kvm10:32:578
29673991514cyclictest32727-21python11:35:257
29673991513cyclictest9989-21python09:35:267
29673991513cyclictest9818-21qemu-kvm08:08:437
29673991513cyclictest32678-21qemu-kvm10:26:367
2967299150cyclictest0-21swapper/410:01:446
2967299150cyclictest0-21swapper/407:59:416
29671991513cyclictest2406-21snmpd08:02:415
29664991513cyclictest2406-21snmpd07:50:264
2965999150cyclictest0-21swapper/107:10:231
2964899150cyclictest0-21swapper/012:02:300
2964899150cyclictest0-21swapper/008:45:590
2964899150cyclictest0-21swapper/008:28:360
2964899150cyclictest0-21swapper/008:20:520
2964899150cyclictest0-21swapper/008:08:340
2968499144cyclictest0-21swapper/912:35:0111
29684991413cyclictest741rcuc/911:33:5011
29684991413cyclictest13587-21chrt08:14:5011
29684991413cyclictest0-21swapper/907:20:1011
29684991412cyclictest26638-21iostat08:30:1811
2968399144cyclictest0-21swapper/809:45:0110
29683991412cyclictest13980-21snmp_easybox.os10:25:2510
2968099145cyclictest0-21swapper/711:35:009
2968099145cyclictest0-21swapper/710:35:019
29673991413cyclictest3832-21timerandwakeup10:55:297
29673991413cyclictest21234-21qemu-kvm09:44:597
2967299147cyclictest0-21swapper/408:55:246
2967299140cyclictest0-21swapper/411:23:336
2967299140cyclictest0-21swapper/410:15:176
2967299140cyclictest0-21swapper/408:10:326
2967299140cyclictest0-21swapper/407:10:426
29664991412cyclictest2406-21snmpd10:59:274
2964899148cyclictest0-21swapper/007:45:250
2964899146cyclictest0-21swapper/012:10:240
2964899146cyclictest0-21swapper/009:57:190
2964899146cyclictest0-21swapper/009:57:190
2964899146cyclictest0-21swapper/009:41:130
2964899146cyclictest0-21swapper/008:35:400
2964899146cyclictest0-21swapper/008:31:110
2964899146cyclictest0-21swapper/007:36:030
2964899145cyclictest0-21swapper/011:54:160
2964899140cyclictest0-21swapper/012:36:380
2964899140cyclictest0-21swapper/012:30:080
2964899140cyclictest0-21swapper/012:23:100
2964899140cyclictest0-21swapper/012:16:570
2964899140cyclictest0-21swapper/011:25:000
2964899140cyclictest0-21swapper/011:20:260
2964899140cyclictest0-21swapper/011:15:190
2964899140cyclictest0-21swapper/010:55:130
2964899140cyclictest0-21swapper/010:45:460
2964899140cyclictest0-21swapper/010:08:310
2964899140cyclictest0-21swapper/008:55:020
2964899140cyclictest0-21swapper/008:44:520
5999130watchdog/70-21swapper/710:47:229
3199130watchdog/30-21swapper/311:32:235
2968699134cyclictest0-21swapper/1112:15:003
2968699133cyclictest0-21swapper/1111:40:103
2968699133cyclictest0-21swapper/1107:38:413
29684991312cyclictest20697-21taskset08:22:3311
29684991312cyclictest0-21swapper/908:00:1311
2968499131cyclictest0-21swapper/911:15:2111
29683991311cyclictest4689-21snmp_rack3slot911:41:5110
29683991311cyclictest4231-21perl08:00:1410
2968099134cyclictest0-21swapper/709:55:019
2968099134cyclictest0-21swapper/707:20:019
2968099133cyclictest0-21swapper/711:49:339
2968099133cyclictest0-21swapper/710:25:019
2967499133cyclictest0-21swapper/612:18:438
2967399133cyclictest0-21swapper/512:00:167
29673991312cyclictest22694-21cpuspeed_turbos09:55:137
29673991312cyclictest22694-21cpuspeed_turbos09:55:127
29673991311cyclictest32677-21qemu-kvm10:08:407
29673991311cyclictest28179-21python10:00:277
2967299131cyclictest0-21swapper/407:19:006
2967299130cyclictest0-21swapper/411:47:276
2967299130cyclictest0-21swapper/409:40:586
2967299130cyclictest0-21swapper/409:39:366
2967299130cyclictest0-21swapper/409:04:436
2967299130cyclictest0-21swapper/407:39:596
2967299130cyclictest0-21swapper/407:30:486
2967299130cyclictest0-21swapper/407:27:216
2967199133cyclictest0-21swapper/308:10:165
2967199130cyclictest0-21swapper/308:27:385
2966499133cyclictest0-21swapper/210:20:174
29664991311cyclictest2443-21lldpd09:46:364
2964899137cyclictest0-21swapper/008:17:140
2964899136cyclictest0-21swapper/012:26:000
2964899136cyclictest0-21swapper/011:56:450
2964899136cyclictest0-21swapper/010:40:050
2964899136cyclictest0-21swapper/010:30:260
2964899136cyclictest0-21swapper/009:52:180
2964899136cyclictest0-21swapper/009:30:100
2964899135cyclictest0-21swapper/012:06:080
2964899135cyclictest0-21swapper/010:51:440
2964899135cyclictest0-21swapper/010:25:400
2964899134cyclictest0-21swapper/011:12:140
2964899134cyclictest0-21swapper/011:05:150
2964899134cyclictest0-21swapper/009:45:260
2964899131cyclictest0-21swapper/011:01:380
2964899131cyclictest0-21swapper/009:20:140
2964899131cyclictest0-21swapper/009:10:230
2964899131cyclictest0-21swapper/007:50:250
2964899131cyclictest0-21swapper/007:40:190
2964899130cyclictest18212-21munin-node09:05:140
2964899130cyclictest0-21swapper/011:39:130
2964899130cyclictest0-21swapper/010:20:220
2964899130cyclictest0-21swapper/009:25:020
2964899130cyclictest0-21swapper/009:02:010
2964899130cyclictest0-21swapper/007:25:130
2964899130cyclictest0-21swapper/007:15:210
7399120watchdog/90-21swapper/910:50:4611
551120ksoftirqd/60-21swapper/607:29:598
551120ksoftirqd/60-21swapper/607:20:518
481120ksoftirqd/50-21swapper/507:18:167
481120ksoftirqd/50-21swapper/507:13:427
4599120watchdog/50-21swapper/511:31:047
2968699124cyclictest0-21swapper/1108:40:003
2968699123cyclictest0-21swapper/1110:40:013
2968699123cyclictest0-21swapper/1109:36:403
2968699122cyclictest0-21swapper/1110:47:373
2968699122cyclictest0-21swapper/1109:55:363
2968699122cyclictest0-21swapper/1109:55:353
2968699122cyclictest0-21swapper/1109:33:403
29686991211cyclictest91602sleep1107:20:463
2968499124cyclictest0-21swapper/907:40:0111
2968499123cyclictest0-21swapper/911:45:0011
2968499122cyclictest0-21swapper/910:26:2611
29684991211cyclictest0-21swapper/908:40:2011
2968499120cyclictest2577-21munin-run08:00:0111
29683991210cyclictest12006-21perl08:55:1510
29683991210cyclictest11957-21snmp_rack3slot911:51:5310
2968099124cyclictest0-21swapper/712:00:009
2968099124cyclictest0-21swapper/711:15:009
2968099124cyclictest0-21swapper/711:05:019
2968099124cyclictest0-21swapper/709:20:029
2968099124cyclictest0-21swapper/709:05:019
2968099124cyclictest0-21swapper/708:45:029
2968099124cyclictest0-21swapper/708:05:019
2968099123cyclictest0-21swapper/708:00:019
2968099122cyclictest0-21swapper/710:16:069
2968099122cyclictest0-21swapper/707:48:519
2967499122cyclictest0-21swapper/612:09:228
2967499122cyclictest0-21swapper/607:32:328
29674991211cyclictest9819-21qemu-kvm08:10:468
29674991211cyclictest9818-21qemu-kvm08:36:528
29674991211cyclictest9818-21qemu-kvm08:33:428
29673991211cyclictest9819-21qemu-kvm08:20:157
29673991211cyclictest6296-21cstates12:30:137
2967299120cyclictest0-21swapper/412:05:366
2967199122cyclictest0-21swapper/312:35:195
2967199122cyclictest0-21swapper/309:55:175
2967199122cyclictest0-21swapper/309:55:175
29664991210cyclictest2406-21snmpd09:35:104
29664991210cyclictest2406-21snmpd09:17:274
2964899123cyclictest5669-21cpuspeed_turbos11:45:130
2964899121cyclictest0-21swapper/011:40:320
2964899120cyclictest0-21swapper/010:15:140
2499120watchdog/20-21swapper/209:24:504
831110ksoftirqd/100-21swapper/1008:55:142
831110ksoftirqd/100-21swapper/1008:20:162
6199110migration/720212-21munin-node09:05:279
551110ksoftirqd/60-21swapper/607:40:118
341110ksoftirqd/30-21swapper/308:55:145
341110ksoftirqd/30-21swapper/308:20:155
2968699114cyclictest0-21swapper/1109:25:183
2968699114cyclictest0-21swapper/1107:41:403
2968699113cyclictest0-21swapper/1111:30:003
2968699113cyclictest0-21swapper/1111:15:013
2968699113cyclictest0-21swapper/1110:05:013
2968699113cyclictest0-21swapper/1108:41:113
2968699113cyclictest0-21swapper/1107:25:163
2968699112cyclictest0-21swapper/1109:47:273
2968699111cyclictest0-21swapper/1107:34:333
2968599119cyclictest7262-21python11:00:272
2968599119cyclictest31855-21python10:05:282
2968599119cyclictest31421-21python09:20:282
2968599119cyclictest27318-1kworker/10:1H12:10:002
2968599119cyclictest27318-1kworker/10:1H09:46:092
2968599119cyclictest27318-1kworker/10:1H09:12:032
2968599119cyclictest27318-1kworker/10:1H08:15:042
2968599119cyclictest27318-1kworker/10:1H07:25:472
2968599119cyclictest27318-1kworker/10:1H07:20:132
2968599119cyclictest25047-21python10:40:312
2968599119cyclictest21410-21python10:35:292
2968599119cyclictest0-21swapper/1011:43:582
2968599119cyclictest0-21swapper/1009:55:362
2968599119cyclictest0-21swapper/1009:55:362
2968599119cyclictest0-21swapper/1007:53:382
29685991110cyclictest10934-21kworker/10:009:10:012
2968499113cyclictest0-21swapper/911:27:4011
2968499113cyclictest0-21swapper/910:33:1211
2968499113cyclictest0-21swapper/909:34:1011
2968499113cyclictest0-21swapper/909:14:1111
2968499113cyclictest0-21swapper/908:59:3111
2968499113cyclictest0-21swapper/908:27:5511
29684991110cyclictest5772-1kworker/9:0H08:51:1711
29684991110cyclictest0-21swapper/911:50:1911
2968499111cyclictest0-21swapper/907:16:3211
2968399119cyclictest5808-21perl10:15:1410
2968399113cyclictest0-21swapper/810:21:0710
2968099114cyclictest0-21swapper/712:35:009
2968099114cyclictest0-21swapper/708:20:019
2968099113cyclictest0-21swapper/712:35:139
2968099113cyclictest0-21swapper/711:34:229
2968099113cyclictest0-21swapper/707:59:519
2968099113cyclictest0-21swapper/707:15:019
2968099111cyclictest0-21swapper/712:02:589
2968099110cyclictest0-21swapper/710:10:229
2968099110cyclictest0-21swapper/710:10:229
2967499119cyclictest0-21swapper/609:15:128
2967499114cyclictest0-21swapper/611:15:278
2967499114cyclictest0-21swapper/608:40:268
2967499113cyclictest0-21swapper/612:27:128
2967499113cyclictest0-21swapper/611:32:098
2967499113cyclictest0-21swapper/611:20:488
2967499113cyclictest0-21swapper/609:41:068
2967499113cyclictest0-21swapper/609:20:018
2967499113cyclictest0-21swapper/609:12:068
2967499113cyclictest0-21swapper/607:50:368
2967499112cyclictest0-21swapper/609:31:378
29674991110cyclictest9819-21qemu-kvm08:21:478
29674991110cyclictest21234-21qemu-kvm09:08:168
2967499110cyclictest0-21swapper/611:55:008
2967399111cyclictest0-21swapper/508:47:117
2967299110cyclictest0-21swapper/411:11:216
2967299110cyclictest0-21swapper/407:43:296
2967199113cyclictest0-21swapper/310:25:185
2967199113cyclictest0-21swapper/308:50:175
2967199113cyclictest0-21swapper/307:55:195
2967199110cyclictest0-21swapper/311:27:555
2966499113cyclictest0-21swapper/210:10:284
2966499113cyclictest0-21swapper/210:10:284
2966499113cyclictest0-21swapper/209:50:184
2965999113cyclictest0-21swapper/110:35:181
2965999113cyclictest0-21swapper/108:55:171
2965999113cyclictest0-21swapper/108:35:191
2965999112cyclictest0-21swapper/108:30:181
2965999110cyclictest0-21swapper/109:25:131
271110ksoftirqd/20-21swapper/208:20:154
201110ksoftirqd/10-21swapper/107:20:131
181110rcuc/132549-21snmp_rack3slot910:07:061
1799110watchdog/10-21swapper/111:11:591
8799100watchdog/110-21swapper/1109:50:593
831100ksoftirqd/100-21swapper/1009:35:152
831100ksoftirqd/100-21swapper/1007:40:112
8099100watchdog/100-21swapper/1009:25:192
7599100migration/94171-21munin-node11:40:3011
621100ksoftirqd/70-21swapper/712:20:199
6199100migration/730096-21taskset12:16:019
6199100migration/724381-21runrttasks08:28:079
5999100watchdog/70-21swapper/711:28:599
551100ksoftirqd/60-21swapper/612:20:188
551100ksoftirqd/60-21swapper/611:25:328
551100ksoftirqd/60-21swapper/609:35:158
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional