You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 13:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Sat Nov 22, 2025 12:46:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501120irq/42-ahci0-21swapper/207:08:074
122501120irq/42-ahci0-21swapper/207:08:074
20484210894sleep80-21swapper/807:05:5310
20484210894sleep80-21swapper/807:05:5310
20641210593sleep70-21swapper/707:07:539
20641210593sleep70-21swapper/707:07:529
12250990irq/42-ahci0-21swapper/307:08:285
12250990irq/42-ahci0-21swapper/307:08:285
3066350940irq/52-eth0-rx-0-21swapper/907:06:1011
3066350940irq/52-eth0-rx-0-21swapper/907:06:1011
12250930irq/42-ahci0-21swapper/107:09:021
12250930irq/42-ahci0-21swapper/107:09:021
3066350880irq/52-eth0-rx-0-21swapper/407:05:136
3066350880irq/52-eth0-rx-0-21swapper/407:05:126
3066350860irq/52-eth0-rx-0-21swapper/507:05:307
3066350860irq/52-eth0-rx-0-21swapper/507:05:307
12250840irq/42-ahci0-21swapper/1107:08:573
12250840irq/42-ahci0-21swapper/1107:08:573
2062526757sleep60-21swapper/607:07:398
2062526757sleep60-21swapper/607:07:398
2053726755sleep00-21swapper/007:06:330
2053726755sleep00-21swapper/007:06:330
12250670irq/42-ahci0-21swapper/1007:05:482
12250670irq/42-ahci0-21swapper/1007:05:482
20959994645cyclictest0-21swapper/009:27:210
20959994025cyclictest0-21swapper/007:17:560
20959993617cyclictest0-21swapper/010:09:090
2095999290cyclictest0-21swapper/008:23:230
12250280irq/42-ahci0-21swapper/909:27:2011
12250280irq/42-ahci0-21swapper/809:27:2210
12250280irq/42-ahci0-21swapper/409:27:206
12250280irq/42-ahci0-21swapper/209:27:374
12250280irq/42-ahci0-21swapper/1009:27:212
12250280irq/42-ahci0-21swapper/109:27:211
20996992515cyclictest0-21swapper/611:29:578
20997992113cyclictest12250irq/42-ahci09:27:219
2095999200cyclictest0-21swapper/010:59:370
2100099192cyclictest12250irq/42-ahci12:01:362
2100099192cyclictest12250irq/42-ahci11:51:452
2100099192cyclictest12250irq/42-ahci11:46:092
2100099192cyclictest12250irq/42-ahci10:04:342
2100099192cyclictest12250irq/42-ahci09:45:492
2100099192cyclictest12250irq/42-ahci09:09:392
2100099192cyclictest12250irq/42-ahci07:56:342
2100099192cyclictest12250irq/42-ahci07:26:162
21002991817cyclictest12250irq/42-ahci10:26:383
21002991817cyclictest12250irq/42-ahci10:21:373
21000991817cyclictest0-21swapper/1009:41:052
21000991817cyclictest0-21swapper/1009:30:512
21000991817cyclictest0-21swapper/1007:20:082
2100099181cyclictest12250irq/42-ahci12:26:532
2100099181cyclictest12250irq/42-ahci12:21:412
2100099181cyclictest12250irq/42-ahci12:16:342
2100099181cyclictest12250irq/42-ahci12:11:232
2100099181cyclictest12250irq/42-ahci12:08:092
2100099181cyclictest12250irq/42-ahci11:55:162
2100099181cyclictest12250irq/42-ahci11:44:572
2100099181cyclictest12250irq/42-ahci11:38:292
2100099181cyclictest12250irq/42-ahci11:31:172
2100099181cyclictest12250irq/42-ahci11:20:092
2100099181cyclictest12250irq/42-ahci11:17:422
2100099181cyclictest12250irq/42-ahci11:10:502
2100099181cyclictest12250irq/42-ahci11:00:352
2100099181cyclictest12250irq/42-ahci10:58:152
2100099181cyclictest12250irq/42-ahci10:54:402
2100099181cyclictest12250irq/42-ahci10:41:312
2100099181cyclictest12250irq/42-ahci10:39:422
2100099181cyclictest12250irq/42-ahci10:27:262
2100099181cyclictest12250irq/42-ahci10:18:302
2100099181cyclictest12250irq/42-ahci10:13:552
2100099181cyclictest12250irq/42-ahci09:58:062
2100099181cyclictest12250irq/42-ahci09:39:592
2100099181cyclictest12250irq/42-ahci09:22:412
2100099181cyclictest12250irq/42-ahci09:16:082
2100099181cyclictest12250irq/42-ahci08:55:202
2100099181cyclictest12250irq/42-ahci08:52:392
2100099181cyclictest12250irq/42-ahci08:47:572
2100099181cyclictest12250irq/42-ahci08:40:262
2100099181cyclictest12250irq/42-ahci08:19:342
2100099181cyclictest12250irq/42-ahci08:11:192
2100099181cyclictest12250irq/42-ahci08:06:292
2100099181cyclictest12250irq/42-ahci08:06:292
2100099181cyclictest12250irq/42-ahci08:04:402
2100099181cyclictest12250irq/42-ahci07:51:172
2100099181cyclictest12250irq/42-ahci07:45:552
2100099181cyclictest12250irq/42-ahci07:43:012
2100099181cyclictest12250irq/42-ahci07:37:422
2100099181cyclictest12250irq/42-ahci07:31:542
2100099181cyclictest12250irq/42-ahci07:13:002
2100099181cyclictest12250irq/42-ahci07:13:002
20998991816cyclictest7448-21snmp_rack3slot910:30:1510
20998991816cyclictest6809-21perl09:00:1710
20998991816cyclictest6306-21snmp_rack3slot907:30:1610
20998991816cyclictest4585-21snmp_rack3slot910:25:1310
20998991816cyclictest32712-21snmp_rack3slot909:35:1810
20998991816cyclictest29171-21snmp_easybox.os08:45:2510
20998991816cyclictest29116-21snmp_easybox.os11:00:1210
20998991816cyclictest28827-21snmp_rack3slot911:41:1310
20998991816cyclictest28622-21snmp_rack3slot909:30:1610
20998991816cyclictest27398-21snmp_rack3slot907:15:2010
20998991816cyclictest22545-21snmp_rack3slot907:10:1610
20998991816cyclictest22545-21snmp_rack3slot907:10:1610
20998991816cyclictest22499-21snmp_rack3slot907:50:4210
20998991816cyclictest21923-21diskstats10:05:1510
20998991816cyclictest19649-21iostat10:45:2010
20998991816cyclictest17039-21ntp_states11:25:2310
20998991816cyclictest15756-21snmp_rack3slot910:40:1710
20998991816cyclictest13183-21munin-run09:55:0110
20996991816cyclictest12405-21qemu-kvm09:09:258
20984991816cyclictest18965-21sensors09:15:366
2095999180cyclictest0-21swapper/009:33:470
21000991716cyclictest0-21swapper/1012:36:012
21000991716cyclictest0-21swapper/1012:36:012
21000991716cyclictest0-21swapper/1011:05:082
21000991716cyclictest0-21swapper/1010:20:282
21000991716cyclictest0-21swapper/1008:35:232
2100099170cyclictest85352sleep1009:01:072
2100099170cyclictest179962sleep1011:28:032
2100099170cyclictest12250irq/42-ahci12:30:482
2100099170cyclictest12250irq/42-ahci10:45:322
2100099170cyclictest12250irq/42-ahci10:30:182
2100099170cyclictest12250irq/42-ahci10:05:132
2100099170cyclictest12250irq/42-ahci09:10:122
2100099170cyclictest12250irq/42-ahci08:30:382
2100099170cyclictest12250irq/42-ahci08:25:342
2100099170cyclictest12250irq/42-ahci08:20:102
2100099170cyclictest12250irq/42-ahci07:15:582
2100099170cyclictest100422sleep1009:50:152
20998991715cyclictest992-21snmp_rack3slot910:20:1510
20998991715cyclictest9364-21snmp_easybox.os09:05:1110
20998991715cyclictest9140-21users09:45:3510
20998991715cyclictest7479-21snmp_rack3slot911:56:3910
20998991715cyclictest6843-21snmp_rack3slot911:10:3610
20998991715cyclictest4861-21snmp_easybox.os09:40:2610
20998991715cyclictest30941-21snmp_rack3slot907:20:1710
20998991715cyclictest30901-21snmp_rack3slot912:30:1610
20998991715cyclictest29462-21snmp_rack3slot912:26:5010
20998991715cyclictest29287-21df_inode10:15:1510
20998991715cyclictest27914-21munin-run10:15:0010
20998991715cyclictest23390-21snmp_easybox.os07:55:1310
20998991715cyclictest21606-21snmp_rack3slot908:35:1710
20998991715cyclictest21289-21snmp_rack3slot909:20:1410
20998991715cyclictest2036-21perl11:50:1510
20998991715cyclictest18855-21snmp_rack3slot911:30:1310
20998991715cyclictest17985-21snmp_rack3slot908:30:1610
20998991715cyclictest12831-21snmp_rack3slot907:40:1410
2099699176cyclictest11429-21munin-run12:05:018
20996991715cyclictest12405-21qemu-kvm09:36:158
20989991716cyclictest1307-21qemu-kvm08:08:457
20989991716cyclictest1307-21qemu-kvm08:08:447
2097799179cyclictest0-21swapper/311:55:265
20977991710cyclictest12250irq/42-ahci09:27:285
20959991716cyclictest0-21swapper/012:26:140
20959991716cyclictest0-21swapper/009:39:240
2095999170cyclictest0-21swapper/011:10:510
12250170irq/42-ahci0-21swapper/1109:27:363
21002991615cyclictest12250irq/42-ahci10:17:243
21002991615cyclictest0-21swapper/1107:19:273
20998991614cyclictest4061-21snmp_easybox.os08:55:2410
20998991614cyclictest3681-21iostat_ios08:10:1910
20998991614cyclictest32512-21snmp_rack3slot911:46:3010
20998991614cyclictest23922-21iostat11:35:1810
20998991614cyclictest2165-21snmp_rack3slot907:25:1410
20998991614cyclictest18504-21perl10:00:1510
20998991614cyclictest1556-21perl11:05:1610
20998991614cyclictest14786-21snmp_rack3slot912:06:1410
20998991614cyclictest13505-21snmp_rack3slot909:10:1110
20998991614cyclictest12952-21iostat_ios11:20:1910
20998991614cyclictest1150-21snmp_rack3slot908:52:0210
20998991614cyclictest10574-21snmp_rack3slot908:20:1610
20996991615cyclictest12406-21qemu-kvm09:21:448
20996991615cyclictest12405-21qemu-kvm09:19:068
20989991615cyclictest1307-21qemu-kvm08:29:547
20984991614cyclictest12943-21sensors10:35:276
2097799160cyclictest0-21swapper/312:07:515
2097799160cyclictest0-21swapper/311:30:095
2097799160cyclictest0-21swapper/310:50:245
2097799160cyclictest0-21swapper/309:47:165
2097799160cyclictest0-21swapper/309:37:195
20971991614cyclictest2406-21snmpd09:18:244
20971991614cyclictest2406-21snmpd07:24:234
20959991615cyclictest12250irq/42-ahci10:24:330
20959991615cyclictest0-21swapper/012:11:030
20959991615cyclictest0-21swapper/008:54:220
20959991614cyclictest4379-21chrt07:28:110
2095999160cyclictest0-21swapper/011:53:340
2100299153cyclictest0-21swapper/1108:37:223
20998991513cyclictest9786-21perl07:35:1510
20998991513cyclictest24975-21iostat_ios08:40:2110
20998991513cyclictest24718-21memory12:20:2110
20998991513cyclictest22090-21snmp_rack3slot912:16:0110
20998991513cyclictest16889-21diskstats07:45:1810
20998991513cyclictest16592-21snmp_rack3slot909:56:5110
2099799151cyclictest0-21swapper/709:01:579
20996991514cyclictest12406-21qemu-kvm09:13:408
20989991514cyclictest1307-21qemu-kvm08:22:337
20989991514cyclictest1306-21qemu-kvm08:45:367
20989991514cyclictest12250irq/42-ahci09:27:217
20989991513cyclictest20702-21qemu-kvm07:30:157
2098999150cyclictest0-21swapper/511:20:257
2098999150cyclictest0-21swapper/510:30:287
2098999150cyclictest0-21swapper/510:20:167
2098999150cyclictest0-21swapper/508:55:007
20984991514cyclictest16601-21sensors10:40:266
20984991513cyclictest8222sleep412:33:526
2097799158cyclictest0-21swapper/308:37:575
2097799157cyclictest0-21swapper/307:57:395
2097799150cyclictest0-21swapper/312:35:235
2097799150cyclictest0-21swapper/312:35:235
2097799150cyclictest0-21swapper/312:19:225
2097799150cyclictest0-21swapper/312:12:035
2097799150cyclictest0-21swapper/311:26:155
2097799150cyclictest0-21swapper/310:13:195
2097799150cyclictest0-21swapper/310:06:125
2097799150cyclictest0-21swapper/309:59:455
2097799150cyclictest0-21swapper/309:42:405
2097799150cyclictest0-21swapper/309:10:495
2097799150cyclictest0-21swapper/307:12:525
2097799150cyclictest0-21swapper/307:12:525
2097199150cyclictest0-21swapper/210:30:284
20960991514cyclictest3066350irq/52-eth0-rx-10:40:261
20960991514cyclictest27595-21python10:55:261
20959991514cyclictest12250irq/42-ahci10:29:400
20959991514cyclictest12250irq/42-ahci10:18:450
12250150irq/42-ahci0-21swapper/912:20:3711
12250150irq/42-ahci0-21swapper/912:11:3311
12250150irq/42-ahci0-21swapper/912:05:0411
12250150irq/42-ahci0-21swapper/911:40:1911
12250150irq/42-ahci0-21swapper/911:17:4411
21002991413cyclictest12250irq/42-ahci08:19:143
20998991412cyclictest8916-21perl11:15:1710
20998991412cyclictest27533-21snmp_rack3slot910:55:2510
20998991412cyclictest2612-21perl12:35:1610
20998991412cyclictest2612-21perl12:35:1610
20998991412cyclictest16467-21diskstats12:10:1510
20998991412cyclictest13928-21diskstats08:25:1510
20998991412cyclictest11160-21snmp_rack3slot912:01:3710
2099799143cyclictest0-21swapper/712:05:019
2099699148cyclictest24102-21qemu-kvm10:30:018
2099699146cyclictest0-21swapper/607:36:428
2099699143cyclictest0-21swapper/612:34:558
2099699143cyclictest0-21swapper/612:23:568
20996991413cyclictest12406-21qemu-kvm09:42:068
20989991413cyclictest15759-21cstates_1111:25:137
2098499140cyclictest0-21swapper/408:15:126
2097799147cyclictest0-21swapper/309:33:505
2097799147cyclictest0-21swapper/308:27:135
2097799140cyclictest0-21swapper/312:04:065
2097799140cyclictest0-21swapper/310:27:095
2097799140cyclictest0-21swapper/308:55:465
2097799140cyclictest0-21swapper/308:16:185
2097799140cyclictest0-21swapper/307:42:085
2097799140cyclictest0-21swapper/307:27:125
20971991412cyclictest2406-21snmpd10:05:544
20971991412cyclictest2406-21snmpd08:07:384
20971991412cyclictest2406-21snmpd08:07:384
2096099145cyclictest0-21swapper/107:25:181
2095999147cyclictest0-21swapper/009:23:170
20959991413cyclictest195762sleep009:16:000
2095999140cyclictest623-21md0_raid108:02:440
2095999140cyclictest0-21swapper/010:35:380
1799140watchdog/10-21swapper/110:53:021
13450140irq/18-ehci_hcd0-21swapper/009:10:250
12250140irq/42-ahci0-21swapper/909:39:3811
2100299133cyclictest0-21swapper/1112:35:013
2100299133cyclictest0-21swapper/1112:35:003
2099899133cyclictest0-21swapper/810:35:0110
2099899133cyclictest0-21swapper/808:20:0110
20998991311cyclictest31573-21diskstats08:05:1610
20998991311cyclictest31573-21diskstats08:05:1610
20998991311cyclictest29011-21snmp_easybox.os08:00:2510
20998991311cyclictest23339-21irqstats10:50:2010
2099799135cyclictest0-21swapper/710:25:189
2099699134cyclictest0-21swapper/611:10:018
2099699133cyclictest0-21swapper/611:20:008
2099699133cyclictest0-21swapper/607:40:018
20996991311cyclictest12406-21qemu-kvm09:45:378
20989991312cyclictest1307-21qemu-kvm08:14:437
2098999130cyclictest0-21swapper/511:45:287
20984991311cyclictest1866-21sensors10:20:256
2097799130cyclictest0-21swapper/311:00:145
2097799130cyclictest0-21swapper/310:48:595
2097799130cyclictest0-21swapper/310:43:595
2097799130cyclictest0-21swapper/308:53:555
2097799130cyclictest0-21swapper/308:30:355
20971991311cyclictest2406-21snmpd11:35:544
2096099133cyclictest0-21swapper/109:50:201
2096099133cyclictest0-21swapper/109:40:191
8799120watchdog/110-21swapper/1108:09:423
8799120watchdog/110-21swapper/1108:09:423
621120ksoftirqd/70-21swapper/711:38:289
551120ksoftirqd/60-21swapper/611:38:288
551120ksoftirqd/60-21swapper/611:31:418
481120ksoftirqd/520716-21qemu-kvm07:26:267
481120ksoftirqd/50-21swapper/511:31:417
481120ksoftirqd/50-21swapper/510:19:367
4599120watchdog/50-21swapper/508:34:267
31120ksoftirqd/00-21swapper/011:38:280
271120ksoftirqd/20-21swapper/211:31:414
2100299122cyclictest0-21swapper/1107:28:573
2099899123cyclictest0-21swapper/809:15:0210
2099799123cyclictest0-21swapper/711:15:009
2099799123cyclictest0-21swapper/708:26:579
2099799122cyclictest0-21swapper/711:35:029
2099799122cyclictest0-21swapper/710:56:319
2099799122cyclictest0-21swapper/707:11:129
2099799122cyclictest0-21swapper/707:11:129
2099699123cyclictest0-21swapper/608:55:018
2099699122cyclictest0-21swapper/608:10:538
20996991211cyclictest12405-21qemu-kvm09:34:118
2099699121cyclictest0-21swapper/610:24:538
2098499123cyclictest0-21swapper/412:25:196
2098499123cyclictest0-21swapper/408:50:186
20984991210cyclictest10910-21sensors07:35:256
2098499120cyclictest0-21swapper/407:55:016
2097799127cyclictest0-21swapper/308:47:175
2097799120cyclictest0-21swapper/308:40:065
2097799120cyclictest0-21swapper/307:18:045
20971991211cyclictest2406-21snmpd10:19:094
2096099123cyclictest0-21swapper/112:30:201
2096099123cyclictest0-21swapper/110:35:011
2096099123cyclictest0-21swapper/110:15:191
2096099123cyclictest0-21swapper/109:30:201
2096099122cyclictest0-21swapper/109:35:211
20960991211cyclictest3066350irq/52-eth0-rx-11:15:281
2096099121cyclictest0-21swapper/108:15:131
2096099120cyclictest623-21md0_raid108:35:151
20959991211cyclictest0-21swapper/012:16:330
20959991210cyclictest0-21swapper/007:20:300
2095999120cyclictest0-21swapper/011:15:250
2095999120cyclictest0-21swapper/011:00:370
2095999120cyclictest0-21swapper/007:55:160
12250120irq/42-ahci26994-1kworker/9:0H12:28:0111
901110ksoftirqd/110-21swapper/1111:45:293
901110ksoftirqd/110-21swapper/1111:31:413
901110ksoftirqd/110-21swapper/1109:55:203
8999110migration/1122993-21munin-node10:05:233
761110ksoftirqd/90-21swapper/911:21:0411
761110ksoftirqd/90-21swapper/907:47:0311
761110ksoftirqd/90-21swapper/907:37:1411
621110ksoftirqd/70-21swapper/712:30:269
621110ksoftirqd/70-21swapper/711:21:049
621110ksoftirqd/70-21swapper/710:52:449
621110ksoftirqd/70-21swapper/710:12:439
621110ksoftirqd/70-21swapper/707:47:039
621110ksoftirqd/70-21swapper/707:25:159
551110ksoftirqd/620715-21qemu-kvm07:25:158
551110ksoftirqd/60-21swapper/612:09:208
551110ksoftirqd/60-21swapper/611:43:028
551110ksoftirqd/60-21swapper/610:34:578
481110ksoftirqd/50-21swapper/510:37:327
481110ksoftirqd/50-21swapper/510:27:387
481110ksoftirqd/50-21swapper/510:11:557
481110ksoftirqd/50-21swapper/510:09:407
481110ksoftirqd/50-21swapper/508:35:527
481110ksoftirqd/50-21swapper/508:18:117
481110ksoftirqd/50-21swapper/507:47:037
411110ksoftirqd/40-21swapper/410:56:086
411110ksoftirqd/40-21swapper/410:52:446
411110ksoftirqd/40-21swapper/410:34:576
411110ksoftirqd/40-21swapper/410:12:436
411110ksoftirqd/40-21swapper/410:03:076
411110ksoftirqd/40-21swapper/407:26:266
341110ksoftirqd/3623-21md0_raid112:30:265
341110ksoftirqd/30-21swapper/307:37:145
3199110watchdog/30-21swapper/307:20:185
31110ksoftirqd/00-21swapper/012:30:260
31110ksoftirqd/00-21swapper/011:21:040
31110ksoftirqd/00-21swapper/010:50:080
31110ksoftirqd/00-21swapper/010:34:570
31110ksoftirqd/00-21swapper/009:45:300
31110ksoftirqd/00-21swapper/008:28:530
31110ksoftirqd/00-21swapper/007:50:330
271110ksoftirqd/20-21swapper/210:12:434
271110ksoftirqd/20-21swapper/209:55:204
2100299119cyclictest0-21swapper/1110:35:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional