You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 08:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Mon Dec 01, 2025 00:46:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501110irq/42-ahci0-21swapper/119:08:241
27221210694sleep70-21swapper/719:07:329
27212210694sleep110-21swapper/1119:07:243
3066350990irq/52-eth0-rx-0-21swapper/419:06:266
12250970irq/42-ahci0-21swapper/219:09:434
3066350890irq/52-eth0-rx-0-21swapper/519:05:217
2715128274sleep00-21swapper/019:06:370
12250800irq/42-ahci0-21swapper/319:08:105
2736826558sleep80-21swapper/819:08:4710
2712626553sleep60-21swapper/619:06:198
2711526455sleep90-21swapper/919:06:1011
2708826355sleep100-21swapper/1019:05:472
2760899380cyclictest0-21swapper/1119:17:513
27608993113cyclictest207362sleep1120:25:203
27608992711cyclictest134302chrt20:15:193
2760899210cyclictest0-21swapper/1120:20:173
2760899210cyclictest0-21swapper/1120:09:333
2760899200cyclictest115162sleep1120:14:553
2760699190cyclictest0-21swapper/922:14:5611
2759799194cyclictest0-21swapper/720:50:019
2759499190cyclictest0-21swapper/421:30:166
3066350180irq/52-eth0-rx-0-21swapper/920:26:1211
27608991817cyclictest0-21swapper/1119:21:563
27607991817cyclictest0-21swapper/1022:45:292
2760799181cyclictest3066350irq/52-eth0-rx-20:36:232
2760799181cyclictest12250irq/42-ahci21:06:542
2760799181cyclictest12250irq/42-ahci21:01:372
2760799181cyclictest12250irq/42-ahci00:20:162
2760699181cyclictest0-21swapper/923:30:1311
2760699181cyclictest0-21swapper/923:30:1211
2760699181cyclictest0-21swapper/922:55:1311
2760699181cyclictest0-21swapper/922:25:3811
27598991816cyclictest6482-21df21:35:1410
27598991816cyclictest6177-21snmp_rack3slot923:45:4810
27598991816cyclictest30964-21snmp_rack3slot922:52:0510
27598991816cyclictest2876-21snmp_rack3slot920:45:1810
27598991816cyclictest24062-21snmp_rack3slot923:26:5210
27598991816cyclictest22635-21snmp_rack3slot919:45:1110
27598991816cyclictest18075-21snmp_rack3slot921:05:2710
27598991816cyclictest16468-21snmp_rack3slot920:20:1510
27598991816cyclictest13424-21snmp_rack3slot921:45:1310
27598991816cyclictest13406-21hddtemp_smartct20:15:1810
27598991816cyclictest11661-21snmp_rack3slot920:56:1410
2759499189cyclictest18347-21sensors21:05:326
2759499180cyclictest0-21swapper/419:10:166
12250180irq/42-ahci0-21swapper/420:27:326
12250180irq/42-ahci0-21swapper/120:26:531
3066350170irq/52-eth0-rx-2760799cyclictest22:05:222
31170ksoftirqd/00-21swapper/022:30:300
27608991715cyclictest30238-21chrt21:24:513
27607991716cyclictest6927-21python20:05:292
27607991716cyclictest27318-1kworker/10:1H00:30:312
27607991716cyclictest23388-21python23:25:312
27607991716cyclictest0-21swapper/1023:15:362
27607991716cyclictest0-21swapper/1021:10:122
2760799170cyclictest77382sleep1000:39:472
2760799170cyclictest61372sleep1023:45:292
2760799170cyclictest57922sleep1023:00:492
2760799170cyclictest56052sleep1022:18:562
2760799170cyclictest48862sleep1021:30:522
2760799170cyclictest32572chrt20:00:292
2760799170cyclictest313322chrt23:36:412
2760799170cyclictest311452sleep1019:14:352
2760799170cyclictest310852sleep1022:53:482
2760799170cyclictest307672sleep1021:25:022
2760799170cyclictest3066450irq/53-eth0-tx-22:20:162
2760799170cyclictest3066350irq/52-eth0-rx-22:30:162
2760799170cyclictest3066350irq/52-eth0-rx-22:13:162
2760799170cyclictest3066350irq/52-eth0-rx-21:50:042
2760799170cyclictest3066350irq/52-eth0-rx-21:45:232
2760799170cyclictest3066350irq/52-eth0-rx-20:55:352
2760799170cyclictest3066350irq/52-eth0-rx-20:10:332
2760799170cyclictest3066350irq/52-eth0-rx-19:52:292
2760799170cyclictest3066350irq/52-eth0-rx-19:35:132
2760799170cyclictest3066350irq/52-eth0-rx-19:30:192
2760799170cyclictest302552sleep1020:40:092
2760799170cyclictest297522chrt19:55:042
2760799170cyclictest297292sleep1000:25:112
2760799170cyclictest28452sleep1019:15:352
2760799170cyclictest280082sleep1021:20:132
2760799170cyclictest27832sleep1023:44:182
2760799170cyclictest27832sleep1023:44:172
2760799170cyclictest261532sleep1023:30:182
2760799170cyclictest261532sleep1023:30:172
2760799170cyclictest253802chrt22:00:172
2760799170cyclictest253532sleep1000:17:092
2760799170cyclictest252722sleep1019:45:502
2760799170cyclictest236592sleep1021:15:132
2760799170cyclictest231552sleep1021:57:062
2760799170cyclictest221502sleep1020:25:552
2760799170cyclictest216622sleep1000:11:212
2760799170cyclictest216482sleep1019:41:032
2760799170cyclictest21122sleep1022:56:252
2760799170cyclictest186782chrt22:35:222
2760799170cyclictest181042sleep1000:07:212
2760799170cyclictest173522sleep1000:03:562
2760799170cyclictest148942sleep1020:16:192
2760799170cyclictest14492sleep1020:45:112
2760799170cyclictest135932sleep1023:57:432
2760799170cyclictest122502sleep1021:42:162
2760799170cyclictest12250irq/42-ahci23:23:242
2760799170cyclictest12250irq/42-ahci22:40:522
2760799170cyclictest12250irq/42-ahci22:27:522
2760799170cyclictest12250irq/42-ahci19:20:062
2760799170cyclictest107062sleep1019:25:382
27606991715cyclictest6951-21taskset20:05:3211
2760699170cyclictest0-21swapper/921:40:3411
2760699170cyclictest0-21swapper/921:30:2211
2760699170cyclictest0-21swapper/920:55:2311
2760699170cyclictest0-21swapper/919:45:1411
2759899175cyclictest0-21swapper/821:20:2010
27598991716cyclictest32583-21snmp_rack3slot900:26:2610
27598991716cyclictest32227-21snmp_rack3slot922:10:1510
27598991716cyclictest28947-21snmp_rack3slot900:21:2510
27598991716cyclictest20322-21snmp_easybox.os21:55:1510
27598991716cyclictest16630-21snmp_easybox.os21:50:1310
27598991716cyclictest16324-21snmp_easybox.os00:00:2510
27598991716cyclictest1448-21memory23:40:2210
27598991716cyclictest1448-21memory23:40:2210
27598991716cyclictest12647-21yum22:25:4110
27598991715cyclictest9402-21snmp_easybox.os21:40:1210
27598991715cyclictest9025-21snmp_rack3slot919:25:1410
27598991715cyclictest7567-21snmp_rack3slot920:06:2410
27598991715cyclictest5839-21perl00:35:1710
27598991715cyclictest3923-21snmp_rack3slot900:32:1610
27598991715cyclictest3850-21diskstats23:00:1510
27598991715cyclictest31662-21snmp_rack3slot921:25:1610
27598991715cyclictest31225-21users23:35:3210
27598991715cyclictest29415-21hddtemp_smartct19:10:1710
27598991715cyclictest27798-21perl20:35:1710
27598991715cyclictest21593-21yum00:10:2710
27598991715cyclictest20966-21snmp_easybox.os20:25:2510
27598991715cyclictest20576-21snmp_rack3slot921:10:1310
27598991715cyclictest20426-21snmp_rack3slot923:21:4910
27598991715cyclictest14991-21perl23:15:1710
27598991715cyclictest1379-21snmp_rack3slot919:15:1810
27598991715cyclictest13515-21snmp_easybox.os19:30:2510
27598991715cyclictest12666-21ntp_states23:55:2310
27598991715cyclictest11185-21users20:10:3510
2759799174cyclictest0-21swapper/719:25:199
2756999170cyclictest0-21swapper/020:26:560
2756999170cyclictest0-21swapper/020:16:060
13450170irq/18-ehci_hcd0-21swapper/021:00:290
12250170irq/42-ahci0-21swapper/920:16:3611
27608991614cyclictest22881-21chrt21:14:033
27607991615cyclictest7419-21expr23:50:142
27606991615cyclictest30794-1kworker/9:2H21:08:4211
27606991614cyclictest29664-21iostat_ios19:10:2111
27606991614cyclictest17111-21taskset22:35:0111
2760699160cyclictest0-21swapper/920:00:1711
27598991615cyclictest4729-21snmp_rack3slot919:20:1510
27598991615cyclictest398-21perl22:55:1510
27598991615cyclictest25889-21perl23:30:1510
27598991615cyclictest25889-21perl23:30:1510
27598991615cyclictest2050-21snmp_easybox.os21:30:1310
27598991615cyclictest15276-21snmp_rack3slot921:00:4910
27598991615cyclictest14596-21perl22:30:1510
27598991614cyclictest7074-21snmp_easybox.os23:50:1310
27598991614cyclictest6984-21snmp_rack3slot922:20:1210
27598991614cyclictest6821-21snmp_easybox.os20:50:2510
27598991614cyclictest3983-21snmp_rack3slot920:02:0210
27598991614cyclictest32135-21snmp_easybox.os20:40:2510
27598991614cyclictest28960-21snmp_rack3slot919:51:1610
27598991614cyclictest23713-21perl00:15:2410
27598991614cyclictest20851-21snmp_easybox.os19:40:2510
2759599160cyclictest0-21swapper/522:20:297
2759499160cyclictest0-21swapper/420:21:236
27591991614cyclictest2406-21snmpd19:55:505
2759199161cyclictest0-21swapper/320:05:125
2759199160cyclictest0-21swapper/322:45:305
27585991614cyclictest2406-21snmpd23:38:504
27585991614cyclictest2406-21snmpd00:19:524
12250160irq/42-ahci0-21swapper/320:16:355
3066450150irq/53-eth0-tx-23672-21munin-node20:30:165
3066350150irq/52-eth0-rx-0-21swapper/420:13:596
3066350150irq/52-eth0-rx-0-21swapper/220:30:194
3066350150irq/52-eth0-rx-0-21swapper/220:21:254
27607991514cyclictest27046-1kworker/10:2H20:30:312
2760699154cyclictest0-21swapper/919:30:0111
27606991513cyclictest30023-21taskset21:21:5211
27598991514cyclictest32626-21snmp_rack3slot919:56:3710
27598991514cyclictest23708-21snmp_rack3slot920:30:1410
27598991513cyclictest20920-21snmp_rack3slot922:40:1410
27598991513cyclictest17190-21snmp_rack3slot922:35:1510
27598991513cyclictest16397-21perl19:35:1610
27598991513cyclictest13401-21munin-run23:15:0110
27597991513cyclictest9595-21seq23:07:419
27596991514cyclictest30607-21qemu-kvm22:16:518
2759599150cyclictest0-21swapper/522:45:377
2759499153cyclictest0-21swapper/423:10:226
2759499153cyclictest0-21swapper/422:20:176
2759499153cyclictest0-21swapper/421:55:216
2759199150cyclictest0-21swapper/322:35:185
2759199150cyclictest0-21swapper/320:55:165
2758599150cyclictest0-21swapper/221:30:154
2758599150cyclictest0-21swapper/219:10:224
27577991514cyclictest12250irq/42-ahci20:13:561
13450150irq/18-ehci_hcd0-21swapper/020:55:310
12250150irq/42-ahci0-21swapper/020:30:040
3066450140irq/53-eth0-tx-0-21swapper/120:24:511
3066350140irq/52-eth0-rx-0-21swapper/720:17:179
3066350140irq/52-eth0-rx-0-21swapper/220:15:594
3066350140irq/52-eth0-rx-0-21swapper/1020:24:062
27608991412cyclictest9217-21chrt22:23:333
27607991413cyclictest0-21swapper/1021:35:212
2760699143cyclictest0-21swapper/922:00:0011
27606991413cyclictest0-21swapper/923:15:2411
27606991413cyclictest0-21swapper/922:47:3411
27606991413cyclictest0-21swapper/921:15:2411
27598991413cyclictest8695-21snmp_easybox.os23:05:2610
27598991413cyclictest30477-21snmp_rack3slot922:07:0310
27598991413cyclictest25609-21snmp_rack3slot922:45:1710
27598991413cyclictest25039-21perl22:00:1510
2759799145cyclictest0-21swapper/720:40:009
2759799144cyclictest0-21swapper/721:55:009
2759799144cyclictest0-21swapper/700:25:019
27597991412cyclictest31105-21hddtemp_smartct19:55:169
27595991413cyclictest8139-21cpuspeed_turbos19:25:127
2758599146cyclictest0-21swapper/219:45:184
27585991412cyclictest2406-21snmpd19:55:044
27585991410cyclictest0-21swapper/220:15:004
2758599140cyclictest0-21swapper/223:51:024
2758599140cyclictest0-21swapper/223:49:574
2758599140cyclictest0-21swapper/221:59:224
2758599140cyclictest0-21swapper/221:48:524
2758599140cyclictest0-21swapper/221:14:164
2758599140cyclictest0-21swapper/221:02:234
2758599140cyclictest0-21swapper/219:36:434
27577991413cyclictest3066350irq/52-eth0-rx-19:30:161
27577991413cyclictest3066350irq/52-eth0-rx-00:10:171
27569991412cyclictest0-21swapper/020:11:140
2756999140cyclictest0-21swapper/020:21:310
12250140irq/42-ahci0-21swapper/520:21:277
12250140irq/42-ahci0-21swapper/420:15:546
12250140irq/42-ahci0-21swapper/220:26:534
12250140irq/42-ahci0-21swapper/120:15:481
3899130watchdog/40-21swapper/420:07:106
2760899134cyclictest0-21swapper/1122:00:013
2760899134cyclictest0-21swapper/1100:00:013
2760899133cyclictest0-21swapper/1122:50:003
2760899133cyclictest0-21swapper/1122:47:213
2760899133cyclictest0-21swapper/1120:57:593
2760899133cyclictest0-21swapper/1100:16:433
27608991312cyclictest3066350irq/52-eth0-rx-22:10:273
27608991312cyclictest12512chrt21:27:273
27607991312cyclictest0-21swapper/1023:13:522
2760699134cyclictest0-21swapper/923:40:0111
2760699134cyclictest0-21swapper/922:40:0211
2760699134cyclictest0-21swapper/921:45:0011
2760699134cyclictest0-21swapper/900:25:0211
2760699133cyclictest0-21swapper/923:20:1211
2760699133cyclictest0-21swapper/922:25:0011
2760699133cyclictest0-21swapper/922:06:1311
2760699133cyclictest0-21swapper/919:55:0111
2760699133cyclictest0-21swapper/900:36:0511
27606991312cyclictest0-21swapper/900:20:3311
27606991312cyclictest0-21swapper/900:15:3411
27606991311cyclictest9446-21taskset23:05:4011
27606991311cyclictest18766-21chrt20:24:3011
27598991311cyclictest18306-21munin-run00:10:0110
2759799134cyclictest0-21swapper/722:45:009
2759799133cyclictest0-21swapper/723:16:429
2759799133cyclictest0-21swapper/722:28:509
2759799133cyclictest0-21swapper/721:35:499
2759799133cyclictest0-21swapper/721:23:139
2759799133cyclictest0-21swapper/721:17:059
2759799133cyclictest0-21swapper/721:10:429
2759799133cyclictest0-21swapper/720:46:109
2759799133cyclictest0-21swapper/720:25:029
2759799133cyclictest0-21swapper/719:35:009
2759799133cyclictest0-21swapper/719:15:349
2759799132cyclictest0-21swapper/720:14:509
27595991312cyclictest30123-21cpuspeed_turbos00:25:137
27595991312cyclictest27318-21qemu-kvm19:39:177
2759599130cyclictest0-21swapper/522:35:367
2759499132cyclictest0-21swapper/419:50:216
2759199133cyclictest0-21swapper/319:20:215
2758599136cyclictest0-21swapper/222:06:534
2758599136cyclictest0-21swapper/221:15:504
2758599135cyclictest0-21swapper/219:51:044
2758599134cyclictest0-21swapper/222:03:314
27585991311cyclictest2406-21snmpd22:40:504
2758599131cyclictest0-21swapper/220:45:254
2758599131cyclictest0-21swapper/220:00:164
2758599131cyclictest0-21swapper/219:30:264
2758599130cyclictest0-21swapper/223:42:104
2758599130cyclictest0-21swapper/223:42:094
2758599130cyclictest0-21swapper/223:20:214
2758599130cyclictest0-21swapper/223:01:004
2758599130cyclictest0-21swapper/221:50:274
2758599130cyclictest0-21swapper/220:56:584
2758599130cyclictest0-21swapper/219:20:424
27577991312cyclictest3066350irq/52-eth0-rx-00:00:331
27569991311cyclictest26437-21chrt21:17:510
31120ksoftirqd/00-21swapper/023:20:270
31120ksoftirqd/00-21swapper/021:30:290
2760899124cyclictest0-21swapper/1121:05:013
2760899123cyclictest0-21swapper/1119:28:353
2760899122cyclictest0-21swapper/1123:31:513
2760899122cyclictest0-21swapper/1123:31:503
2760899122cyclictest0-21swapper/1122:18:363
2760799120cyclictest3066350irq/52-eth0-rx-20:50:372
27606991211cyclictest0-21swapper/920:45:2811
27606991211cyclictest0-21swapper/919:20:1411
2760699121cyclictest0-21swapper/922:31:5111
27598991211cyclictest4588-21snmp_easybox.os22:15:2610
2759799124cyclictest0-21swapper/721:55:009
2759799122cyclictest0-21swapper/723:38:359
2759799122cyclictest0-21swapper/721:48:269
2759799122cyclictest0-21swapper/720:42:359
2759799122cyclictest0-21swapper/719:14:339
2759699120cyclictest0-21swapper/620:50:018
27595991211cyclictest481ksoftirqd/521:05:007
27594991210cyclictest16475-21sensors00:00:266
27594991210cyclictest122442sleep421:42:106
2759199123cyclictest0-21swapper/321:05:205
2758599126cyclictest0-21swapper/221:44:174
2758599126cyclictest0-21swapper/221:24:034
2758599126cyclictest0-21swapper/200:31:434
2758599125cyclictest2406-21snmpd23:26:354
2758599125cyclictest0-21swapper/220:50:164
2758599125cyclictest0-21swapper/200:13:034
2758599125cyclictest0-21swapper/200:00:244
2758599124cyclictest0-21swapper/200:25:174
2758599121cyclictest0-21swapper/223:10:254
2758599121cyclictest0-21swapper/222:25:194
2758599121cyclictest0-21swapper/222:20:124
2758599121cyclictest0-21swapper/222:15:334
2758599121cyclictest0-21swapper/222:10:124
2758599121cyclictest0-21swapper/221:35:214
2758599121cyclictest0-21swapper/221:25:144
2758599121cyclictest0-21swapper/220:35:144
2758599121cyclictest0-21swapper/220:05:364
2758599121cyclictest0-21swapper/219:40:134
2758599121cyclictest0-21swapper/200:20:244
2758599121cyclictest0-21swapper/200:10:004
2758599120cyclictest8351-21cpuspeed_turbos19:25:114
2758599120cyclictest0-21swapper/223:57:594
2758599120cyclictest0-21swapper/223:33:284
2758599120cyclictest0-21swapper/223:33:274
2758599120cyclictest0-21swapper/223:09:464
2758599120cyclictest0-21swapper/222:55:354
2758599120cyclictest0-21swapper/222:53:114
2758599120cyclictest0-21swapper/222:30:214
2758599120cyclictest0-21swapper/221:05:224
2758599120cyclictest0-21swapper/220:40:144
2758599120cyclictest0-21swapper/219:15:024
2758599120cyclictest0-21swapper/200:38:364
27569991210cyclictest226792sleep021:11:150
2756999120cyclictest0-21swapper/019:28:020
14850120irq/18-uhci_hcd0-21swapper/021:55:290
411110ksoftirqd/40-21swapper/422:45:366
31110ksoftirqd/00-21swapper/023:35:380
2760899113cyclictest0-21swapper/1123:50:013
2760899113cyclictest0-21swapper/1123:26:363
2760899113cyclictest0-21swapper/1123:01:413
2760899113cyclictest0-21swapper/1122:45:003
2760899113cyclictest0-21swapper/1122:38:453
2760899113cyclictest0-21swapper/1121:44:113
2760899113cyclictest0-21swapper/1121:06:543
2760899113cyclictest0-21swapper/1120:50:013
2760899113cyclictest0-21swapper/1120:43:353
2760899113cyclictest0-21swapper/1119:47:073
2760899113cyclictest0-21swapper/1100:38:453
2760899113cyclictest0-21swapper/1100:21:263
2760899113cyclictest0-21swapper/1100:14:563
2760899111cyclictest0-21swapper/1123:46:303
2760899111cyclictest0-21swapper/1123:07:123
2760899111cyclictest0-21swapper/1120:02:253
2760899110cyclictest0-21swapper/1122:08:163
2760699114cyclictest0-21swapper/919:15:0011
2760699113cyclictest0-21swapper/923:40:0211
2760699113cyclictest0-21swapper/923:40:0111
2760699113cyclictest0-21swapper/922:50:3511
2760699113cyclictest0-21swapper/922:15:4311
2760699113cyclictest0-21swapper/921:57:4311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional