You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 12:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Fri Nov 28, 2025 00:46:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30664501210irq/53-eth0-tx-0-21swapper/819:05:1410
30663501190irq/52-eth0-rx-0-21swapper/719:09:339
11290210895sleep30-21swapper/319:07:505
11132210394sleep90-21swapper/919:05:4311
11271210291sleep110-21swapper/1119:07:343
3066350950irq/52-eth0-rx-0-21swapper/1019:07:112
3066450940irq/53-eth0-tx-0-21swapper/119:05:261
12250930irq/42-ahci0-21swapper/219:09:034
1145427977sleep40-21swapper/419:09:156
3066350690irq/52-eth0-rx-0-21swapper/519:05:317
1115026654sleep00-21swapper/019:05:570
1125126055sleep60-21swapper/619:07:178
11612995614cyclictest0-21swapper/021:10:270
11654993231cyclictest0-21swapper/1121:10:353
11654993231cyclictest0-21swapper/1121:09:433
1161299310cyclictest0-21swapper/022:14:290
12250280irq/42-ahci0-21swapper/822:14:4310
1165499210cyclictest0-21swapper/1119:14:333
1165499195cyclictest0-21swapper/1120:13:493
1165299181cyclictest16493-21kworker/9:122:00:1411
1165299181cyclictest16493-21kworker/9:122:00:1411
1165299181cyclictest0-21swapper/920:01:2511
11649991816cyclictest24159-21qemu-kvm20:25:158
11649991816cyclictest24159-21qemu-kvm20:08:428
11649991816cyclictest24158-21qemu-kvm20:45:368
11638991816cyclictest17040-21sensors21:25:296
1165399170cyclictest93372sleep1019:46:002
1165399170cyclictest92722sleep1000:15:062
1165399170cyclictest84822sleep1000:11:172
1165399170cyclictest80262sleep1023:25:362
1165399170cyclictest7622sleep1023:16:592
1165399170cyclictest7372sleep1021:50:142
1165399170cyclictest67742sleep1021:11:462
1165399170cyclictest62632sleep1020:26:422
1165399170cyclictest59612chrt19:44:352
1165399170cyclictest59612chrt19:44:352
1165399170cyclictest51812sleep1021:55:142
1165399170cyclictest48152sleep1022:40:082
1165399170cyclictest43972sleep1023:20:362
1165399170cyclictest40872sleep1022:37:082
1165399170cyclictest321762sleep1021:45:392
1165399170cyclictest321762sleep1021:45:392
1165399170cyclictest317922sleep1021:01:442
1165399170cyclictest314522sleep1020:18:162
1165399170cyclictest307982sleep1019:31:102
1165399170cyclictest3066450irq/53-eth0-tx-20:35:112
1165399170cyclictest295362sleep1023:11:232
1165399170cyclictest291822chrt22:27:302
1165399170cyclictest285922sleep1021:41:252
1165399170cyclictest284142sleep1023:55:162
1165399170cyclictest278672sleep1020:55:322
1165399170cyclictest276312chrt20:10:502
1165399170cyclictest266612sleep1000:36:202
1165399170cyclictest26362sleep1020:22:162
1165399170cyclictest262792sleep1019:25:222
1165399170cyclictest259512sleep1023:06:592
1165399170cyclictest254142sleep1022:20:412
1165399170cyclictest239642sleep1020:06:142
1165399170cyclictest238952chrt20:50:312
1165399170cyclictest234872sleep1019:20:452
1165399170cyclictest234872sleep1019:20:442
1165399170cyclictest231792sleep1000:33:212
1165399170cyclictest219932sleep1023:45:282
1165399170cyclictest215852sleep1021:35:012
1165399170cyclictest210282sleep1020:49:192
1165399170cyclictest20312sleep1019:36:352
1165399170cyclictest201252chrt21:30:232
1165399170cyclictest200132sleep1019:18:082
1165399170cyclictest194882sleep1000:27:322
1165399170cyclictest189642sleep1022:15:052
1165399170cyclictest187722chrt22:58:332
1165399170cyclictest177122sleep1021:26:482
1165399170cyclictest174542sleep1020:00:022
1165399170cyclictest173482sleep1020:43:552
1165399170cyclictest157642sleep1000:21:192
1165399170cyclictest153222chrt22:10:042
1165399170cyclictest151752sleep1019:14:212
1165399170cyclictest149472sleep1022:51:092
1165399170cyclictest146702sleep1019:55:132
1165399170cyclictest145622chrt23:35:262
1165399170cyclictest144692sleep1022:06:162
1165399170cyclictest12250irq/42-ahci22:30:452
1165399170cyclictest12250irq/42-ahci21:09:252
1165399170cyclictest118842sleep1023:33:262
1165399170cyclictest114982sleep1022:48:562
1165399170cyclictest111742sleep1021:20:112
1165399170cyclictest107592chrt22:00:512
1165399170cyclictest107592chrt22:00:512
1165399170cyclictest104252sleep1021:16:582
1165399170cyclictest101692sleep1019:50:132
1165299170cyclictest20334-21hdparm22:15:1611
1165299170cyclictest0-21swapper/921:05:2211
11649991715cyclictest24159-21qemu-kvm20:21:498
11649991715cyclictest24158-21qemu-kvm20:35:138
11649991715cyclictest24158-21qemu-kvm20:31:508
11649991715cyclictest24158-21qemu-kvm20:10:138
1164599171cyclictest0-21swapper/519:45:237
13450160irq/18-ehci_hcd0-21swapper/021:40:270
11653991615cyclictest50282sleep1000:08:422
11652991615cyclictest0-21swapper/900:10:1611
11652991614cyclictest24310-21munin-run20:10:0111
11652991614cyclictest18666-21snmp_rack3slot922:57:1411
1164999164cyclictest0-21swapper/619:45:018
1164999164cyclictest0-21swapper/619:45:018
11649991614cyclictest24159-21qemu-kvm20:15:148
11649991614cyclictest24158-21qemu-kvm20:40:178
1161699160cyclictest0-21swapper/121:10:201
1161699160cyclictest0-21swapper/120:17:091
1161699160cyclictest0-21swapper/119:25:251
1161299162cyclictest0-21swapper/022:30:010
13450150irq/18-ehci_hcd0-21swapper/020:15:290
11652991514cyclictest741rcuc/923:08:3411
11652991513cyclictest1363-21taskset00:04:3211
1164999152cyclictest0-21swapper/621:47:368
1164999152cyclictest0-21swapper/621:47:368
1161699151cyclictest0-21swapper/121:01:541
1161699150cyclictest0-21swapper/123:45:131
1161699150cyclictest0-21swapper/120:05:221
1161699150cyclictest0-21swapper/100:21:381
3066450140irq/53-eth0-tx-0-21swapper/222:14:314
3066350140irq/52-eth0-rx-0-21swapper/722:14:289
3066350140irq/52-eth0-rx-0-21swapper/322:14:325
182050140irq/55-eth1-rx-0-21swapper/922:14:2911
13450140irq/18-ehci_hcd0-21swapper/019:40:270
13450140irq/18-ehci_hcd0-21swapper/019:40:270
12250140irq/42-ahci0-21swapper/1122:14:443
1165399144cyclictest0-21swapper/1000:05:012
11652991413cyclictest0-21swapper/922:08:3111
11652991413cyclictest0-21swapper/900:20:2311
11652991412cyclictest8649-21perl21:15:1511
1165299140cyclictest0-21swapper/920:40:1311
11651991412cyclictest9908-21snmp_rack3slot900:15:1410
11651991412cyclictest8467-21snmp_easybox.os22:45:1310
11651991412cyclictest7822-21snmp_easybox.os00:10:2510
11651991412cyclictest7596-21snmp_easybox.os21:15:1210
11651991412cyclictest7229-21snmp_easybox.os23:25:2410
11651991412cyclictest7043-21snmp_easybox.os20:30:1210
11651991412cyclictest5898-21perl22:40:1610
11651991412cyclictest5521-21perl21:55:1510
11651991412cyclictest4860-21snmp_rack3slot900:06:2410
11651991412cyclictest4259-21diskstats20:25:1510
11651991412cyclictest4057-21snmp_rack3slot919:40:1410
11651991412cyclictest4057-21snmp_rack3slot919:40:1310
11651991412cyclictest3959-21snmp_easybox.os21:10:1110
11651991412cyclictest3668-21snmp_easybox.os23:20:2410
11651991412cyclictest32171-21snmp_easybox.os20:20:1310
11651991412cyclictest31930-21perl00:00:1810
11651991412cyclictest30760-21snmp_rack3slot922:30:1410
11651991412cyclictest30684-21memory21:00:2710
11651991412cyclictest30437-21snmp_rack3slot921:45:1510
11651991412cyclictest30437-21snmp_rack3slot921:45:1510
11651991412cyclictest29198-21snmp_easybox.os23:55:2410
11651991412cyclictest28464-21snmp_easybox.os20:15:1010
11651991412cyclictest27974-21snmp_easybox.os19:30:1110
11651991412cyclictest27470-21perl22:25:1610
11651991412cyclictest26759-21snmp_easybox.os23:10:1210
11651991412cyclictest2674-21ntp_states21:50:2610
11651991412cyclictest26602-21diskstats21:40:1410
11651991412cyclictest26578-21users00:35:3010
11651991412cyclictest26426-21iostat_ios20:10:2010
11651991412cyclictest24885-21users21:35:3610
11651991412cyclictest24326-21snmp_easybox.os19:25:1210
11651991412cyclictest23925-21diskstats23:05:1410
11651991412cyclictest23404-21memory20:50:2410
11651991412cyclictest23070-21snmp_rack3slot922:20:1310
11651991412cyclictest22635-21snmp_rack3slot923:46:2610
11651991412cyclictest21057-21diskstats00:30:1810
11651991412cyclictest20694-21snmp_easybox.os19:20:1410
11651991412cyclictest20694-21snmp_easybox.os19:20:1410
11651991412cyclictest19746-21df22:15:1310
11651991412cyclictest1968-21snmp_easybox.os21:05:2510
11651991412cyclictest1955-21df_inode22:35:1510
11651991412cyclictest19339-21diskstats21:30:1510
11651991412cyclictest18466-21snmp_rack3slot900:25:2610
11651991412cyclictest18424-21snmp_rack3slot920:45:1410
11651991412cyclictest17529-21snmp_easybox.os20:00:1110
11651991412cyclictest16532-21df22:55:1310
11651991412cyclictest16446-21ntp_states19:15:1410
11651991412cyclictest16007-21sensors_fan19:55:2510
11651991412cyclictest15727-21snmp_rack3slot900:20:5010
11651991412cyclictest14893-21snmp_easybox.os21:25:1510
11651991412cyclictest14378-21snmp_easybox.os20:40:1110
11651991412cyclictest14110-21iostat_ios23:35:1910
11651991412cyclictest13722-21irqstats22:50:2010
11651991412cyclictest12835-21snmp_rack3slot919:10:1310
11651991412cyclictest12805-21perl22:05:1810
11651991412cyclictest11022-21snmp_rack3slot920:35:1310
11651991412cyclictest10924-21snmp_easybox.os23:30:2410
11651991412cyclictest10631-21snmp_rack3slot919:50:1510
1164999144cyclictest0-21swapper/622:37:218
1161699141cyclictest0-21swapper/120:20:301
1161699140cyclictest0-21swapper/123:51:481
1161699140cyclictest0-21swapper/122:57:131
1161699140cyclictest0-21swapper/119:45:201
1161699140cyclictest0-21swapper/100:31:061
1165499134cyclictest0-21swapper/1120:55:003
1165499134cyclictest0-21swapper/1120:05:013
1165499134cyclictest0-21swapper/1100:05:013
1165499133cyclictest0-21swapper/1121:43:023
1165299135cyclictest0-21swapper/920:13:5511
1165299134cyclictest0-21swapper/919:35:0111
1165299133cyclictest0-21swapper/923:23:2911
11652991312cyclictest741rcuc/922:21:1611
11652991312cyclictest12-21rcu_preempt20:55:2811
11652991312cyclictest0-21swapper/921:50:1411
11652991312cyclictest0-21swapper/920:45:0111
11651991311cyclictest9062-21perl22:00:1410
11651991311cyclictest9062-21perl22:00:1410
11651991311cyclictest28052-21snmp_rack3slot920:56:0110
11651991311cyclictest26333-21snmp_rack3slot923:52:1510
11651991311cyclictest2299-21munin-run19:40:0110
1165099133cyclictest0-21swapper/721:13:299
1165099133cyclictest0-21swapper/720:25:139
1165099132cyclictest0-21swapper/720:35:289
1162599134cyclictest0-21swapper/223:26:304
1162599132cyclictest0-21swapper/220:30:584
1162599132cyclictest0-21swapper/219:43:304
1162599132cyclictest0-21swapper/219:43:294
11616991312cyclictest3066350irq/52-eth0-rx-23:25:381
11616991312cyclictest3066350irq/52-eth0-rx-20:00:111
1161699130cyclictest0-21swapper/123:35:001
1161699130cyclictest0-21swapper/123:06:491
1161699130cyclictest0-21swapper/122:50:121
1161699130cyclictest0-21swapper/122:35:331
1161699130cyclictest0-21swapper/122:30:011
1161699130cyclictest0-21swapper/122:25:521
1161699130cyclictest0-21swapper/122:21:591
1161699130cyclictest0-21swapper/122:02:091
1161699130cyclictest0-21swapper/122:02:091
1161699130cyclictest0-21swapper/121:58:481
1161699130cyclictest0-21swapper/121:55:011
1161699130cyclictest0-21swapper/121:35:321
1161699130cyclictest0-21swapper/121:33:121
1161699130cyclictest0-21swapper/121:20:321
1161699130cyclictest0-21swapper/121:16:051
1161699130cyclictest0-21swapper/121:07:331
1161699130cyclictest0-21swapper/120:45:231
1161699130cyclictest0-21swapper/120:42:391
1161699130cyclictest0-21swapper/119:50:341
1161699130cyclictest0-21swapper/119:34:541
1161699130cyclictest0-21swapper/100:37:331
1161699130cyclictest0-21swapper/100:25:371
1161699130cyclictest0-21swapper/100:10:261
11612991311cyclictest0-21swapper/020:13:320
461120rcuc/50-21swapper/521:59:217
3199120watchdog/30-21swapper/322:59:475
13450120irq/18-ehci_hcd0-21swapper/021:50:290
1165499124cyclictest0-21swapper/1122:00:013
1165499124cyclictest0-21swapper/1120:30:253
1165499122cyclictest0-21swapper/1120:41:303
1165399123cyclictest0-21swapper/1023:50:002
11653991211cyclictest0-21swapper/1020:34:172
1165299124cyclictest0-21swapper/919:35:0111
1165299123cyclictest0-21swapper/900:00:0111
1165299122cyclictest0-21swapper/923:15:3611
1165299122cyclictest0-21swapper/923:10:0611
1165299122cyclictest0-21swapper/921:41:5611
11652991211cyclictest29325-21diskstats20:15:1411
11651991210cyclictest21421-21ntp_states23:00:2410
1164999123cyclictest0-21swapper/623:23:068
1164999122cyclictest0-21swapper/600:34:158
1164599124cyclictest0-21swapper/523:25:197
11645991211cyclictest9334-21switchtime20:30:327
1163899122cyclictest0-21swapper/423:50:206
11638991210cyclictest11062-21sensors23:30:256
1162599124cyclictest0-21swapper/223:50:224
1162599124cyclictest0-21swapper/220:05:364
1162599121cyclictest0-21swapper/223:07:024
1162599120cyclictest0-21swapper/223:31:324
1162599120cyclictest0-21swapper/221:56:484
1162599120cyclictest0-21swapper/221:36:114
1161699120cyclictest0-21swapper/123:40:061
1161699120cyclictest0-21swapper/123:35:571
1161699120cyclictest0-21swapper/123:24:451
1161699120cyclictest0-21swapper/123:18:071
1161699120cyclictest0-21swapper/123:10:251
1161699120cyclictest0-21swapper/123:01:231
1161699120cyclictest0-21swapper/122:40:121
1161699120cyclictest0-21swapper/122:15:361
1161699120cyclictest0-21swapper/122:10:091
1161699120cyclictest0-21swapper/122:08:331
1161699120cyclictest0-21swapper/121:46:191
1161699120cyclictest0-21swapper/121:46:191
1161699120cyclictest0-21swapper/121:41:341
1161699120cyclictest0-21swapper/120:56:441
1161699120cyclictest0-21swapper/120:51:181
1161699120cyclictest0-21swapper/120:35:111
1161699120cyclictest0-21swapper/120:31:151
1161699120cyclictest0-21swapper/120:25:461
1161699120cyclictest0-21swapper/120:10:121
1161699120cyclictest0-21swapper/119:55:181
1161699120cyclictest0-21swapper/119:40:081
1161699120cyclictest0-21swapper/119:40:081
1161699120cyclictest0-21swapper/119:36:291
1161699120cyclictest0-21swapper/119:21:021
1161699120cyclictest0-21swapper/119:21:021
1161699120cyclictest0-21swapper/119:18:311
1161699120cyclictest0-21swapper/119:10:201
1161699120cyclictest0-21swapper/100:19:031
1161699120cyclictest0-21swapper/100:06:111
1161699120cyclictest0-21swapper/100:02:431
1161299129cyclictest0-21swapper/020:45:010
1161299129cyclictest0-21swapper/019:50:160
1161299124cyclictest0-21swapper/019:25:170
1161299122cyclictest0-21swapper/022:15:180
11612991210cyclictest0-21swapper/000:10:150
901110ksoftirqd/110-21swapper/1120:05:223
551110ksoftirqd/60-21swapper/621:35:178
481110ksoftirqd/50-21swapper/522:41:487
341110ksoftirqd/30-21swapper/300:15:145
31110ksoftirqd/00-21swapper/000:15:140
31110ksoftirqd/00-21swapper/000:05:240
2499110watchdog/20-21swapper/223:41:354
14850110irq/18-uhci_hcd0-21swapper/023:05:260
14850110irq/18-uhci_hcd0-21swapper/000:25:310
12250110irq/42-ahci0-21swapper/920:30:4211
12250110irq/42-ahci0-21swapper/000:25:010
1165499117cyclictest0-21swapper/1100:30:243
1165499113cyclictest0-21swapper/1123:25:003
1165499113cyclictest0-21swapper/1122:34:463
1165499113cyclictest0-21swapper/1121:27:503
1165499113cyclictest0-21swapper/1120:20:403
1165499113cyclictest0-21swapper/1120:00:013
1165499113cyclictest0-21swapper/1100:14:143
11654991110cyclictest12250irq/42-ahci19:28:033
1165499111cyclictest0-21swapper/1119:50:113
1165399113cyclictest0-21swapper/1023:02:322
1165299115cyclictest0-21swapper/919:11:3911
1165299114cyclictest0-21swapper/923:51:5911
1165299114cyclictest0-21swapper/922:40:0111
1165299114cyclictest0-21swapper/920:25:0211
1165299113cyclictest0-21swapper/923:48:5911
1165299113cyclictest0-21swapper/923:41:5911
1165299113cyclictest0-21swapper/921:45:3211
1165299113cyclictest0-21swapper/921:45:3111
1165299113cyclictest0-21swapper/919:30:0111
1165299113cyclictest0-21swapper/900:34:4511
1165299112cyclictest0-21swapper/919:50:2911
11652991110cyclictest0-21swapper/921:20:1211
11652991110cyclictest0-21swapper/920:35:1111
1165199119cyclictest24042-21snmp_rack3slot920:07:1510
1165199119cyclictest18942-21snmp_rack3slot923:40:3710
1165199119cyclictest11729-21snmp_rack3slot921:20:1310
1165099119cyclictest0-21swapper/723:35:209
1165099113cyclictest0-21swapper/723:41:339
1165099113cyclictest0-21swapper/722:40:299
1165099113cyclictest0-21swapper/722:20:319
1165099113cyclictest0-21swapper/722:17:599
1165099113cyclictest0-21swapper/722:09:149
1165099113cyclictest0-21swapper/721:57:329
1165099113cyclictest0-21swapper/721:28:309
1165099113cyclictest0-21swapper/721:20:319
1165099113cyclictest0-21swapper/721:15:019
1165099113cyclictest0-21swapper/720:50:309
1165099113cyclictest0-21swapper/720:30:439
1165099113cyclictest0-21swapper/720:15:589
1165099113cyclictest0-21swapper/720:12:279
1165099113cyclictest0-21swapper/719:53:459
1165099113cyclictest0-21swapper/719:30:589
1165099113cyclictest0-21swapper/719:28:589
1165099112cyclictest0-21swapper/722:30:019
1165099111cyclictest0-21swapper/721:32:509
1164999116cyclictest0-21swapper/623:55:018
1164999115cyclictest12250irq/42-ahci20:55:018
1164999114cyclictest0-21swapper/621:45:018
1164999113cyclictest0-21swapper/623:47:318
1164999113cyclictest0-21swapper/622:59:268
1164999113cyclictest0-21swapper/622:08:068
1164999113cyclictest0-21swapper/621:53:028
1164999113cyclictest0-21swapper/620:02:288
1164999113cyclictest0-21swapper/619:56:588
1164999113cyclictest0-21swapper/619:54:598
1164999113cyclictest0-21swapper/600:25:018
1164999113cyclictest0-21swapper/600:13:568
1164999113cyclictest0-21swapper/600:05:018
1164999112cyclictest0-21swapper/619:10:208
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional