You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-23 - 03:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Fri Feb 23, 2024 00:46:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6742210995sleep100-21swapper/1019:07:522
6715210995sleep90-21swapper/919:07:2711
6698210993sleep80-21swapper/819:07:1410
120501060irq/42-ahci0-21swapper/119:09:111
120501020irq/42-ahci0-21swapper/219:08:504
1878050990irq/53-eth0-tx-0-21swapper/519:07:227
1877950990irq/52-eth0-rx-0-21swapper/719:09:189
1877950920irq/52-eth0-rx-0-21swapper/419:05:216
1877950880irq/52-eth0-rx-0-21swapper/1119:07:353
12050780irq/42-ahci0-21swapper/319:08:505
379127364sleep60-21swapper/619:05:138
659525955sleep00-21swapper/019:05:540
7101993213cyclictest0-21swapper/1121:22:033
7101992827cyclictest0-21swapper/1122:23:213
7071992827cyclictest0-21swapper/022:23:130
7071992812cyclictest0-21swapper/019:19:590
7100992710cyclictest831ksoftirqd/1000:35:212
7071992322cyclictest26993-1kworker/0:0H20:21:300
710099192cyclictest831ksoftirqd/1023:35:222
710099192cyclictest831ksoftirqd/1022:50:232
710099192cyclictest1877950irq/52-eth0-rx-23:49:502
7100991817cyclictest831ksoftirqd/1021:55:192
7100991817cyclictest831ksoftirqd/1019:15:152
7100991817cyclictest166942sleep1023:00:322
7100991816cyclictest12561-21users20:40:422
710099181cyclictest831ksoftirqd/1023:50:192
710099181cyclictest831ksoftirqd/1022:30:182
710099181cyclictest831ksoftirqd/1022:10:242
710099181cyclictest831ksoftirqd/1022:00:212
710099181cyclictest831ksoftirqd/1020:55:402
710099181cyclictest831ksoftirqd/1020:45:402
710099181cyclictest831ksoftirqd/1019:55:002
710099181cyclictest831ksoftirqd/1019:30:002
710099181cyclictest831ksoftirqd/1000:05:372
710099181cyclictest1877950irq/52-eth0-rx-22:35:472
709999181cyclictest761ksoftirqd/921:30:3211
709999181cyclictest761ksoftirqd/920:35:3811
709999181cyclictest7244-21kworker/9:023:49:1311
709999181cyclictest16031-21tune2fs23:00:2311
709999181cyclictest0-21swapper/921:14:5611
709999181cyclictest0-21swapper/919:55:2211
709999181cyclictest0-21swapper/900:05:3511
709799185cyclictest0-21swapper/721:30:019
7095991816cyclictest12122-21snmp_rack3slot919:56:337
7089991816cyclictest22636-21snmp_rack3slot922:25:224
7089991816cyclictest21436-21snmp_rack3slot923:06:474
7101991715cyclictest15789-21gltestperf22:15:233
7100991716cyclictest831ksoftirqd/1022:55:162
7100991716cyclictest831ksoftirqd/1020:15:302
7100991716cyclictest3036-21cstates20:30:212
7100991716cyclictest2745-21snmp_easybox.os00:10:362
7100991716cyclictest26901-21cstates00:00:212
7100991716cyclictest21208-21grep23:05:422
7100991716cyclictest0-21swapper/1023:20:232
7100991716cyclictest0-21swapper/1021:20:212
7100991715cyclictest3324-21snmp_rack3slot919:45:222
7100991715cyclictest29377-21diskstats21:50:212
7100991715cyclictest29377-21diskstats21:50:202
7100991715cyclictest23376-21irqstats22:25:282
710099170cyclictest831ksoftirqd/1023:15:332
710099170cyclictest831ksoftirqd/1022:45:222
710099170cyclictest831ksoftirqd/1022:15:202
710099170cyclictest831ksoftirqd/1022:05:212
710099170cyclictest831ksoftirqd/1021:30:192
710099170cyclictest831ksoftirqd/1021:25:002
710099170cyclictest831ksoftirqd/1021:10:182
710099170cyclictest831ksoftirqd/1021:05:222
710099170cyclictest831ksoftirqd/1021:00:192
710099170cyclictest831ksoftirqd/1020:20:212
710099170cyclictest831ksoftirqd/1020:00:212
710099170cyclictest831ksoftirqd/1019:56:242
710099170cyclictest831ksoftirqd/1019:30:002
710099170cyclictest75492sleep1019:10:032
710099170cyclictest72582sleep1023:31:452
710099170cyclictest61152sleep1020:35:022
710099170cyclictest58822sleep1021:16:432
710099170cyclictest57261fschecks_count00:15:232
710099170cyclictest35612sleep1023:26:092
710099170cyclictest306712chrt19:40:082
710099170cyclictest275742chrt21:45:462
710099170cyclictest270372sleep1019:35:062
710099170cyclictest256192sleep1023:59:132
710099170cyclictest252782sleep1023:14:432
710099170cyclictest231102sleep1020:11:102
710099170cyclictest228532chrt21:40:332
710099170cyclictest211542sleep1022:24:022
710099170cyclictest194122sleep1020:06:092
710099170cyclictest189592sleep1019:21:052
710099170cyclictest185402sleep1000:31:162
710099170cyclictest18342sleep1022:40:282
710099170cyclictest175902chrt21:35:082
710099170cyclictest175902chrt21:35:072
710099170cyclictest138102sleep1023:40:362
710099170cyclictest120622sleep1000:25:032
7099991710cyclictest0-21swapper/922:25:2811
7099991710cyclictest0-21swapper/921:35:3111
7099991710cyclictest0-21swapper/921:35:3011
709999170cyclictest8578-1kworker/9:1H00:27:1411
709999170cyclictest0-21swapper/921:20:2311
709999170cyclictest0-21swapper/900:00:2411
709799179cyclictest0-21swapper/723:24:049
709799176cyclictest0-21swapper/720:03:429
7095991710cyclictest22852-21snmp_easybox.os21:40:337
709599170cyclictest0-21swapper/522:45:257
7094991716cyclictest6590-21turbostat22:09:566
7089991715cyclictest5547-21iostat22:45:294
7089991715cyclictest30392-21snmp_rack3slot922:35:264
7089991715cyclictest29130-21snmp_rack3slot900:01:584
7089991715cyclictest11195-21snmp_rack3slot921:25:214
7071991715cyclictest8803-21snmp_rack3slot923:35:210
709999169cyclictest0-21swapper/922:50:2911
709999169cyclictest0-21swapper/920:25:2711
709999169cyclictest0-21swapper/920:03:4311
709999169cyclictest0-21swapper/900:20:3111
7099991615cyclictest0-21swapper/923:40:0111
709799169cyclictest0-21swapper/720:45:109
709799165cyclictest0-21swapper/719:25:279
709799162cyclictest0-21swapper/719:15:269
7095991614cyclictest20722-21munin-node20:55:227
7089991614cyclictest4386-21snmp_rack3slot921:15:264
7089991614cyclictest31790-21snmp_easybox.os21:10:174
7089991614cyclictest18377-21snmp_rack3slot921:35:234
7089991614cyclictest18377-21snmp_rack3slot921:35:224
7089991614cyclictest12849-21snmp_rack3slot900:25:214
709999155cyclictest0-21swapper/923:30:2611
7099991514cyclictest29860-21df_abs22:35:2411
7099991514cyclictest26311-21chrt19:32:0511
7099991514cyclictest12-21rcu_preempt00:35:3011
7099991514cyclictest0-21swapper/922:40:2711
7099991514cyclictest0-21swapper/920:50:3211
7099991514cyclictest0-21swapper/920:20:3211
7099991514cyclictest0-21swapper/920:08:2511
709899154cyclictest0-21swapper/819:25:0110
709799155cyclictest0-21swapper/719:10:029
709799154cyclictest0-21swapper/722:00:019
709699155cyclictest0-21swapper/600:15:008
7096991514cyclictest4075-21processes19:45:358
7096991514cyclictest31366-21if_err_tap000:05:328
7096991514cyclictest10154-21qemu-kvm22:40:228
7096991514cyclictest10154-21qemu-kvm22:08:398
709599152cyclictest0-21swapper/520:05:247
7095991513cyclictest32329-21snmp_rack3slot923:21:467
7095991513cyclictest23419-21munin-node19:30:207
7095991510cyclictest17940-21snmp_rack3slot920:50:217
7089991513cyclictest19380-21snmp_rack3slot922:20:224
7071991514cyclictest0-21swapper/019:20:030
7071991513cyclictest106372sleep019:14:310
14850150irq/18-uhci_hcd0-21swapper/223:30:344
13250150irq/18-ehci_hcd0-21swapper/019:45:510
7101991413cyclictest23733-21netstat22:25:323
7101991413cyclictest2084-21threads21:55:373
7100991413cyclictest831ksoftirqd/1020:25:382
7100991413cyclictest11264-21snmp_rack3slot900:21:102
7100991413cyclictest0-21swapper/1020:50:242
709999145cyclictest0-21swapper/921:46:3111
709999145cyclictest0-21swapper/920:45:3011
7099991413cyclictest741rcuc/922:57:0311
7099991413cyclictest1918-21python21:55:3511
7099991413cyclictest0-21swapper/922:45:2011
7099991413cyclictest0-21swapper/922:00:2611
7099991413cyclictest0-21swapper/919:46:4511
709899144cyclictest0-21swapper/821:10:0210
709899143cyclictest0-21swapper/821:05:0010
709799144cyclictest0-21swapper/723:05:009
709799144cyclictest0-21swapper/722:00:269
709799144cyclictest0-21swapper/721:00:009
709799144cyclictest0-21swapper/720:50:279
709799144cyclictest0-21swapper/720:10:009
709799143cyclictest0-21swapper/723:30:279
709799143cyclictest0-21swapper/723:05:009
709799143cyclictest0-21swapper/700:20:019
709699144cyclictest0-21swapper/619:25:008
709599144cyclictest0-21swapper/523:45:007
7089991413cyclictest2730-21iostat00:10:334
7089991412cyclictest2325-21snmpd19:46:194
7089991412cyclictest22252-21snmp_easybox.os20:10:314
7071991412cyclictest2451-21chrt20:30:150
31140ksoftirqd/00-21swapper/019:34:590
13250140irq/18-ehci_hcd0-21swapper/023:40:350
13250140irq/18-ehci_hcd0-21swapper/021:05:380
7101991311cyclictest138662sleep1122:14:013
709999136cyclictest0-21swapper/919:38:4611
709999133cyclictest0-21swapper/919:15:1611
7099991312cyclictest0-21swapper/921:28:3811
7099991312cyclictest0-21swapper/920:15:2511
7099991312cyclictest0-21swapper/920:15:0011
7099991312cyclictest0-21swapper/900:15:3011
709899135cyclictest0-21swapper/819:30:1410
709899134cyclictest0-21swapper/822:25:0110
709899134cyclictest0-21swapper/822:15:0210
709899134cyclictest0-21swapper/819:25:0110
709899133cyclictest0-21swapper/823:58:4110
709899133cyclictest0-21swapper/823:48:3010
709899133cyclictest0-21swapper/823:15:5110
709899133cyclictest0-21swapper/822:45:0110
709899132cyclictest0-21swapper/820:35:0110
709799136cyclictest0-21swapper/721:51:579
709799136cyclictest0-21swapper/721:51:569
709799136cyclictest0-21swapper/700:00:309
709799135cyclictest0-21swapper/722:48:279
709799135cyclictest0-21swapper/721:15:289
709799134cyclictest0-21swapper/723:40:009
709799133cyclictest0-21swapper/722:50:289
709799133cyclictest0-21swapper/722:40:579
709799133cyclictest0-21swapper/721:30:019
709799133cyclictest0-21swapper/721:06:159
709799132cyclictest0-21swapper/720:30:299
709799132cyclictest0-21swapper/700:20:319
709799131cyclictest0-21swapper/723:48:599
709699134cyclictest0-21swapper/621:00:018
709699132cyclictest0-21swapper/623:19:598
709699132cyclictest0-21swapper/622:14:568
7096991312cyclictest6113-21zgrep22:45:358
7096991312cyclictest31045-21qemu-kvm21:09:448
7096991312cyclictest16023-21if_eth122:15:268
709599133cyclictest0-21swapper/500:30:337
709599132cyclictest0-21swapper/522:25:257
7095991311cyclictest1059-21munin-node00:10:487
709599131cyclictest0-21swapper/523:10:287
7089991312cyclictest14121-21snmp_rack3slot922:56:474
7089991311cyclictest2325-21snmpd20:48:214
7089991311cyclictest13859-21snmp_rack3slot919:15:224
708299130cyclictest0-21swapper/100:00:321
707199133cyclictest0-21swapper/020:50:260
707199133cyclictest0-21swapper/019:40:260
707199133cyclictest0-21swapper/000:15:270
7071991312cyclictest12050irq/42-ahci20:15:570
5999130watchdog/70-21swapper/720:22:329
4599130watchdog/50-21swapper/523:40:097
31130ksoftirqd/00-21swapper/020:00:560
14850130irq/18-uhci_hcd0-21swapper/021:25:330
710199124cyclictest0-21swapper/1123:25:253
710199122cyclictest0-21swapper/1122:35:413
7101991211cyclictest23320-21proc_pri21:40:373
709999125cyclictest0-21swapper/920:30:2811
7099991211cyclictest7844-21cpuspeed_turbos19:10:2111
7099991211cyclictest30075-21open_inodes21:05:3511
7099991211cyclictest21725-21chrt23:50:4511
7099991211cyclictest0-21swapper/922:10:1811
7099991211cyclictest0-21swapper/920:55:2711
7099991211cyclictest0-21swapper/919:44:3111
709899124cyclictest0-21swapper/821:15:0210
709899123cyclictest0-21swapper/821:05:0010
709899123cyclictest0-21swapper/820:55:0110
709899122cyclictest0-21swapper/822:59:4610
709899122cyclictest0-21swapper/821:59:0010
709899122cyclictest0-21swapper/820:32:4510
709899121cyclictest0-21swapper/819:55:0110
709899120cyclictest0-21swapper/822:40:0110
709799125cyclictest465-21sshd00:09:239
709799124cyclictest0-21swapper/722:20:189
709799124cyclictest0-21swapper/700:32:219
709799124cyclictest0-21swapper/700:26:199
709799123cyclictest0-21swapper/723:17:179
709799123cyclictest0-21swapper/722:55:359
709799123cyclictest0-21swapper/722:30:539
709799123cyclictest0-21swapper/721:49:389
709799123cyclictest0-21swapper/721:14:079
709799123cyclictest0-21swapper/720:40:009
709799123cyclictest0-21swapper/720:27:369
709799122cyclictest0-21swapper/719:36:199
7096991211cyclictest19763-21zgrep21:35:368
7096991211cyclictest19763-21zgrep21:35:358
709699120cyclictest26527-21awk19:35:008
709599124cyclictest0-21swapper/519:50:017
7095991211cyclictest7178-21snmp_rack3slot923:30:447
7095991210cyclictest10815-21snmp_rack3slot923:35:457
709599120cyclictest481ksoftirqd/519:19:597
709399123cyclictest0-21swapper/323:40:255
709399122cyclictest0-21swapper/323:50:245
709399122cyclictest0-21swapper/319:55:265
708999123cyclictest0-21swapper/219:10:284
7089991210cyclictest9997-21snmp_rack3slot922:06:484
7089991210cyclictest310-21snmp_rack3slot900:07:144
7089991210cyclictest2325-21snmpd19:20:194
7089991210cyclictest2325-21snmpd00:32:064
708999120cyclictest0-21swapper/221:40:264
707199120cyclictest0-21swapper/022:25:360
707199120cyclictest0-21swapper/022:15:330
3199120watchdog/30-21swapper/320:18:415
31120ksoftirqd/00-21swapper/023:17:310
14850120irq/18-uhci_hcd0-21swapper/022:10:340
14850120irq/18-uhci_hcd0-21swapper/021:11:240
901110ksoftirqd/110-21swapper/1123:35:323
901110ksoftirqd/110-21swapper/1120:05:293
901110ksoftirqd/110-21swapper/1119:30:183
8999110migration/1110648-21munin-node20:40:233
710199119cyclictest1877950irq/52-eth0-rx-20:25:203
710199114cyclictest0-21swapper/1123:57:163
710199113cyclictest0-21swapper/1121:39:213
710199113cyclictest0-21swapper/1121:39:203
710199113cyclictest0-21swapper/1121:32:233
710199113cyclictest0-21swapper/1120:50:013
710199113cyclictest0-21swapper/1120:15:493
710199113cyclictest0-21swapper/1119:53:043
710199113cyclictest0-21swapper/1119:45:193
710199113cyclictest0-21swapper/1119:26:493
7101991110cyclictest27849-21ntp_offset23:15:333
710199110cyclictest0-21swapper/1120:30:223
709999114cyclictest0-21swapper/923:57:1711
709999114cyclictest0-21swapper/923:24:0511
709999114cyclictest0-21swapper/923:16:2211
709999114cyclictest0-21swapper/923:13:2411
709999114cyclictest0-21swapper/921:50:2711
709999114cyclictest0-21swapper/921:50:2611
709999114cyclictest0-21swapper/921:45:0111
709999114cyclictest0-21swapper/919:25:2711
709999113cyclictest0-21swapper/923:06:1111
709999113cyclictest0-21swapper/922:23:0611
709999113cyclictest0-21swapper/922:07:2611
709999112cyclictest0-21swapper/921:05:0111
709999112cyclictest0-21swapper/900:30:0711
7099991110cyclictest7244-21kworker/9:023:28:0011
7099991110cyclictest5546-21latency_hist19:50:0111
7099991110cyclictest27706-21switchtime22:30:3811
709999111cyclictest0-21swapper/922:17:2611
709899115cyclictest0-21swapper/822:22:4410
709899115cyclictest0-21swapper/800:22:3110
709899114cyclictest0-21swapper/823:09:4510
709899114cyclictest0-21swapper/800:35:0010
709899113cyclictest0-21swapper/823:39:3610
709899113cyclictest0-21swapper/823:28:3510
709899113cyclictest0-21swapper/822:47:0110
709899113cyclictest0-21swapper/821:44:5510
709899113cyclictest0-21swapper/821:35:0110
709899113cyclictest0-21swapper/821:35:0010
709899113cyclictest0-21swapper/821:29:2910
709899113cyclictest0-21swapper/821:24:2910
709899113cyclictest0-21swapper/820:15:1410
709899113cyclictest0-21swapper/819:18:3910
709899113cyclictest0-21swapper/819:14:5310
709899113cyclictest0-21swapper/800:05:2010
709899113cyclictest0-21swapper/800:02:5010
709899111cyclictest0-21swapper/800:19:1010
709799118cyclictest0-21swapper/722:35:409
709799114cyclictest0-21swapper/723:56:529
709799114cyclictest0-21swapper/723:27:109
709799114cyclictest0-21swapper/722:11:099
709799114cyclictest0-21swapper/721:36:029
709799114cyclictest0-21swapper/721:36:029
709799114cyclictest0-21swapper/720:36:269
709799114cyclictest0-21swapper/720:17:369
709799114cyclictest0-21swapper/719:30:519
709799114cyclictest0-21swapper/700:36:049
709799113cyclictest0-21swapper/723:52:379
709799113cyclictest0-21swapper/723:40:389
709799113cyclictest0-21swapper/723:11:589
709799113cyclictest0-21swapper/722:05:259
709799113cyclictest0-21swapper/721:20:329
709799113cyclictest0-21swapper/720:10:339
709799113cyclictest0-21swapper/719:56:049
709799113cyclictest0-21swapper/719:45:589
709799113cyclictest0-21swapper/719:40:569
709799113cyclictest0-21swapper/719:21:289
709799112cyclictest0-21swapper/720:56:429
709799112cyclictest0-21swapper/719:51:369
709799112cyclictest0-21swapper/700:10:399
709799110cyclictest0-21swapper/721:40:259
709699115cyclictest6816-21qemu-kvm19:15:018
709699113cyclictest31046-21qemu-kvm21:27:208
709699113cyclictest0-21swapper/622:39:268
709699113cyclictest0-21swapper/622:26:268
709699113cyclictest0-21swapper/620:45:028
709699113cyclictest0-21swapper/620:23:348
709699112cyclictest0-21swapper/619:58:198
7096991110cyclictest1525-21switchtime21:10:388
7096991110cyclictest1298-21ipmi_fans21:55:268
7096991110cyclictest10154-21qemu-kvm22:30:198
709599111cyclictest0-21swapper/522:05:357
709599110cyclictest0-21swapper/522:55:267
709499119cyclictest0-21swapper/420:15:236
709499113cyclictest0-21swapper/421:30:006
709499112cyclictest0-21swapper/421:10:256
709399119cyclictest0-21swapper/320:00:385
709399113cyclictest0-21swapper/321:20:255
709399113cyclictest0-21swapper/320:30:275
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional