You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 05:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Wed Nov 26, 2025 00:46:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501140irq/42-ahci0-21swapper/219:08:154
18652210995sleep90-21swapper/919:06:0411
12250990irq/42-ahci0-21swapper/119:08:291
12250960irq/42-ahci0-21swapper/819:08:5810
12250960irq/42-ahci0-21swapper/319:09:165
3066350920irq/52-eth0-rx-0-21swapper/419:05:326
3066350880irq/52-eth0-rx-0-21swapper/1119:06:273
3066450700irq/53-eth0-tx-0-21swapper/519:05:227
1872626653sleep60-21swapper/619:07:018
1582326253sleep70-21swapper/719:05:039
129950620irq/18-parport00-21swapper/1019:05:292
1804526149sleep00-21swapper/019:05:320
19151993819cyclictest0-21swapper/1120:08:433
1911399310cyclictest0-21swapper/019:26:380
1911399310cyclictest0-21swapper/019:26:370
1915199290cyclictest6654-21chrt20:19:143
12250260irq/42-ahci0-21swapper/022:15:210
19151992322cyclictest4342-1kworker/11:2H19:18:363
19151992322cyclictest4342-1kworker/11:2H19:18:353
19113992318cyclictest0-21swapper/020:08:420
1914999203cyclictest761ksoftirqd/923:43:5411
1914999203cyclictest761ksoftirqd/923:20:2611
1914999203cyclictest761ksoftirqd/923:11:0311
1914999203cyclictest761ksoftirqd/923:02:4611
1914999203cyclictest761ksoftirqd/922:53:0411
1914999203cyclictest761ksoftirqd/922:53:0311
1914999203cyclictest761ksoftirqd/922:30:3811
1914999203cyclictest761ksoftirqd/922:26:0111
1914999203cyclictest761ksoftirqd/922:06:1811
1914999203cyclictest761ksoftirqd/922:06:1711
1914999203cyclictest761ksoftirqd/921:42:3611
1914999203cyclictest761ksoftirqd/920:57:0011
1914999203cyclictest761ksoftirqd/920:52:5811
1914999203cyclictest761ksoftirqd/920:31:2611
1914999203cyclictest761ksoftirqd/920:27:4111
1914999203cyclictest761ksoftirqd/920:00:5011
1914999203cyclictest761ksoftirqd/919:45:2311
1914999203cyclictest761ksoftirqd/919:38:2911
1914999203cyclictest761ksoftirqd/919:38:2811
1914999203cyclictest761ksoftirqd/900:34:0511
1914999203cyclictest761ksoftirqd/900:25:5411
1914999203cyclictest761ksoftirqd/900:14:5011
1914999203cyclictest761ksoftirqd/900:07:4811
1914999203cyclictest761ksoftirqd/900:03:4011
1914999203cyclictest0-21swapper/923:16:4111
1914999203cyclictest0-21swapper/922:18:1311
1914999203cyclictest0-21swapper/919:21:1911
1914999192cyclictest761ksoftirqd/923:55:5311
1914999192cyclictest761ksoftirqd/923:50:5211
1914999192cyclictest761ksoftirqd/923:48:0111
1914999192cyclictest761ksoftirqd/923:35:4811
1914999192cyclictest761ksoftirqd/923:30:4311
1914999192cyclictest761ksoftirqd/923:25:1211
1914999192cyclictest761ksoftirqd/923:05:5311
1914999192cyclictest761ksoftirqd/922:57:0511
1914999192cyclictest761ksoftirqd/922:46:1111
1914999192cyclictest761ksoftirqd/922:46:1111
1914999192cyclictest761ksoftirqd/922:40:0511
1914999192cyclictest761ksoftirqd/922:35:2911
1914999192cyclictest761ksoftirqd/922:35:2911
1914999192cyclictest761ksoftirqd/922:20:4411
1914999192cyclictest761ksoftirqd/922:12:1911
1914999192cyclictest761ksoftirqd/922:00:0511
1914999192cyclictest761ksoftirqd/921:56:4411
1914999192cyclictest761ksoftirqd/921:50:2211
1914999192cyclictest761ksoftirqd/921:45:2811
1914999192cyclictest761ksoftirqd/921:38:1311
1914999192cyclictest761ksoftirqd/921:30:3011
1914999192cyclictest761ksoftirqd/921:26:0411
1914999192cyclictest761ksoftirqd/921:21:2611
1914999192cyclictest761ksoftirqd/921:15:0311
1914999192cyclictest761ksoftirqd/921:13:2011
1914999192cyclictest761ksoftirqd/921:07:5811
1914999192cyclictest761ksoftirqd/921:07:5811
1914999192cyclictest761ksoftirqd/921:00:2411
1914999192cyclictest761ksoftirqd/920:45:3911
1914999192cyclictest761ksoftirqd/920:43:4611
1914999192cyclictest761ksoftirqd/920:35:3711
1914999192cyclictest761ksoftirqd/920:15:1211
1914999192cyclictest761ksoftirqd/920:10:5011
1914999192cyclictest761ksoftirqd/920:05:0611
1914999192cyclictest761ksoftirqd/919:55:0911
1914999192cyclictest761ksoftirqd/919:50:0611
1914999192cyclictest761ksoftirqd/919:40:3711
1914999192cyclictest761ksoftirqd/919:40:3611
1914999192cyclictest761ksoftirqd/919:30:1111
1914999192cyclictest761ksoftirqd/919:30:1011
1914999192cyclictest761ksoftirqd/919:26:2211
1914999192cyclictest761ksoftirqd/919:26:2111
1914999192cyclictest761ksoftirqd/919:15:2311
1914999192cyclictest761ksoftirqd/919:15:2311
1914999192cyclictest761ksoftirqd/919:12:1611
1914999192cyclictest761ksoftirqd/900:38:1311
1914999192cyclictest761ksoftirqd/900:22:2211
1914999192cyclictest761ksoftirqd/900:15:5011
1911399190cyclictest0-21swapper/020:18:310
1914999181cyclictest761ksoftirqd/920:22:3511
19148991816cyclictest5898-21yum19:30:3810
19148991816cyclictest5898-21yum19:30:3810
19148991816cyclictest5897-21snmp_easybox.os00:00:1510
19148991816cyclictest5650-21snmp_rack3slot922:30:1110
19148991816cyclictest4800-21perl20:15:1610
19148991816cyclictest29951-21diskstats20:50:1610
19148991816cyclictest26846-21snmp_rack3slot900:25:4910
19148991816cyclictest26419-21snmp_rack3slot921:30:1410
19148991816cyclictest26060-21snmp_rack3slot922:56:0110
19148991816cyclictest25418-21snmp_rack3slot919:15:1910
19148991816cyclictest25418-21snmp_rack3slot919:15:1910
19148991816cyclictest23926-21iostat_ios21:25:1810
19148991816cyclictest23289-21snmp_rack3slot922:10:1610
19148991816cyclictest21810-21snmp_easybox.os20:40:1110
19148991816cyclictest19633-21snmp_easybox.os19:50:2410
19148991816cyclictest15607-21iostat_ios19:45:1910
19148991816cyclictest1266-21snmp_rack3slot923:50:5910
19148991816cyclictest11737-21diskstats20:25:1410
19148991816cyclictest10263-21perl23:20:1710
1914799183cyclictest0-21swapper/719:27:389
1914799183cyclictest0-21swapper/719:27:389
19138991816cyclictest31648-21qemu-kvm20:39:277
19138991816cyclictest31648-21qemu-kvm20:29:177
19138991816cyclictest31647-21qemu-kvm20:15:077
19138991816cyclictest31647-21qemu-kvm20:08:397
19148991715cyclictest8812-21munin-node19:35:3110
19148991715cyclictest8812-21munin-node19:35:3110
19148991715cyclictest7729-21snmp_rack3slot920:20:1310
19148991715cyclictest6315-21df_inode23:15:1410
19148991715cyclictest3424-21perl23:55:1510
19148991715cyclictest32145-21iostat_ios23:05:2110
19148991715cyclictest30108-21snmp_easybox.os22:20:1210
19148991715cyclictest29750-21snmp_rack3slot923:01:1410
19148991715cyclictest29535-21snmp_rack3slot919:20:1810
19148991715cyclictest27737-21snmp_rack3slot920:00:3810
19148991715cyclictest27657-21perl22:15:1410
19148991715cyclictest26492-21snmp_rack3slot923:41:2510
19148991715cyclictest21596-21iostat23:35:1910
19148991715cyclictest1990-21perl21:40:1610
19148991715cyclictest19711-21snmp_rack3slot921:20:1810
19148991715cyclictest1896-21irqstats20:55:2210
19148991715cyclictest18360-21snmp_easybox.os23:30:2410
19148991715cyclictest17624-21diskstats00:15:1410
19148991715cyclictest17171-21perl22:45:1510
19148991715cyclictest17171-21perl22:45:1410
19148991715cyclictest14576-21snmp_easybox.os20:30:1310
19148991715cyclictest12356-21snmp_rack3slot900:06:3710
19148991715cyclictest12333-21snmp_easybox.os19:40:2410
19148991715cyclictest12333-21snmp_easybox.os19:40:2410
19148991715cyclictest10170-21snmp_rack3slot921:50:2510
1914799173cyclictest0-21swapper/721:22:489
1914799173cyclictest0-21swapper/700:29:459
1914799172cyclictest0-21swapper/719:57:199
1914799172cyclictest0-21swapper/719:35:169
1914799172cyclictest0-21swapper/719:35:159
1913899174cyclictest0-21swapper/521:20:017
19138991715cyclictest31648-21qemu-kvm20:41:027
19138991715cyclictest31648-21qemu-kvm20:34:197
19138991715cyclictest31648-21qemu-kvm20:20:147
19138991715cyclictest31647-21qemu-kvm20:46:057
19138991715cyclictest31647-21qemu-kvm20:10:137
19138991715cyclictest10708-21qemu-kvm21:08:387
19138991715cyclictest10708-21qemu-kvm21:08:387
19148991614cyclictest603-21snmp_rack3slot919:25:1710
19148991614cyclictest603-21snmp_rack3slot919:25:1710
19148991614cyclictest5712-21perl21:45:1710
19148991614cyclictest4609-21yum23:10:3110
19148991614cyclictest3390-21snmp_easybox.os22:25:2410
19148991614cyclictest31614-21snmp_easybox.os21:35:2710
19148991614cyclictest30454-21yum00:30:3110
19148991614cyclictest26945-21snmp_rack3slot920:45:1910
19148991614cyclictest22733-21munin-run22:55:0010
19148991614cyclictest22733-21munin-run22:55:0010
19148991614cyclictest19664-21snmp_rack3slot919:10:1310
19148991614cyclictest15079-21snmp_easybox.os21:15:1310
19148991614cyclictest14746-21snmp_easybox.os23:25:2510
19148991614cyclictest12654-21diskstats21:55:1310
1914799162cyclictest0-21swapper/721:46:509
19146991615cyclictest22154-21qemu-kvm22:27:568
19146991614cyclictest10709-21qemu-kvm21:30:408
19138991615cyclictest8043-21python00:00:307
19138991615cyclictest3335-21cpuspeed_turbos19:30:137
19138991615cyclictest3335-21cpuspeed_turbos19:30:137
19129991614cyclictest2406-21snmpd23:38:184
19113991615cyclictest179892sleep021:17:580
1915199150cyclictest29330-21chrt22:16:403
19148991514cyclictest12984-21iostat_ios21:10:2010
19148991513cyclictest8724-21snmp_easybox.os22:35:1310
19148991513cyclictest8724-21snmp_easybox.os22:35:1210
19148991513cyclictest30604-21snmp_easybox.os20:05:2310
19148991513cyclictest27337-21snmp_easybox.os23:45:1110
19148991513cyclictest22003-21iostat00:20:2010
19148991513cyclictest21288-21munin-run20:39:5910
19148991513cyclictest1730-21snmp_rack3slot900:36:3910
19148991513cyclictest10480-21snmp_rack3slot921:05:4710
19148991513cyclictest10480-21snmp_rack3slot921:05:4710
1914799153cyclictest0-21swapper/721:10:179
1914799152cyclictest0-21swapper/723:53:559
1914799151cyclictest0-21swapper/723:31:239
1914799151cyclictest0-21swapper/723:07:529
1914799151cyclictest0-21swapper/720:11:469
1914799150cyclictest0-21swapper/720:01:539
19124991514cyclictest3066350irq/52-eth0-rx-20:00:161
19113991514cyclictest12250irq/42-ahci19:21:240
3066350140irq/52-eth0-rx-0-21swapper/319:26:355
3066350140irq/52-eth0-rx-0-21swapper/319:26:355
1914899144cyclictest0-21swapper/820:10:0110
19148991412cyclictest24158-21snmp_rack3slot919:56:2510
19148991412cyclictest18307-21snmp_rack3slot922:01:1310
19148991412cyclictest14316-21perl00:10:1410
1914799145cyclictest0-21swapper/719:45:019
1914799145cyclictest0-21swapper/719:45:009
1914799141cyclictest0-21swapper/723:02:229
1914799141cyclictest0-21swapper/721:15:179
1914799140cyclictest0-21swapper/721:35:189
1914699140cyclictest0-21swapper/623:29:178
19138991413cyclictest27166-21python20:00:327
1912999144cyclictest0-21swapper/200:00:214
1912999143cyclictest0-21swapper/223:01:454
19113991413cyclictest2377-21nscd22:30:120
19113991412cyclictest3613-21chrt22:25:280
12250140irq/42-ahci18851-21qemu-kvm19:26:397
12250140irq/42-ahci18851-21qemu-kvm19:26:387
12250140irq/42-ahci0-21swapper/1119:26:393
12250140irq/42-ahci0-21swapper/1119:26:393
5299130watchdog/60-21swapper/620:05:158
1915199134cyclictest0-21swapper/1123:50:013
1915199134cyclictest0-21swapper/1100:00:003
1915199133cyclictest0-21swapper/1100:34:243
19151991312cyclictest12250irq/42-ahci19:11:423
19148991311cyclictest14333-21ntp_states22:40:2610
1914799135cyclictest0-21swapper/721:40:029
1914799134cyclictest0-21swapper/720:10:019
1914799133cyclictest0-21swapper/723:25:009
1914799133cyclictest0-21swapper/723:00:029
1914799133cyclictest0-21swapper/722:25:009
1914799133cyclictest0-21swapper/719:32:499
1914799133cyclictest0-21swapper/719:32:489
1914799133cyclictest0-21swapper/700:09:249
1914699133cyclictest0-21swapper/620:31:488
1914699133cyclictest0-21swapper/619:49:168
1914699133cyclictest0-21swapper/600:39:178
1912999133cyclictest0-21swapper/222:20:194
1912999133cyclictest0-21swapper/200:10:184
1912999131cyclictest0-21swapper/221:42:064
1912999131cyclictest0-21swapper/220:06:474
19124991312cyclictest3066350irq/52-eth0-rx-20:33:161
1911399133cyclictest31ksoftirqd/023:25:260
1911399132cyclictest31ksoftirqd/022:20:270
1915199124cyclictest0-21swapper/1119:35:023
1915199124cyclictest0-21swapper/1119:35:013
1915199124cyclictest0-21swapper/1100:13:303
1915199123cyclictest0-21swapper/1123:16:313
1915199123cyclictest0-21swapper/1100:35:023
1915199123cyclictest0-21swapper/1100:25:003
1914899124cyclictest0-21swapper/822:10:0210
1914899124cyclictest0-21swapper/822:10:0110
1914799128cyclictest12250irq/42-ahci20:40:009
1914799124cyclictest0-21swapper/723:45:019
1914799123cyclictest0-21swapper/722:40:029
1914799123cyclictest0-21swapper/722:40:019
1914799123cyclictest0-21swapper/719:15:029
1914799123cyclictest0-21swapper/719:15:019
1914699125cyclictest0-21swapper/620:58:038
1914699124cyclictest0-21swapper/600:20:158
1914699123cyclictest0-21swapper/600:30:008
1914699122cyclictest0-21swapper/623:49:178
1914699122cyclictest0-21swapper/600:16:248
1914699122cyclictest0-21swapper/600:09:328
1914699121cyclictest0-21swapper/619:35:008
1914699121cyclictest0-21swapper/619:35:008
1914699121cyclictest0-21swapper/600:00:008
1913899122cyclictest0-21swapper/521:55:187
1913699123cyclictest0-21swapper/300:15:185
1912999123cyclictest0-21swapper/223:22:144
1912999123cyclictest0-21swapper/220:30:024
1912999123cyclictest0-21swapper/220:21:314
1912999123cyclictest0-21swapper/219:27:464
1912999123cyclictest0-21swapper/219:27:454
1912999123cyclictest0-21swapper/219:15:254
1912999123cyclictest0-21swapper/219:15:244
19124991211cyclictest3066350irq/52-eth0-rx-23:22:201
1911399123cyclictest0-21swapper/019:55:200
1911399122cyclictest0-21swapper/020:45:200
1911399122cyclictest0-21swapper/000:30:000
19113991211cyclictest12250irq/42-ahci19:10:210
19113991210cyclictest0-21swapper/022:40:290
1911399120cyclictest0-21swapper/022:50:230
1911399120cyclictest0-21swapper/022:50:230
881110rcuc/110-21swapper/1119:20:143
201110ksoftirqd/10-21swapper/122:50:121
201110ksoftirqd/10-21swapper/122:50:121
1915199113cyclictest0-21swapper/1123:01:323
1915199113cyclictest0-21swapper/1122:57:203
1915199113cyclictest0-21swapper/1121:27:163
1915199113cyclictest0-21swapper/1121:09:473
1915199113cyclictest0-21swapper/1121:09:463
1915199113cyclictest0-21swapper/1120:49:163
1915199113cyclictest0-21swapper/1120:26:153
1915199113cyclictest0-21swapper/1120:03:143
1915199113cyclictest0-21swapper/1119:45:463
1915199113cyclictest0-21swapper/1119:44:003
1915199113cyclictest0-21swapper/1119:44:003
1914799115cyclictest0-21swapper/720:54:179
1914799115cyclictest0-21swapper/719:47:029
1914799113cyclictest0-21swapper/722:41:519
1914799113cyclictest0-21swapper/722:20:009
1914799113cyclictest0-21swapper/721:32:589
1914799111cyclictest0-21swapper/723:55:209
1914799111cyclictest0-21swapper/723:22:529
1914799111cyclictest0-21swapper/722:12:179
1914799111cyclictest0-21swapper/720:23:009
1914699115cyclictest0-21swapper/621:50:048
1914699115cyclictest0-21swapper/620:39:458
1914699115cyclictest0-21swapper/619:59:238
1914699114cyclictest0-21swapper/622:41:148
1914699114cyclictest0-21swapper/622:20:028
1914699113cyclictest0-21swapper/623:36:328
1914699113cyclictest0-21swapper/623:10:358
1914699113cyclictest0-21swapper/623:05:508
1914699113cyclictest0-21swapper/622:56:598
1914699113cyclictest0-21swapper/622:51:108
1914699113cyclictest0-21swapper/622:51:098
1914699113cyclictest0-21swapper/622:38:058
1914699113cyclictest0-21swapper/622:38:048
1914699113cyclictest0-21swapper/622:06:178
1914699113cyclictest0-21swapper/622:06:168
1914699113cyclictest0-21swapper/621:36:298
1914699113cyclictest0-21swapper/621:19:178
1914699113cyclictest0-21swapper/620:25:028
1914699113cyclictest0-21swapper/620:17:448
1914699113cyclictest0-21swapper/620:01:188
1914699113cyclictest0-21swapper/619:52:298
1914699113cyclictest0-21swapper/619:27:468
1914699113cyclictest0-21swapper/619:27:458
1914699112cyclictest0-21swapper/622:01:318
1914699112cyclictest0-21swapper/621:23:198
1914699111cyclictest0-21swapper/623:31:338
1914699111cyclictest0-21swapper/623:23:318
1914699111cyclictest0-21swapper/623:03:178
1913899113cyclictest0-21swapper/523:10:177
1913899113cyclictest0-21swapper/500:05:207
1913699113cyclictest0-21swapper/322:00:195
1913699113cyclictest0-21swapper/300:20:185
1912999115cyclictest0-21swapper/220:15:204
1912999115cyclictest0-21swapper/219:53:324
1912999113cyclictest0-21swapper/222:45:184
1912999113cyclictest0-21swapper/222:45:174
1912999113cyclictest0-21swapper/222:25:174
1912999110cyclictest0-21swapper/222:30:134
1912999110cyclictest0-21swapper/221:34:084
1912999110cyclictest0-21swapper/220:28:214
1912999110cyclictest0-21swapper/219:33:154
1912999110cyclictest0-21swapper/219:33:154
1912499119cyclictest0-21swapper/119:15:171
1912499119cyclictest0-21swapper/119:15:171
1912499113cyclictest0-21swapper/122:30:171
1912499113cyclictest0-21swapper/122:15:181
1911399113cyclictest0-21swapper/023:50:000
1911399113cyclictest0-21swapper/022:45:170
1911399113cyclictest0-21swapper/022:45:170
1911399113cyclictest0-21swapper/021:28:400
1911399113cyclictest0-21swapper/020:40:190
1911399113cyclictest0-21swapper/000:30:180
1911399112cyclictest0-21swapper/021:05:180
1911399112cyclictest0-21swapper/021:05:180
1911399110cyclictest0-21swapper/019:15:010
1911399110cyclictest0-21swapper/019:15:000
901100ksoftirqd/110-21swapper/1123:20:143
901100ksoftirqd/110-21swapper/1122:50:133
901100ksoftirqd/110-21swapper/1122:50:133
8999100migration/1121153-21latency_hist19:55:013
8999100migration/1112738-21munin-node21:10:163
881100rcuc/110-21swapper/1119:30:013
881100rcuc/110-21swapper/1119:30:013
8799100watchdog/110-21swapper/1123:45:473
8799100watchdog/110-21swapper/1119:53:073
831100ksoftirqd/100-21swapper/1021:30:162
831100ksoftirqd/100-21swapper/1019:35:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional