You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 07:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3.osadl.org (updated Tue Nov 18, 2025 00:46:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
122501140irq/42-ahci0-21swapper/219:09:194
17635210896sleep90-21swapper/919:07:3311
17598210894sleep30-21swapper/319:07:035
17494210592sleep70-21swapper/719:05:439
3066350970irq/52-eth0-rx-0-21swapper/419:05:236
3066350930irq/52-eth0-rx-0-21swapper/119:05:191
3066350920irq/52-eth0-rx-0-21swapper/1119:06:153
1784928280sleep80-21swapper/819:09:4510
1763128272sleep60-21swapper/619:07:308
13450750irq/18-ehci_hcd0-21swapper/519:05:287
1741026253sleep100-21swapper/1019:05:342
1470226055sleep00-21swapper/019:05:120
17971993517cyclictest0-21swapper/022:22:440
18013992726cyclictest7104-21kworker/11:121:14:023
1798499193cyclictest0-21swapper/323:55:265
1798499193cyclictest0-21swapper/323:20:275
1798499193cyclictest0-21swapper/323:00:025
1798499193cyclictest0-21swapper/322:15:205
1798499193cyclictest0-21swapper/322:05:185
1798499193cyclictest0-21swapper/321:56:525
1798499193cyclictest0-21swapper/320:00:305
1798499193cyclictest0-21swapper/319:55:265
1798499193cyclictest0-21swapper/319:50:325
1798499193cyclictest0-21swapper/300:15:375
1801299181cyclictest3066350irq/52-eth0-rx-21:30:252
1801299181cyclictest124050irq/18-i801_smb20:50:292
1801299181cyclictest12250irq/42-ahci21:08:122
1801299180cyclictest12250irq/42-ahci22:08:282
1801199181cyclictest0-21swapper/922:30:2411
18010991816cyclictest4410-21snmp_rack3slot921:45:1510
18010991816cyclictest330-21perl20:55:1910
18010991816cyclictest3208-21snmp_rack3slot923:55:2510
18010991816cyclictest31568-21snmp_rack3slot923:05:2910
18010991816cyclictest31568-21snmp_rack3slot923:05:2810
18010991816cyclictest3145-21snmp_rack3slot919:30:1610
18010991816cyclictest31193-21snmp_easybox.os20:10:1310
18010991816cyclictest31193-21snmp_easybox.os20:10:1310
18010991816cyclictest30684-21snmp_easybox.os19:25:1310
18010991816cyclictest29863-21snmp_rack3slot922:20:1410
18010991816cyclictest29552-21perl21:35:1710
18010991816cyclictest27970-21snmp_easybox.os20:50:1310
18010991816cyclictest27347-21iostat_ios23:00:2010
18010991816cyclictest26438-21iostat_ios21:30:1910
18010991816cyclictest25328-21snmp_rack3slot923:41:0410
18010991816cyclictest22123-21snmp_rack3slot900:21:2510
18010991816cyclictest18509-21snmp_rack3slot900:16:3710
18010991816cyclictest13179-21snmp_easybox.os22:40:2810
1798499183cyclictest0-21swapper/320:51:285
1798499182cyclictest0-21swapper/323:25:375
1798499182cyclictest0-21swapper/323:06:305
1798499182cyclictest0-21swapper/323:06:305
1798499182cyclictest0-21swapper/319:20:235
1798499182cyclictest0-21swapper/300:28:145
17971991817cyclictest2691-21runrttasks20:23:190
17971991817cyclictest2691-21runrttasks20:23:190
1797199180cyclictest0-21swapper/019:13:530
18012991716cyclictest32632-1kworker/10:2H19:35:402
18012991716cyclictest0-21swapper/1020:08:072
1801299170cyclictest89682sleep1020:21:302
1801299170cyclictest89682sleep1020:21:302
1801299170cyclictest78152sleep1000:04:432
1801299170cyclictest72792sleep1023:17:492
1801299170cyclictest68852chrt22:33:222
1801299170cyclictest6652chrt00:37:472
1801299170cyclictest61972sleep1021:45:512
1801299170cyclictest59022sleep1021:03:102
1801299170cyclictest53002sleep1020:16:062
1801299170cyclictest50652chrt19:34:492
1801299170cyclictest34852sleep1023:10:382
1801299170cyclictest320242sleep1022:24:072
1801299170cyclictest31062sleep1022:26:202
1801299170cyclictest3066350irq/52-eth0-rx-22:58:132
1801299170cyclictest306482sleep1023:50:172
1801299170cyclictest300502sleep1019:23:362
1801299170cyclictest295492sleep1000:33:342
1801299170cyclictest287502chrt23:03:002
1801299170cyclictest283622sleep1022:18:552
1801299170cyclictest274982sleep1023:45:162
1801299170cyclictest271292sleep1020:45:572
1801299170cyclictest268242chrt20:03:042
1801299170cyclictest253582sleep1023:41:292
1801299170cyclictest25282sleep1021:40:512
1801299170cyclictest245342sleep1022:11:182
1801299170cyclictest241052sleep1021:27:142
1801299170cyclictest230242sleep1019:56:272
1801299170cyclictest220532sleep1000:20:342
1801299170cyclictest218582chrt23:38:162
1801299170cyclictest215762sleep1022:54:592
1801299170cyclictest20992sleep1020:57:352
1801299170cyclictest204042sleep1021:21:252
1801299170cyclictest200432sleep1020:38:322
1801299170cyclictest193942sleep1019:51:272
1801299170cyclictest184192sleep1000:15:322
1801299170cyclictest176282chrt22:45:582
1801299170cyclictest176282chrt22:45:582
1801299170cyclictest174332chrt22:04:422
1801299170cyclictest167502chrt21:16:252
1801299170cyclictest16022chrt20:11:282
1801299170cyclictest16022chrt20:11:282
1801299170cyclictest157182sleep1019:46:132
1801299170cyclictest148662sleep1000:11:432
1801299170cyclictest146162sleep1023:28:512
1801299170cyclictest140172sleep1022:41:222
1801299170cyclictest130902chrt21:11:122
1801299170cyclictest126962sleep1020:27:442
1801299170cyclictest12250irq/42-ahci21:56:072
1801299170cyclictest12250irq/42-ahci20:40:482
1801299170cyclictest12250irq/42-ahci19:15:002
1801299170cyclictest121612sleep1019:42:252
1801299170cyclictest110552sleep1023:24:512
1801299170cyclictest104782chrt22:37:452
1801299170cyclictest100762chrt21:54:042
1801199172cyclictest0-21swapper/923:00:1611
18011991715cyclictest9703-21chrt21:09:4611
18011991715cyclictest4692-21chrt00:00:0511
18011991715cyclictest20648-21taskset21:24:4811
1801199170cyclictest0-21swapper/920:20:1411
1801199170cyclictest0-21swapper/920:20:1411
1801199170cyclictest0-21swapper/900:10:1411
18010991715cyclictest9435-21snmp_rack3slot921:06:5010
18010991715cyclictest8623-21iostat_ios21:50:2110
18010991715cyclictest7578-21snmp_rack3slot900:01:2610
18010991715cyclictest691-21snmp_rack3slot921:40:1610
18010991715cyclictest4678-21df22:30:1410
18010991715cyclictest333-21snmp_rack3slot922:25:1410
18010991715cyclictest28185-21perl19:20:1710
18010991715cyclictest26205-21snmp_easybox.os23:45:1210
18010991715cyclictest24988-21snmp_rack3slot922:56:5010
18010991715cyclictest2493-21snmp_easybox.os20:15:1210
18010991715cyclictest23404-21snmp_rack3slot919:15:1610
18010991715cyclictest22546-21snmp_rack3slot922:10:1510
18010991715cyclictest22024-21snmp_rack3slot920:40:1710
18010991715cyclictest21749-21snmp_rack3slot923:36:4910
18010991715cyclictest21368-21snmp_rack3slot922:52:0210
18010991715cyclictest20906-21snmp_rack3slot922:07:1610
18010991715cyclictest19519-21snmp_rack3slot919:10:1610
18010991715cyclictest19211-21snmp_easybox.os21:20:2610
18010991715cyclictest18090-21snmp_rack3slot923:31:2510
18010991715cyclictest15855-21snmp_rack3slot922:45:1910
18010991715cyclictest15855-21snmp_rack3slot922:45:1910
18010991715cyclictest15011-21irqstats20:30:2310
18010991715cyclictest14019-21perl19:45:1810
18010991715cyclictest13533-21snmp_rack3slot921:56:3910
18010991715cyclictest11342-21perl21:10:1810
18010991715cyclictest10356-21snmp_easybox.os00:05:2610
18005991715cyclictest21025-21qemu-kvm22:23:188
17984991715cyclictest6397-21latency00:00:215
17984991715cyclictest21848-21fschecks_time20:40:165
1798499171cyclictest0-21swapper/323:45:195
1798499171cyclictest0-21swapper/323:35:195
1798499171cyclictest0-21swapper/322:56:595
1798499171cyclictest0-21swapper/322:46:265
1798499171cyclictest0-21swapper/322:46:265
1798499171cyclictest0-21swapper/322:34:265
1798499171cyclictest0-21swapper/321:20:105
1798499171cyclictest0-21swapper/321:10:035
1798499171cyclictest0-21swapper/321:07:035
1798499171cyclictest0-21swapper/320:45:305
1798499171cyclictest0-21swapper/320:30:225
1798499171cyclictest0-21swapper/320:27:205
1798499171cyclictest0-21swapper/320:05:385
1798499171cyclictest0-21swapper/319:45:095
1798499171cyclictest0-21swapper/319:25:025
1798499171cyclictest0-21swapper/319:15:285
1798499171cyclictest0-21swapper/300:39:575
1798499171cyclictest0-21swapper/300:34:455
1798499171cyclictest0-21swapper/300:20:165
1798499171cyclictest0-21swapper/300:14:535
17974991715cyclictest2406-21snmpd23:53:424
17971991716cyclictest300082sleep019:23:020
18012991615cyclictest0-21swapper/1020:30:122
18011991615cyclictest13718-21df_abs19:45:1611
18011991615cyclictest0-21swapper/923:44:0411
18011991615cyclictest0-21swapper/920:25:1411
18011991614cyclictest6381-21chrt21:48:1511
1801199160cyclictest23957-1kworker/9:2H20:03:2111
1801199160cyclictest0-21swapper/922:10:2311
18010991614cyclictest6967-21df_inode20:20:1610
18010991614cyclictest6967-21df_inode20:20:1510
18010991614cyclictest6323-21snmp_easybox.os23:15:2510
18010991614cyclictest4105-21perl21:00:1710
18010991614cyclictest29322-21users00:30:4910
18010991614cyclictest25777-21snmp_rack3slot900:26:3910
18010991614cyclictest2213-21iostat_ios23:10:1910
18010991614cyclictest22053-21df_inode21:25:1710
18010991614cyclictest15068-21perl21:15:1610
18010991614cyclictest11819-21snmp_rack3slot923:25:1410
18010991614cyclictest10871-21snmp_rack3slot923:22:1410
18010991614cyclictest10525-21df20:25:1510
18005991615cyclictest30514-21qemu-kvm20:08:588
17992991615cyclictest27537-21sensors22:15:316
1798499163cyclictest0-21swapper/322:27:205
1798499163cyclictest0-21swapper/321:01:525
1798499163cyclictest0-21swapper/320:15:375
1798499163cyclictest0-21swapper/319:40:385
1798499163cyclictest0-21swapper/319:30:425
1798499163cyclictest0-21swapper/300:06:525
1798499161cyclictest0-21swapper/323:50:515
1798499161cyclictest0-21swapper/322:51:455
1798499161cyclictest0-21swapper/322:40:445
1798499161cyclictest0-21swapper/321:41:225
1798499161cyclictest0-21swapper/320:38:235
1798499161cyclictest0-21swapper/319:37:365
1798499160cyclictest0-21swapper/323:40:085
1798499160cyclictest0-21swapper/323:34:385
1798499160cyclictest0-21swapper/322:22:185
1798499160cyclictest0-21swapper/322:00:175
1798499160cyclictest0-21swapper/321:50:215
1798499160cyclictest0-21swapper/321:35:155
1798499160cyclictest0-21swapper/321:30:355
1798499160cyclictest0-21swapper/320:12:275
1798499160cyclictest0-21swapper/320:12:275
1798499160cyclictest0-21swapper/319:12:265
1797499163cyclictest0-21swapper/222:12:234
1797499163cyclictest0-21swapper/220:29:084
1797499162cyclictest0-21swapper/222:41:394
17974991614cyclictest2406-21snmpd23:59:304
18012991514cyclictest32632-1kworker/10:2H19:12:462
1801199154cyclictest0-21swapper/922:05:0111
1801199154cyclictest0-21swapper/920:45:0211
18011991514cyclictest741rcuc/921:40:3811
18011991514cyclictest23957-1kworker/9:2H23:31:1511
18011991514cyclictest0-21swapper/923:55:1911
18010991513cyclictest7701-21snmp_rack3slot922:35:1510
18010991513cyclictest5611-21snmp_easybox.os19:35:1310
18010991513cyclictest32264-21snmp_easybox.os00:35:2710
18010991513cyclictest13853-21snmp_easybox.os00:10:2310
18010991513cyclictest10501-21snmp_rack3slot919:40:1810
1800599153cyclictest0-21swapper/623:39:568
17992991513cyclictest17016-21sensors22:45:326
17992991513cyclictest17016-21sensors22:45:326
1798499150cyclictest0-21swapper/323:16:545
1798499150cyclictest0-21swapper/322:38:005
1798499150cyclictest0-21swapper/322:13:445
1798499150cyclictest0-21swapper/321:16:245
1798499150cyclictest0-21swapper/320:59:445
1798499150cyclictest0-21swapper/320:21:485
1798499150cyclictest0-21swapper/320:21:485
1797499153cyclictest0-21swapper/223:46:164
1797499153cyclictest0-21swapper/200:10:074
1797499152cyclictest0-21swapper/200:34:454
17972991513cyclictest9733-21turbostat21:14:561
18012991413cyclictest27249-1kworker/10:0H21:38:312
1801199144cyclictest0-21swapper/919:15:0011
18011991413cyclictest0-21swapper/900:32:0311
18011991412cyclictest32219-21taskset23:05:4711
18011991412cyclictest32219-21taskset23:05:4711
18010991412cyclictest30972-21perl23:50:1810
18010991412cyclictest26705-21snmp_rack3slot920:01:2610
18010991412cyclictest26477-21perl22:15:1910
18010991412cyclictest18156-21irqstats19:50:2310
1798499141cyclictest0-21swapper/323:10:025
1797499145cyclictest0-21swapper/223:42:034
1797499143cyclictest0-21swapper/223:26:214
1797499143cyclictest0-21swapper/222:47:124
1797499143cyclictest0-21swapper/222:47:114
1797499143cyclictest0-21swapper/222:17:134
1797499143cyclictest0-21swapper/220:22:014
1797499143cyclictest0-21swapper/220:22:014
1797499143cyclictest0-21swapper/219:35:054
1797499142cyclictest0-21swapper/200:39:004
17971991413cyclictest3066350irq/52-eth0-rx-21:28:550
8799130watchdog/110-21swapper/1100:24:403
3899130watchdog/40-21swapper/423:09:276
3899130watchdog/40-21swapper/423:09:276
1801299134cyclictest0-21swapper/1023:35:012
1801299133cyclictest0-21swapper/1000:08:232
1801199134cyclictest0-21swapper/920:05:0111
1801199134cyclictest0-21swapper/900:25:0111
1801199132cyclictest0-21swapper/920:40:0111
18011991312cyclictest23957-1kworker/9:2H22:22:3611
18011991312cyclictest0-21swapper/923:35:2411
18011991312cyclictest0-21swapper/922:40:3011
18011991312cyclictest0-21swapper/922:40:0111
18011991311cyclictest3096-21taskset22:26:0911
18011991311cyclictest28822-21df_inode20:50:1511
18011991311cyclictest25181-21chrt22:59:2311
18011991311cyclictest23465-21snmp_easybox.os19:15:1611
18011991311cyclictest23023-21chrt19:56:2611
18010991311cyclictest18651-21iostat_ios20:35:2410
18009991312cyclictest0-21swapper/721:14:569
1800599137cyclictest531rcuc/623:00:018
1800599133cyclictest0-21swapper/621:47:528
1800599133cyclictest0-21swapper/619:26:138
1799899133cyclictest0-21swapper/520:45:227
17998991312cyclictest7437-21latency_hist23:20:017
17992991312cyclictest28337-21sensors23:45:266
1798499133cyclictest0-21swapper/321:45:165
1798499130cyclictest0-21swapper/321:25:245
1797499135cyclictest0-21swapper/223:05:234
1797499135cyclictest0-21swapper/223:05:234
1797499135cyclictest0-21swapper/200:05:244
1797499133cyclictest0-21swapper/223:39:454
1797499133cyclictest0-21swapper/223:23:504
1797499133cyclictest0-21swapper/223:19:184
1797499133cyclictest0-21swapper/222:57:504
1797499133cyclictest0-21swapper/222:52:524
1797499133cyclictest0-21swapper/222:31:264
1797499133cyclictest0-21swapper/222:25:084
1797499133cyclictest0-21swapper/220:59:174
1797499133cyclictest0-21swapper/220:54:334
1797499133cyclictest0-21swapper/220:49:164
1797499133cyclictest0-21swapper/220:40:564
1797499133cyclictest0-21swapper/220:14:364
1797499133cyclictest0-21swapper/220:14:364
1797499133cyclictest0-21swapper/220:06:014
1797499133cyclictest0-21swapper/220:02:354
1797499133cyclictest0-21swapper/219:15:224
1797499133cyclictest0-21swapper/200:16:344
1797499133cyclictest0-21swapper/200:01:094
1797499132cyclictest0-21swapper/222:20:414
1797499132cyclictest0-21swapper/220:38:544
1797499131cyclictest0-21swapper/223:32:234
1797499131cyclictest0-21swapper/220:32:054
1797499131cyclictest0-21swapper/200:26:324
1797499130cyclictest0-21swapper/219:32:334
17971991312cyclictest306682sleep019:25:020
3899120watchdog/40-21swapper/422:38:396
31120ksoftirqd/00-21swapper/019:30:280
1801399124cyclictest0-21swapper/1123:50:013
1801399122cyclictest0-21swapper/1121:48:113
1801399122cyclictest0-21swapper/1100:15:263
1801399122cyclictest0-21swapper/1100:10:413
1801399122cyclictest0-21swapper/1100:02:413
1801399120cyclictest0-21swapper/1123:02:263
18012991211cyclictest3274-21python23:55:282
18012991211cyclictest229052sleep1000:25:102
1801199123cyclictest0-21swapper/919:35:0111
18011991211cyclictest27953-21awk21:35:0011
18011991211cyclictest0-21swapper/923:50:1811
18011991211cyclictest0-21swapper/923:29:2611
18011991211cyclictest0-21swapper/920:40:1211
1801199120cyclictest0-21swapper/919:30:1411
1800599124cyclictest0-21swapper/623:00:028
1800599124cyclictest0-21swapper/622:51:578
1800599124cyclictest0-21swapper/600:35:008
1800599122cyclictest0-21swapper/623:51:228
1800599122cyclictest0-21swapper/623:30:338
1800599122cyclictest0-21swapper/620:56:018
1800599122cyclictest0-21swapper/620:28:468
17998991211cyclictest481ksoftirqd/519:11:457
1797499123cyclictest0-21swapper/223:00:044
1797499123cyclictest0-21swapper/222:36:554
1797499123cyclictest0-21swapper/222:05:144
1797499123cyclictest0-21swapper/222:03:024
1797499123cyclictest0-21swapper/221:26:094
1797499123cyclictest0-21swapper/221:15:374
1797499123cyclictest0-21swapper/221:11:434
1797499123cyclictest0-21swapper/220:15:084
1797499123cyclictest0-21swapper/219:49:064
1797499123cyclictest0-21swapper/219:41:514
1797499123cyclictest0-21swapper/219:25:094
1797499123cyclictest0-21swapper/219:24:334
1797499123cyclictest0-21swapper/219:10:274
1797499122cyclictest0-21swapper/221:04:064
1797499120cyclictest0-21swapper/200:23:594
17972991210cyclictest29257-21turbostat23:54:561
17971991211cyclictest3066350irq/52-eth0-rx-21:18:050
1797199120cyclictest0-21swapper/020:45:020
1797199120cyclictest0-21swapper/019:15:300
621110ksoftirqd/70-21swapper/719:11:459
551110ksoftirqd/60-21swapper/620:00:358
481110ksoftirqd/50-21swapper/522:10:317
3899110watchdog/40-21swapper/419:43:026
31110ksoftirqd/00-21swapper/022:10:310
31110ksoftirqd/00-21swapper/020:55:370
31110ksoftirqd/00-21swapper/020:30:300
31110ksoftirqd/00-21swapper/020:00:350
31110ksoftirqd/00-21swapper/000:20:250
31110ksoftirqd/00-21swapper/000:05:290
201110ksoftirqd/10-21swapper/119:11:451
1801399113cyclictest0-21swapper/1123:49:563
1801399113cyclictest0-21swapper/1123:19:413
1801399113cyclictest0-21swapper/1122:41:503
1801399113cyclictest0-21swapper/1122:26:563
1801399113cyclictest0-21swapper/1121:56:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional