You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-22 - 09:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Wed Oct 22, 2025 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31468991919,0cyclictest66850irq/125-lan21:56:122
31468991918,1cyclictest66850irq/125-lan23:24:372
31468991918,1cyclictest66850irq/125-lan23:00:132
31468991918,1cyclictest66850irq/125-lan22:50:172
31468991918,1cyclictest66850irq/125-lan22:35:152
31468991918,1cyclictest66850irq/125-lan22:26:212
31468991918,1cyclictest66850irq/125-lan22:20:202
31468991918,1cyclictest66850irq/125-lan22:00:032
31468991918,1cyclictest66850irq/125-lan21:45:372
31468991918,1cyclictest66850irq/125-lan21:30:212
31468991918,1cyclictest66850irq/125-lan21:05:152
31468991918,1cyclictest66850irq/125-lan00:14:242
31468991818,0cyclictest66850irq/125-lan23:50:072
31468991818,0cyclictest66850irq/125-lan23:28:102
31468991818,0cyclictest66850irq/125-lan21:17:412
31468991818,0cyclictest66850irq/125-lan00:00:442
31468991817,1cyclictest66850irq/125-lan23:42:472
31468991817,1cyclictest66850irq/125-lan23:30:142
31468991817,1cyclictest66850irq/125-lan21:38:382
31468991817,1cyclictest66850irq/125-lan00:35:192
31468991817,1cyclictest66850irq/125-lan00:32:502
31468991717,0cyclictest66850irq/125-lan23:35:182
31468991717,0cyclictest66850irq/125-lan21:53:412
31468991717,0cyclictest66850irq/125-lan21:40:182
31468991717,0cyclictest66850irq/125-lan21:25:182
31468991717,0cyclictest66850irq/125-lan21:20:512
31468991717,0cyclictest66850irq/125-lan20:50:202
31468991717,0cyclictest66850irq/125-lan19:48:322
31468991717,0cyclictest66850irq/125-lan19:38:282
31468991717,0cyclictest66850irq/125-lan00:20:082
31468991716,1cyclictest66850irq/125-lan23:55:022
31468991716,1cyclictest66850irq/125-lan23:45:162
31468991716,1cyclictest66850irq/125-lan23:05:192
31468991716,1cyclictest66850irq/125-lan22:58:152
31468991716,1cyclictest66850irq/125-lan22:45:592
31468991716,1cyclictest66850irq/125-lan22:32:162
31468991716,1cyclictest66850irq/125-lan22:10:212
31468991716,1cyclictest66850irq/125-lan22:08:522
31468991716,1cyclictest66850irq/125-lan20:30:202
31468991716,1cyclictest66850irq/125-lan20:30:192
31468991716,1cyclictest66850irq/125-lan19:58:542
31468991716,1cyclictest66850irq/125-lan19:50:212
31468991716,1cyclictest66850irq/125-lan00:05:462
31468991616,0cyclictest66850irq/125-lan22:15:222
31468991616,0cyclictest66850irq/125-lan20:00:442
31468991616,0cyclictest66850irq/125-lan00:16:152
31468991615,1cyclictest66850irq/125-lan23:15:192
31468991615,1cyclictest66850irq/125-lan23:10:032
31468991615,1cyclictest66850irq/125-lan21:11:222
31468991615,1cyclictest66850irq/125-lan20:45:162
31468991615,1cyclictest66850irq/125-lan20:15:232
31468991615,1cyclictest66850irq/125-lan00:25:572
31463991616,0cyclictest771-21snmpd20:29:311
31463991616,0cyclictest771-21snmpd19:35:231
31463991616,0cyclictest22612-21cat00:05:171
31463991615,1cyclictest749-21ntpd22:35:461
31468991515,0cyclictest66850irq/125-lan20:36:202
31468991515,0cyclictest66850irq/125-lan19:30:132
31468991515,0cyclictest66850irq/125-lan19:15:122
31468991515,0cyclictest66850irq/125-lan19:10:152
31468991514,1cyclictest66850irq/125-lan22:42:552
31468991514,1cyclictest66850irq/125-lan20:25:172
31468991514,1cyclictest66850irq/125-lan19:22:452
31463991515,0cyclictest771-21snmpd22:33:531
31463991515,0cyclictest771-21snmpd22:23:531
31463991515,0cyclictest771-21snmpd20:38:521
31463991515,0cyclictest771-21snmpd20:02:021
31463991515,0cyclictest23794-21cat23:00:151
31468991414,0cyclictest66850irq/125-lan20:20:212
31463991414,0cyclictest771-21snmpd23:50:261
31463991414,0cyclictest771-21snmpd23:36:321
31463991414,0cyclictest771-21snmpd22:46:201
31463991414,0cyclictest771-21snmpd22:41:271
31463991414,0cyclictest771-21snmpd21:34:481
31463991414,0cyclictest771-21snmpd20:23:491
31463991414,0cyclictest771-21snmpd19:50:241
31463991414,0cyclictest771-21snmpd00:31:041
31463991414,0cyclictest13624-21cat21:05:171
31463991414,0cyclictest10642-21cat23:20:171
31463991313,0cyclictest771-21snmpd20:54:291
31463991313,0cyclictest771-21snmpd20:30:301
31463991313,0cyclictest771-21snmpd20:30:301
31463991313,0cyclictest14197-21cat22:50:191
31468991212,0cyclictest66850irq/125-lan19:40:422
31468991211,1cyclictest66850irq/125-lan20:10:172
31468991211,1cyclictest66850irq/125-lan19:25:192
31468991111,0cyclictest66850irq/125-lan20:42:062
314739998,1cyclictest771-21snmpd22:57:413
314739998,1cyclictest771-21snmpd20:42:223
314739998,1cyclictest771-21snmpd00:23:213
314739998,1cyclictest769-21lldpd22:46:043
314689999,0cyclictest66850irq/125-lan20:57:082
314639998,1cyclictest771-21snmpd23:07:131
314599998,1cyclictest771-21snmpd19:25:440
314599998,1cyclictest2958-21ntp_kernel_pll_23:45:180
314739988,0cyclictest771-21snmpd21:13:413
314739988,0cyclictest771-21snmpd20:04:443
314739988,0cyclictest771-21snmpd19:38:473
314739988,0cyclictest769-21lldpd23:29:063
314739988,0cyclictest769-21lldpd23:09:503
314739988,0cyclictest769-21lldpd22:42:493
314739988,0cyclictest769-21lldpd21:19:583
314739988,0cyclictest769-21lldpd20:09:253
314739988,0cyclictest769-21lldpd19:11:373
314739987,1cyclictest771-21snmpd23:23:163
314739987,1cyclictest771-21snmpd22:34:333
314739987,1cyclictest769-21lldpd23:31:553
314739987,1cyclictest524-21avahi-daemon21:56:493
314739987,1cyclictest524-21avahi-daemon21:37:263
314739987,1cyclictest517-21systemd-resolve22:26:433
314739986,2cyclictest771-21snmpd20:52:323
314739986,2cyclictest771-21snmpd00:28:193
314689988,0cyclictest66850irq/125-lan20:07:022
314689987,1cyclictest771-21snmpd21:00:232
314639988,0cyclictest771-21snmpd21:18:291
314639988,0cyclictest769-21lldpd23:16:491
314639987,1cyclictest771-21snmpd20:49:411
314639987,1cyclictest771-21snmpd20:15:301
314639987,1cyclictest771-21snmpd19:15:101
314639987,1cyclictest524-21avahi-daemon23:27:151
314639987,1cyclictest524-21avahi-daemon00:10:461
314639987,1cyclictest517-21systemd-resolve00:03:561
314639986,2cyclictest771-21snmpd21:37:221
314639986,2cyclictest771-21snmpd19:10:131
314639986,2cyclictest771-21snmpd00:35:491
314639986,2cyclictest771-21snmpd00:19:121
314639986,2cyclictest749-21ntpd23:59:041
314639986,2cyclictest749-21ntpd23:46:221
314639986,2cyclictest749-21ntpd23:40:211
314639986,2cyclictest749-21ntpd23:31:181
314639986,2cyclictest749-21ntpd23:11:511
314639986,2cyclictest749-21ntpd22:55:391
314639986,2cyclictest749-21ntpd22:13:041
314639986,2cyclictest749-21ntpd21:59:581
314639986,2cyclictest749-21ntpd21:51:041
314639986,2cyclictest749-21ntpd21:28:441
314639986,2cyclictest13111-21gltestperf22:15:151
314599988,0cyclictest771-21snmpd23:10:460
314599988,0cyclictest771-21snmpd22:02:180
314599988,0cyclictest771-21snmpd21:29:570
314599988,0cyclictest771-21snmpd21:19:380
314599988,0cyclictest771-21snmpd20:40:380
314599988,0cyclictest771-21snmpd19:14:440
314599988,0cyclictest771-21snmpd00:10:210
314599988,0cyclictest769-21lldpd22:51:360
314599988,0cyclictest769-21lldpd22:09:410
314599988,0cyclictest769-21lldpd21:35:570
314599987,1cyclictest771-21snmpd23:32:530
314599987,1cyclictest771-21snmpd21:53:580
314599987,1cyclictest771-21snmpd19:57:090
314599987,1cyclictest524-21avahi-daemon21:23:490
314599987,1cyclictest12156-21fschecks_count23:55:140
314599986,2cyclictest771-21snmpd23:41:200
314599986,2cyclictest771-21snmpd23:16:570
314599986,2cyclictest5127-21cat20:45:120
314599984,3cyclictest255-20systemd-journal22:25:260
314739977,0cyclictest769-21lldpd23:00:503
314739977,0cyclictest769-21lldpd20:19:173
314739977,0cyclictest769-21lldpd20:11:173
314739976,1cyclictest771-21snmpd19:51:153
314739976,1cyclictest771-21snmpd00:12:483
314739976,1cyclictest25128-21cpuspeed_turbos21:20:163
314739975,2cyclictest771-21snmpd21:07:113
314739972,4cyclictest550-21NetworkManager20:20:073
314639977,0cyclictest769-21lldpd21:41:161
314639977,0cyclictest769-21lldpd19:40:551
314639976,1cyclictest771-21snmpd20:56:251
314639976,1cyclictest749-21ntpd22:00:331
314639975,2cyclictest749-21ntpd22:27:401
314639975,2cyclictest749-21ntpd21:21:031
314639975,2cyclictest749-21ntpd21:10:221
314639975,2cyclictest749-21ntpd00:20:381
314639974,3cyclictest4225-21ntp_states22:05:211
314599977,0cyclictest771-21snmpd23:50:230
314599977,0cyclictest771-21snmpd21:12:120
314599977,0cyclictest771-21snmpd21:03:130
314599977,0cyclictest769-21lldpd23:05:520
314599977,0cyclictest769-21lldpd19:47:300
314599976,1cyclictest771-21snmpd20:21:450
314599976,1cyclictest769-21lldpd20:25:320
314599976,1cyclictest517-21systemd-resolve00:00:380
314599975,2cyclictest771-21snmpd22:40:210
314599975,2cyclictest771-21snmpd20:05:530
314599975,2cyclictest771-21snmpd19:19:490
314599975,2cyclictest771-21snmpd00:29:100
314599973,3cyclictest13691-21ssh23:23:400
314739966,0cyclictest771-21snmpd23:55:573
314739966,0cyclictest771-21snmpd00:18:473
314739966,0cyclictest769-21lldpd21:25:463
314739966,0cyclictest769-21lldpd20:34:243
314739966,0cyclictest769-21lldpd20:34:233
314739966,0cyclictest769-21lldpd19:26:313
314739965,1cyclictest771-21snmpd19:20:173
314739965,1cyclictest517-21systemd-resolve23:38:153
314739965,1cyclictest29620-21cpuspeed_turbos20:25:143
314739964,2cyclictest771-21snmpd21:02:173
314739964,2cyclictest550-21NetworkManager21:48:463
314739964,2cyclictest550-21NetworkManager21:32:043
314739963,3cyclictest9147-21apt-get20:55:093
314739963,3cyclictest550-21NetworkManager22:35:113
314739963,2cyclictest32312-21ssh23:42:203
314739963,2cyclictest255-20systemd-journal23:17:083
314739962,3cyclictest13620-21sh22:50:043
314739962,1cyclictest55050NetworkManager22:19:203
314639965,1cyclictest749-21ntpd21:47:281
314639965,1cyclictest584-21in:imuxsock21:01:391
314639965,1cyclictest584-21in:imuxsock19:33:151
314639965,1cyclictest524-21avahi-daemon19:25:441
314639964,2cyclictest749-21ntpd20:05:211
314639962,3cyclictest4718-21seq19:20:501
314599966,0cyclictest771-21snmpd21:40:340
314599966,0cyclictest771-21snmpd20:12:430
314599966,0cyclictest771-21snmpd19:33:580
314599966,0cyclictest524-21avahi-daemon00:07:290
314599965,1cyclictest8230-21fschecks_count22:10:160
314599965,1cyclictest771-21snmpd22:21:590
314599965,1cyclictest769-21lldpd22:17:440
314599965,1cyclictest524-21avahi-daemon21:58:240
314599965,1cyclictest26358-21ntpq20:15:200
314599964,2cyclictest771-21snmpd23:26:130
314599964,2cyclictest771-21snmpd20:33:220
314599964,2cyclictest771-21snmpd20:33:210
314599964,2cyclictest0-21swapper/022:36:240
314599963,3cyclictest550-21NetworkManager00:21:510
314599963,2cyclictest255-20systemd-journal22:57:490
314599962,3cyclictest28582-21sh22:30:260
314599962,3cyclictest16861-21seq21:45:150
314739955,0cyclictest771-21snmpd22:22:143
314739955,0cyclictest769-21lldpd23:54:473
314739955,0cyclictest769-21lldpd19:16:223
314739954,1cyclictest771-21snmpd19:32:133
314739954,1cyclictest769-21lldpd00:05:393
314739954,1cyclictest524-21avahi-daemon19:42:183
314739954,1cyclictest511ktimers/321:53:513
314739954,1cyclictest11615-21cpuspeed_turbos21:40:123
314739953,2cyclictest771-21snmpd22:12:313
314739953,2cyclictest771-21snmpd00:00:233
314739953,2cyclictest2585-21cat23:45:153
314739952,3cyclictest31250-21perl22:00:143
314739952,3cyclictest2249-21grep20:35:173
314739952,2cyclictest17693-21fschecks_count00:35:143
314739952,2cyclictest13796-21sed00:30:183
314739952,2cyclictest13777-21sed19:45:133
314739951,3cyclictest4288-21sed22:05:213
314739950,4cyclictest1385-21ntpq23:10:203
314639954,1cyclictest771-21snmpd20:44:431
314639953,2cyclictest13848-21cat19:45:131
314639952,3cyclictest541-21systemd-logind20:10:221
314639952,2cyclictest586-21rs:main1
314639950,4cyclictest18140-21ntpq19:55:181
314599955,0cyclictest769-21lldpd19:52:140
314599954,1cyclictest771-21snmpd20:03:210
314599954,1cyclictest24932-21gltestperf23:35:140
314599954,1cyclictest13473-21gltestperf00:30:180
314599953,2cyclictest771-21snmpd19:39:340
314599953,2cyclictest550-21NetworkManager00:15:100
314599953,2cyclictest13958-21ntpq21:05:190
314599952,2cyclictest6444-21sed21:34:590
314599952,2cyclictest255-20systemd-journal19:40:540
314599952,2cyclictest20117-21ssh00:37:400
314599952,2cyclictest10675-21ssh22:45:510
314739942,2cyclictest771-21snmpd19:57:573
314739941,2cyclictest4785-21cut20:45:103
314599941,2cyclictest7678-21cat20:50:170
314599941,2cyclictest23237-21sh23:00:050
314599941,2cyclictest1847-21cut20:35:140
314599940,3cyclictest9375-21/usr/sbin/munin20:55:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional