You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 06:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Tue Dec 02, 2025 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3538992019,1cyclictest66850irq/125-lan22:55:141
3538991919,0cyclictest66850irq/125-lan21:16:071
3538991919,0cyclictest66850irq/125-lan00:16:021
3538991918,1cyclictest66850irq/125-lan22:50:141
3538991918,1cyclictest66850irq/125-lan22:27:231
3538991918,1cyclictest66850irq/125-lan00:25:141
3544991818,0cyclictest771-21snmpd23:05:482
3544991818,0cyclictest771-21snmpd21:41:292
3544991818,0cyclictest771-21snmpd20:57:062
3544991818,0cyclictest771-21snmpd20:43:152
3544991818,0cyclictest771-21snmpd20:10:082
3538991818,0cyclictest66850irq/125-lan23:35:151
3538991818,0cyclictest66850irq/125-lan23:17:231
3538991818,0cyclictest66850irq/125-lan23:03:591
3538991818,0cyclictest66850irq/125-lan22:42:321
3538991818,0cyclictest66850irq/125-lan21:41:271
3538991818,0cyclictest66850irq/125-lan21:26:591
3538991818,0cyclictest66850irq/125-lan20:15:241
3538991818,0cyclictest66850irq/125-lan00:00:131
3538991817,1cyclictest66850irq/125-lan23:55:191
3538991817,1cyclictest66850irq/125-lan23:50:161
3538991817,1cyclictest66850irq/125-lan22:20:191
3538991817,1cyclictest66850irq/125-lan22:19:331
3538991817,1cyclictest66850irq/125-lan22:10:201
3538991817,1cyclictest66850irq/125-lan21:20:201
3538991817,1cyclictest66850irq/125-lan00:10:561
3544991717,0cyclictest771-21snmpd20:47:462
3538991717,0cyclictest66850irq/125-lan23:49:461
3538991717,0cyclictest66850irq/125-lan22:36:051
3538991717,0cyclictest66850irq/125-lan22:07:361
3538991717,0cyclictest66850irq/125-lan21:50:131
3538991717,0cyclictest66850irq/125-lan20:10:161
3538991717,0cyclictest66850irq/125-lan19:45:141
3538991717,0cyclictest66850irq/125-lan19:31:131
3538991717,0cyclictest66850irq/125-lan19:10:131
3538991717,0cyclictest66850irq/125-lan00:23:441
3538991716,1cyclictest66850irq/125-lan23:25:141
3538991716,1cyclictest66850irq/125-lan23:22:151
3538991716,1cyclictest66850irq/125-lan23:10:151
3538991716,1cyclictest66850irq/125-lan23:05:171
3538991716,1cyclictest66850irq/125-lan22:00:191
3538991716,1cyclictest66850irq/125-lan21:55:101
3538991716,1cyclictest66850irq/125-lan21:45:171
3538991716,1cyclictest66850irq/125-lan21:30:161
3538991716,1cyclictest66850irq/125-lan21:00:231
3538991716,1cyclictest66850irq/125-lan20:40:221
3538991716,1cyclictest66850irq/125-lan20:35:161
3538991716,1cyclictest66850irq/125-lan20:30:201
3538991716,1cyclictest66850irq/125-lan00:36:321
3538991716,1cyclictest66850irq/125-lan00:30:561
3538991716,1cyclictest66850irq/125-lan00:05:191
3544991616,0cyclictest771-21snmpd23:03:472
3538991616,0cyclictest66850irq/125-lan23:40:301
3538991616,0cyclictest66850irq/125-lan23:31:361
3538991616,0cyclictest66850irq/125-lan21:10:041
3538991616,0cyclictest66850irq/125-lan20:50:131
3538991616,0cyclictest66850irq/125-lan19:25:151
3538991615,1cyclictest66850irq/125-lan22:45:031
3538991615,1cyclictest66850irq/125-lan22:31:581
3538991615,1cyclictest66850irq/125-lan21:35:061
3538991615,1cyclictest66850irq/125-lan19:40:161
3538991515,0cyclictest66850irq/125-lan20:25:161
3538991515,0cyclictest66850irq/125-lan20:00:211
3538991515,0cyclictest66850irq/125-lan19:20:161
3538991514,1cyclictest66850irq/125-lan19:15:171
3538991414,0cyclictest66850irq/125-lan21:05:211
3538991414,0cyclictest66850irq/125-lan20:20:201
3538991414,0cyclictest66850irq/125-lan20:20:191
3538991414,0cyclictest66850irq/125-lan19:55:141
3538991414,0cyclictest66850irq/125-lan19:50:161
3544991311,2cyclictest749-21ntpd22:36:142
3538991313,0cyclictest66850irq/125-lan20:55:241
3538991313,0cyclictest66850irq/125-lan20:05:221
3538991312,1cyclictest66850irq/125-lan19:35:151
3534991211,1cyclictest749-21ntpd23:19:300
3534991110,1cyclictest10032-21turbostat23:09:540
35499998,1cyclictest771-21snmpd21:04:173
35499998,1cyclictest771-21snmpd00:17:553
35499998,1cyclictest769-21lldpd22:58:193
35499998,1cyclictest769-21lldpd22:11:383
35449998,1cyclictest584-21in:imuxsock21:30:202
35449996,2cyclictest749-21ntpd22:21:182
35349998,1cyclictest771-21snmpd21:57:090
35349998,1cyclictest769-21lldpd22:03:400
35499988,0cyclictest771-21snmpd23:53:553
35499988,0cyclictest771-21snmpd21:13:223
35499988,0cyclictest769-21lldpd21:22:293
35499988,0cyclictest769-21lldpd19:42:543
35499988,0cyclictest769-21lldpd19:34:203
35499987,1cyclictest771-21snmpd21:15:443
35499987,1cyclictest524-21avahi-daemon22:37:183
35499987,1cyclictest524-21avahi-daemon00:39:503
35499986,2cyclictest771-21snmpd23:47:063
35499986,2cyclictest771-21snmpd19:28:513
35449988,0cyclictest769-21lldpd20:00:222
35449988,0cyclictest769-21lldpd00:27:062
35449988,0cyclictest769-21lldpd00:19:182
35449987,1cyclictest771-21snmpd21:24:372
35449987,1cyclictest771-21snmpd19:31:452
35449987,1cyclictest771-21snmpd19:22:062
35449987,1cyclictest771-21snmpd19:16:452
35449986,2cyclictest771-21snmpd23:36:282
35449986,2cyclictest771-21snmpd23:20:062
35449986,2cyclictest771-21snmpd21:06:162
35449986,2cyclictest771-21snmpd19:41:432
35449986,2cyclictest749-21ntpd23:45:202
35449986,2cyclictest749-21ntpd23:41:542
35449986,2cyclictest749-21ntpd23:27:122
35449986,2cyclictest749-21ntpd23:15:092
35449986,2cyclictest749-21ntpd23:11:252
35449986,2cyclictest749-21ntpd22:50:162
35449986,2cyclictest749-21ntpd22:43:412
35449986,2cyclictest749-21ntpd22:33:492
35449986,2cyclictest749-21ntpd22:10:322
35449986,2cyclictest749-21ntpd21:35:202
35449986,2cyclictest749-21ntpd21:28:422
35449986,2cyclictest749-21ntpd21:15:332
35449986,2cyclictest749-21ntpd21:10:292
35449986,2cyclictest749-21ntpd00:21:562
35449986,2cyclictest749-21ntpd00:14:142
35449984,3cyclictest1147-21ntp_states22:55:182
35389988,0cyclictest66850irq/125-lan20:45:131
35349988,0cyclictest771-21snmpd23:14:000
35349988,0cyclictest771-21snmpd22:34:100
35349988,0cyclictest771-21snmpd20:03:370
35349988,0cyclictest771-21snmpd19:49:130
35349988,0cyclictest769-21lldpd21:20:220
35349988,0cyclictest769-21lldpd19:34:040
35349987,1cyclictest524-21avahi-daemon22:23:060
35349987,1cyclictest524-21avahi-daemon21:18:100
35349987,1cyclictest255-20systemd-journal00:32:070
35349987,1cyclictest22026-21sh22:12:500
35349986,2cyclictest771-21snmpd22:48:460
35349986,2cyclictest771-21snmpd21:49:140
35349986,2cyclictest771-21snmpd20:31:140
35499977,0cyclictest771-21snmpd23:06:453
35499977,0cyclictest771-21snmpd22:34:433
35499977,0cyclictest771-21snmpd20:51:213
35499977,0cyclictest771-21snmpd19:57:193
35499977,0cyclictest769-21lldpd22:46:533
35499977,0cyclictest769-21lldpd19:38:483
35499977,0cyclictest769-21lldpd19:19:153
35499977,0cyclictest769-21lldpd00:28:543
35499976,1cyclictest771-21snmpd22:28:433
35499976,1cyclictest771-21snmpd20:22:083
35499976,1cyclictest771-21snmpd20:22:083
35499976,1cyclictest771-21snmpd00:01:393
35499975,2cyclictest771-21snmpd23:00:443
35499975,2cyclictest771-21snmpd21:25:313
35499973,3cyclictest28669-21fw_conntrack20:10:143
35449977,0cyclictest771-21snmpd21:02:142
35449977,0cyclictest771-21snmpd19:47:012
35449977,0cyclictest769-21lldpd20:36:272
35449977,0cyclictest769-21lldpd00:33:542
35449977,0cyclictest769-21lldpd00:08:032
35449976,1cyclictest771-21snmpd19:11:092
35449976,1cyclictest749-21ntpd23:55:302
35449976,1cyclictest749-21ntpd19:25:182
35449976,1cyclictest524-21avahi-daemon23:33:262
35449976,1cyclictest12435-21gltestperf20:50:152
35449975,2cyclictest749-21ntpd23:52:072
35449975,2cyclictest749-21ntpd22:05:072
35449975,2cyclictest749-21ntpd22:04:262
35449973,3cyclictest2232-21fw_forwarded_lo22:25:142
35349977,0cyclictest771-21snmpd23:31:230
35349977,0cyclictest771-21snmpd20:56:180
35349977,0cyclictest769-21lldpd23:38:550
35349977,0cyclictest769-21lldpd22:09:510
35349977,0cyclictest769-21lldpd21:00:550
35349977,0cyclictest769-21lldpd19:42:020
35349977,0cyclictest769-21lldpd19:27:480
35349976,1cyclictest771-21snmpd23:50:330
35349976,1cyclictest771-21snmpd19:17:030
35349976,1cyclictest771-21snmpd00:23:160
35349976,1cyclictest29735-21cpuspeed_turbos21:50:130
35349975,2cyclictest771-21snmpd22:17:120
35349975,2cyclictest771-21snmpd21:38:060
35349975,2cyclictest771-21snmpd20:46:450
35349975,2cyclictest771-21snmpd00:09:250
35349975,2cyclictest12656-21cat22:35:170
35499966,0cyclictest771-21snmpd23:27:503
35499966,0cyclictest769-21lldpd22:16:003
35499966,0cyclictest769-21lldpd20:36:173
35499966,0cyclictest3803-21sh20:30:003
35499965,1cyclictest769-21lldpd22:02:583
35499965,1cyclictest517-21systemd-resolve21:06:083
35499964,2cyclictest771-21snmpd23:56:573
35499964,2cyclictest771-21snmpd19:22:213
35499963,3cyclictest550-21NetworkManager22:44:363
35499963,2cyclictest9052-21ssh00:05:563
35499963,2cyclictest30656-21ssh00:30:043
35499963,2cyclictest20082-21ssh23:14:473
35499963,1cyclictest9350-21cut21:30:143
35499962,3cyclictest4364-21fw_forwarded_lo20:30:153
35499962,3cyclictest25744-21ssh00:24:573
35499962,3cyclictest14670-21ssh00:12:113
35499962,3cyclictest11922-21ssh23:37:363
35499961,4cyclictest32212-21cat20:20:003
35449966,0cyclictest771-21snmpd19:58:552
35449965,1cyclictest5817-21munin-plugin-st20:35:012
35449965,1cyclictest26637-21fschecks_time20:05:192
35449965,1cyclictest2548-21needreboot00:00:152
35449965,1cyclictest0-21swapper/220:15:222
35449964,2cyclictest749-21ntpd19:36:142
35449963,3cyclictest550-21NetworkManager22:47:382
35449962,3cyclictest4740-21ssh00:35:492
35449960,4cyclictest27545-21sh22:18:202
35349966,0cyclictest771-21snmpd20:35:210
35349966,0cyclictest769-21lldpd22:26:260
35349966,0cyclictest769-21lldpd21:44:140
35349966,0cyclictest769-21lldpd20:15:040
35349966,0cyclictest769-21lldpd00:18:560
35349966,0cyclictest22671-21gltestperf19:55:150
35349966,0cyclictest18823-21idleruntime-cro23:45:000
35349965,1cyclictest771-21snmpd21:34:530
35349965,1cyclictest771-21snmpd20:23:410
35349965,1cyclictest771-21snmpd20:23:410
35349965,1cyclictest769-21lldpd21:05:220
35349965,1cyclictest749-21ntpd23:58:090
35349965,1cyclictest6184-21ntp_kernel_pll_23:00:190
35349965,1cyclictest255-20systemd-journal23:26:510
35349964,2cyclictest749-21ntpd23:22:360
35349964,2cyclictest749-21ntpd20:50:170
35349964,2cyclictest586-21rs:main0
35349963,2cyclictest29963-21ssh22:51:410
35349962,3cyclictest8999-21open_files20:40:180
35349962,3cyclictest21047-21ssh23:45:550
35499955,0cyclictest769-21lldpd21:55:183
35499955,0cyclictest21636-21turbostat19:55:003
35499954,1cyclictest524-21avahi-daemon23:19:523
35499954,1cyclictest511ktimers/323:31:583
35499954,1cyclictest511ktimers/322:20:123
35499954,1cyclictest511ktimers/322:05:583
35499954,1cyclictest14770-21needreboot21:35:153
35499953,2cyclictest24651-21fw_packets20:00:163
35499952,3cyclictest10127-21apt-get20:45:143
35499952,2cyclictest29966-21fw_conntrack21:50:143
35499952,2cyclictest28497-21ntp_states22:50:193
35499952,2cyclictest26607-21ssh23:20:173
35499952,2cyclictest26530-21df20:05:173
35499952,2cyclictest255-20systemd-journal21:41:333
35499952,2cyclictest25185-21ssh21:45:183
35499952,2cyclictest14811-21cut23:40:163
35449955,0cyclictest769-21lldpd20:23:372
35449955,0cyclictest769-21lldpd20:23:362
35449954,1cyclictest749-21ntpd21:45:202
35449952,2cyclictest6213-21ssh21:58:332
35449952,2cyclictest255-20systemd-journal20:25:152
35449950,2cyclictest29796-21kworker/2:0+mm_percpu_wq21:50:212
35349953,2cyclictest771-21snmpd00:35:160
35349953,2cyclictest749-21ntpd20:26:430
35349952,2cyclictest8314-21grep19:20:150
35349952,2cyclictest3560-21ssh00:00:390
35349952,2cyclictest3022-21ssh22:57:170
35349952,2cyclictest17557-21cut22:40:140
35349951,3cyclictest4399-21load21:25:170
35349951,3cyclictest29979-21ssh00:29:220
35349950,4cyclictest13892-21cat19:35:120
35499942,2cyclictest771-21snmpd20:40:063
35499942,2cyclictest586-21rs:main3
35499942,2cyclictest4494-21ntpq19:10:163
35499941,2cyclictest18016-21tr19:45:123
35449942,2cyclictest749-21ntpd19:50:092
35349944,0cyclictest27181-21ntpq20:05:220
35349943,1cyclictest20207-21gltestperf19:50:140
35349942,2cyclictest749-21ntpd20:10:160
35349941,3cyclictest550-21NetworkManager19:14:480
35349940,4cyclictest21252-21ntpq21:10:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional