You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-04 - 20:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot6s.osadl.org (updated Sun May 04, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26483991919,0cyclictest65850irq/126-lan12:39:312
26483991918,1cyclictest65850irq/126-lan12:12:072
26483991818,0cyclictest765-21snmpd12:04:492
26483991818,0cyclictest765-21snmpd07:53:072
26483991818,0cyclictest65850irq/126-lan12:25:172
26483991818,0cyclictest65850irq/126-lan11:58:262
26483991818,0cyclictest65850irq/126-lan11:45:122
26483991818,0cyclictest65850irq/126-lan11:43:022
26483991818,0cyclictest65850irq/126-lan11:36:112
26483991818,0cyclictest65850irq/126-lan11:10:152
26483991818,0cyclictest65850irq/126-lan11:01:592
26483991818,0cyclictest65850irq/126-lan09:41:012
26483991818,0cyclictest65850irq/126-lan09:25:092
26483991818,0cyclictest65850irq/126-lan09:24:302
26483991817,1cyclictest9870-21kworker/2:2+events10:57:302
26483991817,1cyclictest9870-21kworker/2:2+events10:53:212
26483991817,1cyclictest9870-21kworker/2:2+events10:37:252
26483991817,1cyclictest65850irq/126-lan12:19:392
26483991817,1cyclictest65850irq/126-lan11:20:032
26483991817,1cyclictest65850irq/126-lan10:30:132
26483991817,1cyclictest65850irq/126-lan10:15:422
26483991817,1cyclictest65850irq/126-lan10:03:392
26483991817,1cyclictest65850irq/126-lan07:25:152
26483991717,0cyclictest765-21snmpd09:07:542
26483991717,0cyclictest765-21snmpd08:21:022
26483991717,0cyclictest765-21snmpd08:06:442
26483991717,0cyclictest65850irq/126-lan12:34:552
26483991717,0cyclictest65850irq/126-lan12:22:492
26483991717,0cyclictest65850irq/126-lan11:51:112
26483991717,0cyclictest65850irq/126-lan11:34:122
26483991717,0cyclictest65850irq/126-lan11:25:172
26483991717,0cyclictest65850irq/126-lan11:05:192
26483991717,0cyclictest65850irq/126-lan10:47:102
26483991717,0cyclictest65850irq/126-lan10:26:392
26483991717,0cyclictest65850irq/126-lan10:21:102
26483991717,0cyclictest65850irq/126-lan10:10:142
26483991717,0cyclictest65850irq/126-lan10:05:032
26483991717,0cyclictest65850irq/126-lan09:01:422
26483991717,0cyclictest65850irq/126-lan08:25:202
26483991717,0cyclictest65850irq/126-lan07:49:172
26483991716,1cyclictest65850irq/126-lan10:41:372
26483991716,1cyclictest65850irq/126-lan09:55:232
26483991716,1cyclictest65850irq/126-lan09:50:192
26483991716,1cyclictest65850irq/126-lan09:37:472
26483991716,1cyclictest65850irq/126-lan09:30:182
26483991716,1cyclictest65850irq/126-lan09:16:502
26483991616,0cyclictest65850irq/126-lan09:45:172
26483991616,0cyclictest65850irq/126-lan09:12:362
26483991616,0cyclictest65850irq/126-lan07:15:012
26483991615,1cyclictest65850irq/126-lan12:05:442
26483991615,1cyclictest65850irq/126-lan11:17:492
26483991615,1cyclictest65850irq/126-lan08:35:142
26483991515,0cyclictest65850irq/126-lan08:55:182
26483991515,0cyclictest65850irq/126-lan08:05:002
26483991515,0cyclictest65850irq/126-lan07:35:222
26483991414,0cyclictest65850irq/126-lan08:33:302
26483991414,0cyclictest65850irq/126-lan07:42:512
26483991414,0cyclictest65850irq/126-lan07:20:202
26483991413,1cyclictest65850irq/126-lan07:30:132
26483991413,1cyclictest65850irq/126-lan07:10:182
26483991313,0cyclictest65850irq/126-lan08:53:172
26483991313,0cyclictest65850irq/126-lan08:45:162
26483991312,1cyclictest65850irq/126-lan08:18:592
26473991111,0cyclictest765-21snmpd08:46:380
26473991111,0cyclictest765-21snmpd07:53:090
26473991111,0cyclictest27500-21cat10:20:140
26483991010,0cyclictest65850irq/126-lan08:12:422
2647399109,0cyclictest765-21snmpd10:04:060
264779998,1cyclictest765-21snmpd09:56:001
264779998,1cyclictest765-21snmpd08:29:091
264779998,1cyclictest763-21lldpd11:58:531
264779998,1cyclictest19985-21cut11:29:591
264739999,0cyclictest765-21snmpd09:00:510
264739999,0cyclictest765-21snmpd07:24:420
264739999,0cyclictest739-21ntpd09:25:140
264739998,1cyclictest765-21snmpd10:59:430
264739998,1cyclictest765-21snmpd10:30:250
264739998,1cyclictest765-21snmpd08:06:250
264739998,1cyclictest763-21lldpd08:43:170
264739998,1cyclictest506-21avahi-daemon10:49:340
264889988,0cyclictest765-21snmpd11:27:363
264889988,0cyclictest765-21snmpd11:05:403
264889988,0cyclictest765-21snmpd10:56:533
264889988,0cyclictest765-21snmpd08:45:053
264889988,0cyclictest763-21lldpd10:40:193
264889988,0cyclictest763-21lldpd10:02:593
264889988,0cyclictest763-21lldpd09:06:273
264889988,0cyclictest763-21lldpd08:06:323
264889988,0cyclictest763-21lldpd07:10:363
264889987,1cyclictest765-21snmpd09:22:153
264889987,1cyclictest765-21snmpd07:34:543
264889987,1cyclictest763-21lldpd12:39:503
264889987,1cyclictest763-21lldpd08:56:283
264889987,1cyclictest506-21avahi-daemon08:10:103
264889986,2cyclictest765-21snmpd10:21:003
264889986,2cyclictest765-21snmpd09:33:583
264889986,2cyclictest765-21snmpd09:26:113
264779988,0cyclictest765-21snmpd12:10:461
264779988,0cyclictest765-21snmpd11:17:391
264779988,0cyclictest763-21lldpd11:39:251
264779988,0cyclictest763-21lldpd08:58:411
264779988,0cyclictest763-21lldpd08:16:371
264779987,1cyclictest765-21snmpd07:55:281
264779987,1cyclictest765-21snmpd07:27:261
264779987,1cyclictest763-21lldpd10:09:511
264779987,1cyclictest551-21in:imuxsock08:05:141
264779987,1cyclictest506-21avahi-daemon12:25:301
264779987,1cyclictest506-21avahi-daemon09:12:141
264779987,1cyclictest506-21avahi-daemon09:05:171
264779986,2cyclictest765-21snmpd09:46:051
264779986,2cyclictest765-21snmpd09:43:051
264779986,2cyclictest765-21snmpd08:52:411
264779986,2cyclictest765-21snmpd08:37:321
264779986,2cyclictest765-21snmpd08:10:381
264779986,2cyclictest765-21snmpd08:04:041
264779986,2cyclictest765-21snmpd07:22:511
264779984,3cyclictest14344-21diskstats10:45:141
264739988,0cyclictest765-21snmpd11:32:360
264739988,0cyclictest765-21snmpd10:40:300
264739988,0cyclictest765-21snmpd10:10:500
264739988,0cyclictest765-21snmpd08:52:260
264739988,0cyclictest765-21snmpd08:20:340
264739988,0cyclictest765-21snmpd08:11:440
264739988,0cyclictest763-21lldpd11:58:310
264739988,0cyclictest763-21lldpd10:29:010
264739988,0cyclictest739-21ntpd12:13:010
264739987,1cyclictest499-21systemd-resolve11:36:480
264739987,1cyclictest20651-21ntp_kernel_pll_12:00:210
264739986,2cyclictest765-21snmpd09:46:470
264739986,2cyclictest765-21snmpd08:34:080
264889977,0cyclictest765-21snmpd10:25:553
264889977,0cyclictest765-21snmpd10:09:443
264889977,0cyclictest765-21snmpd09:53:473
264889977,0cyclictest765-21snmpd08:03:463
264889977,0cyclictest763-21lldpd11:22:123
264889977,0cyclictest763-21lldpd10:19:253
264889977,0cyclictest763-21lldpd09:11:333
264889977,0cyclictest763-21lldpd07:59:213
264889977,0cyclictest763-21lldpd07:23:203
264889977,0cyclictest20650-21kworker/u16:3-events_unbound10:54:523
264889976,1cyclictest765-21snmpd11:57:243
264889976,1cyclictest765-21snmpd10:49:353
264889976,1cyclictest765-21snmpd10:32:093
264889976,1cyclictest765-21snmpd08:23:293
264889976,1cyclictest506-21avahi-daemon11:33:303
264889976,1cyclictest506-21avahi-daemon08:38:323
264889975,2cyclictest765-21snmpd09:35:253
264889975,2cyclictest765-21snmpd09:00:093
264889975,2cyclictest6948-21cat07:45:173
264889974,2cyclictest31786-21ssh11:40:153
264839976,1cyclictest9870-21kworker/2:2+events08:40:142
264839973,3cyclictest10161-21fw_forwarded_lo07:55:142
264779977,0cyclictest765-21snmpd09:36:341
264779977,0cyclictest765-21snmpd07:31:081
264779977,0cyclictest763-21lldpd12:01:401
264779977,0cyclictest763-21lldpd10:44:191
264779977,0cyclictest763-21lldpd10:02:141
264779977,0cyclictest763-21lldpd07:54:511
264779976,1cyclictest765-21snmpd09:18:541
264779976,1cyclictest506-21avahi-daemon09:28:311
264779976,1cyclictest499-21systemd-resolve11:22:551
264779976,1cyclictest499-21systemd-resolve10:21:561
264779976,1cyclictest23984-21cat12:35:161
264779975,2cyclictest765-21snmpd07:47:061
264779975,2cyclictest765-21snmpd07:41:091
264779975,2cyclictest765-21snmpd07:19:081
264779973,3cyclictest254-20systemd-journal10:55:171
264739977,0cyclictest765-21snmpd12:37:150
264739977,0cyclictest765-21snmpd11:41:390
264739977,0cyclictest765-21snmpd11:01:440
264739977,0cyclictest765-21snmpd10:38:180
264739977,0cyclictest765-21snmpd10:07:500
264739977,0cyclictest765-21snmpd08:18:490
264739977,0cyclictest765-21snmpd07:28:140
264739977,0cyclictest765-21snmpd07:18:020
264739977,0cyclictest765-21snmpd07:12:350
264739977,0cyclictest763-21lldpd09:51:530
264739977,0cyclictest763-21lldpd09:42:400
264739977,0cyclictest763-21lldpd08:57:090
264739977,0cyclictest739-21ntpd10:54:560
264739976,1cyclictest765-21snmpd07:48:240
264739976,1cyclictest506-21avahi-daemon11:54:050
264739976,1cyclictest506-21avahi-daemon10:15:370
264739976,1cyclictest254-20systemd-journal12:30:000
264739975,2cyclictest765-21snmpd12:28:280
264739975,2cyclictest765-21snmpd09:23:390
264889966,0cyclictest765-21snmpd12:02:193
264889966,0cyclictest765-21snmpd08:41:203
264889966,0cyclictest763-21lldpd08:28:423
264889966,0cyclictest763-21lldpd07:25:113
264889965,1cyclictest8943-21nvmesmart_nvme007:50:213
264889965,1cyclictest765-21snmpd11:46:583
264889964,2cyclictest765-21snmpd12:15:113
264889964,2cyclictest765-21snmpd09:41:473
264889964,2cyclictest32407-21cat09:45:153
264889963,3cyclictest530-21NetworkManager12:14:543
264889963,3cyclictest530-21NetworkManager11:52:503
264889963,3cyclictest530-21NetworkManager09:16:173
264889963,2cyclictest20694-21ssh12:31:553
264889961,4cyclictest530-21NetworkManager07:40:583
264779966,0cyclictest763-21lldpd10:13:381
264779966,0cyclictest763-21lldpd07:39:521
264779965,1cyclictest765-21snmpd11:41:151
264779965,1cyclictest765-21snmpd08:47:561
264779965,1cyclictest506-21avahi-daemon11:03:421
264779965,1cyclictest499-21systemd-resolve12:34:051
264779965,1cyclictest499-21systemd-resolve11:30:481
264779963,3cyclictest530-21NetworkManager10:52:051
264779963,3cyclictest1965-21ssh10:29:081
264779963,2cyclictest554-21rs:main1
264779963,2cyclictest2770-21ssh11:10:181
264779963,2cyclictest254-20systemd-journal12:07:351
264779963,2cyclictest254-20systemd-journal11:06:011
264779963,2cyclictest12868-21ssh09:21:161
264779962,3cyclictest25416-21ssh10:18:011
264779961,4cyclictest7815-21cat12:20:001
264739966,0cyclictest765-21snmpd11:23:270
264739966,0cyclictest765-21snmpd11:12:400
264739966,0cyclictest739-21ntpd12:05:480
264739966,0cyclictest739-21ntpd11:15:090
264739966,0cyclictest739-21ntpd11:06:450
264739966,0cyclictest506-21avahi-daemon07:40:110
264739965,1cyclictest765-21snmpd09:19:150
264739965,1cyclictest171ktimers/009:36:000
264739965,1cyclictest171ktimers/009:32:010
264739964,2cyclictest765-21snmpd09:59:380
264739964,2cyclictest765-21snmpd07:39:360
264739962,3cyclictest6457-21ssh11:47:030
264889954,1cyclictest17465-21cat08:15:193
264889953,2cyclictest765-21snmpd07:15:323
264889953,2cyclictest739-21ntpd12:23:003
264889953,2cyclictest739-21ntpd12:08:173
264889953,2cyclictest254-20systemd-journal10:37:213
264889952,2cyclictest7083-21ssh11:15:393
264889952,2cyclictest3970-21find07:35:223
264889952,2cyclictest29817-21sed08:50:213
264889952,2cyclictest29670-21ssh11:38:323
264889952,2cyclictest26360-21apt-get11:00:153
264889951,3cyclictest8001-21grep09:55:203
264889951,3cyclictest3324-21ssh11:10:503
264779955,0cyclictest763-21lldpd11:46:241
264779955,0cyclictest763-21lldpd09:33:551
264779953,2cyclictest765-21snmpd10:35:361
264779953,2cyclictest26943-21dpkg07:10:001
264779952,3cyclictest739-21ntpd08:30:461
264779952,3cyclictest18880-21df_inode08:20:131
264779952,2cyclictest6280-21ssh10:34:381
264779952,2cyclictest1002-21awk09:00:191
264779951,3cyclictest3361-21ssh09:50:121
264779951,3cyclictest10261-21ntpq11:50:171
264739955,0cyclictest763-21lldpd09:05:180
264739955,0cyclictest763-21lldpd07:33:350
264739955,0cyclictest747-21in:imuxsock08:30:000
264739955,0cyclictest499-21systemd-resolve07:58:170
264739953,2cyclictest765-21snmpd12:17:400
264739952,2cyclictest254-20systemd-journal11:28:370
264739950,5cyclictest254-20systemd-journal12:20:410
264889940,4cyclictest22070-21apt-get08:30:123
264889940,3cyclictest13298-21df12:25:123
264779940,3cyclictest26062-21ntpq08:40:181
264739942,2cyclictest530-21NetworkManager08:35:130
264739941,2cyclictest4131-21iwlist09:10:180
264739940,3cyclictest11859-21fw_conntrack08:00:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional