You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-07 - 02:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sun Jul 06, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26187993535,0cyclictest12313-21fwupd07:54:371
26180991918,1cyclictest66850irq/125-lan12:15:200
26180991918,1cyclictest66850irq/125-lan11:08:510
26180991918,1cyclictest66850irq/125-lan09:25:220
26180991818,0cyclictest66850irq/125-lan12:34:370
26180991818,0cyclictest66850irq/125-lan11:55:180
26180991818,0cyclictest66850irq/125-lan11:51:000
26180991818,0cyclictest66850irq/125-lan11:25:230
26180991818,0cyclictest66850irq/125-lan11:15:240
26180991818,0cyclictest66850irq/125-lan10:21:590
26180991818,0cyclictest66850irq/125-lan09:30:150
26180991818,0cyclictest66850irq/125-lan08:26:360
26180991817,1cyclictest66850irq/125-lan11:45:180
26180991817,1cyclictest66850irq/125-lan11:40:250
26180991817,1cyclictest66850irq/125-lan11:20:150
26180991817,1cyclictest66850irq/125-lan10:54:030
26180991817,1cyclictest66850irq/125-lan10:47:040
26180991817,1cyclictest66850irq/125-lan10:35:130
26180991817,1cyclictest66850irq/125-lan10:28:470
26180991717,0cyclictest66850irq/125-lan12:37:400
26180991717,0cyclictest66850irq/125-lan12:26:220
26180991717,0cyclictest66850irq/125-lan12:05:170
26180991717,0cyclictest66850irq/125-lan12:00:190
26180991717,0cyclictest66850irq/125-lan11:32:380
26180991717,0cyclictest66850irq/125-lan11:01:460
26180991717,0cyclictest66850irq/125-lan10:43:160
26180991717,0cyclictest66850irq/125-lan10:13:370
26180991717,0cyclictest66850irq/125-lan09:47:060
26180991717,0cyclictest66850irq/125-lan09:10:150
26180991717,0cyclictest66850irq/125-lan08:10:180
26180991716,1cyclictest66850irq/125-lan12:12:050
26180991716,1cyclictest66850irq/125-lan11:35:130
26180991716,1cyclictest66850irq/125-lan11:10:140
26180991716,1cyclictest66850irq/125-lan10:55:100
26180991716,1cyclictest66850irq/125-lan10:02:220
26180991716,1cyclictest66850irq/125-lan09:35:320
26180991716,1cyclictest66850irq/125-lan08:50:180
26180991716,1cyclictest66850irq/125-lan07:40:160
26180991616,0cyclictest771-21snmpd09:20:200
26180991616,0cyclictest66850irq/125-lan12:21:280
26180991616,0cyclictest66850irq/125-lan10:17:440
26180991616,0cyclictest66850irq/125-lan09:59:100
26180991616,0cyclictest66850irq/125-lan09:50:200
26180991616,0cyclictest66850irq/125-lan08:45:220
26180991615,1cyclictest66850irq/125-lan10:07:540
26180991615,1cyclictest66850irq/125-lan09:41:460
26180991615,1cyclictest66850irq/125-lan09:15:190
26180991515,0cyclictest66850irq/125-lan10:30:030
26180991515,0cyclictest66850irq/125-lan08:20:180
26180991515,0cyclictest66850irq/125-lan08:19:290
26180991514,1cyclictest66850irq/125-lan07:14:140
26180991414,0cyclictest66850irq/125-lan08:55:200
26180991414,0cyclictest66850irq/125-lan08:30:210
26180991414,0cyclictest66850irq/125-lan07:15:140
26180991413,1cyclictest66850irq/125-lan09:00:190
26180991413,1cyclictest66850irq/125-lan07:35:210
26180991313,0cyclictest66850irq/125-lan07:55:160
26180991312,1cyclictest66850irq/125-lan07:35:000
26180991212,0cyclictest66850irq/125-lan09:05:050
26180991212,0cyclictest66850irq/125-lan08:36:310
26180991212,0cyclictest66850irq/125-lan08:05:210
26180991212,0cyclictest66850irq/125-lan07:20:570
26180991211,1cyclictest66850irq/125-lan07:50:270
26180991111,0cyclictest66850irq/125-lan08:40:170
26180991111,0cyclictest66850irq/125-lan07:45:160
26180991110,1cyclictest66850irq/125-lan07:25:160
261979998,1cyclictest771-21snmpd09:08:503
261979998,1cyclictest769-21lldpd09:31:113
261979996,2cyclictest771-21snmpd08:25:043
261929998,1cyclictest771-21snmpd07:11:132
261929998,1cyclictest769-21lldpd12:18:502
261929998,1cyclictest769-21lldpd10:46:192
261929998,1cyclictest524-21avahi-daemon09:47:592
261929998,1cyclictest255-20systemd-journal12:33:052
261879998,1cyclictest771-21snmpd08:17:301
261979988,0cyclictest771-21snmpd11:54:193
261979988,0cyclictest771-21snmpd11:01:473
261979988,0cyclictest771-21snmpd10:36:313
261979988,0cyclictest771-21snmpd09:53:153
261979988,0cyclictest769-21lldpd12:11:483
261979988,0cyclictest769-21lldpd09:39:323
261979988,0cyclictest524-21avahi-daemon07:16:353
261979987,1cyclictest771-21snmpd12:31:383
261979987,1cyclictest771-21snmpd10:06:003
261979987,1cyclictest524-21avahi-daemon09:12:223
261979987,1cyclictest524-21avahi-daemon07:40:233
261979987,1cyclictest29503-21ntp_kernel_pll_11:15:203
261979987,1cyclictest13198-21ntp_offset11:30:173
261979986,2cyclictest771-21snmpd10:19:463
261979986,2cyclictest771-21snmpd08:45:003
261929988,0cyclictest771-21snmpd12:26:082
261929988,0cyclictest771-21snmpd10:00:182
261929988,0cyclictest771-21snmpd09:43:542
261929988,0cyclictest769-21lldpd11:48:492
261929988,0cyclictest769-21lldpd09:30:312
261929988,0cyclictest769-21lldpd08:57:312
261929988,0cyclictest769-21lldpd07:57:582
261929988,0cyclictest524-21avahi-daemon11:15:382
261929987,1cyclictest771-21snmpd10:26:102
261929987,1cyclictest771-21snmpd07:41:262
261929987,1cyclictest771-21snmpd07:38:112
261929987,1cyclictest769-21lldpd11:10:412
261929987,1cyclictest524-21avahi-daemon12:22:562
261929986,2cyclictest771-21snmpd08:44:102
261929986,2cyclictest771-21snmpd07:17:112
261879988,0cyclictest771-21snmpd11:04:411
261879988,0cyclictest771-21snmpd10:21:101
261879988,0cyclictest771-21snmpd10:14:521
261879988,0cyclictest771-21snmpd09:17:261
261879988,0cyclictest771-21snmpd09:02:111
261879988,0cyclictest771-21snmpd07:48:481
261879988,0cyclictest769-21lldpd09:27:071
261879988,0cyclictest769-21lldpd08:41:211
261879987,1cyclictest771-21snmpd11:14:291
261879987,1cyclictest771-21snmpd10:37:581
261879987,1cyclictest517-21systemd-resolve11:53:301
261879987,1cyclictest517-21systemd-resolve10:45:321
261879986,2cyclictest771-21snmpd11:47:151
261879986,2cyclictest771-21snmpd10:32:221
261879986,2cyclictest771-21snmpd09:05:441
261879986,2cyclictest771-21snmpd07:22:201
261879986,1cyclictest771-21snmpd12:28:111
261879985,2cyclictest8263-21grep10:55:141
261979977,0cyclictest771-21snmpd11:29:573
261979977,0cyclictest771-21snmpd08:36:163
261979977,0cyclictest771-21snmpd08:02:293
261979977,0cyclictest771-21snmpd07:20:563
261979977,0cyclictest769-21lldpd12:02:223
261979977,0cyclictest524-21avahi-daemon08:44:543
261979976,1cyclictest771-21snmpd12:26:413
261979976,1cyclictest771-21snmpd07:46:013
261979976,1cyclictest771-21snmpd07:32:503
261979976,1cyclictest524-21avahi-daemon10:01:153
261979976,1cyclictest23397-21needreboot08:20:143
261979976,1cyclictest19734-21needreboot09:55:153
261979975,2cyclictest771-21snmpd10:43:373
261979975,2cyclictest771-21snmpd10:33:133
261979975,2cyclictest771-21snmpd10:10:163
261979973,3cyclictest19047-21ssh09:19:083
261979973,3cyclictest13731-21ssh09:49:273
261929977,0cyclictest771-21snmpd11:34:152
261929977,0cyclictest771-21snmpd10:20:012
261929977,0cyclictest769-21lldpd11:58:142
261929977,0cyclictest769-21lldpd11:25:172
261929977,0cyclictest769-21lldpd09:15:472
261929976,1cyclictest769-21lldpd10:18:062
261929976,1cyclictest524-21avahi-daemon10:39:432
261929976,1cyclictest524-21avahi-daemon07:30:282
261929976,1cyclictest3371-21fschecks_time10:50:142
261929976,1cyclictest0-21swapper/209:55:112
261929975,2cyclictest771-21snmpd11:52:322
261929975,2cyclictest771-21snmpd09:36:022
261929975,2cyclictest771-21snmpd08:18:212
261929975,2cyclictest771-21snmpd08:03:302
261929974,3cyclictest771-21snmpd12:12:462
261929974,3cyclictest550-21NetworkManager12:38:562
261929973,3cyclictest13224-21ssh09:11:372
261879977,0cyclictest771-21snmpd11:38:331
261879977,0cyclictest771-21snmpd09:22:261
261879977,0cyclictest769-21lldpd09:12:071
261879977,0cyclictest769-21lldpd08:25:591
261879977,0cyclictest769-21lldpd07:40:301
261879977,0cyclictest769-21lldpd07:13:411
261879977,0cyclictest679-21head09:35:141
261879976,1cyclictest771-21snmpd12:01:281
261879976,1cyclictest771-21snmpd10:50:231
261879976,1cyclictest771-21snmpd08:11:571
261879976,1cyclictest769-21lldpd11:43:111
261879976,1cyclictest524-21avahi-daemon12:10:411
261879976,1cyclictest524-21avahi-daemon07:32:441
261879975,2cyclictest771-21snmpd11:05:231
261879975,2cyclictest771-21snmpd08:47:011
261879975,2cyclictest771-21snmpd07:19:251
261879974,3cyclictest771-21snmpd08:52:431
261879974,2cyclictest255-20systemd-journal10:18:571
261879973,3cyclictest4372-21fw_forwarded_lo07:35:141
261809977,0cyclictest769-21lldpd08:02:440
261979966,0cyclictest771-21snmpd09:25:233
261979966,0cyclictest771-21snmpd08:10:423
261979966,0cyclictest771-21snmpd08:05:033
261979966,0cyclictest769-21lldpd07:26:323
261979965,1cyclictest9449-21cut10:20:163
261979965,1cyclictest771-21snmpd07:55:483
261979964,2cyclictest771-21snmpd12:37:473
261979964,2cyclictest771-21snmpd08:33:213
261979964,2cyclictest2780-21unixbench_singl11:20:223
261979963,2cyclictest586-21rs:main3
261979963,2cyclictest30567-21sed11:45:173
261979963,2cyclictest16150-21ssh10:27:193
261979962,3cyclictest23207-21ssh09:23:573
261979962,3cyclictest21578-21ssh11:07:533
261979962,3cyclictest20778-21ssh11:36:533
261979962,3cyclictest12301-21seq07:54:333
261979962,3cyclictest10102-21ssh12:24:593
261929966,0cyclictest769-21lldpd11:24:532
261929966,0cyclictest769-21lldpd10:43:022
261929966,0cyclictest769-21lldpd07:20:342
261929966,0cyclictest524-21avahi-daemon07:51:272
261929965,1cyclictest8254-21cpuspeed_turbos07:45:122
261929965,1cyclictest771-21snmpd10:07:152
261929965,1cyclictest769-21lldpd10:30:182
261929965,1cyclictest524-21avahi-daemon12:07:282
261929965,1cyclictest428-21cpuspeed_turbos10:10:132
261929963,3cyclictest771-21snmpd08:05:302
261929963,3cyclictest550-21NetworkManager12:00:092
261929963,2cyclictest14737-21ssh11:00:492
261929962,3cyclictest3703-21grep08:50:252
261929962,3cyclictest27326-21memory08:30:152
261929962,3cyclictest25624-21awk09:25:222
261929962,3cyclictest24626-21cat08:25:002
261929962,3cyclictest24155-21sed11:40:152
261929962,3cyclictest19189-21ld11:05:172
261879966,0cyclictest769-21lldpd07:56:541
261879965,1cyclictest771-21snmpd11:28:391
261879965,1cyclictest771-21snmpd07:26:561
261879965,1cyclictest524-21avahi-daemon12:35:331
261879963,3cyclictest550-21NetworkManager09:56:531
261879963,2cyclictest586-21rs:main1
261879962,3cyclictest4630-21ssh09:40:061
261879962,3cyclictest28930-21fw_forwarded_lo09:30:121
261879962,3cyclictest16961-21cut08:05:141
261879961,5cyclictest550-21NetworkManager12:07:271
261879960,2cyclictest1129750ssh11:56:251
261979955,0cyclictest769-21lldpd07:37:483
261979954,1cyclictest0-21swapper/308:50:253
261979953,2cyclictest771-21snmpd08:57:583
261979953,2cyclictest550-21NetworkManager12:16:513
261979952,2cyclictest586-21rs:main3
261979952,2cyclictest255-20systemd-journal09:42:553
261979952,2cyclictest24829-21cat11:10:233
261979952,2cyclictest22027-21grep08:15:243
261979952,2cyclictest20420-21cat12:05:133
261979952,2cyclictest1610-21rm10:48:133
261979951,3cyclictest9372-21users10:55:223
261929955,0cyclictest771-21snmpd09:02:352
261929954,1cyclictest1153-21ntpq08:45:172
261929953,2cyclictest771-21snmpd07:27:022
261929952,3cyclictest550-21NetworkManager08:37:102
261929952,2cyclictest8757-21ssh10:55:172
261929952,2cyclictest8264-21https09:05:002
261929952,2cyclictest19559-21ld08:10:192
261929952,2cyclictest15890-21ssh09:50:282
261929952,2cyclictest10946-21kworker/u16:1+nfsiod09:20:022
261929951,3cyclictest18415-21expr11:35:162
261879955,0cyclictest26809-21gltestperf10:40:141
261879953,2cyclictest771-21snmpd10:28:161
261879952,2cyclictest586-21rs:main1
261879952,2cyclictest4736-21ssh12:20:021
261879952,2cyclictest30108-21unixbench_singl11:15:251
261879952,2cyclictest2155-21ld11:20:171
261879951,3cyclictest27274-21sed08:30:151
261879951,3cyclictest14962-21fw_packets08:00:131
261879951,3cyclictest14716-21fw_forwarded_lo09:50:141
261879950,4cyclictest32545-21ntpq12:15:181
261879950,4cyclictest24487-21ntpq10:00:171
261879950,4cyclictest13221-21ntpq11:30:171
261879950,3cyclictest24663-21cat08:25:001
261979942,2cyclictest771-21snmpd09:00:063
261979941,2cyclictest26678-21gltestperf07:10:153
261979941,2cyclictest10660-21rm11:55:393
261929940,3cyclictest25502-21ntpq08:25:162
261879942,2cyclictest5261-21ntpq08:55:191
261879941,2cyclictest28930-21expr08:35:121
261879941,2cyclictest28512-21sed10:05:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional