You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 11:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sat Jul 12, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24806994444,0cyclictest12512-21fwupd21:42:181
24814991919,0cyclictest66850irq/125-lan21:55:253
24814991918,1cyclictest66850irq/125-lan22:25:133
24814991918,1cyclictest66850irq/125-lan21:51:403
24814991918,1cyclictest66850irq/125-lan21:15:173
24814991918,1cyclictest66850irq/125-lan19:55:163
24814991918,1cyclictest66850irq/125-lan00:21:143
24814991918,1cyclictest66850irq/125-lan00:15:173
24814991818,0cyclictest66850irq/125-lan23:35:053
24814991818,0cyclictest66850irq/125-lan23:10:183
24814991818,0cyclictest66850irq/125-lan22:46:253
24814991818,0cyclictest66850irq/125-lan22:15:243
24814991818,0cyclictest66850irq/125-lan22:11:483
24814991818,0cyclictest66850irq/125-lan21:44:523
24814991818,0cyclictest66850irq/125-lan21:26:023
24814991818,0cyclictest66850irq/125-lan21:05:193
24814991818,0cyclictest66850irq/125-lan19:20:193
24814991818,0cyclictest66850irq/125-lan19:10:123
24814991817,1cyclictest66850irq/125-lan22:05:223
24814991817,1cyclictest66850irq/125-lan22:00:243
24814991817,1cyclictest66850irq/125-lan00:32:073
24814991817,1cyclictest66850irq/125-lan00:05:133
24814991717,0cyclictest66850irq/125-lan23:45:043
24814991717,0cyclictest66850irq/125-lan23:42:393
24814991717,0cyclictest66850irq/125-lan23:25:163
24814991717,0cyclictest66850irq/125-lan22:59:493
24814991717,0cyclictest66850irq/125-lan22:30:123
24814991717,0cyclictest66850irq/125-lan19:35:123
24814991717,0cyclictest66850irq/125-lan00:10:153
24814991717,0cyclictest66850irq/125-lan00:00:053
24814991716,1cyclictest66850irq/125-lan23:30:173
24814991716,1cyclictest66850irq/125-lan22:42:133
24814991716,1cyclictest66850irq/125-lan22:35:573
24814991716,1cyclictest66850irq/125-lan22:20:033
24814991716,1cyclictest66850irq/125-lan21:45:133
24814991716,1cyclictest66850irq/125-lan21:36:463
24814991716,1cyclictest66850irq/125-lan21:32:243
24814991716,1cyclictest66850irq/125-lan00:25:183
24814991616,0cyclictest771-21snmpd20:55:073
24814991616,0cyclictest66850irq/125-lan23:55:143
24814991616,0cyclictest66850irq/125-lan23:50:153
24814991616,0cyclictest66850irq/125-lan23:06:043
24814991616,0cyclictest66850irq/125-lan23:00:003
24814991616,0cyclictest66850irq/125-lan22:50:193
24814991616,0cyclictest66850irq/125-lan20:20:183
24814991616,0cyclictest66850irq/125-lan19:30:213
24814991615,1cyclictest66850irq/125-lan23:21:533
24814991615,1cyclictest66850irq/125-lan21:20:203
24814991615,1cyclictest66850irq/125-lan21:12:303
24814991615,1cyclictest66850irq/125-lan20:00:373
24814991615,1cyclictest66850irq/125-lan19:19:103
24814991515,0cyclictest771-21snmpd21:01:283
24814991515,0cyclictest771-21snmpd20:25:513
24814991515,0cyclictest771-21snmpd19:42:103
24814991515,0cyclictest66850irq/125-lan23:15:343
24814991515,0cyclictest66850irq/125-lan20:40:173
24814991515,0cyclictest66850irq/125-lan20:35:153
24814991514,1cyclictest66850irq/125-lan20:50:493
24814991514,1cyclictest66850irq/125-lan20:06:543
24814991514,1cyclictest66850irq/125-lan19:45:163
24814991514,1cyclictest66850irq/125-lan00:35:463
24814991414,0cyclictest66850irq/125-lan20:30:223
24814991414,0cyclictest66850irq/125-lan20:10:173
24814991414,0cyclictest66850irq/125-lan19:29:263
24814991313,0cyclictest66850irq/125-lan19:50:173
24814991111,0cyclictest66850irq/125-lan20:48:083
24814991111,0cyclictest66850irq/125-lan20:15:223
248119998,1cyclictest524-21avahi-daemon21:43:282
248069998,1cyclictest771-21snmpd23:40:131
248069998,1cyclictest769-21lldpd22:55:211
248069998,1cyclictest769-21lldpd21:46:111
247999998,1cyclictest524-21avahi-daemon23:49:430
248119988,0cyclictest769-21lldpd22:07:282
248119988,0cyclictest769-21lldpd21:05:272
248119988,0cyclictest769-21lldpd00:21:112
248119987,1cyclictest771-21snmpd22:26:592
248119987,1cyclictest771-21snmpd20:28:302
248119987,1cyclictest771-21snmpd20:20:072
248119987,1cyclictest524-21avahi-daemon23:09:232
248119987,1cyclictest524-21avahi-daemon21:29:182
248119987,1cyclictest524-21avahi-daemon21:15:382
248119986,2cyclictest771-21snmpd23:25:012
248119986,2cyclictest771-21snmpd20:45:272
248119986,2cyclictest771-21snmpd19:39:132
248119986,2cyclictest771-21snmpd19:19:292
248119986,2cyclictest771-21snmpd19:13:582
248119986,2cyclictest771-21snmpd00:39:362
248119986,2cyclictest771-21snmpd00:01:252
248069988,0cyclictest771-21snmpd23:13:581
248069988,0cyclictest771-21snmpd23:08:051
248069988,0cyclictest771-21snmpd22:41:021
248069988,0cyclictest769-21lldpd22:11:261
248069988,0cyclictest769-21lldpd20:02:161
248069988,0cyclictest769-21lldpd19:29:351
248069987,1cyclictest771-21snmpd21:00:041
248069987,1cyclictest771-21snmpd19:56:141
248069987,1cyclictest771-21snmpd19:50:331
248069987,1cyclictest771-21snmpd19:43:131
248069987,1cyclictest771-21snmpd00:11:381
248069987,1cyclictest749-21ntpd21:12:261
248069987,1cyclictest584-21in:imuxsock21:57:111
248069987,1cyclictest524-21avahi-daemon22:25:581
248069987,1cyclictest517-21systemd-resolve22:33:211
248069987,1cyclictest32233-21ntp_kernel_pll_23:35:211
248069986,2cyclictest771-21snmpd21:15:011
248069986,2cyclictest771-21snmpd19:48:471
248069986,2cyclictest771-21snmpd19:18:141
247999988,0cyclictest771-21snmpd19:43:070
247999988,0cyclictest771-21snmpd00:14:540
247999988,0cyclictest769-21lldpd23:31:010
247999988,0cyclictest769-21lldpd21:50:090
247999988,0cyclictest769-21lldpd21:04:580
247999987,1cyclictest771-21snmpd23:01:020
247999987,1cyclictest771-21snmpd20:48:150
247999987,1cyclictest524-21avahi-daemon00:20:320
247999986,2cyclictest771-21snmpd23:20:450
247999986,2cyclictest771-21snmpd21:28:330
247999986,2cyclictest771-21snmpd20:27:510
247999985,3cyclictest13164-21cat22:45:140
248119977,0cyclictest771-21snmpd22:58:302
248119977,0cyclictest771-21snmpd22:44:472
248119977,0cyclictest771-21snmpd20:32:152
248119977,0cyclictest769-21lldpd22:03:572
248119977,0cyclictest769-21lldpd20:50:192
248119977,0cyclictest769-21lldpd19:30:502
248119976,1cyclictest771-21snmpd23:45:402
248119976,1cyclictest771-21snmpd21:10:012
248119976,1cyclictest771-21snmpd21:01:462
248119976,1cyclictest524-21avahi-daemon23:19:372
248119976,1cyclictest524-21avahi-daemon21:55:522
248119976,1cyclictest517-21systemd-resolve22:51:232
248119975,2cyclictest771-21snmpd23:52:102
248119975,2cyclictest771-21snmpd23:12:212
248069977,0cyclictest771-21snmpd23:53:491
248069977,0cyclictest771-21snmpd21:23:511
248069977,0cyclictest771-21snmpd20:56:551
248069977,0cyclictest769-21lldpd23:27:221
248069977,0cyclictest769-21lldpd22:18:061
248069977,0cyclictest769-21lldpd22:01:051
248069977,0cyclictest769-21lldpd19:37:171
248069977,0cyclictest769-21lldpd19:32:391
248069977,0cyclictest524-21avahi-daemon20:05:401
248069976,1cyclictest8951-21fschecks_count21:05:141
248069976,1cyclictest771-21snmpd22:08:311
248069976,1cyclictest771-21snmpd19:12:191
248069976,1cyclictest584-21in:imuxsock23:23:441
248069976,1cyclictest524-21avahi-daemon22:53:171
248069976,1cyclictest27054-21ntp_kernel_pll_23:30:161
248069975,2cyclictest771-21snmpd23:18:511
248069975,2cyclictest771-21snmpd20:54:111
248069975,2cyclictest771-21snmpd00:28:241
248069974,2cyclictest27866-21ntp_states20:30:171
248069974,2cyclictest255-20systemd-journal22:35:111
248069974,2cyclictest10729-21dpkg00:20:001
247999977,0cyclictest771-21snmpd23:13:220
247999977,0cyclictest771-21snmpd22:36:220
247999977,0cyclictest771-21snmpd22:22:130
247999977,0cyclictest771-21snmpd22:07:540
247999977,0cyclictest771-21snmpd22:00:240
247999977,0cyclictest771-21snmpd21:23:030
247999977,0cyclictest771-21snmpd21:05:260
247999977,0cyclictest771-21snmpd20:10:110
247999977,0cyclictest769-21lldpd23:15:120
247999977,0cyclictest769-21lldpd22:44:050
247999977,0cyclictest769-21lldpd22:29:270
247999977,0cyclictest769-21lldpd21:15:390
247999977,0cyclictest769-21lldpd20:37:130
247999977,0cyclictest769-21lldpd19:16:550
247999977,0cyclictest769-21lldpd19:11:190
247999976,1cyclictest524-21avahi-daemon21:43:550
247999976,1cyclictest517-21systemd-resolve23:51:560
247999976,1cyclictest31554-21cpuspeed_turbos19:25:130
247999975,2cyclictest771-21snmpd21:45:110
247999975,2cyclictest771-21snmpd20:20:190
247999975,2cyclictest771-21snmpd00:35:260
247999975,2cyclictest31880-21cat23:35:180
248119966,0cyclictest771-21snmpd21:32:502
248119966,0cyclictest769-21lldpd21:48:012
248119966,0cyclictest769-21lldpd20:41:502
248119966,0cyclictest769-21lldpd19:43:072
248119966,0cyclictest769-21lldpd00:27:102
248119965,1cyclictest771-21snmpd19:47:512
248119965,1cyclictest517-21systemd-resolve22:46:192
248119965,1cyclictest31554-21cpuspeed_turbos19:25:132
248119965,1cyclictest15723-21cpuspeed_turbos20:05:142
248119964,2cyclictest9002-21cat22:10:152
248119964,2cyclictest771-21snmpd20:55:222
248119964,2cyclictest550-21NetworkManager22:36:202
248119963,3cyclictest550-21NetworkManager22:15:002
248119963,3cyclictest550-21NetworkManager21:39:292
248119963,2cyclictest8440-21ssh00:16:482
248119963,2cyclictest7705-21ssh23:43:222
248119963,2cyclictest32503-21ssh00:07:512
248119963,2cyclictest23848-21ssh23:59:422
248119962,3cyclictest31628-21ssh22:31:082
248119962,3cyclictest11932-21grep19:55:172
248069966,0cyclictest769-21lldpd23:04:581
248069965,1cyclictest255-20systemd-journal21:36:331
248069965,1cyclictest19461-21cpuspeed_turbos22:20:131
248069964,2cyclictest771-21snmpd23:46:531
248069964,2cyclictest771-21snmpd20:40:361
248069964,2cyclictest771-21snmpd20:21:131
248069964,2cyclictest255-20systemd-journal21:51:411
248069963,3cyclictest771-21snmpd20:12:441
248069963,3cyclictest550-21NetworkManager00:32:091
248069963,3cyclictest13586-21ntpq22:45:171
248069963,2cyclictest255-20systemd-journal00:06:501
248069963,2cyclictest1793-21ssh21:31:491
248069962,3cyclictest25382-21sh00:35:461
247999966,0cyclictest771-21snmpd20:50:430
247999966,0cyclictest771-21snmpd20:16:020
247999966,0cyclictest769-21lldpd21:34:270
247999966,0cyclictest769-21lldpd00:29:380
247999965,1cyclictest771-21snmpd23:55:380
247999965,1cyclictest771-21snmpd22:13:310
247999965,1cyclictest3410-21gltestperf19:35:150
247999965,1cyclictest171ktimers/022:50:140
247999964,2cyclictest517-21systemd-resolve00:09:480
247999963,3cyclictest7912-21ssh21:38:100
247999963,3cyclictest550-21NetworkManager23:41:310
247999963,3cyclictest11770-21df_inode19:55:150
247999963,2cyclictest24721-21sshd23:28:260
247999961,5cyclictest550-21NetworkManager22:30:450
247999961,5cyclictest550-21NetworkManager20:59:290
247999960,3cyclictest19008-21awk22:20:000
248119955,0cyclictest771-21snmpd19:20:402
248119955,0cyclictest769-21lldpd00:30:032
248119954,1cyclictest26492-21fschecks_time23:30:132
248119953,2cyclictest771-21snmpd20:00:392
248119953,2cyclictest23674-21ssh21:22:012
248119952,3cyclictest9338-21diskstats19:50:142
248119952,3cyclictest29318-21fw_conntrack20:35:152
248119952,3cyclictest16003-21grep23:20:132
248119952,2cyclictest3226-21awk00:10:222
248119952,2cyclictest31668-21ssh23:03:442
248119952,2cyclictest21206-21ssh22:20:532
248119952,2cyclictest1163-21rm23:36:222
248119951,3cyclictest20096-21apt20:15:112
248069954,1cyclictest524-21avahi-daemon00:00:061
248069952,3cyclictest6585-21fw_packets00:15:151
248069952,2cyclictest550-21NetworkManager21:25:581
248069952,2cyclictest21040-21ssh23:55:391
248069952,2cyclictest1992-21sed20:45:221
248069951,3cyclictest30140-21grep19:20:131
248069951,3cyclictest29575-21sed20:35:161
248069951,3cyclictest20652-21memory20:15:151
247999955,0cyclictest769-21lldpd21:59:120
247999955,0cyclictest517-21systemd-resolve20:08:160
247999955,0cyclictest11537-21needreboot21:10:150
247999954,1cyclictest20559-21cut00:30:170
247999953,2cyclictest771-21snmpd20:01:350
247999953,2cyclictest0-21swapper/020:30:110
247999952,2cyclictest900-21cut23:05:120
247999952,2cyclictest27567-21sshd00:02:370
247999952,2cyclictest255-20systemd-journal22:58:400
247999952,2cyclictest255-20systemd-journal00:18:430
247999952,2cyclictest1441-21grep19:30:130
248119942,2cyclictest20580-21cat21:50:182
248119940,3cyclictest18375-21ntpq20:10:162
248069941,3cyclictest550-21NetworkManager20:28:471
247999942,2cyclictest550-21NetworkManager19:51:130
247999941,2cyclictest29733-21grep19:20:120
247999941,1cyclictest217-20jbd2/nvme0n1p2-819:46:510
247999940,3cyclictest31228-21diskstats20:40:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional