You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 06:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Thu Dec 04, 2025 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31100991919,0cyclictest66850irq/125-lan23:16:451
31100991919,0cyclictest66850irq/125-lan20:25:101
31100991919,0cyclictest66850irq/125-lan00:22:571
31100991918,1cyclictest66850irq/125-lan22:45:121
31100991918,1cyclictest66850irq/125-lan21:23:471
31100991918,1cyclictest66850irq/125-lan21:10:521
31100991818,0cyclictest66850irq/125-lan22:14:381
31100991818,0cyclictest66850irq/125-lan22:06:391
31100991818,0cyclictest66850irq/125-lan21:39:361
31100991818,0cyclictest66850irq/125-lan21:39:361
31100991818,0cyclictest66850irq/125-lan21:27:241
31100991818,0cyclictest66850irq/125-lan20:30:541
31100991818,0cyclictest66850irq/125-lan00:19:421
31100991817,1cyclictest66850irq/125-lan23:50:261
31100991817,1cyclictest66850irq/125-lan23:40:211
31100991817,1cyclictest66850irq/125-lan23:30:171
31100991817,1cyclictest66850irq/125-lan23:20:411
31100991817,1cyclictest66850irq/125-lan23:01:401
31100991817,1cyclictest66850irq/125-lan22:55:391
31100991817,1cyclictest66850irq/125-lan22:30:121
31100991817,1cyclictest66850irq/125-lan22:00:211
31100991817,1cyclictest66850irq/125-lan21:51:391
31100991817,1cyclictest66850irq/125-lan21:15:301
31100991817,1cyclictest66850irq/125-lan20:55:191
31100991817,1cyclictest66850irq/125-lan19:50:211
31100991817,1cyclictest66850irq/125-lan19:35:161
31100991817,1cyclictest66850irq/125-lan00:11:241
31100991817,1cyclictest66850irq/125-lan00:02:091
31100991717,0cyclictest66850irq/125-lan23:57:371
31100991717,0cyclictest66850irq/125-lan23:45:391
31100991717,0cyclictest66850irq/125-lan22:41:191
31100991717,0cyclictest66850irq/125-lan22:25:181
31100991717,0cyclictest66850irq/125-lan22:18:021
31100991717,0cyclictest66850irq/125-lan21:55:201
31100991717,0cyclictest66850irq/125-lan21:46:481
31100991717,0cyclictest66850irq/125-lan00:30:161
31100991717,0cyclictest66850irq/125-lan00:27:281
31100991716,1cyclictest66850irq/125-lan23:25:101
31100991716,1cyclictest66850irq/125-lan23:11:401
31100991716,1cyclictest66850irq/125-lan23:05:141
31100991716,1cyclictest66850irq/125-lan22:37:371
31100991716,1cyclictest66850irq/125-lan22:23:051
31100991716,1cyclictest66850irq/125-lan21:43:551
31100991716,1cyclictest66850irq/125-lan21:30:251
31100991716,1cyclictest66850irq/125-lan20:15:151
31100991716,1cyclictest66850irq/125-lan00:05:181
31100991616,0cyclictest66850irq/125-lan22:50:181
31100991616,0cyclictest66850irq/125-lan20:40:151
31100991616,0cyclictest66850irq/125-lan20:23:381
31100991616,0cyclictest66850irq/125-lan19:58:361
31100991616,0cyclictest66850irq/125-lan19:34:511
31100991616,0cyclictest66850irq/125-lan00:38:021
31100991615,1cyclictest66850irq/125-lan23:37:131
31100991615,1cyclictest66850irq/125-lan21:00:121
31100991615,1cyclictest66850irq/125-lan19:45:121
31100991615,1cyclictest66850irq/125-lan19:10:201
31100991515,0cyclictest66850irq/125-lan19:42:371
31100991414,0cyclictest66850irq/125-lan21:05:221
31100991414,0cyclictest66850irq/125-lan20:45:131
31100991413,1cyclictest66850irq/125-lan19:20:131
31100991313,0cyclictest66850irq/125-lan19:25:171
31100991312,1cyclictest66850irq/125-lan20:50:191
31100991312,1cyclictest66850irq/125-lan20:05:221
31100991312,1cyclictest66850irq/125-lan20:00:121
31100991110,1cyclictest66850irq/125-lan19:15:151
311069998,1cyclictest612-21gmain23:36:273
311069998,1cyclictest524-21avahi-daemon00:25:193
311009999,0cyclictest66850irq/125-lan20:35:131
310959998,1cyclictest771-21snmpd20:26:010
310959998,1cyclictest769-21lldpd20:46:390
311069988,0cyclictest771-21snmpd23:30:363
311069988,0cyclictest771-21snmpd19:13:263
311069988,0cyclictest771-21snmpd00:22:293
311069988,0cyclictest771-21snmpd00:04:113
311069988,0cyclictest769-21lldpd23:25:093
311069988,0cyclictest769-21lldpd22:12:163
311069988,0cyclictest769-21lldpd20:40:363
311069987,1cyclictest771-21snmpd20:28:073
311069987,1cyclictest771-21snmpd20:04:063
311069987,1cyclictest769-21lldpd20:22:293
311069986,2cyclictest771-21snmpd23:10:153
311069986,2cyclictest771-21snmpd19:47:273
311069986,2cyclictest16463-21cat21:10:143
311039988,0cyclictest771-21snmpd23:52:132
311039988,0cyclictest771-21snmpd21:13:442
311039988,0cyclictest771-21snmpd20:29:582
311039988,0cyclictest771-21snmpd20:09:052
311039988,0cyclictest771-21snmpd00:35:512
311039988,0cyclictest771-21snmpd00:33:092
311039988,0cyclictest769-21lldpd23:58:112
311039988,0cyclictest769-21lldpd23:09:532
311039988,0cyclictest769-21lldpd22:09:222
311039988,0cyclictest769-21lldpd21:42:572
311039988,0cyclictest769-21lldpd20:10:502
311039988,0cyclictest769-21lldpd19:42:082
311039987,1cyclictest771-21snmpd22:03:402
311039987,1cyclictest771-21snmpd20:47:202
311039987,1cyclictest524-21avahi-daemon23:31:482
311039987,1cyclictest524-21avahi-daemon22:40:072
311039986,2cyclictest771-21snmpd23:27:512
311039986,2cyclictest771-21snmpd21:05:292
311039986,2cyclictest771-21snmpd19:39:562
310959988,0cyclictest771-21snmpd20:34:040
310959988,0cyclictest771-21snmpd20:24:430
310959988,0cyclictest771-21snmpd20:08:000
310959988,0cyclictest771-21snmpd00:34:510
310959988,0cyclictest769-21lldpd23:29:190
310959988,0cyclictest769-21lldpd22:07:300
310959988,0cyclictest769-21lldpd22:01:120
310959988,0cyclictest769-21lldpd21:47:210
310959988,0cyclictest769-21lldpd20:41:160
310959987,1cyclictest771-21snmpd21:31:180
310959987,1cyclictest771-21snmpd19:48:120
310959987,1cyclictest771-21snmpd00:11:080
310959987,1cyclictest584-21in:imuxsock00:35:190
311069977,0cyclictest771-21snmpd21:08:163
311069977,0cyclictest769-21lldpd23:21:573
311069977,0cyclictest769-21lldpd22:45:163
311069977,0cyclictest769-21lldpd21:24:083
311069977,0cyclictest769-21lldpd20:39:053
311069977,0cyclictest769-21lldpd19:28:093
311069977,0cyclictest769-21lldpd19:24:103
311069977,0cyclictest524-21avahi-daemon22:52:513
311069977,0cyclictest524-21avahi-daemon22:29:453
311069976,1cyclictest9688-21gltestperf20:55:153
311069976,1cyclictest771-21snmpd23:02:383
311069976,1cyclictest771-21snmpd20:09:533
311069976,1cyclictest771-21snmpd19:43:393
311069976,1cyclictest769-21lldpd23:41:213
311069976,1cyclictest769-21lldpd00:17:543
311069976,1cyclictest7667-21fschecks_count19:30:143
311069976,1cyclictest524-21avahi-daemon23:47:063
311069976,1cyclictest524-21avahi-daemon21:43:403
311069976,1cyclictest511ktimers/300:14:213
311069976,1cyclictest26503-21unixbench_multi22:20:213
311069976,1cyclictest255-20systemd-journal22:43:393
311069975,2cyclictest771-21snmpd23:16:363
311069973,3cyclictest8949-21ssh00:07:433
311069973,3cyclictest3232-21ssh21:59:483
311039977,0cyclictest771-21snmpd20:37:212
311039977,0cyclictest771-21snmpd19:25:502
311039977,0cyclictest769-21lldpd23:14:542
311039977,0cyclictest769-21lldpd22:59:512
311039977,0cyclictest769-21lldpd21:31:092
311039977,0cyclictest769-21lldpd21:27:112
311039977,0cyclictest769-21lldpd19:55:282
311039976,1cyclictest771-21snmpd23:04:452
311039976,1cyclictest771-21snmpd00:26:472
311039976,1cyclictest24795-21grep22:50:162
311039975,2cyclictest771-21snmpd21:17:442
311039975,2cyclictest771-21snmpd19:12:412
311039974,3cyclictest771-21snmpd22:35:202
311039974,2cyclictest586-21rs:main2
311039974,2cyclictest586-21rs:main2
311039973,3cyclictest255-20systemd-journal21:46:362
310959977,0cyclictest771-21snmpd22:26:400
310959977,0cyclictest771-21snmpd21:01:230
310959976,1cyclictest771-21snmpd22:21:190
310959976,1cyclictest771-21snmpd21:18:230
310959976,1cyclictest524-21avahi-daemon23:55:160
310959976,1cyclictest524-21avahi-daemon00:03:460
310959976,1cyclictest24024-21cat23:20:150
310959975,2cyclictest771-21snmpd21:38:030
310959975,2cyclictest771-21snmpd21:38:020
310959975,2cyclictest771-21snmpd19:18:200
310959975,2cyclictest26175-21cat21:50:150
310959974,2cyclictest550-21NetworkManager22:59:480
310959974,2cyclictest255-20systemd-journal21:24:140
310959973,3cyclictest2554-21sed20:35:220
310959973,3cyclictest12766-21diskstats23:10:130
311069965,1cyclictest8467-21cpuspeed_turbos22:35:133
311069965,1cyclictest771-21snmpd21:48:323
311069965,1cyclictest524-21avahi-daemon21:36:183
311069965,1cyclictest524-21avahi-daemon21:36:183
311069965,1cyclictest3871-21cpuspeed_turbos22:00:123
311069965,1cyclictest255-20systemd-journal22:15:313
311069965,1cyclictest1-21systemd23:59:103
311069963,2cyclictest26847-21ssh21:50:213
311069963,2cyclictest2518-21ssh21:28:043
311069962,3cyclictest7305-21ssh21:32:283
311069962,3cyclictest22761-21fw_conntrack23:50:143
311069962,3cyclictest21916-21grep21:15:183
311039966,0cyclictest771-21snmpd20:51:202
311039966,0cyclictest771-21snmpd19:50:172
311039966,0cyclictest771-21snmpd19:19:412
311039966,0cyclictest769-21lldpd21:53:042
311039965,1cyclictest7884-21cut23:35:202
311039965,1cyclictest771-21snmpd20:19:402
311039965,1cyclictest524-21avahi-daemon23:45:182
311039965,1cyclictest524-21avahi-daemon23:43:242
311039965,1cyclictest30015-21sh22:25:012
311039965,1cyclictest255-20systemd-journal23:21:242
311039965,1cyclictest255-20systemd-journal22:15:472
311039965,1cyclictest11497-21cpuspeed_turbos21:00:132
311039964,2cyclictest771-21snmpd20:20:422
311039964,2cyclictest14770-21cat22:10:172
311039963,2cyclictest586-21rs:main2
311039963,2cyclictest255-20systemd-journal00:12:312
311039963,2cyclictest20339-21ntpq20:00:172
311039962,3cyclictest32260-21sed21:55:202
311039962,3cyclictest255-20systemd-journal23:16:062
311039962,3cyclictest21678-21ssh00:20:392
311009966,0cyclictest66850irq/125-lan20:10:161
310959966,0cyclictest771-21snmpd20:11:250
310959966,0cyclictest769-21lldpd22:45:380
310959966,0cyclictest769-21lldpd00:26:040
310959966,0cyclictest769-21lldpd00:07:180
310959965,1cyclictest771-21snmpd20:17:550
310959965,1cyclictest5500-21cpuspeed_turbos19:25:110
310959965,1cyclictest3475-21cut22:30:130
310959965,1cyclictest13867-21cat21:05:150
310959964,2cyclictest771-21snmpd22:12:430
310959964,2cyclictest771-21snmpd19:41:570
310959964,2cyclictest3087-21tr19:20:130
310959964,2cyclictest171ktimers/019:30:120
310959963,3cyclictest550-21NetworkManager23:15:160
310959963,2cyclictest255-20systemd-journal23:45:380
310959963,2cyclictest255-20systemd-journal23:34:420
310959963,2cyclictest17925-21ssh00:17:180
310959962,3cyclictest5484-21ssh23:02:450
310959962,3cyclictest22315-21ssh00:21:420
311069955,0cyclictest769-21lldpd20:54:263
311069954,1cyclictest515-21haveged00:30:123
311069954,1cyclictest511ktimers/321:00:183
311069954,1cyclictest255-20systemd-journal19:50:413
311069953,2cyclictest17784-21grep19:55:143
311069952,3cyclictest24446-21ntpq20:10:173
311069952,2cyclictest7376-21ssh23:05:073
311069952,2cyclictest6282-21sed20:45:183
311069952,2cyclictest586-21rs:main3
311069952,2cyclictest4806-21ssh00:39:053
311069952,2cyclictest31942-21grep20:30:133
311069952,2cyclictest30677-21ssh22:55:203
311069952,2cyclictest10855-21ssh22:05:503
311069951,3cyclictest255-20systemd-journal20:16:113
311039954,1cyclictest4856-21apt-key19:24:592
311039953,2cyclictest771-21snmpd00:00:132
311039953,2cyclictest550-21NetworkManager00:17:172
311039953,2cyclictest3526-21gltestperf22:30:142
311039950,4cyclictest10348-21ntpq21:35:172
311039950,4cyclictest10348-21ntpq21:35:172
310959955,0cyclictest771-21snmpd21:26:570
310959955,0cyclictest524-21avahi-daemon19:12:520
310959954,1cyclictest8041-21grep20:50:160
310959953,2cyclictest771-21snmpd23:51:070
310959952,2cyclictest9860-21ssh22:35:230
310959952,2cyclictest8717-21cat23:05:210
310959952,2cyclictest22145-21ssh22:16:260
310959952,2cyclictest20435-21ld20:00:180
310959952,2cyclictest16923-21ssh21:10:180
310959952,2cyclictest15579-21grep21:40:140
310959952,2cyclictest14508-21ntp_offset22:40:160
310959952,2cyclictest13230-21grep23:40:210
310959952,2cyclictest10320-21ntpq19:35:180
310959951,3cyclictest9687-21fw_packets20:55:150
310959951,3cyclictest27171-21ssh22:52:430
310959951,3cyclictest255-20systemd-journal23:39:500
310959950,5cyclictest550-21NetworkManager21:57:530
311069941,3cyclictest9587-21cut19:35:123
311069941,2cyclictest2420-21uname19:17:053
311039942,2cyclictest32055-21tail20:30:142
311039941,2cyclictest3547-21tune2fs20:40:142
311039941,2cyclictest31241-21rm22:25:172
311039941,2cyclictest13940-21basename19:45:142
311039940,3cyclictest9559-21diskstats20:55:142
311039940,3cyclictest7753-21fw_conntrack19:30:142
310959942,2cyclictest17826-21grep19:55:140
310959941,2cyclictest15654-21grep19:50:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional