You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-16 - 05:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Wed Sep 16, 2026 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
65729910491048,1cyclictest7046-21kworker/3:1+events20:28:053
65729910481047,1cyclictest22523-21kworker/3:2+events22:14:463
65569910481044,4cyclictest29140-21turbostat20:09:550
65689910461045,1cyclictest12038-21kworker/2:1+events21:35:222
65729910441042,1cyclictest27352-21kworker/3:2+events20:01:273
65569910441043,1cyclictest26935-21kworker/0:2+events20:39:030
65689910431041,2cyclictest13985-21kworker/2:0+events23:42:002
65689910411040,1cyclictest12038-21kworker/2:1+events00:39:362
65569910371036,1cyclictest12789-21kworker/0:1+events19:34:020
65689910351034,1cyclictest18089-21kworker/2:0+events00:15:442
65569910351034,1cyclictest9509-21kworker/0:0+events20:45:440
65689910341031,3cyclictest25055-21turbostat22:19:552
65569910331032,1cyclictest3662-21kworker/0:2+events19:28:190
65569910311030,1cyclictest1019-21kworker/0:2+events22:55:440
65729910301029,1cyclictest23432-21kworker/3:0+events23:28:273
65729910301028,2cyclictest27998-21turbostat22:54:553
65689910301029,1cyclictest19507-21kworker/2:2+events23:47:122
65729910281027,1cyclictest8502-21turbostat19:19:563
65569910261024,2cyclictest20801-21turbostat23:19:550
65729910251024,1cyclictest7771-21kworker/3:2+events22:35:553
65729910231023,0cyclictest23432-21kworker/3:0+events23:35:213
65729910231023,0cyclictest23432-21kworker/3:0+events23:35:213
65569910231022,1cyclictest12976-21kworker/0:1+events23:07:590
65729910211020,1cyclictest7046-21kworker/3:1+events21:10:433
65729910201019,1cyclictest17258-21kworker/3:1+events22:17:323
65629910201019,1cyclictest19507-21kworker/1:0+events19:14:251
65729910181017,1cyclictest32212-21kworker/3:2+events21:53:383
65729910181017,1cyclictest25478-21kworker/3:2+events23:33:193
65729910171016,1cyclictest2920-21kworker/3:2+events20:18:403
65729910161014,2cyclictest10531-21turbostat19:24:563
65729910151014,1cyclictest6928-21kworker/3:0+events21:56:513
65729910121011,1cyclictest7046-21kworker/3:1+events21:25:223
65729910121011,1cyclictest2920-21kworker/3:2+events20:23:153
65689910121011,1cyclictest4604-21kworker/2:2+events00:02:122
65689910101007,2cyclictest17352-21turbostat20:59:552
65629910091008,1cyclictest19507-21kworker/1:0+events21:45:221
65729910081007,1cyclictest6928-21kworker/3:0+events22:01:293
65729910071005,2cyclictest7257-21turbostat20:34:553
65729910061005,1cyclictest27352-21kworker/3:2+events20:05:223
65729910051004,1cyclictest30787-21kworker/3:0+events20:10:463
65729910051004,1cyclictest27899-21kworker/3:0+events22:42:353
65729910051004,1cyclictest25902-21kworker/3:0+events21:21:493
65729910051004,1cyclictest23432-21kworker/3:0+events23:21:583
65729910041003,1cyclictest27899-21kworker/3:0+events22:34:473
65729910031003,0cyclictest7046-21kworker/3:1+events21:04:463
65729910031002,1cyclictest7046-21kworker/3:1+events21:17:073
65729910011000,1cyclictest25478-21kworker/3:2+events22:46:513
657299998997,1cyclictest6928-21kworker/3:0+events22:08:513
657299997995,2cyclictest22210-21turbostat21:44:553
657299996995,1cyclictest19623-21kworker/3:2+events21:06:063
656299996994,2cyclictest2107-21kworker/1:3+events00:13:521
655699996995,1cyclictest20356-21kworker/0:0+events19:46:380
655699996993,3cyclictest2769-21turbostat22:29:550
656299995994,1cyclictest29596-21kworker/1:2+events00:28:191
656899994992,2cyclictest31143-21turbostat20:14:552
656299993991,2cyclictest24657-21turbostat19:59:551
657299991989,2cyclictest5170-21turbostat00:39:553
655699990988,2cyclictest1019-21kworker/0:2+events23:00:430
656299988987,1cyclictest17636-21kworker/1:1+events19:17:461
657299981978,3cyclictest6933-21turbostat00:09:553
656299980979,1cyclictest30030-21kworker/1:1+events23:12:351
656299976972,2cyclictest19507-21kworker/1:0+events22:25:211
656899974973,1cyclictest4604-21kworker/2:2+events00:08:172
656299974972,2cyclictest12728-21kworker/1:0+events00:23:191
656299974972,1cyclictest2480-21kworker/1:0+events00:34:351
656299970969,1cyclictest4740-21kworker/1:2+events23:16:521
656299970969,1cyclictest30030-21kworker/1:1+events22:23:171
655699970968,2cyclictest16991-21turbostat00:19:550
656899968967,1cyclictest31671-21kworker/2:0+events23:57:302
656899967966,1cyclictest12038-21kworker/2:1+events21:34:232
657299960958,2cyclictest10738-21turbostat23:09:563
656299959958,1cyclictest19507-21kworker/1:0+events19:21:161
656899958957,1cyclictest12038-21kworker/2:1+events23:50:442
656299943941,2cyclictest3199-21turbostat20:24:551
655699939936,2cyclictest12789-21kworker/0:1+events19:39:260
655699936935,1cyclictest26935-21kworker/0:2+events20:50:420
655699924924,0cyclictest12789-21kworker/0:1+events19:40:210
655699922921,1cyclictest24457-21kworker/0:1+events19:56:510
655699918917,1cyclictest26935-21kworker/0:2+events20:42:120
655699912911,1cyclictest17070-21kworker/0:0+events20:59:380
655699908905,3cyclictest5236-21turbostat20:29:550
655699906904,2cyclictest22406-21kworker/0:2+events19:50:200
656899857854,3cyclictest9019-21turbostat23:39:552
656899857854,3cyclictest9019-21turbostat23:39:552
655699853850,3cyclictest27256-21turbostat21:49:550
657299698694,4cyclictest27256-21turbostat21:49:553
655699696693,3cyclictest27998-21turbostat22:54:560
655699682679,3cyclictest10531-21turbostat19:24:560
655699673670,3cyclictest7257-21turbostat20:34:550
656299672670,2cyclictest10738-21turbostat23:09:551
657299670668,2cyclictest20801-21turbostat23:19:553
656899666662,4cyclictest11966-21turbostat00:14:552
655699656654,2cyclictest5170-21turbostat00:39:540
656899638635,3cyclictest6649-21turbostat21:29:552
657299599597,2cyclictest17352-21turbostat20:59:553
657299591588,3cyclictest24657-21turbostat19:59:553
656299582579,3cyclictest25055-21turbostat22:19:561
656299573569,4cyclictest16991-21turbostat00:19:551
657299549548,1cyclictest17156-21turbostat21:39:563
657299533532,1cyclictest2769-21turbostat22:29:553
656899518515,3cyclictest32519-21turbostat00:34:542
656299509507,2cyclictest22210-21turbostat21:44:551
656299491489,2cyclictest6933-21turbostat00:09:551
6562991919,0cyclictest67750irq/123-lan21:20:181
6562991918,1cyclictest67750irq/123-lan23:55:121
6562991818,0cyclictest67750irq/123-lan23:36:131
6562991818,0cyclictest67750irq/123-lan23:36:131
6562991818,0cyclictest67750irq/123-lan22:51:371
6562991818,0cyclictest67750irq/123-lan22:42:481
6562991818,0cyclictest67750irq/123-lan22:37:501
6562991818,0cyclictest67750irq/123-lan22:33:231
6562991818,0cyclictest67750irq/123-lan22:11:491
6562991818,0cyclictest67750irq/123-lan21:50:101
6562991818,0cyclictest67750irq/123-lan00:02:381
6562991817,1cyclictest67750irq/123-lan23:50:181
6562991817,1cyclictest67750irq/123-lan23:30:171
6562991817,1cyclictest67750irq/123-lan23:00:181
6562991817,1cyclictest67750irq/123-lan22:08:011
6562991817,1cyclictest67750irq/123-lan21:58:491
6562991817,1cyclictest67750irq/123-lan21:30:161
6562991817,1cyclictest67750irq/123-lan21:25:181
6562991817,1cyclictest67750irq/123-lan21:10:131
6562991817,1cyclictest67750irq/123-lan21:09:321
6562991817,1cyclictest67750irq/123-lan00:35:201
6562991717,0cyclictest67750irq/123-lan23:25:261
6562991717,0cyclictest67750irq/123-lan23:23:551
6562991717,0cyclictest67750irq/123-lan22:00:131
6562991717,0cyclictest67750irq/123-lan21:18:541
6562991716,1cyclictest67750irq/123-lan23:48:521
6562991716,1cyclictest67750irq/123-lan22:55:171
6562991716,1cyclictest67750irq/123-lan22:45:171
6562991716,1cyclictest67750irq/123-lan21:35:131
6562991716,1cyclictest67750irq/123-lan20:50:211
6562991616,0cyclictest67750irq/123-lan23:40:111
6562991616,0cyclictest67750irq/123-lan21:00:211
6562991616,0cyclictest67750irq/123-lan20:55:171
6562991616,0cyclictest67750irq/123-lan19:30:221
6562991615,1cyclictest67750irq/123-lan20:45:141
6562991615,1cyclictest67750irq/123-lan20:40:181
6562991615,1cyclictest67750irq/123-lan20:15:131
6562991615,1cyclictest67750irq/123-lan20:00:171
6562991514,1cyclictest67750irq/123-lan19:45:131
6562991514,1cyclictest67750irq/123-lan19:40:171
6562991514,1cyclictest67750irq/123-lan19:25:181
6562991414,0cyclictest67750irq/123-lan20:25:191
6562991413,1cyclictest67750irq/123-lan19:35:171
6562991313,0cyclictest67750irq/123-lan20:13:351
6562991312,1cyclictest67750irq/123-lan20:05:161
6562991111,0cyclictest67750irq/123-lan20:30:211
656299109,1cyclictest67750irq/123-lan19:54:351
65729998,1cyclictest784-21snmpd00:15:343
65689997,2cyclictest762-21ntpd23:08:172
65629998,1cyclictest67750irq/123-lan20:35:201
65569998,1cyclictest784-21snmpd22:07:080
65729988,0cyclictest784-21snmpd23:56:513
65729988,0cyclictest784-21snmpd20:38:263
65729988,0cyclictest507-21systemd-resolve00:27:013
65729987,1cyclictest782-21lldpd19:41:183
65729986,2cyclictest784-21snmpd23:40:543
65689988,0cyclictest784-21snmpd23:17:222
65689988,0cyclictest784-21snmpd21:11:182
65689988,0cyclictest784-21snmpd19:44:372
65689988,0cyclictest782-21lldpd22:28:162
65689988,0cyclictest782-21lldpd22:11:482
65689988,0cyclictest782-21lldpd21:53:502
65689988,0cyclictest782-21lldpd19:46:502
65689987,1cyclictest784-21snmpd23:25:182
65689987,1cyclictest784-21snmpd20:19:262
65689987,1cyclictest572-21in:imuxsock23:32:522
65689987,1cyclictest507-21systemd-resolve23:01:102
65689986,2cyclictest784-21snmpd22:41:152
65689986,2cyclictest784-21snmpd21:24:582
65689986,2cyclictest784-21snmpd19:22:482
65689986,2cyclictest784-21snmpd00:20:242
65689986,2cyclictest762-21ntpd22:00:032
65569988,0cyclictest784-21snmpd23:59:030
65569988,0cyclictest784-21snmpd21:14:330
65569988,0cyclictest782-21lldpd21:03:350
65569988,0cyclictest782-21lldpd19:16:030
65569987,1cyclictest784-21snmpd22:11:080
65569987,1cyclictest784-21snmpd20:01:040
65569987,1cyclictest526-21avahi-daemon22:01:330
65569986,2cyclictest784-21snmpd00:25:570
65569985,3cyclictest784-21snmpd00:12:150
65729977,0cyclictest784-21snmpd21:34:583
65729977,0cyclictest782-21lldpd20:45:573
65729977,0cyclictest782-21lldpd20:44:473
65729976,1cyclictest784-21snmpd19:36:213
65729976,1cyclictest526-21avahi-daemon00:31:513
65729976,1cyclictest2542-21fschecks_time00:00:153
65729976,0cyclictest191rcu_preempt23:53:593
65729976,0cyclictest191rcu_preempt20:55:003
65729976,0cyclictest191rcu_preempt19:32:343
65729975,2cyclictest784-21snmpd19:26:543
65729974,2cyclictest0-21swapper/300:21:033
65729974,1cyclictest191rcu_preempt23:00:193
65729973,2cyclictest52-21ksoftirqd/323:10:163
65689977,0cyclictest784-21snmpd20:40:142
65689977,0cyclictest784-21snmpd19:35:272
65689977,0cyclictest782-21lldpd23:22:312
65689977,0cyclictest782-21lldpd20:45:352
65689977,0cyclictest782-21lldpd20:39:012
65689977,0cyclictest782-21lldpd19:28:142
65689976,1cyclictest784-21snmpd22:55:072
65689976,1cyclictest782-21lldpd21:08:452
65689976,1cyclictest526-21avahi-daemon21:45:452
65689975,2cyclictest784-21snmpd21:04:242
65689975,2cyclictest762-21ntpd20:20:192
65689974,3cyclictest762-21ntpd22:24:592
65689974,3cyclictest547-21NetworkManager00:26:372
65689973,3cyclictest24261-21ntp_states22:45:182
65569977,0cyclictest784-21snmpd23:20:010
65569977,0cyclictest784-21snmpd21:50:350
65569977,0cyclictest782-21lldpd23:52:510
65569977,0cyclictest782-21lldpd23:43:050
65569977,0cyclictest782-21lldpd23:28:190
65569977,0cyclictest782-21lldpd23:12:150
65569976,1cyclictest784-21snmpd00:08:510
65569976,1cyclictest526-21avahi-daemon23:31:560
65569976,1cyclictest507-21systemd-resolve22:36:010
65569976,1cyclictest171ktimers/021:15:300
65569973,3cyclictest2874-21ntp_states21:20:180
65569973,3cyclictest22899-21ssh22:44:560
65729966,0cyclictest782-21lldpd23:47:093
65729966,0cyclictest782-21lldpd19:11:563
65729965,0cyclictest191rcu_preempt00:10:453
65729964,0cyclictest191rcu_preempt22:55:213
65729964,0cyclictest191rcu_preempt22:24:113
65729961,3cyclictest24640-21chrt19:54:583
65689966,0cyclictest784-21snmpd23:13:372
65689966,0cyclictest784-21snmpd19:30:152
65689965,1cyclictest784-21snmpd20:05:252
65689965,1cyclictest526-21avahi-daemon22:36:132
65689965,1cyclictest28838-21/usr/sbin/munin21:15:192
65689964,2cyclictest784-21snmpd19:59:072
65689964,2cyclictest245-20systemd-journal22:52:542
65569966,0cyclictest784-21snmpd22:16:240
65569966,0cyclictest784-21snmpd21:56:500
65569966,0cyclictest784-21snmpd19:10:450
65569965,1cyclictest526-21avahi-daemon00:01:520
65569965,1cyclictest171ktimers/021:37:210
65569965,1cyclictest171ktimers/021:27:300
65569963,3cyclictest547-21NetworkManager22:47:420
65569963,2cyclictest8154-21ssh22:30:080
65569962,3cyclictest15941-21ssh21:33:160
65569960,4cyclictest26994-21sh00:24:180
65569960,3cyclictest5410-21cat20:25:000
65729955,0cyclictest191rcu_preempt19:45:103
65689954,1cyclictest784-21snmpd22:30:312
65689953,2cyclictest762-21ntpd22:05:592
65689953,2cyclictest762-21ntpd20:25:012
65689952,3cyclictest762-21ntpd20:32:032
65689952,2cyclictest5426-21tr21:55:112
65689952,2cyclictest28175-21ld20:00:172
65689952,2cyclictest10003-21sed19:15:222
65689950,5cyclictest547-21NetworkManager20:54:022
65569954,1cyclictest526-21avahi-daemon21:40:590
65569954,1cyclictest10284-21ntp_offset23:35:180
65569954,1cyclictest10284-21ntp_offset23:35:180
65569952,3cyclictest4067-21ssh00:33:250
65569952,2cyclictest245-20systemd-journal22:20:090
65569952,2cyclictest22425-21ld21:05:160
65569952,2cyclictest22112-21ssh23:47:000
65689940,3cyclictest8503-21ls19:15:012
65689940,3cyclictest23610-21ntpq19:50:162
65689940,3cyclictest22549-21ssh21:40:112
65569943,1cyclictest171ktimers/020:10:140
65569941,2cyclictest1444-21apt-get20:15:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional