You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 23:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat Jul 04, 2026 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
231572178104,28sleep10-21swapper/107:08:231
232772175119,48sleep00-21swapper/007:09:550
231082161103,30sleep20-21swapper/207:07:442
230252161104,27sleep30-21swapper/307:06:413
273432670,2sleep20-21swapper/211:48:252
324252620,2sleep20-21swapper/211:27:302
313062610,2sleep10-21swapper/108:40:231
168922520,8sleep228-21ksoftirqd/210:27:072
297602510,1sleep30-21swapper/310:36:263
115132510,1sleep10-21swapper/111:35:451
20372480,1sleep00-21swapper/011:05:130
206052470,1sleep20-21swapper/208:15:252
278532460,2sleep10-21swapper/111:24:361
116022430,1sleep00-21swapper/011:11:360
241052420,1sleep20-21swapper/209:44:292
21972420,1sleep22198-21rm12:18:062
203192390,1sleep20-21swapper/211:18:432
2356199351,14cyclictest27824-21ssh11:00:122
23562993212,10cyclictest34-21ksoftirqd/310:58:103
2356199321,23cyclictest0-21swapper/207:30:052
208162320,1sleep00-21swapper/009:16:570
23562993014,7cyclictest29739-21ntp_states11:25:213
2356299300,2cyclictest141rcu_preempt07:49:153
316652290,2sleep10-21swapper/110:14:261
23560992920,6cyclictest24371-21/usr/sbin/munin12:35:211
2355999291,26cyclictest0-21swapper/008:05:040
260182280,2chrt0-21swapper/010:10:110
2356299289,4cyclictest34-21ksoftirqd/309:35:303
2356099281,25cyclictest0-21swapper/107:49:351
2356099281,25cyclictest0-21swapper/107:37:481
2356099281,25cyclictest0-21swapper/107:30:121
2356099281,25cyclictest0-21swapper/107:23:411
2355999281,25cyclictest0-21swapper/007:58:500
2355999281,25cyclictest0-21swapper/007:37:390
2355999281,25cyclictest0-21swapper/007:30:350
11592280,3chrt0-21swapper/308:45:233
23562992722,3cyclictest6274-21munin-run07:45:003
23562992713,4cyclictest34-21ksoftirqd/312:04:503
2356299271,24cyclictest0-21swapper/308:14:093
23562992711,5cyclictest34-21ksoftirqd/312:19:223
23562992711,4cyclictest34-21ksoftirqd/309:53:283
23562992710,4cyclictest34-21ksoftirqd/311:17:473
23562992710,4cyclictest34-21ksoftirqd/309:11:053
2356099271,24cyclictest0-21swapper/108:15:581
2356099271,24cyclictest0-21swapper/108:13:451
2356099271,24cyclictest0-21swapper/108:08:251
2356099271,24cyclictest0-21swapper/107:57:251
2356099271,24cyclictest0-21swapper/107:28:511
2355999271,2cyclictest0-21swapper/008:00:420
2355999271,24cyclictest0-21swapper/008:28:030
2355999271,24cyclictest0-21swapper/008:13:360
2355999271,24cyclictest0-21swapper/007:50:480
2355999271,24cyclictest0-21swapper/007:40:300
244582260,2sleep10-21swapper/108:25:181
2356299269,5cyclictest34-21ksoftirqd/312:09:583
2356299269,4cyclictest34-21ksoftirqd/309:16:053
2356299269,2cyclictest141rcu_preempt10:00:203
2356299268,4cyclictest34-21ksoftirqd/312:39:583
2356299268,4cyclictest34-21ksoftirqd/311:14:373
2356299267,4cyclictest34-21ksoftirqd/309:44:453
2356299264,12cyclictest34-21ksoftirqd/311:22:193
23562992622,2cyclictest0-21swapper/307:50:103
23562992619,5cyclictest34-21ksoftirqd/308:59:593
23562992617,7cyclictest34-21ksoftirqd/311:40:013
23562992612,4cyclictest34-21ksoftirqd/310:05:343
2356299261,24cyclictest0-21swapper/307:38:473
2356299261,23cyclictest0-21swapper/308:09:133
2356299261,23cyclictest0-21swapper/307:56:083
2356299261,23cyclictest0-21swapper/307:31:313
2356299261,23cyclictest0-21swapper/307:24:053
23562992610,4cyclictest34-21ksoftirqd/310:17:063
2356199261,24cyclictest0-21swapper/207:37:082
2356199261,23cyclictest0-21swapper/207:51:232
23560992624,1cyclictest0-21swapper/108:04:501
23560992624,1cyclictest0-21swapper/107:41:031
2356099261,1cyclictest0-21swapper/108:51:181
2355999263,17cyclictest30257-21ssh12:15:120
23559992624,1cyclictest0-21swapper/007:27:550
2355999261,24cyclictest0-21swapper/008:15:540
2355999261,1cyclictest0-21swapper/007:45:060
95202250,2sleep30-21swapper/309:33:103
2356299258,5cyclictest34-21ksoftirqd/311:40:043
2356299254,11cyclictest34-21ksoftirqd/310:53:013
23562992515,7cyclictest23999-21df_inode08:25:133
2356299251,23cyclictest0-21swapper/308:00:313
2356299251,13cyclictest34-21ksoftirqd/310:30:043
2356299250,23cyclictest0-21swapper/308:15:063
2356299250,23cyclictest0-21swapper/307:25:153
23561992523,1cyclictest0-21swapper/207:40:092
2356199252,16cyclictest19381-21ssh10:05:132
2356199251,23cyclictest0-21swapper/208:00:462
2356199251,23cyclictest0-21swapper/207:46:502
2356199251,23cyclictest0-21swapper/207:28:422
2356199251,18cyclictest0-21swapper/207:59:382
23560992522,2cyclictest0-21swapper/107:50:571
2356099250,10cyclictest130422ssh10:48:541
2355999251,19cyclictest17789-21ssh12:30:120
214392250,2chrt0-21swapper/109:17:361
2356299247,4cyclictest34-21ksoftirqd/309:45:073
2356299246,4cyclictest34-21ksoftirqd/311:50:253
23562992410,4cyclictest34-21ksoftirqd/311:57:503
23561992422,1cyclictest0-21swapper/208:20:522
23561992422,1cyclictest0-21swapper/208:13:162
23561992422,1cyclictest0-21swapper/208:06:582
2356099242,15cyclictest8616-21ssh11:10:031
2356099241,9cyclictest20392ssh11:05:141
230082240,1sleep00-21swapper/012:09:470
184932240,2sleep00-21swapper/010:04:330
302922230,1sleep30-21swapper/308:40:143
2356299238,4cyclictest34-21ksoftirqd/312:13:553
2356299236,4cyclictest34-21ksoftirqd/311:46:203
2356299236,4cyclictest34-21ksoftirqd/309:58:353
23562992311,4cyclictest34-21ksoftirqd/312:28:413
23562992311,4cyclictest34-21ksoftirqd/311:06:063
2356299231,13cyclictest34-21ksoftirqd/309:20:553
23562992310,2cyclictest141rcu_preempt10:40:153
2356199232,15cyclictest1339-21ssh10:40:132
23560992312,7cyclictest22809-21grep11:45:141
23559992310,3cyclictest141rcu_preempt12:25:180
71262220,2sleep00-21swapper/011:57:060
57052220,1sleep00-21swapper/012:20:210
2356299228,4cyclictest34-21ksoftirqd/312:22:403
2356299224,3cyclictest141rcu_preempt10:10:233
23562992215,6cyclictest34-21ksoftirqd/309:05:003
23562992215,5cyclictest34-21ksoftirqd/311:35:003
23562992213,7cyclictest7940-21awk10:20:203
23562992211,4cyclictest34-21ksoftirqd/310:25:433
2356099222,14cyclictest31663-21diskmemload09:50:101
23560992211,7cyclictest3993-21expr11:55:151
2355999222,16cyclictest729-21kworker/u8:209:35:120
23559992220,1cyclictest0-21swapper/007:23:430
23559992214,5cyclictest30479-21sh09:25:010
23559992213,6cyclictest22197-21cpuspeed_turbos11:20:150
178432220,2sleep00-21swapper/010:52:050
2356299217,4cyclictest34-21ksoftirqd/312:33:363
2356299217,4cyclictest34-21ksoftirqd/311:00:453
2356299216,8cyclictest34-21ksoftirqd/309:27:403
2356299212,10cyclictest34-21ksoftirqd/310:48:563
23562992119,1cyclictest0-21swapper/308:35:493
23562992118,2cyclictest0-21swapper/308:51:213
23561992113,6cyclictest6147-21sh11:56:062
23561992113,5cyclictest9048-21pmu-power10:45:232
23561992112,7cyclictest5409-21cat12:20:192
2356099219,8cyclictest16152-21apt-get11:40:121
2356099212,13cyclictest5893-21apt-get09:55:121
23559992119,1cyclictest0-21swapper/008:51:460
23559992114,4cyclictest636-21runrttasks11:51:250
23559992114,4cyclictest27565-21sh11:48:380
23559992113,6cyclictest18331-21sh10:28:360
23559992113,6cyclictest16895-21seq09:14:460
23559992113,6cyclictest10272-21seq09:07:180
23559992112,7cyclictest18782-21entropy09:40:140
23559992112,6cyclictest793-21memory08:45:190
23559992111,7cyclictest21847-21df08:20:160
2355999211,16cyclictest53050irq/142-enp3s0-12:35:250
210512210,4chrt21054-21head10:05:260
23562992015,4cyclictest34-21ksoftirqd/309:10:003
23562992014,5cyclictest34-21ksoftirqd/308:35:013
2356199208,10cyclictest0-21swapper/212:27:522
2356199202,13cyclictest8784-21taskset11:10:132
2356199202,13cyclictest75-21kswapd010:15:132
23561992012,6cyclictest26822-21sed10:10:172
23561992012,6cyclictest20437-21cat10:30:122
23561992012,6cyclictest13760-21ntp_states10:00:182
23561992010,9cyclictest12960-21ssh12:01:172
2356199200,9cyclictest274442entropy10:35:172
2356099209,8cyclictest18372-21sed12:30:161
2356099209,7cyclictest52850irq/140-enp3s0-09:10:241
2356099202,13cyclictest30922-21ssh09:25:131
23560992014,3cyclictest13096-21turbostat.cron10:25:011
23560992012,4cyclictest0-21swapper/110:35:011
23560992010,7cyclictest956-21ntp_states08:45:221
2355999209,9cyclictest16355-21cstates11:40:140
2355999209,6cyclictest53050irq/142-enp3s0-09:25:120
23559992013,5cyclictest4252-21seq09:53:450
23559992012,6cyclictest28510-21seq12:13:390
23559992012,6cyclictest27054-21apt10:35:130
23559992012,6cyclictest11292-21seq11:35:300
23559992011,7cyclictest3542-21cat11:30:180
23559992011,7cyclictest28860-21ntp_states08:35:200
23559992011,6cyclictest25573-21fw_packets09:45:150
97012190,2sleep20-21swapper/209:33:222
278872190,4sleep0131rcuc/008:35:010
2356199197,8cyclictest52850irq/140-enp3s0-09:10:132
2356199192,12cyclictest2015-21apt-get11:05:132
23561991917,1cyclictest0-21swapper/208:28:042
23561991917,1cyclictest0-21swapper/207:23:472
23561991911,6cyclictest6622-21expr09:00:132
23561991911,6cyclictest4388-21cat11:30:252
23561991911,6cyclictest24488-21wc12:35:132
23561991911,6cyclictest17866-21sed12:05:172
2356199191,13cyclictest9500-21ssh11:35:122
23561991910,6cyclictest6417-21gltestperf09:55:162
23561991910,6cyclictest1357-21cat08:45:242
2356199190,7cyclictest193582sed12:30:242
2356199190,4cyclictest0-21swapper/208:32:512
2356099199,6cyclictest52850irq/140-enp3s0-12:25:121
2356099192,11cyclictest18382-21apt-get09:40:111
2356099191,5cyclictest10501-21diskmemload09:33:581
23560991914,3cyclictest28360-21grep11:00:161
23560991913,4cyclictest31548-21sh11:50:421
23560991913,4cyclictest15493-21tr11:15:141
23560991912,5cyclictest14414-21load10:25:211
23560991912,4cyclictest10501-21diskmemload10:00:181
23560991911,6cyclictest24782-21ls07:10:241
23560991911,6cyclictest1819-21grep10:15:251
23560991911,5cyclictest20380-21awk10:05:191
23559991914,3cyclictest31885-21sh11:26:570
23559991914,3cyclictest26330-21sh10:58:570
23559991914,3cyclictest23353-21sh10:31:420
23559991913,4cyclictest8005-21sh09:55:500
23559991913,4cyclictest53050irq/142-enp3s0-10:15:150
23559991913,4cyclictest5207-21sh10:42:400
23559991913,4cyclictest11876-21sh10:23:460
23559991912,5cyclictest5339-21expr09:30:140
23559991912,5cyclictest10316-21seq12:00:060
23559991911,6cyclictest6866-21cut09:00:160
23559991911,6cyclictest16318-21ls11:15:200
23559991910,7cyclictest8377-21tune2fs10:45:150
23559991910,7cyclictest6299-21cat09:00:000
23559991910,6cyclictest24295-21latency07:10:190
2356299187,4cyclictest34-21ksoftirqd/308:21:113
23562991811,5cyclictest24843-21grep07:10:253
2356199189,7cyclictest52950irq/141-enp3s0-09:25:242
2356199188,7cyclictest23636-21seq11:20:272
23561991813,3cyclictest29607-21cpu11:50:132
23561991813,3cyclictest28210-21sh09:22:432
23561991811,5cyclictest2824-21awk08:50:172
23561991811,5cyclictest28-21ksoftirqd/209:36:322
23561991811,5cyclictest26508-21wc07:15:192
23561991811,4cyclictest23626-21/usr/sbin/munin12:10:142
23561991810,6cyclictest9172-21smartctl09:05:162
23561991810,6cyclictest6852-21cat10:20:102
23561991810,6cyclictest4664-21forks08:55:152
23561991810,6cyclictest32672-21ntpq09:50:202
23561991810,6cyclictest17575-21ls11:40:252
2356199180,16cyclictest22704-21cat10:55:262
2356099186,9cyclictest0-21swapper/109:45:201
2356099182,5cyclictest11732-21apt-get09:35:121
2356099182,12cyclictest27058-21ssh10:35:121
2356099181,8cyclictest24406-21irqstats12:10:181
23560991813,3cyclictest31144-21sh11:26:111
23560991811,5cyclictest27689-21cron08:35:001
2356099181,15cyclictest2509-21ssh10:40:241
23560991811,4cyclictest636-21runrttasks12:00:561
23560991811,4cyclictest636-21runrttasks10:55:241
2356099181,13cyclictest31819-21ssh12:15:241
23560991810,6cyclictest8459-21wc09:05:001
23560991810,6cyclictest29821-21cron08:40:011
2355999189,5cyclictest53050irq/142-enp3s0-11:00:250
23562991710,5cyclictest25979-21expr07:15:153
2356199179,6cyclictest560-21dbus-daemon08:40:012
2356199172,11cyclictest17548-21apt-get09:15:112
23561991712,3cyclictest25990-21needreboot09:45:192
23561991711,3cyclictest10501-21diskmemload10:50:012
2356099179,6cyclictest24344-21cat09:20:121
2356099179,6cyclictest17967-21sed12:05:191
2356099173,8cyclictest2895-21apt-get11:30:121
23560991712,3cyclictest5820-21smartctl12:20:241
23560991712,3cyclictest14501-21/usr/sbin/munin10:50:251
23560991711,4cyclictest26193-21cut07:15:161
2355999179,6cyclictest26102-21uniq07:15:160
2355999178,7cyclictest31764-21seq08:42:160
227342170,2sleep10-21swapper/108:20:251
2356199162,9cyclictest52750irq/139-enp3s0-07:10:142
23561991611,3cyclictest30464-21fschecks_count08:40:152
23560991610,4cyclictest4884-21hwlatdetect08:55:171
2356099157,6cyclictest9104-21expr09:05:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional