You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 02:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sat Aug 29, 2026 12:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
99942180106,28sleep10-21swapper/107:07:491
100252167111,30sleep00-21swapper/007:08:130
98992163108,28sleep30-21swapper/307:06:353
100422160103,28sleep20-21swapper/207:08:282
2619821460,1sleep20-21swapper/212:19:062
124762720,3sleep0131rcuc/011:45:000
307672680,3sleep334-21ksoftirqd/311:34:283
140362660,8sleep334-21ksoftirqd/312:10:073
279732580,2sleep30-21swapper/310:44:123
296172570,2sleep2271rcuc/209:10:012
296172570,2sleep2271rcuc/209:10:002
151812550,4sleep215183-21latency_hist10:35:012
26992520,2sleep00-21swapper/010:25:180
23872520,1sleep10-21swapper/112:00:341
118412520,6sleep0141rcu_preempt09:44:260
191662500,2sleep20-21swapper/211:00:522
147022500,2sleep10-21swapper/111:45:381
324342490,1sleep10-21swapper/109:59:521
184522470,4sleep1211rcuc/110:13:041
22112460,1sleep32178-21timerwakeupswit10:00:253
137172420,2chrt0-21swapper/311:45:193
290942370,1sleep30-21swapper/307:50:243
1043999336,10cyclictest34-21ksoftirqd/311:09:163
138002320,3sleep1211rcuc/109:45:191
19572310,4sleep1221ksoftirqd/110:00:231
1043999308,9cyclictest34-21ksoftirqd/312:24:223
10439993025,3cyclictest331rcuc/310:24:313
179642290,1sleep20-21swapper/211:48:572
1043999291,4cyclictest34-21ksoftirqd/311:40:013
58662280,1sleep20-21swapper/212:27:452
244022280,2sleep10-21swapper/111:29:491
1043999288,10cyclictest34-21ksoftirqd/309:36:223
1043999287,9cyclictest34-21ksoftirqd/311:02:143
1043799281,25cyclictest0-21swapper/107:25:371
1043699281,25cyclictest0-21swapper/008:39:390
1043699281,25cyclictest0-21swapper/007:28:300
296022270,2sleep10-21swapper/109:33:271
226072270,10chrt34-21ksoftirqd/312:15:303
1043999279,10cyclictest52850irq/140-enp3s0-12:30:133
1043999278,9cyclictest34-21ksoftirqd/311:43:533
1043999278,8cyclictest34-21ksoftirqd/310:34:193
1043999277,9cyclictest34-21ksoftirqd/312:08:033
1043999277,9cyclictest34-21ksoftirqd/311:50:053
1043999276,9cyclictest34-21ksoftirqd/310:54:113
1043999276,8cyclictest34-21ksoftirqd/311:17:013
1043999274,7cyclictest34-21ksoftirqd/309:55:263
1043999274,11cyclictest34-21ksoftirqd/311:55:573
10439992720,5cyclictest34-21ksoftirqd/309:25:003
10439992712,4cyclictest34-21ksoftirqd/312:29:183
1043899271,24cyclictest0-21swapper/207:34:362
1043899271,24cyclictest0-21swapper/207:26:162
1043799271,24cyclictest0-21swapper/108:00:031
1043799271,24cyclictest0-21swapper/107:50:041
1043799271,24cyclictest0-21swapper/107:33:211
1043699271,24cyclictest0-21swapper/007:46:320
1043699271,24cyclictest0-21swapper/007:37:110
1043699271,24cyclictest0-21swapper/007:30:340
1043699271,24cyclictest0-21swapper/007:20:450
1043999268,8cyclictest34-21ksoftirqd/309:52:433
1043999268,8cyclictest34-21ksoftirqd/309:52:423
1043999267,9cyclictest34-21ksoftirqd/310:39:163
1043999266,9cyclictest34-21ksoftirqd/310:28:573
1043999266,10cyclictest34-21ksoftirqd/311:29:153
1043999265,9cyclictest34-21ksoftirqd/310:18:183
1043999265,6cyclictest34-21ksoftirqd/312:35:213
1043999264,10cyclictest34-21ksoftirqd/310:49:333
1043999261,23cyclictest0-21swapper/308:39:463
1043999261,23cyclictest0-21swapper/308:33:493
1043999261,23cyclictest0-21swapper/308:16:213
1043999261,23cyclictest0-21swapper/307:33:183
1043999261,23cyclictest0-21swapper/307:20:273
1043899261,23cyclictest0-21swapper/208:41:342
1043899261,23cyclictest0-21swapper/208:38:102
1043899261,23cyclictest0-21swapper/208:34:532
1043899261,23cyclictest0-21swapper/207:59:452
10437992624,1cyclictest0-21swapper/108:39:181
10437992623,2cyclictest0-21swapper/107:47:281
1043799261,24cyclictest0-21swapper/107:37:431
1043799261,24cyclictest0-21swapper/107:24:301
1043699261,1cyclictest0-21swapper/008:14:440
1043699261,1cyclictest0-21swapper/007:54:520
1043699261,1cyclictest0-21swapper/007:44:090
1043699261,12cyclictest7412threads11:35:250
201262250,2sleep10-21swapper/109:50:171
201262250,2sleep10-21swapper/109:50:161
1043999258,9cyclictest34-21ksoftirqd/309:16:573
1043999258,8cyclictest34-21ksoftirqd/309:32:023
1043999257,8cyclictest34-21ksoftirqd/311:21:293
1043999257,8cyclictest34-21ksoftirqd/311:21:293
1043999257,8cyclictest34-21ksoftirqd/311:11:063
1043999256,9cyclictest34-21ksoftirqd/309:47:203
1043999256,8cyclictest34-21ksoftirqd/312:01:223
1043999255,8cyclictest34-21ksoftirqd/310:08:003
10439992513,4cyclictest34-21ksoftirqd/310:11:063
1043999251,23cyclictest0-21swapper/307:27:413
1043999251,1cyclictest0-21swapper/308:10:103
1043999250,23cyclictest0-21swapper/307:37:503
1043899251,1cyclictest0-21swapper/207:48:522
1043899250,23cyclictest0-21swapper/207:37:462
1043899250,23cyclictest0-21swapper/207:24:362
10437992523,1cyclictest0-21swapper/108:33:491
10436992523,1cyclictest0-21swapper/007:59:420
10436992514,7cyclictest636-21runrttasks07:15:120
86622240,2sleep00-21swapper/010:53:520
1043999245,9cyclictest34-21ksoftirqd/310:55:493
10439992422,1cyclictest0-21swapper/307:47:373
1043999241,7cyclictest24436-21munin-run09:30:013
10439992417,4cyclictest34-21ksoftirqd/307:45:013
1043999241,6cyclictest34-21ksoftirqd/307:55:173
1043999240,6cyclictest34-21ksoftirqd/309:40:223
1043999240,1cyclictest0-21swapper/308:00:073
10438992422,1cyclictest0-21swapper/208:00:092
1043899241,22cyclictest0-21swapper/207:54:292
1043899241,1cyclictest0-21swapper/208:27:422
1043899241,19cyclictest1042-21fschecks_count10:00:162
10437992422,1cyclictest0-21swapper/107:44:381
10436992422,1cyclictest0-21swapper/008:34:150
1043999236,9cyclictest34-21ksoftirqd/309:13:053
10439992321,1cyclictest0-21swapper/308:40:423
10439992321,1cyclictest0-21swapper/308:26:143
10439992317,4cyclictest34-21ksoftirqd/308:50:003
10439992312,1cyclictest141rcu_preempt08:08:283
10438992321,1cyclictest0-21swapper/208:14:342
10438992314,7cyclictest5845-21ssh12:04:042
10438992314,6cyclictest15917-21sh10:59:232
1043899230,6cyclictest17867-21cmp09:25:012
10437992321,1cyclictest0-21swapper/108:12:031
1043699239,3cyclictest12-21ksoftirqd/011:00:130
1043699233,14cyclictest18347-21ssh11:25:120
260292220,3sleep012-21ksoftirqd/011:55:040
10439992218,3cyclictest331rcuc/307:10:063
10439992218,2cyclictest331rcuc/308:53:593
10439992215,5cyclictest34-21ksoftirqd/309:10:013
10439992215,5cyclictest34-21ksoftirqd/309:10:003
1043999220,5cyclictest0-21swapper/308:55:223
1043899221,4cyclictest0-21swapper/209:57:302
10438992212,8cyclictest32198-21tune2fs11:35:162
1043899220,9cyclictest123282latency_hist11:45:002
10437992213,7cyclictest560-21dbus-daemon12:40:001
10437992213,7cyclictest17756-21ntpq08:40:231
10437992212,8cyclictest26905-21users09:00:261
10437992212,7cyclictest7960-21sendmail-msp08:20:001
1043699221,9cyclictest151992ssh11:46:080
10436992214,6cyclictest12521-21ssh10:55:590
10436992213,7cyclictest2253-21df12:25:160
10436992213,6cyclictest8725-21python310:05:250
256012210,2sleep3331rcuc/309:00:143
10439992117,3cyclictest331rcuc/307:15:083
1043999211,4cyclictest34-21ksoftirqd/308:25:013
10438992119,1cyclictest0-21swapper/207:44:202
10438992114,5cyclictest19354-21ntp_kernel_pll_11:25:192
10438992112,6cyclictest25131-21cat11:30:142
10437992113,6cyclictest6446-21ls11:15:281
10437992113,6cyclictest13531-21ntpq07:15:341
10437992113,6cyclictest11581-21ls08:25:261
1043699212,14cyclictest22513-21ssh10:40:130
1043699212,13cyclictest17956-21ssh09:25:040
1043699212,12cyclictest636-21runrttasks09:10:130
10436992119,1cyclictest0-21swapper/008:00:340
1043699211,17cyclictest13222-21ssh11:20:250
1043699211,17cyclictest13222-21ssh11:20:240
1043899209,3cyclictest141rcu_preempt10:40:002
10438992013,4cyclictest15673-21sh09:46:302
10438992012,6cyclictest6622-21fw_packets09:40:142
10438992012,6cyclictest28344-21irqstats10:20:192
10438992011,6cyclictest6021-21cat08:15:112
10438992010,7cyclictest30456-21diskstats09:10:152
10438992010,7cyclictest21231-21entropy10:15:172
1043799209,7cyclictest31421-21sort11:35:011
1043799203,11cyclictest30060-21apt-get09:10:121
10437992018,1cyclictest0-21swapper/108:09:471
10437992018,1cyclictest0-21swapper/107:59:521
10437992013,4cyclictest32504-21kernelversion11:35:181
10437992013,4cyclictest29084-21sh10:45:051
10437992013,4cyclictest22767-21sh11:04:311
10437992013,4cyclictest14636-21sh12:10:161
10437992013,4cyclictest10514-21sh12:06:481
10437992012,6cyclictest28450-21ls10:20:201
10437992012,6cyclictest11107-21seq09:20:091
10437992011,6cyclictest19546-21cat11:50:121
1043699202,13cyclictest15839-21ssh10:35:150
1043699202,12cyclictest23742-21sh11:05:030
10436992014,4cyclictest27361-21sh12:20:020
10436992013,5cyclictest12-21ksoftirqd/009:35:000
10436992013,5cyclictest12-21ksoftirqd/009:20:000
10436992013,4cyclictest18902-21sh12:37:210
10436992011,6cyclictest5566-21seq12:03:470
1043699201,16cyclictest17688-21ssh09:25:000
1043699201,11cyclictest0-21swapper/010:31:530
10436992010,7cyclictest10690-21cat08:25:160
261222190,10sleep228-21ksoftirqd/211:55:102
1043899199,7cyclictest1-21systemd12:10:072
1043899192,13cyclictest7261-21ssh12:05:032
1043899192,13cyclictest31177-21apt-get09:35:022
1043899192,12cyclictest3550-21ssh10:50:122
10438991913,4cyclictest26183-21sh10:42:242
10438991913,4cyclictest21291-21sh12:39:462
10438991913,4cyclictest10294-21sh09:19:362
10438991912,5cyclictest30264-21ntp_kernel_err10:45:222
10438991912,5cyclictest13711-21rt-features07:15:352
10438991912,4cyclictest11230-21latency_hist11:20:002
10438991911,6cyclictest8637-21grep08:20:152
10438991911,6cyclictest25363-21tune2fs09:30:142
10438991911,6cyclictest14045-21cat10:10:102
10438991911,5cyclictest26159-21seq09:00:202
1043899191,12cyclictest23747-21ssh11:05:032
1043899190,7cyclictest20852-21ssh09:50:242
1043899190,7cyclictest20852-21ssh09:50:232
1043799198,7cyclictest52950irq/141-enp3s0-10:25:121
1043799192,5cyclictest22-21ksoftirqd/110:40:131
1043799191,9cyclictest22252-21cat10:40:011
10437991913,4cyclictest9520-21sh10:54:451
10437991913,4cyclictest8020-21sh09:17:201
10437991913,4cyclictest692-21sendmail_mailtr09:35:231
10437991913,4cyclictest5472-21sh12:27:211
10437991913,4cyclictest25537-21tail11:05:251
10437991912,5cyclictest27627-21cut09:05:141
10437991912,5cyclictest27627-21cut09:05:131
10437991912,5cyclictest20598-21seq08:46:551
10437991912,5cyclictest10737-21seq11:43:251
10437991912,4cyclictest29425-21diskmemload12:21:531
10437991912,4cyclictest11621-21/usr/sbin/munin11:20:131
10437991912,4cyclictest11621-21/usr/sbin/munin11:20:121
10437991911,6cyclictest24137-21ntpq08:55:201
10437991911,6cyclictest23107-21cut08:55:011
10437991911,5cyclictest52950irq/141-enp3s0-09:40:141
10437991911,5cyclictest52950irq/141-enp3s0-09:25:121
10437991910,5cyclictest29425-21diskmemload11:55:201
1043699191,6cyclictest0-21swapper/009:45:190
1043699191,5cyclictest0-21swapper/011:31:310
1043699191,5cyclictest0-21swapper/010:18:360
10436991913,4cyclictest5296-21sh10:03:240
10436991912,5cyclictest32122-21ls09:35:150
10436991912,5cyclictest12128-21seq07:12:100
10436991911,6cyclictest6789-21seq11:15:360
10436991911,6cyclictest560-21dbus-daemon12:35:000
10436991911,6cyclictest22866-21seq08:53:380
10436991911,6cyclictest22232-21cat12:15:250
10436991911,6cyclictest15171-21uname12:10:200
10436991911,6cyclictest12-21ksoftirqd/010:10:190
10436991910,7cyclictest27142-21seq09:03:210
10436991910,6cyclictest6416-21uniq08:15:150
85542181,1sleep10-21swapper/112:30:131
306412180,1sleep00-21swapper/010:21:450
1043899186,7cyclictest52750irq/139-enp3s0-11:10:142
1043899186,3cyclictest0-21swapper/211:50:192
1043899181,8cyclictest10968-21ssh09:20:022
10438991812,4cyclictest29500-21sh12:20:382
10438991812,3cyclictest12952-21mailstats11:20:232
10438991812,3cyclictest12952-21mailstats11:20:222
10438991811,5cyclictest2150-21expr10:25:152
10438991810,6cyclictest8928-21entropy12:30:162
10438991810,6cyclictest7739-21expr10:05:152
1043799189,6cyclictest9355-21python308:20:231
1043799189,6cyclictest278-21systemd-journal07:14:591
1043799181,8cyclictest22753-21ssh10:15:371
1043799181,7cyclictest21450-21gltestperf12:15:161
10437991813,3cyclictest12068-21sh10:31:551
10437991812,4cyclictest12787-21sh10:56:141
10437991811,4cyclictest7178-21sh10:05:031
10436991813,3cyclictest9537-21sh12:05:500
10436991813,3cyclictest28109-21hwlatdetect09:05:170
10436991813,3cyclictest28109-21hwlatdetect09:05:170
10436991813,3cyclictest2116-21sh11:12:390
10436991812,4cyclictest29425-21diskmemload10:45:060
10436991811,5cyclictest1829-21cat08:05:140
10436991811,4cyclictest21492-21sh09:50:370
10436991811,4cyclictest21492-21sh09:50:370
10436991810,6cyclictest25011-21seq08:58:420
1043899179,6cyclictest24132-21ntp_kernel_pll_08:55:192
1043899179,6cyclictest19033-21cat08:45:142
10438991712,3cyclictest1734-21/usr/sbin/munin08:05:162
10438991710,5cyclictest21638-21expr08:50:172
10438991710,5cyclictest11385-21ntpq07:10:212
10437991710,5cyclictest31718-21open_inodes11:10:231
1043699179,6cyclictest27889-21cat09:55:250
1043699178,6cyclictest16921-21cpuspeed_turbos08:40:140
1043699176,4cyclictest21459-21ssh11:50:430
10436991711,4cyclictest19140-21cstates08:45:150
10436991710,5cyclictest9833-21seq08:22:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional