You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 11:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Jun 30, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308122175139,25sleep20-21swapper/219:09:302
30744215799,49sleep10-21swapper/119:08:371
30681214798,23sleep30-21swapper/319:07:463
306312136101,24sleep00-21swapper/019:07:070
254572710,5sleep20-21swapper/200:05:202
283152690,5sleep20-21swapper/200:31:212
90692650,2sleep334-21ksoftirqd/323:28:333
254532650,2sleep325449-21ssh23:40:213
268032610,2sleep0131rcuc/000:30:220
41232600,1sleep04126-21awk20:35:220
64722580,1sleep10-21swapper/121:49:551
222292560,2sleep30-21swapper/321:12:333
202552560,3sleep0131rcuc/022:48:170
161912560,2sleep30-21swapper/322:20:233
324512550,6sleep032448-21ssh21:20:160
177902550,1sleep00-21swapper/023:10:200
68832530,8sleep012-21ksoftirqd/021:50:060
252292530,2sleep20-21swapper/222:27:362
252292530,2sleep20-21swapper/222:27:362
216272520,2sleep20-21swapper/223:13:392
232392510,2sleep20-21swapper/200:03:402
151552510,1sleep20-21swapper/221:00:242
97322500,3sleep39730-21ssh22:40:173
51592500,4sleep00-21swapper/020:40:120
276232500,2sleep227624-21unixbench-2d21:40:252
276232500,2sleep227624-21unixbench-2d21:40:252
115322500,1sleep00-21swapper/020:55:000
115322500,1sleep00-21swapper/020:55:000
14902490,2sleep10-21swapper/100:35:251
31126994513,10cyclictest12311-21chrt21:54:001
197162450,2sleep10-21swapper/121:35:141
69432420,1chrt0-21swapper/200:15:202
31126994211,16cyclictest17974-21diskmemload23:53:111
119512420,1sleep10-21swapper/123:05:321
3112699418,13cyclictest15958-21rm23:58:001
127062410,1sleep00-21swapper/000:20:150
3112699409,12cyclictest22923-21tune2fs20:05:151
220162400,1sleep20-21swapper/221:36:242
31126993913,11cyclictest18077-21rm21:58:041
31126993910,9cyclictest17974-21diskmemload00:27:281
270892390,2chrt0-21swapper/022:29:310
270892390,2chrt0-21swapper/022:29:310
3112699388,9cyclictest13615-21ssh21:30:191
31126993810,11cyclictest31702-21ssh00:34:461
31126993810,10cyclictest16250-21ssh23:34:061
3112699379,11cyclictest17974-21diskmemload22:09:261
3112699379,10cyclictest11635-21cat19:40:011
31126993711,9cyclictest1827-21rm21:21:091
31126993711,10cyclictest32621-21sh22:57:091
31126993710,9cyclictest19671-21ssh00:00:211
31126993710,15cyclictest24821-21ssh22:03:071
3112699370,23cyclictest0-21swapper/122:20:111
3112699370,11cyclictest560-21dbus-daemon19:30:211
3112699369,9cyclictest17974-21diskmemload23:43:591
3112699369,15cyclictest648-21ntpd22:36:581
3112699367,11cyclictest22941-21ssh21:13:161
3112699367,10cyclictest9291-21ssh23:04:361
31126993611,9cyclictest0-21swapper/123:19:571
31126993611,10cyclictest648-21ntpd00:15:351
31126993610,8cyclictest17974-21diskmemload21:19:571
62142350,2chrt0-21swapper/023:25:370
3112699359,14cyclictest26280-21ssh22:52:281
3112699359,10cyclictest11723-21ssh22:41:241
31126993511,8cyclictest32209-21ssh21:44:541
31126993511,8cyclictest32209-21ssh21:44:541
31126993510,9cyclictest31858-21expr00:10:171
31126993510,8cyclictest0-21swapper/123:20:011
31126993510,8cyclictest0-21swapper/100:23:261
298872350,9sleep012-21ksoftirqd/021:18:320
273462350,2sleep00-21swapper/023:17:430
3112699349,15cyclictest8073-21ssh21:25:441
3112699348,14cyclictest4190-21ssh23:49:321
31126993410,9cyclictest3400-21ssh22:10:431
31126993410,8cyclictest21953-21ssh23:38:091
31126993410,8cyclictest14252-21ssh22:19:591
31126993311,6cyclictest19489-21ssh23:11:281
31126993311,10cyclictest10428-21chrt23:29:561
31126993310,8cyclictest28733-21ssh00:07:301
322082320,2sleep30-21swapper/323:20:513
3112699328,15cyclictest17574-21ssh22:45:341
3112699328,14cyclictest0-21swapper/120:45:201
3112699326,9cyclictest24609-21ssh22:26:591
3112699326,9cyclictest24609-21ssh22:26:591
3112699321,14cyclictest0-21swapper/122:30:121
31126993210,9cyclictest24713-21cpu20:10:121
3112799309,5cyclictest18730-21latency_hist00:25:012
31125993019,2cyclictest141rcu_preempt23:32:110
31125993014,5cyclictest12-21ksoftirqd/022:17:320
31126992816,9cyclictest20170-21sendmail20:00:001
3112599287,5cyclictest12-21ksoftirqd/021:35:400
31125992815,2cyclictest141rcu_preempt00:19:020
31125992812,5cyclictest12-21ksoftirqd/023:42:540
31125992812,4cyclictest12-21ksoftirqd/022:13:040
31125992811,5cyclictest12-21ksoftirqd/000:14:230
3112599280,7cyclictest12-21ksoftirqd/021:50:000
3112799271,11cyclictest122512grep20:55:162
31126992723,2cyclictest31338-21cstates19:10:151
31126992716,8cyclictest6636-21head20:40:261
31126992716,8cyclictest31596-21fschecks_time20:25:171
3112599277,5cyclictest12-21ksoftirqd/023:03:200
31125992716,2cyclictest141rcu_preempt23:36:340
31125992716,2cyclictest141rcu_preempt23:09:130
31125992716,2cyclictest141rcu_preempt21:43:520
31125992716,2cyclictest141rcu_preempt21:43:520
31125992716,2cyclictest141rcu_preempt21:34:190
31125992713,4cyclictest12-21ksoftirqd/023:51:340
31125992712,4cyclictest12-21ksoftirqd/022:44:000
3112899262,6cyclictest568-21hddtemp_smartct00:35:173
31126992617,6cyclictest4879-21cron20:40:001
31126992616,7cyclictest945-21tr20:30:131
31126992616,7cyclictest4489-21tail19:20:271
31126992615,8cyclictest29880-21awk20:20:221
31126992614,9cyclictest16886-21sort19:50:171
3112599269,9cyclictest12-21ksoftirqd/023:23:420
31125992610,4cyclictest12-21ksoftirqd/022:09:310
275522260,2chrt0-21swapper/322:53:463
138562260,8sleep012-21ksoftirqd/023:55:510
3112899251,7cyclictest4399-21tail23:00:183
31128992515,8cyclictest0-21swapper/322:09:473
3112599259,2cyclictest141rcu_preempt00:03:330
3112599257,4cyclictest12-21ksoftirqd/000:38:500
3112599253,16cyclictest13770-21ssh21:55:130
31125992512,2cyclictest141rcu_preempt22:03:480
3112599250,1cyclictest0-21swapper/019:10:490
18212250,1sleep30-21swapper/322:58:473
151442250,10sleep228-21ksoftirqd/223:57:102
31128992422,1cyclictest0-21swapper/319:10:593
31128992416,6cyclictest17172-21munin-run23:35:003
31127992422,1cyclictest0-21swapper/219:10:302
31127992414,7cyclictest1895-21cut22:10:152
31126992417,5cyclictest9926-21tail20:50:181
31126992417,5cyclictest9926-21tail20:50:181
31126992416,5cyclictest6579-21wc19:25:241
31126992415,6cyclictest12673-21grep19:40:181
31126992414,7cyclictest20557-21expr20:00:141
31126992414,7cyclictest12230-21fschecks_time20:55:161
31126992413,8cyclictest27674-21cut20:15:181
31126992413,8cyclictest2090-21sed19:15:261
3112599249,3cyclictest12-21ksoftirqd/021:25:010
3112599248,4cyclictest12-21ksoftirqd/022:59:550
3112599248,4cyclictest12-21ksoftirqd/022:53:450
3112599247,4cyclictest12-21ksoftirqd/021:10:410
3112599246,4cyclictest12-21ksoftirqd/022:22:200
31125992410,4cyclictest12-21ksoftirqd/000:26:450
3112799231,10cyclictest41262awk20:35:212
31126992317,4cyclictest14996-21wc19:45:211
31126992313,8cyclictest16984-21ntpq21:05:201
3112699230,8cyclictest14267-21df21:00:141
3112599239,2cyclictest141rcu_preempt00:09:390
3112599238,4cyclictest12-21ksoftirqd/022:39:150
3112599238,3cyclictest12-21ksoftirqd/022:34:240
3112799221,5cyclictest0-21swapper/222:16:472
31127992213,6cyclictest19298-21sed21:10:192
3112599220,11cyclictest12-21ksoftirqd/023:48:200
3112899214,4cyclictest17974-21diskmemload00:18:233
31128992116,3cyclictest331rcuc/322:20:003
31128992114,4cyclictest17974-21diskmemload00:10:413
31128992113,6cyclictest8853-21ls19:30:253
31128992113,6cyclictest14250-21perl21:55:163
3112899211,17cyclictest17974-21diskmemload22:03:583
31128992111,7cyclictest11607-21cat23:05:253
3112799214,3cyclictest17974-21diskmemload00:38:412
3112799214,2cyclictest141rcu_preempt23:25:152
3112799213,13cyclictest12707-21apt-get21:30:112
3112799211,8cyclictest26234-21cut21:15:232
3112799211,6cyclictest12112-21awk23:30:202
31127992114,5cyclictest28-21ksoftirqd/219:40:012
31127992113,6cyclictest2884-21sed19:20:002
31127992111,8cyclictest560-21dbus-daemon20:35:002
3112799211,11cyclictest327192diskmemload22:09:202
31125992113,6cyclictest374-21seq20:27:480
3112599211,2cyclictest141rcu_preempt19:40:010
31125992112,6cyclictest28389-21seq20:16:000
31125992112,6cyclictest278-21systemd-journal20:00:000
113352210,1sleep20-21swapper/222:41:012
3112899203,2cyclictest141rcu_preempt20:55:003
3112899203,2cyclictest141rcu_preempt20:55:003
3112899201,4cyclictest29336-21pmu-power22:30:213
31128992014,4cyclictest6399-21pmu-power19:25:223
31128992014,4cyclictest23828-21sh22:26:103
31128992014,4cyclictest23828-21sh22:26:103
3112899201,3cyclictest141rcu_preempt21:43:413
3112899201,3cyclictest141rcu_preempt21:43:413
31128992012,6cyclictest14031-21awk19:45:123
31128992012,6cyclictest13285-21users20:55:273
31128992011,6cyclictest31729-21cat20:25:183
31128992010,7cyclictest20743-21df_inode20:00:153
3112799202,4cyclictest32733-21apt-get21:45:122
3112799201,9cyclictest29193-21apt-get22:55:112
31127992012,5cyclictest9705-21cron23:05:002
31127992011,6cyclictest560-21dbus-daemon21:10:002
31127992010,6cyclictest7931-21sh21:50:192
3112599202,4cyclictest0-21swapper/019:50:230
3112599202,4cyclictest0-21swapper/019:40:210
3112599201,6cyclictest12-21ksoftirqd/019:15:210
31125992014,4cyclictest12-21ksoftirqd/020:05:000
31125992014,3cyclictest6162-21missed_timers19:25:190
31052200,2sleep30-21swapper/319:20:143
214432200,9sleep334-21ksoftirqd/300:01:483
208282200,2sleep30-21swapper/321:35:243
3112899199,7cyclictest1196-21cat20:30:153
3112899196,3cyclictest141rcu_preempt22:10:163
3112899194,6cyclictest34-21ksoftirqd/321:21:403
3112899194,3cyclictest17974-21diskmemload23:15:513
3112899192,3cyclictest141rcu_preempt23:45:123
3112899192,12cyclictest13941-21apt-get21:00:123
3112899191,6cyclictest7628-21diskstats20:45:133
3112899191,4cyclictest141rcu_preempt21:53:363
31128991914,3cyclictest6402-21timerandwakeup20:40:243
31128991914,3cyclictest10329-21interrupts19:35:183
31128991913,4cyclictest755-21grep21:45:153
31128991913,4cyclictest31184-21sh21:19:503
31128991913,3cyclictest17974-21diskmemload23:51:033
31128991913,3cyclictest17974-21diskmemload23:10:083
31128991913,3cyclictest17974-21diskmemload00:27:063
3112899191,2cyclictest141rcu_preempt00:30:143
31128991912,5cyclictest52950irq/141-enp3s0-21:25:253
31128991912,4cyclictest11627-21sh23:55:083
31128991911,6cyclictest2732-21cat22:35:133
31128991911,6cyclictest26111-21cat00:05:263
31128991911,6cyclictest16513-21perl21:05:153
31128991910,7cyclictest13015-21grep00:20:183
3112799199,7cyclictest560-21dbus-daemon23:50:222
3112799197,10cyclictest28-21ksoftirqd/222:50:012
3112799197,10cyclictest0-21swapper/222:35:112
3112799193,11cyclictest20416-21apt-get20:00:142
3112799191,6cyclictest31736-21unixbench_singl23:20:262
3112799191,6cyclictest0-21swapper/221:23:262
3112799191,3cyclictest141rcu_preempt23:40:202
31127991912,5cyclictest9829-21cut20:50:172
31127991912,5cyclictest9829-21cut20:50:172
31127991912,5cyclictest29779-21sed20:20:212
31127991911,6cyclictest6487-21cut21:25:162
3112599199,7cyclictest14307-21df_inode19:45:130
3112599196,4cyclictest12-21ksoftirqd/019:30:110
3112599195,2cyclictest22557-21cpu20:05:130
31125991913,4cyclictest12-21ksoftirqd/019:25:000
31125991912,5cyclictest26260-21seq20:11:070
31125991911,6cyclictest16075-21apt21:05:100
3112899189,6cyclictest27775-21pluginstate20:15:203
3112899187,9cyclictest0-21swapper/323:35:133
31128991813,3cyclictest18811-21sh19:55:163
31128991813,3cyclictest16684-21sh21:32:403
31128991811,5cyclictest3560-21smartctl20:35:163
31128991811,5cyclictest29572-21idleruntime20:20:193
31128991811,5cyclictest2194-21cat19:15:263
31128991811,5cyclictest11991-21expr19:40:133
31128991811,4cyclictest15877-21cstates22:45:123
3112899181,12cyclictest24688-21apt-get20:10:113
3112799188,7cyclictest15473-21kworker/u8:100:25:242
3112799186,10cyclictest200032sleep222:24:042
3112799182,12cyclictest268292chrt20:15:102
3112799182,11cyclictest13978-21apt-get19:45:122
3112799181,6cyclictest0-21swapper/222:52:182
3112799181,5cyclictest14283-21fw_packets21:55:162
31127991811,5cyclictest18484-21expr19:55:132
31127991811,4cyclictest23808-21ls23:40:002
31127991810,6cyclictest6265-21tr20:40:232
31127991810,6cyclictest28580-21df22:30:132
3112599188,2cyclictest141rcu_preempt20:45:180
3112599185,4cyclictest131rcuc/021:00:140
3112599185,4cyclictest12-21ksoftirqd/020:34:570
31125991811,5cyclictest11979-21expr20:55:140
263022181,2sleep226318-21ssh22:04:382
3112899179,6cyclictest16673-21smartctl19:50:163
31128991710,5cyclictest22681-21cut20:05:133
3112799178,7cyclictest6401-21pmu-power19:25:222
3112799178,7cyclictest278-21systemd-journal20:30:002
3112799171,3cyclictest17974-21diskmemload23:18:062
31127991712,3cyclictest8215-21ntp_states20:45:202
31127991711,4cyclictest636-21runrttasks20:07:322
31127991711,4cyclictest17974-21diskmemload00:10:032
31127991710,5cyclictest4490-21sed19:20:272
31127991710,5cyclictest11973-21expr19:40:132
31127991710,5cyclictest11802-21grep23:05:262
3112599174,3cyclictest12-21ksoftirqd/020:20:190
3112799169,5cyclictest8980-21awk19:30:262
31127991611,3cyclictest16286-21cstates19:50:132
31127991610,4cyclictest32399-21timerwakeupswit23:45:252
31127991510,3cyclictest636-21runrttasks20:10:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional