You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 18:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Apr 08, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21902202163,26sleep30-21swapper/307:08:423
20912202164,26sleep00-21swapper/007:07:270
21832201166,23sleep10-21swapper/107:08:361
21572201164,25sleep20-21swapper/207:08:182
1741521790,4sleep2238499cyclictest11:20:232
2951821720,5sleep2238499cyclictest09:20:242
313221650,5sleep3239199cyclictest07:10:153
1706421460,4sleep2238499cyclictest07:40:162
146502790,4sleep00-21swapper/011:15:160
146502790,4sleep00-21swapper/011:15:160
51742610,1sleep30-21swapper/307:15:043
2384996057,2cyclictest0-21swapper/209:11:192
196182590,4sleep3239199cyclictest07:45:183
274512580,1sleep00-21swapper/011:45:130
86182510,5sleep20-21swapper/208:35:132
42162510,2sleep30-21swapper/310:50:223
176472500,3sleep1237999cyclictest12:35:191
71672440,5sleep1237999cyclictest09:45:151
239199370,25cyclictest0-21swapper/309:29:443
239199361,27cyclictest0-21swapper/309:55:103
2374993611,12cyclictest26876-21munin-run11:45:010
239199331,25cyclictest0-21swapper/308:04:403
239199331,21cyclictest0-21swapper/312:03:523
239199320,24cyclictest0-21swapper/309:02:483
239199320,12cyclictest0-21swapper/308:26:043
237999324,21cyclictest3469-21latency_hist12:05:011
237499320,25cyclictest6843-21kworker/0:011:00:450
239199310,23cyclictest0-21swapper/311:24:163
239199310,23cyclictest0-21swapper/308:45:123
239199310,22cyclictest0-21swapper/310:20:293
239199301,19cyclictest0-21swapper/311:31:183
239199301,14cyclictest0-21swapper/308:24:253
239199301,11cyclictest0-21swapper/308:31:243
239199301,10cyclictest23439-21diskmemload10:05:533
239199301,10cyclictest23439-21diskmemload10:05:523
239199300,23cyclictest0-21swapper/312:37:133
239199300,22cyclictest0-21swapper/309:44:083
239199300,13cyclictest0-21swapper/312:28:083
237499305,12cyclictest10872-21munin-run09:55:010
239199290,23cyclictest0-21swapper/308:15:373
239199290,12cyclictest0-21swapper/309:51:023
239199281,9cyclictest0-21swapper/307:20:283
239199281,12cyclictest10476-21df12:20:123
239199281,12cyclictest0-21swapper/308:35:403
239199281,11cyclictest715-21taskset11:56:163
239199281,11cyclictest0-21swapper/312:17:063
239199281,10cyclictest0-21swapper/308:59:563
239199281,10cyclictest0-21swapper/308:59:563
239199280,17cyclictest0-21swapper/309:45:443
239199280,11cyclictest0-21swapper/311:45:083
239199280,11cyclictest0-21swapper/309:21:523
238499281,10cyclictest0-21swapper/211:30:502
237499282,11cyclictest6843-21kworker/0:010:57:250
237499281,9cyclictest12982-21kworker/0:310:06:240
237499281,9cyclictest12982-21kworker/0:310:06:240
239199271,12cyclictest0-21swapper/311:03:363
239199271,12cyclictest0-21swapper/309:14:003
239199270,12cyclictest0-21swapper/310:56:483
239199270,11cyclictest0-21swapper/311:12:283
239199270,11cyclictest0-21swapper/309:06:083
239199270,10cyclictest0-21swapper/311:44:503
237999271,16cyclictest0-21swapper/110:15:331
237499278,16cyclictest32238-21munin-run09:30:010
237499271,10cyclictest3428-21kworker/0:308:24:040
239199261,9cyclictest0-21swapper/309:39:483
239199260,18cyclictest0-21swapper/312:34:343
239199260,18cyclictest0-21swapper/312:34:343
239199260,12cyclictest0-21swapper/312:06:103
239199260,11cyclictest0-21swapper/310:36:053
239199260,11cyclictest0-21swapper/309:33:123
239199260,11cyclictest0-21swapper/309:19:203
239199260,11cyclictest0-21swapper/308:15:003
239199260,10cyclictest0-21swapper/307:54:283
237999266,6cyclictest31145-21systemd-journal11:53:351
237999261,17cyclictest11045-21load12:20:171
237499268,16cyclictest27853-21idleruntime-cro09:20:010
237499267,16cyclictest26261-21munin-run10:30:010
237499267,16cyclictest14035-21munin-run07:35:010
237499262,11cyclictest23917-21kworker/0:210:31:080
237499262,11cyclictest14247-21kworker/0:012:34:360
237499262,11cyclictest14247-21kworker/0:012:34:360
237499262,10cyclictest20044-21kworker/0:211:31:160
237499261,13cyclictest13073-21awk10:00:010
237499261,12cyclictest4273-21munin-run09:40:010
237499261,10cyclictest18304-21kworker/0:207:46:360
2374992610,3cyclictest3428-21kworker/0:308:34:400
239199257,3cyclictest0-21swapper/307:35:303
239199251,7cyclictest0-21swapper/307:34:123
239199251,10cyclictest0-21swapper/310:40:523
239199250,9cyclictest0-21swapper/310:03:023
239199250,16cyclictest0-21swapper/311:50:323
239199250,11cyclictest0-21swapper/311:18:403
239199250,11cyclictest0-21swapper/311:18:403
239199250,11cyclictest0-21swapper/310:17:493
239199250,11cyclictest0-21swapper/308:43:043
239199250,10cyclictest0-21swapper/310:31:413
237999252,13cyclictest25277-40dnf09:10:301
2379992517,2cyclictest0-21swapper/111:05:211
237999251,5cyclictest0-21swapper/111:24:461
237999251,19cyclictest0-21swapper/110:14:091
237999251,12cyclictest0-21swapper/107:47:341
237999250,16cyclictest87550irq/128-enp1s0-09:55:181
237499252,9cyclictest18767-21kworker/0:110:15:560
237499251,12cyclictest11496-21munin-run11:10:020
237499251,11cyclictest12982-21kworker/0:310:04:520
237499251,10cyclictest21222-21kworker/0:009:31:560
237499250,11cyclictest22773-21kworker/0:107:58:440
239199241,9cyclictest0-21swapper/310:12:353
239199240,7cyclictest0-21swapper/312:10:323
239199240,11cyclictest0-21swapper/311:38:043
239199240,10cyclictest0-21swapper/308:05:043
2379992418,3cyclictest17195-21cpuspeed_turbos08:55:111
2379992418,3cyclictest17195-21cpuspeed_turbos08:55:111
237999241,3cyclictest0-21swapper/109:41:291
237999241,14cyclictest19227-21memory11:25:201
237999241,12cyclictest0-21swapper/109:37:041
237999241,10cyclictest16510-21perf12:35:011
237999241,10cyclictest16510-21perf12:35:011
237999240,17cyclictest6747-21kworker/u8:111:38:591
237499249,3cyclictest21222-21kworker/0:009:41:360
237499247,15cyclictest29431-21awk08:10:010
237499246,10cyclictest0-21swapper/012:01:570
237499242,10cyclictest21222-21kworker/0:009:46:360
2374992411,3cyclictest25970-21kworker/0:211:52:040
2374992411,2cyclictest30998-21kworker/0:012:09:120
237499240,12cyclictest0-21swapper/009:05:010
239199238,3cyclictest0-21swapper/311:25:343
239199237,3cyclictest0-21swapper/307:41:443
239199231,9cyclictest0-21swapper/307:25:163
239199231,10cyclictest0-21swapper/310:46:443
238499236,4cyclictest827-21runrttasks10:35:132
237999239,4cyclictest17470-21munin-run10:10:011
237999239,4cyclictest17470-21munin-run10:10:011
237999233,15cyclictest13496-21kworker/u8:011:12:421
2379992317,3cyclictest15112-21sort07:35:161
237999231,4cyclictest23747-21kworker/u8:210:41:131
2379992314,2cyclictest8307-21strings11:00:181
237999231,3cyclictest3984-21ntpq10:50:191
237999231,16cyclictest6747-21kworker/u8:112:05:451
237999231,16cyclictest0-21swapper/111:57:551
237999231,16cyclictest0-21swapper/111:32:351
237999231,16cyclictest0-21swapper/111:19:561
237999231,16cyclictest0-21swapper/111:19:551
237999231,13cyclictest2896-21sed10:50:021
237999231,13cyclictest22158-21kworker/u8:211:41:511
237499238,2cyclictest9872-21kworker/0:212:23:400
237499232,9cyclictest18767-21kworker/0:110:12:480
237499232,7cyclictest21222-21kworker/0:009:24:440
237499232,11cyclictest30884-40dnf08:10:220
237499232,11cyclictest22413-21kworker/0:011:39:360
237499232,10cyclictest21222-21kworker/0:009:11:400
237499231,9cyclictest23917-21kworker/0:210:40:080
237499231,8cyclictest11976-21kworker/0:312:27:240
237499230,11cyclictest13709-21perf11:15:010
237499230,10cyclictest11427-21kworker/0:111:23:320
239199228,3cyclictest16330-21/usr/sbin/munin08:50:243
239199221,7cyclictest0-21swapper/310:25:093
239199220,9cyclictest0-21swapper/307:58:263
239199220,10cyclictest0-21swapper/311:07:503
238499221,7cyclictest0-21swapper/210:01:342
238499221,5cyclictest0-21swapper/210:17:362
237999223,10cyclictest1-21systemd07:30:001
237999221,8cyclictest0-21swapper/112:15:241
237999221,14cyclictest22158-21kworker/u8:211:46:231
237999221,13cyclictest6030-21memory10:55:171
237999221,12cyclictest13818-21cut10:00:141
2379992210,3cyclictest26821-21df10:30:121
237999220,5cyclictest0-21swapper/110:24:501
237999220,2cyclictest0-21swapper/108:05:011
237999220,10cyclictest0-21swapper/107:40:061
237499222,10cyclictest7624-21kworker/0:112:15:000
237499221,9cyclictest3428-21kworker/0:308:35:200
237499221,8cyclictest23917-21kworker/0:210:38:200
237499221,10cyclictest4205-21kworker/0:307:11:240
2374992210,2cyclictest11976-21kworker/0:312:38:040
238499211,8cyclictest5582-21tune2fs10:55:132
238499211,7cyclictest0-21swapper/209:37:432
238499211,6cyclictest7351-21sed09:45:172
238499211,19cyclictest0-21swapper/209:27:202
238499211,12cyclictest16409-21memory10:05:182
238499211,12cyclictest16409-21memory10:05:182
238499211,12cyclictest0-21swapper/208:00:132
2379992115,3cyclictest28644-21/usr/sbin/munin09:20:161
2379992114,4cyclictest827-21runrttasks09:00:161
237999211,3cyclictest31173-21memory09:25:191
237999211,18cyclictest0-21swapper/108:20:191
237999211,18cyclictest0-21swapper/108:08:401
237999211,17cyclictest23439-21diskmemload09:15:121
237999211,11cyclictest473-21fschecks_time09:30:141
237999211,11cyclictest0-21swapper/109:52:501
237999210,14cyclictest0-21swapper/112:11:251
237499219,2cyclictest4199-21kworker/0:007:27:260
237499218,2cyclictest14501-21kworker/0:108:51:090
237499212,10cyclictest30998-21kworker/0:012:15:280
237499211,11cyclictest30998-21kworker/0:011:56:360
237499211,10cyclictest31955-21kworker/0:010:46:440
230232212,15sleep223050-21diskstats11:35:132
238499208,3cyclictest0-21swapper/208:28:522
238499202,6cyclictest32458-21/usr/sbin/munin09:30:242
238499201,7cyclictest1164-21fschecks_time10:45:142
2384992013,4cyclictest5144-21sed09:40:162
2384992013,4cyclictest19035-21turbostat.cron09:00:012
2384992013,4cyclictest19035-21turbostat.cron09:00:012
238499201,12cyclictest0-21swapper/212:36:582
238499201,12cyclictest0-21swapper/212:19:162
238499201,10cyclictest0-21swapper/209:03:292
238499201,10cyclictest0-21swapper/208:33:572
238499201,10cyclictest0-21swapper/207:23:012
238499200,11cyclictest0-21swapper/208:13:512
237999208,2cyclictest0-21swapper/110:25:151
237999204,9cyclictest0-21swapper/110:38:181
237999201,7cyclictest2575-21/usr/sbin/munin07:10:111
2379992016,3cyclictest0-21swapper/107:22:391
237999201,18cyclictest0-21swapper/108:27:121
237999201,18cyclictest0-21swapper/107:34:001
237999201,11cyclictest23051-21acpi07:55:101
237999201,11cyclictest20846-21/usr/sbin/munin07:50:211
237999201,10cyclictest21470-21/usr/sbin/munin09:05:251
237999200,18cyclictest0-21swapper/108:49:081
237999200,15cyclictest0-21swapper/112:27:441
237499204,4cyclictest18968-21kworker/0:209:07:280
237499202,9cyclictest4199-21kworker/0:007:16:000
237499202,11cyclictest4199-21kworker/0:007:23:000
237499201,9cyclictest2354-21kworker/0:110:50:280
237499201,9cyclictest18767-21kworker/0:110:24:320
238499195,3cyclictest87550irq/128-enp1s0-09:15:152
238499192,5cyclictest5523-21awk12:10:012
238499192,3cyclictest16228-21munin-run07:40:022
238499192,15cyclictest0-21swapper/208:44:242
238499192,15cyclictest0-21swapper/208:15:342
238499191,6cyclictest31493-21/usr/sbin/munin11:55:252
238499191,5cyclictest13017-21diskstats08:45:132
238499191,5cyclictest0-21swapper/208:51:252
238499191,4cyclictest11027-21grep11:05:242
2384991913,3cyclictest18680-21grep10:10:182
238499190,10cyclictest0-21swapper/208:22:352
237999194,9cyclictest0-21swapper/108:10:141
2379991917,1cyclictest0-21swapper/107:15:111
2379991912,3cyclictest0-21swapper/108:40:221
237999191,17cyclictest0-21swapper/108:39:511
237999191,17cyclictest0-21swapper/108:19:131
237999191,15cyclictest15936-21ntp_states08:50:201
2374991914,3cyclictest5850-21munin-plugin-st08:30:000
186472191,14sleep218661-21diskstats11:25:142
238499184,4cyclictest12245-21proc_pri09:55:222
238499183,4cyclictest827-21runrttasks07:45:232
238499183,3cyclictest13307-21users11:10:262
238499182,9cyclictest827-21runrttasks12:12:262
238499182,4cyclictest30348-21ntp_offset11:50:202
238499182,3cyclictest0-21swapper/212:02:542
238499181,9cyclictest0-21swapper/209:53:182
238499181,6cyclictest30589-21kworker/u8:310:54:332
238499181,5cyclictest26485-21/usr/sbin/munin10:30:132
238499181,5cyclictest25009-21latency_hist08:00:012
238499181,5cyclictest24899-21/usr/sbin/munin11:40:142
238499181,5cyclictest22533-21ntp_states09:05:202
238499181,5cyclictest10538-21hddtemp_smartct07:25:162
238499181,4cyclictest0-21swapper/211:48:452
238499181,15cyclictest0-21swapper/207:16:072
237999181,15cyclictest0-21swapper/108:30:111
237499187,2cyclictest16748-21kworker/0:008:55:170
237499187,2cyclictest16748-21kworker/0:008:55:160
237499186,2cyclictest18304-21kworker/0:207:50:240
237499182,10cyclictest7840-21kworker/0:008:47:290
237499182,10cyclictest7344-21kworker/0:107:39:160
238499173,7cyclictest0-21swapper/210:29:002
238499171,9cyclictest0-21swapper/210:20:402
238499171,5cyclictest14535-21/usr/sbin/munin12:30:122
238499171,5cyclictest14535-21/usr/sbin/munin12:30:112
238499171,5cyclictest0-21swapper/212:26:192
238499171,4cyclictest10350-21expr12:20:122
238499171,3cyclictest0-21swapper/211:16:172
238499171,3cyclictest0-21swapper/211:16:172
238499171,3cyclictest0-21swapper/208:05:222
238499171,15cyclictest0-21swapper/210:41:412
2384991710,4cyclictest8830-21grep11:00:242
2384991710,4cyclictest21339-21tune2fs07:50:132
237499176,2cyclictest87350irq/126-enp1s0-08:18:280
237499173,12cyclictest10471-21/usr/sbin/munin08:40:240
2374991712,3cyclictest25242-21/usr/sbin/munin08:00:230
238499169,4cyclictest2591-21cpu07:10:112
2384991612,3cyclictest0-21swapper/207:30:552
237499162,11cyclictest11427-21kworker/0:111:25:160
237499161,6cyclictest0-21swapper/007:40:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional