You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 00:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sun Jun 07, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3039628843,40sleep30-21swapper/307:09:243
3039628843,40sleep30-21swapper/307:09:233
3014828878,6sleep00-21swapper/007:06:110
3014828878,6sleep00-21swapper/007:06:110
3021728558,9sleep20-21swapper/207:07:052
3021728558,9sleep20-21swapper/207:07:052
3019128564,6sleep10-21swapper/107:06:441
3019128564,6sleep10-21swapper/107:06:441
293792530,0sleep20-21swapper/211:09:472
30530993029,1cyclictest15732-21kworker/u8:212:13:001
30532992724,2cyclictest3145-21kworker/u8:012:12:582
3053299260,4cyclictest0-21swapper/210:10:122
3052699260,1cyclictest0-21swapper/011:41:040
30530992421,2cyclictest0-21swapper/111:04:111
30536992322,1cyclictest0-21swapper/309:50:173
3053299230,4cyclictest0-21swapper/212:15:352
3053299230,23cyclictest0-21swapper/209:37:222
3053299230,0cyclictest0-21swapper/210:57:422
3052699230,22cyclictest0-21swapper/010:33:510
30536992221,1cyclictest0-21swapper/312:20:013
3053299220,22cyclictest0-21swapper/210:01:202
3053099220,22cyclictest0-21swapper/109:40:201
3053099220,20cyclictest0-21swapper/112:25:041
30532992120,1cyclictest0-21swapper/209:24:092
30532992119,1cyclictest0-21swapper/210:08:052
3053299210,21cyclictest0-21swapper/212:33:422
3053299210,1cyclictest141rcu_preempt11:23:422
30530992120,1cyclictest0-21swapper/110:05:371
3053099210,17cyclictest0-21swapper/111:25:111
3053299200,18cyclictest0-21swapper/211:12:392
3053299200,16cyclictest0-21swapper/208:40:122
30530992019,1cyclictest0-21swapper/111:55:141
3053099201,1cyclictest19697-21ssh09:26:091
3052699206,7cyclictest20214-21munin-run12:40:000
3052699200,20cyclictest0-21swapper/011:57:060
3053299190,19cyclictest0-21swapper/212:39:022
30530991910,1cyclictest141rcu_preempt11:42:321
3053099190,18cyclictest0-21swapper/110:54:111
3053099190,17cyclictest0-21swapper/111:08:211
30536991812,3cyclictest18853-21missed_timers09:45:203
3053699180,18cyclictest0-21swapper/310:13:353
3053299182,3cyclictest32-21ksoftirqd/210:45:522
3053299180,18cyclictest0-21swapper/210:41:522
3053299180,18cyclictest0-21swapper/210:39:282
3053299180,16cyclictest0-21swapper/212:25:202
30530991817,1cyclictest25-21ksoftirqd/109:59:591
3053099181,2cyclictest141rcu_preempt10:23:081
3053099180,18cyclictest0-21swapper/110:36:351
3052699187,6cyclictest447-21awk09:35:000
3052699180,1cyclictest0-21swapper/011:25:170
3053099173,14cyclictest241rcuc/110:46:271
3053099170,16cyclictest0-21swapper/109:23:551
3052699175,6cyclictest21381-21ssh11:05:000
3052699170,4cyclictest0-21swapper/010:14:260
3052699170,1cyclictest0-21swapper/011:10:390
3052699170,17cyclictest0-21swapper/010:17:070
3053699163,13cyclictest0-21swapper/310:09:033
3053299162,13cyclictest0-21swapper/211:44:162
3053099162,14cyclictest0-21swapper/109:45:351
30530991613,1cyclictest141rcu_preempt10:58:191
3053099160,1cyclictest0-21swapper/112:07:301
3053099160,1cyclictest0-21swapper/111:22:561
3052699164,2cyclictest0-21swapper/012:10:130
30526991612,4cyclictest0-21swapper/011:07:530
3053699154,9cyclictest25683-21grep12:05:153
3053699153,5cyclictest3581-21ssh10:54:153
30536991510,4cyclictest455-21dbus-daemon12:12:443
30532991514,1cyclictest32-21ksoftirqd/210:28:012
3053299151,13cyclictest0-21swapper/207:50:132
3053299150,15cyclictest0-21swapper/211:54:402
3053299150,14cyclictest0-21swapper/211:32:562
3053099159,4cyclictest25-21ksoftirqd/111:46:411
3053099152,9cyclictest12468-21wget12:35:141
30530991512,2cyclictest0-21swapper/110:14:261
30530991511,1cyclictest25-21ksoftirqd/109:32:551
3053099150,1cyclictest0-21swapper/110:04:181
3052699153,7cyclictest4579-21perf10:55:000
3052699150,14cyclictest0-21swapper/012:02:080
3053699145,2cyclictest8592-21apt-get11:55:133
30536991411,3cyclictest20460-21kworker/3:010:20:233
30536991411,3cyclictest0-21swapper/311:36:013
30536991411,2cyclictest15387-21grep11:00:253
30536991411,2cyclictest11984-21cut12:15:163
30536991410,3cyclictest9572-21wc09:20:183
30536991410,3cyclictest6838-21sh11:35:003
30536991410,3cyclictest28779-21cut12:25:163
3053699140,2cyclictest0-21swapper/310:59:033
3053299142,8cyclictest6732-21dnf11:15:162
30532991411,2cyclictest10437-21grep07:35:012
3053099149,4cyclictest24033-21sed08:00:261
3053099148,6cyclictest25-21ksoftirqd/112:15:341
3053099148,5cyclictest13826-21grep10:40:211
30530991412,1cyclictest25-21ksoftirqd/109:54:241
30530991410,3cyclictest25-21ksoftirqd/107:15:001
30530991410,3cyclictest20015-21ssh12:20:141
30530991410,3cyclictest11995-21cat07:35:261
3053099140,14cyclictest22686-21kworker/1:010:30:331
3052699141,5cyclictest475-21in:imuxsock10:24:360
3053699139,3cyclictest20580-21grep07:55:153
3053699139,3cyclictest17387-21cron07:49:593
3053699136,5cyclictest9283-21apt-get09:40:123
3053699132,10cyclictest32669-21apache_volume11:50:123
30536991312,1cyclictest0-21swapper/311:06:303
30536991310,2cyclictest9584-21wc07:30:243
30536991310,2cyclictest426-21sh09:35:003
30536991310,2cyclictest30038-21head10:30:243
30536991310,2cyclictest17062-21wc09:25:123
30536991310,2cyclictest10915-21basename10:20:003
3053699130,1cyclictest0-21swapper/310:28:473
3053699130,12cyclictest17361-21tr08:55:203
3053299139,3cyclictest22096-21cat09:05:202
3053299137,5cyclictest5408-21ntp_kernel_pll_08:30:192
3053299132,11cyclictest0-21swapper/209:45:172
30532991311,2cyclictest32-21ksoftirqd/212:22:182
30532991310,2cyclictest8223-21wc08:35:252
30532991310,2cyclictest29297-21apt08:15:002
30532991310,2cyclictest2895-21wc08:25:192
30532991310,2cyclictest18030-21tr09:25:202
3053299130,2cyclictest0-21swapper/207:10:142
3053299130,13cyclictest0-21swapper/210:53:202
3053299130,0cyclictest0-21swapper/211:39:442
3053099139,3cyclictest7594-21cron07:30:001
3053099138,4cyclictest13530-21chrt08:49:251
3053099138,2cyclictest25-21ksoftirqd/109:38:091
3053099138,1cyclictest141rcu_preempt12:34:571
3053099136,6cyclictest27448-21ssh09:11:511
3053099136,4cyclictest25-21ksoftirqd/112:03:091
30530991312,1cyclictest3000-21kworker/1:010:17:411
30530991311,2cyclictest25-21ksoftirqd/111:51:581
30530991310,3cyclictest25-21ksoftirqd/110:25:091
30530991310,2cyclictest29689-21grep08:15:161
3053099130,2cyclictest0-21swapper/111:18:321
3052699131,1cyclictest0-21swapper/011:49:010
30526991310,2cyclictest8571-21grep07:30:140
30526991310,2cyclictest3141-21cat09:55:180
30526991310,2cyclictest17021-21grep09:25:120
3052699130,13cyclictest0-21swapper/007:35:340
3053699129,2cyclictest9783-21cut08:40:163
3053699129,2cyclictest8717-21cron09:40:003
3053699129,2cyclictest7638-21wc07:30:003
3053699129,2cyclictest4269-21tr12:30:153
3053699129,2cyclictest3455-21ls09:55:213
3053699129,2cyclictest26126-21wc08:05:233
3053699129,2cyclictest25546-21tail09:10:263
3053699129,2cyclictest22758-21head10:45:273
3053699129,2cyclictest20460-21kworker/3:010:03:423
3053699129,2cyclictest14514-21grep07:40:263
3053699129,2cyclictest14172-21wc12:35:263
3053699129,2cyclictest12791-21grep10:40:143
3053699128,3cyclictest7529-21wc11:15:223
3053699128,3cyclictest32277-21cat11:50:003
3053699128,2cyclictest11621-21cut08:45:143
3053699124,7cyclictest19759-21cron07:54:593
3053699124,7cyclictest0-21swapper/307:15:003
3053299129,2cyclictest9456-21tail11:55:182
3053299129,2cyclictest26934-21sort09:50:162
3053299129,2cyclictest23938-21tr09:10:142
3053299129,2cyclictest22175-21grep08:00:002
3053299129,2cyclictest154872sleep210:22:032
3053299129,2cyclictest1431-21tail07:15:172
3053299129,2cyclictest0-21swapper/212:00:132
3053299128,4cyclictest55550irq/126-enp2s0-09:15:542
3053299128,3cyclictest17186-21cat08:55:182
3053299128,3cyclictest11563-21basename08:45:132
3053299120,12cyclictest0-21swapper/209:57:062
3053299120,11cyclictest25172-21tail11:45:172
3053299120,11cyclictest0-21swapper/208:20:142
3053099129,2cyclictest493-21cat08:20:191
3053099129,2cyclictest3590-21grep08:25:251
3053099129,2cyclictest32499-21tr09:15:161
3053099129,2cyclictest19755-21cron07:54:591
3053099129,2cyclictest19354-21grep09:00:161
3053099125,6cyclictest38150irq/131-snd_hda11:11:121
30530991210,1cyclictest21510-21cat07:55:261
3052699129,2cyclictest9784-21sort08:40:160
3052699129,2cyclictest8622-21grep09:20:130
3052699129,2cyclictest29467-21wc12:25:210
3052699129,2cyclictest26624-21cat10:10:000
3052699129,2cyclictest26562-21expr09:50:140
3052699129,2cyclictest26415-21grep08:05:280
3052699129,2cyclictest24534-21sh08:05:000
3052699129,2cyclictest23184-21wc09:10:000
3052699129,2cyclictest2156-21sed11:50:260
3052699129,2cyclictest2055-21cat09:35:190
3052699129,2cyclictest2046-21cut08:25:130
3052699129,2cyclictest19983-21tr12:20:150
3052699129,2cyclictest13682-21head07:40:170
3052699129,2cyclictest10677-21tr09:40:210
3052699128,3cyclictest21139-21ls10:25:210
3052699128,2cyclictest3874-21sed07:20:170
3052699124,1cyclictest141rcu_preempt12:05:470
3052699123,5cyclictest28398-21tr08:10:220
3052699120,7cyclictest28676-21perf10:50:000
3052699120,4cyclictest0-21swapper/011:19:330
3053699118,2cyclictest5950-21sed10:35:263
3053699118,2cyclictest5875-21ssh09:18:393
3053699118,2cyclictest5635-21which08:30:213
3053699118,2cyclictest31055-21sed11:10:183
3053699118,2cyclictest22910-21ssh11:25:083
3053699118,2cyclictest20232-21grep09:00:253
3053699118,2cyclictest17523-21ssh12:00:153
3053699118,2cyclictest16771-21head11:40:173
3053699118,2cyclictest14943-21grep08:50:203
3053699118,2cyclictest14800-21ssh11:20:143
3053299119,1cyclictest144942sleep209:42:562
3053299118,2cyclictest27934-21ssh10:30:042
3053299118,2cyclictest27011-21tail09:30:252
3053299118,2cyclictest25684-21tr12:05:152
3053299118,2cyclictest15706-21taskset08:51:502
3053299118,2cyclictest14724-21ssh11:00:192
3053299118,2cyclictest13668-21head07:40:182
3053299118,2cyclictest12458-21chrt07:38:022
3053299118,2cyclictest10257-21ssh10:19:302
3053299110,4cyclictest225-21systemd-journal07:26:352
3053099118,2cyclictest9637-21ssh11:35:361
3053099118,2cyclictest655-21ls11:30:251
3053099117,3cyclictest1503-21head07:15:181
3053099116,4cyclictest17145-21taskset07:47:001
30530991111,0cyclictest0-21swapper/108:52:041
3052699119,1cyclictest8257-21ssh10:37:110
3052699119,1cyclictest14088-21grep08:50:140
3052699118,2cyclictest918-21grep11:30:260
3052699118,2cyclictest7704-21sed08:35:180
3052699118,2cyclictest7655-21tr11:35:140
3052699118,2cyclictest4624-21chrt12:30:160
3052699118,2cyclictest3150-21ssh09:16:320
3052699118,2cyclictest30590-21wc08:15:230
3052699118,2cyclictest27939-21ssh09:12:140
3052699118,2cyclictest2246-21cat07:15:260
3052699118,2cyclictest20296-21tr07:55:160
3052699118,2cyclictest19757-21cron07:54:590
3052699118,2cyclictest18583-21taskset09:00:020
3052699118,2cyclictest18368-21cut09:45:170
3052699118,2cyclictest16704-21ssh10:03:470
3052699118,2cyclictest12983-21basename11:00:000
3052699118,2cyclictest12788-21cut10:40:140
3052699118,2cyclictest11335-21tr12:15:140
3052699111,6cyclictest225-21systemd-journal08:35:000
3053699108,1cyclictest271342sleep308:10:053
3053699108,1cyclictest19072sleep308:25:113
3053699108,1cyclictest124012sleep307:37:193
3053699107,2cyclictest7478-21sort08:35:183
3053699107,2cyclictest4251-21ntpq07:20:193
3053699107,2cyclictest24040-21grep08:00:263
3053699107,2cyclictest21256-21cut09:05:143
3053699107,2cyclictest1208-21df07:15:163
3053699100,9cyclictest31430-21taskset08:17:063
3053299108,1cyclictest260842sleep208:05:232
3053299108,1cyclictest16074-21which07:45:172
3053299107,2cyclictest25271-21ssh11:25:362
3053299107,2cyclictest22458-21grep08:00:132
3053299102,2cyclictest3257-21grep07:20:132
3053299102,2cyclictest225-21systemd-journal08:18:442
3053299100,9cyclictest18862-21expr09:00:152
3053099108,1cyclictest8013-21cat07:30:021
3053099108,1cyclictest3946-21cat07:20:171
3053099108,1cyclictest25935-21ntpq08:05:211
3053099108,1cyclictest16890-21cut08:55:161
3053099107,2cyclictest22898-21chrt09:07:281
3053099107,2cyclictest15011-21cron07:45:001
3053099105,4cyclictest7745-21tr08:35:191
3053099104,5cyclictest29306-21sleep08:15:001
3053099100,4cyclictest4370-21aten2_r7power_p08:30:121
3052699108,1cyclictest5780-21grep07:25:140
3052699108,1cyclictest17981-21taskset08:55:330
3052699108,1cyclictest11876-21expr08:45:140
3052699107,2cyclictest32664-21head08:20:170
3052699107,2cyclictest14855-21tr11:20:140
3052699106,3cyclictest16900-21grep07:45:260
305369997,1cyclictest31954-21apache_processe08:20:123
305309997,1cyclictest9392-21expr08:40:141
305269997,1cyclictest32568-21chrt07:11:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional