You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-19 - 15:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed Aug 19, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3197629785,8sleep00-21swapper/007:09:390
3181129669,9sleep20-21swapper/207:07:322
3194028665,6sleep10-21swapper/107:09:121
2986828370,9sleep30-21swapper/307:05:043
138422760,0sleep20-21swapper/211:37:112
3207799299,13cyclictest9046-21munin-run09:20:010
3208699250,25cyclictest0-21swapper/210:47:532
3208199250,24cyclictest0-21swapper/112:04:561
3209099240,1cyclictest0-21swapper/309:29:033
3208699230,23cyclictest0-21swapper/212:04:462
3208699220,22cyclictest0-21swapper/211:55:492
3208699220,22cyclictest0-21swapper/211:49:432
3208699220,20cyclictest0-21swapper/209:59:112
3208699220,0cyclictest0-21swapper/209:53:302
3207799220,17cyclictest0-21swapper/009:36:530
3208699210,20cyclictest0-21swapper/209:35:092
3208699210,1cyclictest0-21swapper/211:21:012
3208199210,1cyclictest0-21swapper/108:40:261
3208699200,20cyclictest0-21swapper/211:03:382
3208699200,19cyclictest0-21swapper/209:15:082
3208699200,18cyclictest0-21swapper/210:01:562
32081992019,1cyclictest0-21swapper/109:30:141
3208199201,19cyclictest0-21swapper/110:29:321
3208199200,1cyclictest0-21swapper/112:22:371
3208199200,1cyclictest0-21swapper/112:22:371
3207799200,3cyclictest141rcu_preempt10:41:480
158012200,0sleep30-21swapper/310:01:593
3209099190,1cyclictest0-21swapper/312:04:563
3208699190,2cyclictest0-21swapper/208:10:122
3208699190,19cyclictest0-21swapper/210:58:202
3208699190,19cyclictest0-21swapper/210:58:202
3208699190,14cyclictest0-21swapper/211:54:482
3208199192,16cyclictest0-21swapper/108:05:141
3209099180,2cyclictest0-21swapper/311:28:523
3208699180,17cyclictest0-21swapper/210:34:552
3208699180,15cyclictest0-21swapper/212:11:102
3208199180,1cyclictest0-21swapper/111:15:241
3208199180,17cyclictest0-21swapper/110:05:141
3207799182,16cyclictest24723-21diskmemload12:20:550
3207799182,16cyclictest24723-21diskmemload12:20:550
3208699170,17cyclictest0-21swapper/212:29:082
3208699170,16cyclictest0-21swapper/209:24:212
3207799170,16cyclictest0-21swapper/011:26:470
32090991610,5cyclictest4250-21head11:50:253
3209099160,16cyclictest0-21swapper/310:28:363
3209099160,14cyclictest0-21swapper/311:04:213
3208699160,1cyclictest0-21swapper/210:17:232
3208699160,15cyclictest0-21swapper/209:28:592
3208199169,5cyclictest0-21swapper/110:44:141
3208199163,9cyclictest225-21systemd-journal08:50:001
3208199163,9cyclictest225-21systemd-journal08:50:001
3208199160,2cyclictest0-21swapper/109:16:021
3208199160,16cyclictest0-21swapper/111:06:381
3208199160,12cyclictest0-21swapper/110:54:001
3208199160,0cyclictest0-21swapper/109:47:211
3207799164,7cyclictest12625-21perf08:45:000
3207799164,6cyclictest25848-21awk11:44:590
3209099153,10cyclictest12871-21apt-get08:45:143
3209099153,10cyclictest12871-21apt-get08:45:143
3209099152,6cyclictest0-21swapper/308:00:143
3209099152,12cyclictest0-21swapper/311:45:113
32090991512,2cyclictest1797-21basename09:34:593
3209099151,13cyclictest28510-21ssh12:25:143
3209099150,7cyclictest0-21swapper/309:54:463
3209099150,2cyclictest0-21swapper/311:10:483
3208699150,15cyclictest0-21swapper/210:40:362
32081991514,1cyclictest0-21swapper/111:36:061
32081991511,3cyclictest0-21swapper/109:24:361
3207799157,6cyclictest18796-21sh12:19:230
3209099148,5cyclictest24723-21diskmemload11:31:223
3209099145,8cyclictest0-21swapper/311:37:163
3209099144,8cyclictest16437-21latency_hist11:20:013
32090991412,1cyclictest0-21swapper/312:15:453
32090991411,2cyclictest9220-21ls07:30:003
3209099141,11cyclictest0-21swapper/310:30:133
3208699143,9cyclictest32-21ksoftirqd/210:10:142
32086991412,1cyclictest26981-21sh09:30:162
32086991411,2cyclictest22862-21wc07:55:252
3208699140,2cyclictest0-21swapper/209:10:042
3208699140,14cyclictest0-21swapper/211:17:462
3208699140,14cyclictest0-21swapper/210:29:242
3208699140,12cyclictest0-21swapper/210:54:382
32081991410,1cyclictest0-21swapper/112:19:191
3208199140,14cyclictest0-21swapper/111:34:211
3208199140,14cyclictest0-21swapper/110:45:501
32077991413,1cyclictest32075-21cyclictest12:08:440
32077991411,2cyclictest0-21swapper/011:00:000
32077991411,2cyclictest0-21swapper/011:00:000
32077991410,3cyclictest28766-21tr12:25:150
3207799140,1cyclictest0-21swapper/010:04:160
3209099137,5cyclictest0-21swapper/312:11:143
3209099134,5cyclictest12585-21sh10:58:003
3209099134,5cyclictest12585-21sh10:58:003
3209099133,7cyclictest24723-21diskmemload10:15:253
3209099131,6cyclictest17963-21/usr/sbin/munin11:40:213
32090991312,1cyclictest0-21swapper/309:42:143
32090991310,2cyclictest30796-21ssh10:10:353
3208699139,3cyclictest4042-21head07:15:262
3208699139,3cyclictest28714-21ntpq12:05:202
3208699139,3cyclictest27732-21wc08:05:272
3208699131,8cyclictest225-21systemd-journal07:15:012
32086991310,2cyclictest311rcuc/212:35:262
32086991310,2cyclictest27127-21grep11:25:262
32086991310,2cyclictest13060-21sed07:35:232
3208199138,4cyclictest23519-21sh11:05:001
3208199130,12cyclictest0-21swapper/110:33:051
3207799139,3cyclictest25888-21sh09:30:000
3207799139,3cyclictest18575-21cat08:55:190
3207799139,3cyclictest12639-21grep09:40:260
3207799133,6cyclictest19619-21df_abs09:45:180
3207799132,9cyclictest15649-21/usr/sbin/munin11:00:160
3207799131,6cyclictest14087-21munin-run10:40:000
32077991312,1cyclictest0-21swapper/010:20:130
32077991310,2cyclictest4284-21grep09:55:170
32077991310,2cyclictest32039-21grep08:15:210
32077991310,2cyclictest30498-21grep10:10:260
32077991310,2cyclictest26479-21grep09:10:200
32077991310,2cyclictest25387-21sort08:00:260
32077991310,2cyclictest24321-21grep09:05:280
32077991310,2cyclictest22925-21cat07:55:250
32077991310,2cyclictest1773-21ssh09:35:000
32077991310,2cyclictest17348-21grep07:45:170
3209099129,2cyclictest8223-21ssh10:35:273
3209099129,2cyclictest4702-21chrt09:35:423
3209099129,2cyclictest4635-21wc09:55:213
3209099129,2cyclictest4501-21ssh12:30:043
3209099129,2cyclictest31915-21basename11:10:003
3209099129,2cyclictest25480-21grep09:10:153
3209099129,2cyclictest24818-21wc09:10:003
3209099129,2cyclictest22723-21ssh10:45:053
3209099129,2cyclictest22046-21ssh09:46:133
3209099129,2cyclictest21327-21cat12:20:203
3209099129,2cyclictest21327-21cat12:20:203
3209099129,2cyclictest18707-21sshd07:49:143
3209099129,2cyclictest17599-21head11:20:163
3209099129,2cyclictest14539-21grep07:40:153
3209099123,8cyclictest22812-21python07:55:243
3209099122,7cyclictest15244-21sh10:40:153
3209099120,7cyclictest0-21swapper/312:35:303
3209099120,7cyclictest0-21swapper/308:05:133
3209099120,12cyclictest0-21swapper/309:24:293
3209099120,0cyclictest0-21swapper/310:05:143
3208699129,2cyclictest29577-21seq11:08:122
3208699129,2cyclictest20685-21sort09:00:162
3208699129,2cyclictest15096-21sed10:20:262
3208699129,2cyclictest1454-21grep11:30:142
3208699129,2cyclictest10193-21grep07:30:162
3208699128,3cyclictest3486-21ssh11:11:352
3208699123,6cyclictest0-21swapper/210:05:232
3208199129,2cyclictest29307-21grep08:10:161
3208199129,2cyclictest28656-21grep12:25:141
3208199129,2cyclictest23547-21perf08:00:001
3208199129,2cyclictest19274-21grep11:40:221
3208199129,2cyclictest12599-21sed11:55:281
3208199129,2cyclictest12536-21rm10:57:581
3208199129,2cyclictest12536-21rm10:57:571
3208199129,2cyclictest11608-21sh09:40:181
3208199128,3cyclictest14931-21ssh12:35:261
3208199128,2cyclictest5689-21sh11:51:311
3208199128,2cyclictest0-21swapper/109:25:331
3208199125,7cyclictest3798-21apache211:48:491
3208199122,10cyclictest29642-21ssh09:12:201
3208199121,3cyclictest24723-21diskmemload10:15:081
3208199121,3cyclictest225-21systemd-journal09:00:001
3208199120,3cyclictest0-21swapper/112:33:521
3208199120,12cyclictest162922sleep110:21:171
3208199120,11cyclictest8231-21grep07:25:201
3207799129,2cyclictest4573-21grep08:25:220
3207799129,2cyclictest32493-21rm11:10:140
3207799129,2cyclictest31315-21chrt10:50:130
3207799129,2cyclictest29317-21sed08:10:160
3207799129,2cyclictest26570-21tr11:45:130
3207799129,2cyclictest24977-21sh11:05:170
3207799129,2cyclictest24682-21cron11:25:010
3207799129,2cyclictest21050-21grep12:00:290
3207799120,7cyclictest0-21swapper/009:53:390
3209099119,2cyclictest29417-21mii-tool08:10:173
3209099119,2cyclictest18678-21cat08:55:203
3209099119,2cyclictest18543-21rm10:23:013
3209099119,1cyclictest5692-21acpi08:30:113
3209099119,1cyclictest467-21uniq07:10:163
3209099119,1cyclictest4025-21ssh09:16:063
3209099119,1cyclictest30895-21seq08:15:053
3209099119,1cyclictest27319-21taskset12:05:083
3209099119,1cyclictest19371-21expr07:50:143
3209099118,2cyclictest31315-21taskset10:50:133
3209099118,2cyclictest20756-21grep09:00:163
3209099118,2cyclictest17343-21cron08:55:003
3209099118,2cyclictest13977-21munin-plugin-st07:40:003
3209099118,2cyclictest12063-21cat08:40:263
3209099118,2cyclictest10087-21chrt08:38:363
3209099118,2cyclictest10038-21grep07:30:153
3209099110,1cyclictest0-21swapper/311:55:083
3208699119,1cyclictest23220-21cut09:05:172
3208699118,2cyclictest638-21run-parts08:20:002
3208699118,2cyclictest4068-21tail08:25:182
3208699118,2cyclictest19235-21rm09:45:162
3208699118,2cyclictest18163-21cut11:40:162
3208699118,2cyclictest15854-21cut08:50:162
3208699118,2cyclictest15521-21ssh12:16:472
3208699118,2cyclictest14779-21ssh09:42:042
3208699118,2cyclictest11076-21ssh12:33:522
3208699118,2cyclictest10342-21ssh10:37:062
3208699116,2cyclictest20509-21cat12:20:152
3208699116,2cyclictest20509-21cat12:20:152
3208199119,1cyclictest9361-21wc08:35:221
3208199119,1cyclictest12572-21tail07:35:181
3208199118,2cyclictest7390-21cat10:35:201
3208199118,2cyclictest6626-21ssh12:11:231
3208199118,2cyclictest4712-21grep09:55:201
3208199118,2cyclictest4342-21wc08:25:201
3208199118,2cyclictest4024-21head07:15:261
3208199118,2cyclictest32195-21ssh09:52:361
3208199118,2cyclictest28371-21chrt10:10:031
3208199118,2cyclictest28318-21head12:05:171
3208199118,2cyclictest21502-21cat09:00:251
3208199118,2cyclictest2040-21ssh09:35:021
3208199118,2cyclictest18178-21wc07:45:261
3208199118,2cyclictest17585-21grep11:20:161
3208199118,2cyclictest15973-21grep08:50:171
3208199118,2cyclictest15648-21sensors07:40:241
3208199118,2cyclictest13772-21sed10:00:271
3208199118,2cyclictest1169-21tr08:20:151
3208199118,2cyclictest10161-21cut07:30:161
3208199117,3cyclictest4726-21basename07:20:121
3208199117,3cyclictest28988-21ssh11:26:511
3208199117,3cyclictest1527-21grep07:10:261
3208199111,9cyclictest1047-21irqstats11:10:191
3208199110,10cyclictest19050-21aten2_r7power_c07:50:131
3207799118,2cyclictest5960-21rm10:15:230
3207799118,2cyclictest5827-21ssh12:10:450
3207799118,2cyclictest4472-21ssh11:31:250
3207799118,2cyclictest30987-21head10:30:170
3207799118,2cyclictest26774-21ssh10:27:460
3207799118,2cyclictest26648-21expr08:05:150
3207799118,2cyclictest2476-21ssh11:50:130
3207799118,2cyclictest24355-21head10:45:230
3207799118,2cyclictest20376-21cut10:05:150
3207799118,2cyclictest14688-21expr07:40:150
3207799118,2cyclictest14305-21cut12:35:190
3207799118,2cyclictest13640-21taskset07:35:550
3207799118,2cyclictest10716-21awk07:30:200
3207799118,2cyclictest10619-21ssh12:33:300
3207799117,3cyclictest9597-21chrt11:35:130
3207799117,3cyclictest11048-21/usr/sbin/munin11:55:160
3209099109,1cyclictest4180-21taskset07:16:433
3209099109,1cyclictest17182sleep308:20:173
3209099108,2cyclictest6591-21chrt07:21:553
3209099108,1cyclictest3639-21cat08:25:153
3208699108,1cyclictest7794-21cut07:25:162
3208699108,1cyclictest161992sleep207:42:302
3208699107,2cyclictest8804-21grep08:35:162
3208699107,2cyclictest6402-21sort08:30:162
3208699107,2cyclictest20666-21sed07:50:262
3208699107,2cyclictest1856-21uname08:20:192
3208699107,2cyclictest17634-21cut08:55:142
3208699107,2cyclictest17387-21tail07:45:172
3208699107,2cyclictest13613-21grep08:45:172
3208699107,2cyclictest13613-21grep08:45:162
3208699107,2cyclictest10555-21basename08:40:142
3208699106,2cyclictest5308-21df_inode07:20:162
3208699100,1cyclictest141rcu_preempt08:00:182
3208199108,1cyclictest31986-21sed08:15:201
3208199107,2cyclictest6012-21tr08:30:141
3208199107,2cyclictest25946-21sleep08:05:001
3208199107,2cyclictest23593-21sort09:05:211
3207799108,1cyclictest8791-21cstates11:15:140
3207799108,1cyclictest4993-21expr07:20:140
3207799108,1cyclictest42252sleep007:17:160
3207799108,1cyclictest20807-21hostname09:00:170
3207799108,1cyclictest1884-21taskset07:13:310
3207799107,2cyclictest8122-21cut08:35:130
3207799107,2cyclictest7394-21grep08:30:250
3207799107,2cyclictest7380-21tr07:25:140
3207799107,2cyclictest2567-21tail08:20:260
3207799107,2cyclictest19376-21expr07:50:140
3207799107,2cyclictest17240-21chrt08:53:440
3207799107,2cyclictest14598-21who08:45:280
3207799107,2cyclictest14598-21who08:45:280
3207799100,9cyclictest9997-21ssh09:20:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional