You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-09 - 05:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Aug 09, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
925829242,45sleep20-21swapper/219:07:202
917729170,16sleep00-21swapper/019:06:180
730628364,18sleep30-21swapper/319:05:123
941728064,12sleep10-21swapper/119:09:251
954799260,25cyclictest0-21swapper/223:16:522
954799260,25cyclictest0-21swapper/223:16:512
953699261,3cyclictest141rcu_preempt00:10:560
954799220,22cyclictest0-21swapper/223:53:432
954799220,22cyclictest0-21swapper/221:56:032
954199220,0cyclictest0-21swapper/122:41:021
953699224,12cyclictest0-21swapper/020:15:150
954799210,21cyclictest0-21swapper/223:33:562
955099200,20cyclictest0-21swapper/323:53:433
9547992020,0cyclictest0-21swapper/221:13:292
954799200,20cyclictest0-21swapper/221:17:342
954799200,18cyclictest0-21swapper/222:18:192
954199201,1cyclictest26913-21kworker/1:200:23:131
954799190,19cyclictest0-21swapper/222:49:122
9541991917,1cyclictest0-21swapper/123:48:181
9541991915,3cyclictest2285-21diskmemload22:46:161
9550991818,0cyclictest0-21swapper/322:36:213
955099180,3cyclictest0-21swapper/300:13:283
954799180,18cyclictest0-21swapper/200:35:562
954799180,0cyclictest0-21swapper/200:15:332
9541991817,1cyclictest0-21swapper/100:33:201
954199181,1cyclictest0-21swapper/122:59:341
954199180,17cyclictest0-21swapper/100:27:431
9550991716,1cyclictest0-21swapper/323:29:433
955099170,14cyclictest9349-21apt-get22:50:143
954799170,17cyclictest0-21swapper/222:21:352
9541991716,1cyclictest31740-21kworker/1:200:08:131
9541991715,1cyclictest2285-21diskmemload23:41:171
9550991614,1cyclictest12366-21/usr/sbin/munin21:35:193
954799163,10cyclictest13084-21apache200:10:192
954799162,10cyclictest27386-21seq21:24:542
954799160,1cyclictest0-21swapper/223:20:502
954799160,16cyclictest0-21swapper/221:38:372
954199160,15cyclictest0-21swapper/100:35:051
955099151,10cyclictest10970-31find00:08:553
9550991510,1cyclictest25449-21apt-get20:50:143
955099150,4cyclictest0-21swapper/322:01:543
954799152,1cyclictest23592-21ssh22:00:202
9547991515,0cyclictest0-21swapper/222:28:402
9547991514,1cyclictest0-21swapper/223:48:462
954799150,2cyclictest0-21swapper/223:12:122
954799150,14cyclictest0-21swapper/222:59:202
954199150,13cyclictest0-21swapper/121:35:191
953699154,5cyclictest2720-21awk23:25:000
953699150,13cyclictest29014-21rm21:45:000
955099149,3cyclictest7010-21ssh22:10:163
955099149,3cyclictest10740-21awk21:15:013
9550991410,3cyclictest2272-21sh22:45:213
9550991410,2cyclictest23809-21cat00:35:003
9547991413,1cyclictest2285-21diskmemload23:26:462
954799141,2cyclictest141rcu_preempt21:29:152
954799141,1cyclictest0-21swapper/200:03:202
9547991410,3cyclictest31436-21grep21:00:222
954799140,2cyclictest0-21swapper/223:35:582
954799140,1cyclictest0-21swapper/223:56:032
954799140,14cyclictest0-21swapper/222:38:042
954799140,13cyclictest0-21swapper/223:04:482
954799140,13cyclictest0-21swapper/222:05:022
954199143,6cyclictest4278-21rt-features21:10:231
954199142,8cyclictest8093-21sh22:30:141
9541991412,1cyclictest0-21swapper/121:30:071
9541991412,1cyclictest0-21swapper/121:30:061
954199141,1cyclictest2285-21diskmemload22:24:571
954199141,13cyclictest0-21swapper/100:17:341
954199140,13cyclictest11120-21cron23:30:011
954199140,13cyclictest0-21swapper/123:05:321
953699149,3cyclictest8131-21grep22:30:140
953699141,7cyclictest30944-21awk19:55:000
9536991413,1cyclictest0-21swapper/023:18:490
9536991413,1cyclictest0-21swapper/023:18:490
9536991410,3cyclictest17830-21grep22:35:210
953699140,11cyclictest0-21swapper/000:33:540
955099139,3cyclictest12342-21sort23:30:183
955099139,2cyclictest10267-21grep20:15:293
955099138,4cyclictest28506-21diskmemload21:25:163
955099131,10cyclictest0-21swapper/323:20:113
9550991310,2cyclictest8067-21grep00:25:183
9550991310,2cyclictest29661-21ssh00:00:153
9550991310,2cyclictest27353-21cut23:00:203
9550991310,2cyclictest24644-21cut22:20:183
9550991310,2cyclictest16673-21ls19:25:003
955099130,1cyclictest0-21swapper/322:57:233
955099130,12cyclictest1584-21wc21:05:243
954799139,3cyclictest13370-21wc21:55:002
954799139,3cyclictest1142-21runrttasks00:32:492
954799138,3cyclictest5804-21wc21:30:262
954799138,3cyclictest5804-21wc21:30:262
9547991310,2cyclictest28620-21cat19:50:002
9547991310,2cyclictest21781-21grep19:35:152
9547991310,2cyclictest20535-21grep19:30:202
9547991310,2cyclictest16469-21grep20:30:182
9547991310,2cyclictest11407-21grep19:10:272
954799130,13cyclictest0-21swapper/222:12:402
954799130,12cyclictest0-21swapper/223:42:372
954199139,3cyclictest14636-21grep23:50:251
954199139,3cyclictest10722-21tr23:10:141
954199136,5cyclictest32503-21taskset23:03:531
954199132,1cyclictest2285-21diskmemload21:58:021
9541991310,3cyclictest0-21swapper/123:21:291
9541991310,2cyclictest7219-21grep21:50:231
9541991310,2cyclictest5646-21cron20:10:011
9541991310,2cyclictest31442-21grep00:00:281
9541991310,2cyclictest1815-21grep21:05:251
953699139,3cyclictest31871-21df00:20:150
953699134,6cyclictest14820-21cron22:14:590
953699131,6cyclictest25188-21munin-run20:50:000
9536991310,2cyclictest3921-21ls21:10:190
9536991310,2cyclictest31241-21grep21:45:250
9536991310,2cyclictest29914-21cron21:00:000
9536991310,2cyclictest25182-21ls22:20:220
9536991310,2cyclictest19767-21grep21:20:140
9536991310,2cyclictest14053-21grep21:55:160
953699130,8cyclictest10972-21apt-get20:20:130
953699130,13cyclictest0-21swapper/000:07:320
955099129,2cyclictest6852-21wc00:25:013
955099129,2cyclictest6296-21wc22:10:003
955099129,2cyclictest431-21sed19:55:273
955099129,2cyclictest4264-21cut21:30:143
955099129,2cyclictest4264-21cut21:30:143
955099129,2cyclictest3312-21uname23:05:203
955099129,2cyclictest28068-21grep20:55:143
955099129,2cyclictest24613-21wc20:45:263
955099129,2cyclictest23107-21grep23:55:263
955099129,2cyclictest22585-21sh20:45:003
955099129,2cyclictest21743-21cut21:40:173
955099129,2cyclictest20895-21grep21:20:213
955099129,2cyclictest19801-21grep20:35:263
955099129,2cyclictest13809-21grep19:15:273
955099129,2cyclictest13082-21basename20:24:593
955099129,2cyclictest10054-21grep19:10:163
955099128,3cyclictest4897-21ssh21:50:003
955099128,3cyclictest17408-21grep20:30:273
955099121,10cyclictest11417-21grep21:15:143
9550991210,1cyclictest16623-21tr22:15:213
955099120,10cyclictest0-21swapper/323:38:413
954799129,2cyclictest6357-21grep00:05:202
954799129,2cyclictest32704-21wc19:55:262
954799129,2cyclictest30437-21grep19:50:282
954799129,2cyclictest2862-21tail23:05:182
954799129,2cyclictest23837-21grep20:45:182
954799129,2cyclictest19757-21tail20:35:262
954799129,2cyclictest14103-21sed20:25:182
954799129,2cyclictest1399-21grep20:00:162
954799129,2cyclictest10112-21grep22:30:262
954799128,3cyclictest29975-21wc21:00:002
954799120,8cyclictest0-21swapper/200:21:482
954199129,2cyclictest31529-21tail22:05:171
954199129,2cyclictest222892sleep122:19:171
954199129,2cyclictest20561-21ssh21:20:181
954199129,2cyclictest12108-21sed21:15:171
954199128,3cyclictest6171-21tr20:10:151
954199128,3cyclictest12910-21tail23:30:211
954199120,11cyclictest22063-21ssh22:00:021
953699129,2cyclictest7186-21chrt21:31:240
953699129,2cyclictest7186-21chrt21:31:230
953699129,2cyclictest5526-21grep21:50:110
953699129,2cyclictest27166-21uname23:00:190
953699129,2cyclictest27054-21sed20:50:260
953699129,2cyclictest19309-21grep20:35:210
953699129,2cyclictest18649-21grep19:25:280
953699129,2cyclictest17063-21ssh21:37:310
953699129,2cyclictest16427-21grep20:30:190
953699129,2cyclictest10257-21chrt23:48:280
953699128,4cyclictest13-21ksoftirqd/000:26:450
953699126,2cyclictest225-21systemd-journal19:35:170
953699124,3cyclictest30709-21ssh00:00:200
9536991210,1cyclictest5929-21basename20:10:140
953699120,12cyclictest0-21swapper/023:39:490
955099118,2cyclictest9784-21ssh23:48:063
955099118,2cyclictest4905-21awk20:05:223
955099118,2cyclictest32290-21taskset21:04:443
955099118,2cyclictest28175-21ssh23:40:043
955099118,2cyclictest28080-21ssh00:17:573
955099118,2cyclictest25708-21grep22:40:183
955099118,2cyclictest21250-21ssh23:15:463
955099118,2cyclictest21250-21ssh23:15:453
955099118,2cyclictest20768-21ssh21:59:223
955099118,2cyclictest18569-21grep19:25:293
955099118,2cyclictest12205-21ssh22:32:023
955099118,2cyclictest11687-21tr23:10:193
955099117,3cyclictest14565-21head20:25:233
955099116,4cyclictest15065-21kworker/3:122:25:033
954799118,2cyclictest7488-21egrep00:25:152
954799118,2cyclictest7244-21which20:10:232
954799118,2cyclictest676-21ssh21:46:492
954799118,2cyclictest3577-21cut20:05:142
954799118,2cyclictest27094-21tail20:50:272
954799118,2cyclictest26381-21grep22:40:232
954799118,2cyclictest25295-21sh21:42:152
954799118,2cyclictest21798-21fgrep20:40:222
954799118,2cyclictest14287-21turbostat19:20:012
954799118,2cyclictest11048-21ssh22:50:252
954799118,2cyclictest1081-21uname21:05:182
954799118,2cyclictest10693-21ls20:20:002
954199118,2cyclictest9311-21grep20:15:201
954199118,2cyclictest38150irq/131-snd_hda22:54:101
954199118,2cyclictest32410-21tr22:25:161
954199118,2cyclictest29769-21cat21:45:141
954199118,2cyclictest29292-21cut21:25:211
954199118,2cyclictest24508-21sh21:41:381
954199118,2cyclictest23295-21cut20:45:151
954199118,2cyclictest22561-21ssh23:16:461
954199118,2cyclictest22561-21ssh23:16:461
954199118,2cyclictest22097-21ssh23:35:311
954199118,2cyclictest21599-21cat23:55:151
954199118,2cyclictest17911-21ssh00:11:451
954199118,2cyclictest16262-21grep20:30:171
954199118,2cyclictest15140-21taskset20:26:121
954199118,2cyclictest12057-21cut20:20:201
954199118,2cyclictest10566-21ssh22:11:411
954199117,3cyclictest18816-21grep20:35:161
954199116,4cyclictest18198-21ssh22:35:261
954199113,4cyclictest9788-21apt-get19:10:141
953699118,2cyclictest7361-21ssh23:27:090
953699118,2cyclictest6940-21ssh22:29:300
953699118,2cyclictest5054-21ssh22:09:090
953699118,2cyclictest4516-21sh23:05:490
953699118,2cyclictest3961-21ssh22:46:140
953699118,2cyclictest31824-21ssh21:26:520
953699118,2cyclictest30662-21ssh00:38:490
953699118,2cyclictest25996-21ssh22:01:430
953699118,2cyclictest24907-21ssh22:40:110
953699118,2cyclictest24186-21sh23:56:150
953699118,2cyclictest23596-21ssh22:58:300
953699118,2cyclictest20053-21expr19:30:150
953699118,2cyclictest16460-21ssh22:15:190
953699118,2cyclictest15640-21needreboot19:20:190
953699118,2cyclictest15092-21ssh23:12:290
953699118,2cyclictest14983-21ssh23:31:300
953699118,2cyclictest11534-21grep21:15:150
953699118,2cyclictest10165-21grep22:50:160
953699118,1cyclictest0-21swapper/023:41:170
953699110,8cyclictest12746-21ssh23:50:130
953699110,6cyclictest22610-21idleruntime-cro20:45:010
955099108,1cyclictest7721-21taskset20:11:133
955099108,1cyclictest3007-21taskset20:01:393
955099108,1cyclictest26417-21cat19:45:123
955099108,1cyclictest24258-21chrt00:35:113
955099108,1cyclictest20171-21cat19:30:163
955099107,2cyclictest5164-21ssh21:50:083
955099107,2cyclictest30195-21grep19:50:253
955099107,2cyclictest25432-21mailstats19:40:233
955099107,2cyclictest21697-21cut19:35:143
954799107,2cyclictest0-21swapper/219:25:222
954799103,5cyclictest26188-21munin-run19:45:002
954799100,4cyclictest225-21systemd-journal19:22:372
954199108,1cyclictest323062sleep121:04:561
954199108,1cyclictest31381-21egrep19:55:151
954199108,1cyclictest305762sleep119:50:391
954199108,1cyclictest222282sleep120:40:321
954199108,1cyclictest18987-21chrt19:29:311
954199107,2cyclictest2769-21date20:00:281
954199107,2cyclictest24947-21cat19:40:191
954199107,2cyclictest19558-21grep19:30:131
954199107,2cyclictest14599-21aten2_r7power_e19:20:131
954199106,3cyclictest14261-21cron19:20:011
954199100,9cyclictest27790-21cat20:55:111
954199100,9cyclictest26650-21grep19:45:151
954199100,9cyclictest25490-21cut20:50:151
953699108,1cyclictest324-21cat21:05:140
953699108,1cyclictest31143-21grep21:00:200
953699108,1cyclictest26810-21expr19:45:150
953699108,1cyclictest25005-21uname19:40:200
953699108,1cyclictest1649-21expr20:00:170
953699108,1cyclictest15642-21cut20:30:000
953699108,1cyclictest12518-21cat19:15:150
953699107,2cyclictest24777-21ssh00:15:280
95479997,1cyclictest12000-21ls20:20:202
95419991,3cyclictest22393-21tail19:35:181
95369997,1cyclictest600-21taskset19:56:430
95369997,1cyclictest3962-21expr20:05:160
95369997,1cyclictest11606-21chrt19:11:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional