You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 04:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Thu Jun 18, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
423329667,9sleep30-21swapper/319:07:393
411429078,8sleep00-21swapper/019:06:080
424228673,9sleep20-21swapper/219:07:462
434328366,13sleep10-21swapper/119:09:061
450799241,21cyclictest0-21swapper/223:10:132
450799240,22cyclictest0-21swapper/200:23:582
4511992316,2cyclictest39-21ksoftirqd/323:50:583
4504992322,1cyclictest0-21swapper/123:02:051
4511992221,1cyclictest381rcuc/321:10:153
450799220,1cyclictest0-21swapper/200:05:002
451199214,13cyclictest39-21ksoftirqd/300:35:013
4511992119,2cyclictest39-21ksoftirqd/322:30:183
451199210,19cyclictest0-21swapper/300:20:133
4511992019,1cyclictest0-21swapper/300:17:103
4511992019,1cyclictest0-21swapper/300:17:093
451199201,2cyclictest0-21swapper/322:42:453
451199200,19cyclictest0-21swapper/323:49:403
450799200,3cyclictest0-21swapper/221:33:562
450299208,6cyclictest5212-21ssh23:10:000
153272200,1chrt0-21swapper/321:59:013
451199194,4cyclictest39-21ksoftirqd/322:57:183
451199193,13cyclictest29497-21diskmemload21:21:213
451199192,7cyclictest141rcu_preempt22:54:373
4511991917,1cyclictest39-21ksoftirqd/322:09:263
4511991915,1cyclictest141rcu_preempt21:53:583
450799190,19cyclictest0-21swapper/223:33:362
450799190,19cyclictest0-21swapper/223:08:012
450799190,17cyclictest0-21swapper/222:47:152
4504991918,1cyclictest0-21swapper/121:05:201
451199185,2cyclictest141rcu_preempt23:15:583
451199182,9cyclictest39-21ksoftirqd/322:22:343
4511991817,1cyclictest39-21ksoftirqd/322:25:213
4511991811,4cyclictest39-21ksoftirqd/321:05:003
450799180,18cyclictest0-21swapper/222:19:232
450799180,17cyclictest0-21swapper/223:17:502
450799180,16cyclictest0-21swapper/221:38:222
4504991817,1cyclictest25-21ksoftirqd/120:40:001
4504991817,1cyclictest25-21ksoftirqd/120:40:001
4504991817,1cyclictest11811-21kworker/1:122:01:081
285022180,1sleep339-21ksoftirqd/323:04:283
4511991714,2cyclictest39-21ksoftirqd/322:15:163
451199171,3cyclictest141rcu_preempt23:14:333
4511991713,3cyclictest7322-21kworker/u8:222:47:453
451199171,1cyclictest18514-21cut22:00:203
451199170,2cyclictest141rcu_preempt23:26:053
450499172,1cyclictest141rcu_preempt23:22:261
4504991714,2cyclictest0-21swapper/123:47:451
450499171,13cyclictest141rcu_preempt23:07:301
451199167,3cyclictest39-21ksoftirqd/322:12:323
451199163,4cyclictest39-21ksoftirqd/321:37:063
4511991611,4cyclictest39-21ksoftirqd/322:35:123
4511991611,2cyclictest39-21ksoftirqd/321:27:253
450799160,16cyclictest0-21swapper/223:29:592
450799160,15cyclictest0-21swapper/223:46:082
450799160,15cyclictest0-21swapper/221:17:522
450499169,1cyclictest141rcu_preempt00:35:101
4504991616,0cyclictest0-21swapper/122:26:011
4504991615,1cyclictest38150irq/131-snd_hda22:08:291
4504991615,1cyclictest12631-21kworker/u8:022:19:011
450499160,16cyclictest29497-21diskmemload23:50:451
451199159,5cyclictest39-21ksoftirqd/300:10:173
4511991512,3cyclictest39-21ksoftirqd/300:32:323
4511991511,3cyclictest39-21ksoftirqd/321:15:213
451199150,4cyclictest0-21swapper/323:42:263
4507991513,2cyclictest32-21ksoftirqd/200:11:592
450799150,15cyclictest0-21swapper/222:28:582
450499157,1cyclictest141rcu_preempt23:15:161
450499154,3cyclictest25-21ksoftirqd/100:14:441
450499154,0cyclictest141rcu_preempt21:29:221
4504991513,1cyclictest25-21ksoftirqd/100:22:361
450499150,1cyclictest0-21swapper/121:47:061
4502991510,3cyclictest27727-21df00:20:150
4502991510,1cyclictest0-21swapper/023:11:130
450299150,14cyclictest0-21swapper/023:30:040
451199149,2cyclictest39-21ksoftirqd/321:32:293
451199145,6cyclictest39-21ksoftirqd/323:21:063
4507991413,1cyclictest29497-21diskmemload21:58:262
4507991410,3cyclictest23017-21grep19:45:242
4507991410,2cyclictest23873-21grep19:50:152
450499143,1cyclictest0-21swapper/121:30:431
4504991412,1cyclictest25-21ksoftirqd/100:30:291
4504991410,3cyclictest15179-21ssh23:35:021
4504991410,3cyclictest13979-21wc22:55:191
450499140,2cyclictest0-21swapper/121:56:451
4502991411,2cyclictest6193-21ssh00:07:180
4502991411,2cyclictest18999-21grep23:55:260
450299140,1cyclictest0-21swapper/021:41:240
450299140,1cyclictest0-21swapper/021:17:320
450299140,13cyclictest0-21swapper/022:22:310
451199139,3cyclictest5441-21basename20:20:013
451199139,3cyclictest30016-21wc20:00:253
451199139,3cyclictest26773-21cut19:55:173
451199139,3cyclictest15970-21sed20:40:173
451199139,2cyclictest39-21ksoftirqd/300:00:063
451199139,2cyclictest24444-21cron21:00:003
451199133,1cyclictest141rcu_preempt23:30:513
4511991310,2cyclictest7352-21grep20:20:263
4511991310,2cyclictest39-21ksoftirqd/323:59:133
4511991310,2cyclictest32236-21munin-run21:50:003
4511991310,2cyclictest18358-21sed20:45:163
4511991310,1cyclictest39-21ksoftirqd/300:05:213
450799139,3cyclictest23061-21tr21:25:132
450799134,3cyclictest225-21systemd-journal20:15:002
4507991313,0cyclictest0-21swapper/220:45:122
4507991310,2cyclictest7833-21basename20:25:002
4507991310,2cyclictest4332-21wc20:15:182
4507991310,2cyclictest30805-21grep21:10:162
4507991310,2cyclictest28301-21cron20:00:002
4507991310,2cyclictest24557-21chrt23:20:262
4507991310,2cyclictest20128-21grep19:40:202
4507991310,2cyclictest16230-21grep23:35:162
450799130,12cyclictest0-21swapper/222:20:152
450499139,3cyclictest31267-21uname21:10:191
4504991312,1cyclictest29497-21diskmemload21:20:101
4504991311,1cyclictest5057-21cut22:50:171
450499130,12cyclictest8723-21grep19:15:251
450299139,3cyclictest6477-21cron23:30:000
450299139,3cyclictest29002-21grep21:05:250
450299139,3cyclictest26186-21sed21:45:240
450299139,3cyclictest22296-21turbostat.cron21:25:000
450299139,3cyclictest20629-21grep19:40:280
450299139,3cyclictest18745-21munin-run19:40:000
4502991310,2cyclictest373-21grep21:50:120
4502991310,2cyclictest30751-21cron21:29:590
4502991310,2cyclictest3035-21grep22:30:130
4502991310,2cyclictest27372-21grep19:55:220
4502991310,2cyclictest26441-21sed00:00:200
4502991310,2cyclictest25247-21tr23:40:200
4502991310,2cyclictest16599-21cron22:00:000
4502991310,2cyclictest10765-21ssh23:50:360
450299130,2cyclictest0-21swapper/022:07:390
450299130,12cyclictest1687-21head20:10:160
451199129,2cyclictest4021-21tr00:25:163
451199129,2cyclictest21189-21sed20:50:233
451199129,2cyclictest17177-21grep19:35:153
451199129,2cyclictest14966-21tail19:30:153
451199123,5cyclictest381rcuc/320:30:203
451199120,1cyclictest0-21swapper/323:38:203
450799129,2cyclictest9339-21grep23:50:172
450799129,2cyclictest8680-21sort20:25:162
450799129,2cyclictest3759-21head00:05:282
450799129,2cyclictest3406-21sed22:30:142
450799129,2cyclictest30220-21grep20:00:252
450799129,2cyclictest20017-21grep22:40:162
450799129,2cyclictest18370-21cut23:55:212
450799129,2cyclictest1585-21ssh22:10:132
450799129,2cyclictest14977-21grep19:30:152
450799129,2cyclictest13670-21head00:30:252
450799129,2cyclictest12151-21tr19:25:152
450799128,3cyclictest25356-21wc23:40:202
450799128,3cyclictest22062-21grep23:00:162
450799128,3cyclictest20680-21grep20:50:172
450799124,4cyclictest225-21systemd-journal00:40:002
4507991210,1cyclictest14681-21grep22:55:252
4507991210,1cyclictest0-21swapper/200:15:022
4507991210,1cyclictest0-21swapper/200:15:022
450799120,3cyclictest0-21swapper/222:37:522
450799120,2cyclictest0-21swapper/222:06:182
450799120,1cyclictest0-21swapper/221:53:302
450799120,0cyclictest0-21swapper/221:43:532
450499129,2cyclictest25770-21wc21:00:191
450499129,2cyclictest2424-21grep22:10:171
450499129,2cyclictest22092-21awk20:55:001
450499129,2cyclictest20474-21cut22:40:181
450499129,2cyclictest19465-21sh22:20:181
450499124,4cyclictest225-21systemd-journal19:30:001
450499124,2cyclictest225-21systemd-journal20:25:001
450499122,3cyclictest23010-21ssh00:17:101
450499122,3cyclictest23010-21ssh00:17:101
4504991211,1cyclictest0-21swapper/122:37:461
450499121,10cyclictest29497-21diskmemload00:02:181
4504991210,1cyclictest5015-21cut19:10:141
450499120,1cyclictest0-21swapper/123:42:001
450499120,11cyclictest7084-21ls23:10:221
450499120,11cyclictest4370-21wc22:30:211
450299129,2cyclictest9560-21cat20:25:250
450299129,2cyclictest3696-21wc22:50:000
450299129,2cyclictest28843-21grep20:00:130
450299129,2cyclictest27089-21tr22:25:130
450299129,2cyclictest23806-21tr19:50:140
450299129,2cyclictest23497-21cron19:50:000
450299129,2cyclictest21729-21grep22:40:280
450299129,2cyclictest14450-21sed20:35:260
450299129,2cyclictest14450-21sed20:35:260
450299129,2cyclictest13006-21cat22:35:250
450299128,3cyclictest9492-21ssh22:15:000
450299128,3cyclictest20457-21grep20:50:150
450299128,3cyclictest18737-21ssh22:58:290
4502991210,1cyclictest14002-21perf19:29:590
451199118,3cyclictest39-21ksoftirqd/319:10:163
451199118,2cyclictest9546-21tr19:20:143
451199118,2cyclictest29506-21sh23:05:053
451199118,2cyclictest22485-21tail19:45:183
451199118,2cyclictest18982-21chrt19:40:043
451199111,7cyclictest39-21ksoftirqd/321:40:133
451199110,11cyclictest0-21swapper/320:35:133
451199110,11cyclictest0-21swapper/320:35:133
450799118,2cyclictest5359-21grep19:10:172
450799118,2cyclictest4278-21ssh22:50:092
450799118,2cyclictest307-21taskset20:06:102
450799118,2cyclictest24357-21taskset20:58:552
450799118,2cyclictest24123-21sh21:45:062
450799118,2cyclictest17975-21sed22:00:162
450799118,2cyclictest16595-21taskset19:35:102
450799118,2cyclictest15154-21sort21:20:152
450799118,2cyclictest14907-21taskset20:39:492
450799118,2cyclictest14907-21taskset20:39:492
450799117,3cyclictest29610-21ls21:10:012
450499118,2cyclictest8333-21tr20:25:141
450499118,2cyclictest8317-21ssh23:30:211
450499118,2cyclictest24761-21cat19:50:201
450499118,2cyclictest19160-21wget19:40:151
450499118,2cyclictest1168-21grep20:10:141
450499118,1cyclictest0-21swapper/121:36:331
450499117,3cyclictest31162-21egrep20:05:141
450499117,3cyclictest25-21ksoftirqd/100:06:281
450499117,3cyclictest1688-21grep21:50:191
450499114,4cyclictest225-21systemd-journal20:35:011
450499113,4cyclictest225-21systemd-journal20:55:001
450499110,1cyclictest29497-21diskmemload22:48:511
450299118,2cyclictest7440-21ssh21:35:100
450299118,2cyclictest651-21sh20:10:010
450299118,2cyclictest6350-21grep19:10:260
450299118,2cyclictest5159-21sh22:50:160
450299118,2cyclictest4525-21sh00:25:200
450299118,2cyclictest32275-21ssh23:45:020
450299118,2cyclictest28988-21ssh23:23:480
450299118,2cyclictest27454-21ssh23:03:420
450299118,2cyclictest25851-21cut21:00:190
450299118,2cyclictest2347-21ssh21:12:350
450299118,2cyclictest22202-21ssh00:16:330
450299118,2cyclictest22202-21ssh00:16:330
450299118,2cyclictest20383-21cut00:35:120
450299118,2cyclictest19858-21ssh23:37:190
450299118,2cyclictest18963-21ssh23:17:380
450299118,2cyclictest18821-21cat22:00:250
450299118,2cyclictest11919-21grep22:15:250
450299118,2cyclictest11597-21sh00:10:210
450299118,2cyclictest11562-21rm00:30:060
450299118,2cyclictest10236-21grep19:20:180
450299113,7cyclictest9241-21munin-run19:20:000
450299111,8cyclictest31498-21ssh21:30:130
451199108,1cyclictest28800-21wc21:05:243
451199108,1cyclictest1832-21cat20:10:183
451199108,1cyclictest137702sleep319:27:183
451199107,2cyclictest9304-21tr20:25:213
451199107,2cyclictest8761-21tail19:15:253
451199100,7cyclictest537-21snmpd20:05:103
450799108,1cyclictest9537-21cut19:20:142
450799108,1cyclictest3226-21cut00:25:132
450799108,1cyclictest16676-21sed20:40:252
450799107,2cyclictest7332-21grep19:15:132
450799107,2cyclictest12447-21taskset20:33:412
450499109,1cyclictest6787-21head21:15:171
450499109,1cyclictest19808-21cat20:50:001
450499108,2cyclictest5553-21ssh00:25:411
450499108,2cyclictest32418-21grep23:25:211
450499108,2cyclictest30329-21who20:00:261
450499108,1cyclictest9976-21cat19:20:151
450499108,1cyclictest3396-21cut20:15:131
450499108,1cyclictest16977-21wc21:40:181
450499108,1cyclictest12956-21cat19:25:201
450499107,2cyclictest23321-21seq19:47:471
450499104,2cyclictest0-21swapper/123:55:131
450299108,1cyclictest5954-21cut20:20:140
450299108,1cyclictest19640-21seq20:49:210
450299108,1cyclictest14692-21expr19:30:130
450299107,2cyclictest3529-21tr20:15:130
450299107,2cyclictest22409-21grep20:55:130
45119996,2cyclictest24321-21grep19:50:173
45079997,1cyclictest25079-21expr21:00:142
45049998,1cyclictest26565-21cat19:55:151
45049997,1cyclictest16058-21cat20:40:191
45049991,4cyclictest14123-21kworker/1:219:35:181
45029997,1cyclictest15550-21expr20:40:160
45029996,2cyclictest10471-21aten2_r7power_c20:30:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional