You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 13:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue May 26, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
361129470,18sleep20-21swapper/219:05:232
406329178,9sleep30-21swapper/319:05:563
418129078,8sleep00-21swapper/019:07:270
417728164,13sleep10-21swapper/119:07:231
265442720,0sleep00-21swapper/000:20:170
99362550,1sleep20-21swapper/219:20:142
109842500,0sleep10-21swapper/121:17:381
446799280,1cyclictest0-21swapper/122:36:271
446799270,22cyclictest0-21swapper/123:58:001
446799240,3cyclictest0-21swapper/121:31:521
447299230,1cyclictest0-21swapper/323:40:503
447299230,1cyclictest0-21swapper/322:14:493
446999230,22cyclictest0-21swapper/200:20:212
446799230,4cyclictest0-21swapper/122:25:131
446799230,1cyclictest0-21swapper/122:07:381
4463992322,1cyclictest0-21swapper/021:52:480
447299220,3cyclictest0-21swapper/300:24:093
446399220,1cyclictest0-21swapper/000:06:250
446999210,19cyclictest0-21swapper/223:27:112
446799211,2cyclictest0-21swapper/121:10:121
446799210,1cyclictest0-21swapper/100:20:131
447299201,1cyclictest0-21swapper/321:46:593
446799200,19cyclictest0-21swapper/123:44:531
447299190,7cyclictest0-21swapper/321:35:183
447299190,1cyclictest0-21swapper/323:11:413
446799190,18cyclictest13348-21needreboot22:55:191
447299185,12cyclictest15442-21apache223:32:373
447299180,2cyclictest0-21swapper/323:15:533
447299180,16cyclictest0-21swapper/322:07:383
446799180,2cyclictest0-21swapper/123:13:371
446399185,7cyclictest7131-21awk23:50:000
446399180,2cyclictest0-21swapper/022:12:450
446399180,1cyclictest0-21swapper/000:39:550
311952180,0sleep30-21swapper/322:27:313
447299175,7cyclictest29731-21munin-run21:10:003
447299170,17cyclictest0-21swapper/322:17:353
446799170,8cyclictest0-21swapper/100:05:321
446799170,4cyclictest0-21swapper/100:04:281
446799170,16cyclictest0-21swapper/100:14:451
446399170,2cyclictest0-21swapper/021:20:280
446399170,14cyclictest0-21swapper/021:15:150
447299161,0cyclictest0-21swapper/300:18:033
447299160,5cyclictest0-21swapper/323:01:323
447299160,4cyclictest0-21swapper/323:49:573
447299160,1cyclictest0-21swapper/323:20:043
447299160,1cyclictest0-21swapper/321:41:383
447299160,12cyclictest0-21swapper/319:50:053
446999163,6cyclictest0-21swapper/200:19:442
4469991614,1cyclictest0-21swapper/223:58:012
446999160,1cyclictest0-21swapper/200:35:022
446999160,15cyclictest0-21swapper/223:16:162
446999160,15cyclictest0-21swapper/200:05:082
446799165,10cyclictest1771-21apt-get00:25:141
446799162,3cyclictest141rcu_preempt22:53:021
4467991615,1cyclictest0-21swapper/122:18:001
4467991615,1cyclictest0-21swapper/121:23:371
4467991614,1cyclictest10691-21kworker/1:221:39:101
446799160,2cyclictest0-21swapper/119:50:041
446799160,14cyclictest0-21swapper/122:49:471
446799160,13cyclictest29733-21diskmemload23:04:271
4463991613,3cyclictest13-21ksoftirqd/000:18:030
446399161,1cyclictest0-21swapper/023:10:240
4472991510,2cyclictest39-21ksoftirqd/300:35:223
447299150,1cyclictest0-21swapper/322:02:173
447299150,1cyclictest0-21swapper/300:04:283
446999151,8cyclictest225-21systemd-journal22:49:172
4469991512,2cyclictest455-21dbus-daemon23:39:592
4469991510,2cyclictest0-21swapper/221:30:452
446999150,5cyclictest0-21swapper/219:37:422
4467991514,1cyclictest0-21swapper/121:46:511
4467991511,4cyclictest1286-21ssh21:50:181
446799150,2cyclictest0-21swapper/123:54:521
446799150,1cyclictest0-21swapper/123:46:511
446799150,1cyclictest0-21swapper/121:27:301
446799150,12cyclictest29733-21diskmemload22:02:371
4463991515,0cyclictest0-21swapper/022:37:310
4463991515,0cyclictest0-21swapper/021:45:360
4463991511,3cyclictest8032-21idleruntime-cro20:25:000
446399150,1cyclictest0-21swapper/023:28:110
446399150,14cyclictest0-21swapper/022:19:460
447299141,6cyclictest0-21swapper/321:18:363
4472991412,1cyclictest0-21swapper/300:33:253
4472991412,1cyclictest0-21swapper/300:33:243
447299140,2cyclictest0-21swapper/323:05:123
447299140,1cyclictest0-21swapper/322:54:573
447299140,0cyclictest0-21swapper/322:57:223
446999145,8cyclictest18859-21head20:45:182
4469991411,2cyclictest1864-21tail20:10:172
4469991410,3cyclictest32078-21wc21:50:002
446999140,1cyclictest0-21swapper/221:18:312
446999140,14cyclictest0-21swapper/221:23:342
4467991413,1cyclictest0-21swapper/123:21:401
446799141,1cyclictest20084-21sshd23:19:081
4467991410,4cyclictest0-21swapper/121:55:321
446799140,6cyclictest0-21swapper/122:24:551
446799140,13cyclictest0-21swapper/100:36:181
446399144,5cyclictest225-21systemd-journal00:35:010
446399144,5cyclictest225-21systemd-journal00:35:000
4463991414,0cyclictest0-21swapper/022:51:320
4463991413,1cyclictest0-21swapper/000:29:340
446399141,2cyclictest0-21swapper/022:21:070
4463991410,2cyclictest4930-21grep19:10:130
447299139,3cyclictest6131-21ssh21:34:283
447299137,4cyclictest30214-21apt-get21:10:133
4472991310,2cyclictest12887-21cat20:35:003
447299130,8cyclictest7227-31find00:09:563
447299130,5cyclictest0-21swapper/321:51:463
447299130,1cyclictest0-21swapper/322:39:233
446999139,3cyclictest7277-21grep23:30:182
446999139,3cyclictest30252-21tr23:05:192
4469991310,2cyclictest8698-21ssh00:10:042
4469991310,2cyclictest8685-21grep19:15:282
4469991310,2cyclictest8301-21sort23:50:162
4469991310,2cyclictest5673-21grep22:50:252
4469991310,2cyclictest28399-21ssh22:25:262
4469991310,2cyclictest21456-21grep22:40:262
4469991310,2cyclictest12130-21tr19:25:162
4469991310,2cyclictest1168-21grep23:45:262
4469991310,2cyclictest10590-21cat22:35:012
446799132,5cyclictest29733-21diskmemload23:05:061
446799131,11cyclictest38150irq/131-snd_hda21:41:521
446799130,3cyclictest5980-21/usr/sbin/munin23:30:181
446799130,10cyclictest0-21swapper/100:17:341
446399139,2cyclictest3265-21sleep20:15:010
446399139,2cyclictest16216-21grep20:40:150
4463991310,2cyclictest28620-21cat20:00:010
4463991310,2cyclictest24421-21tr21:25:230
447299129,2cyclictest8494-21wc19:15:263
447299129,2cyclictest29106-21grep22:45:213
447299129,2cyclictest25699-21expr21:00:163
447299129,2cyclictest21529-21sshd23:58:083
447299129,2cyclictest15143-21cron20:40:003
447299129,2cyclictest14337-21ssh21:20:023
447299129,2cyclictest11333-21ssh23:51:473
447299129,2cyclictest10512-21awk00:10:213
447299128,3cyclictest22749-21ssh22:22:273
447299127,3cyclictest29848-21sh23:25:023
447299121,3cyclictest29733-21diskmemload22:41:323
447299120,1cyclictest0-21swapper/321:55:313
446999129,2cyclictest4192-21cut20:15:172
446999129,2cyclictest31443-21tr20:05:162
446999129,2cyclictest30969-21grep21:10:162
446999129,2cyclictest28286-21grep21:05:172
446999129,2cyclictest2594-21uname22:10:192
446999129,2cyclictest25153-21wc21:00:002
446999129,2cyclictest23369-21tr23:40:162
446999129,2cyclictest208142sleep221:42:512
446999129,2cyclictest19014-21sed22:20:182
446999129,2cyclictest17889-21tail22:00:162
446999129,2cyclictest17149-21sed20:40:242
446999129,2cyclictest15535-21wc19:30:202
446999129,2cyclictest13271-21tr20:35:162
446999129,2cyclictest10525-21head22:15:162
446999128,3cyclictest7541-21sed20:20:262
446999128,3cyclictest14615-21chrt22:55:472
446999125,6cyclictest13076-21ssh21:57:222
446999120,2cyclictest0-21swapper/223:13:342
446799129,2cyclictest6009-21basename20:20:151
446799129,2cyclictest20894-21grep19:40:251
446799129,2cyclictest15925-21tail19:30:261
446799124,4cyclictest29733-21diskmemload22:30:331
446799122,6cyclictest29742-21awk21:10:001
4467991210,1cyclictest32581-21sensors_fan20:05:231
446399129,2cyclictest9362-21grep21:55:190
446399129,2cyclictest8162-21grep21:35:170
446399129,2cyclictest29773-21grep22:45:260
446399129,2cyclictest29308-21ssh23:05:140
446399129,2cyclictest28479-21grep22:25:270
446399129,2cyclictest2754-21tr22:30:140
446399129,2cyclictest27452-21wc21:05:000
446399129,2cyclictest23886-21grep23:40:180
446399129,2cyclictest21860-21ssh23:20:130
446399129,2cyclictest21716-21wc20:50:230
446399129,2cyclictest18226-21tr20:45:140
446399128,3cyclictest28477-21sh23:05:010
446399128,3cyclictest20118-21sleep23:00:000
446399128,2cyclictest29601-21grep20:00:170
446399126,2cyclictest22857-21aten2_r7power_c20:55:140
446399123,2cyclictest21208-21/usr/sbin/munin22:40:260
446399122,5cyclictest225-21systemd-journal23:39:590
4463991210,1cyclictest9326-21ssh00:10:130
447299119,2cyclictest26845-21rm21:26:533
447299119,1cyclictest5473-21chrt20:18:013
447299119,1cyclictest30434-21grep20:00:263
447299119,1cyclictest26455-21cut19:55:133
447299119,1cyclictest23580-21taskset19:48:053
447299119,1cyclictest19678-21expr19:40:143
447299119,1cyclictest18164-21aten2_r7power_p20:45:153
447299119,1cyclictest16827-21taskset19:35:073
447299119,1cyclictest10773-21ps19:20:213
447299119,1cyclictest10420-21sh20:30:003
447299118,2cyclictest3636-21grep00:25:273
447299118,2cyclictest15697-21sort20:40:123
447299118,2cyclictest1534-21expr20:10:153
447299118,2cyclictest15300-21needreboot19:30:183
447299118,2cyclictest14167-21ssh23:35:013
447299118,2cyclictest11919-21tr19:25:163
447299116,3cyclictest0-21swapper/322:31:283
447299110,11cyclictest7010-21cat20:20:203
447299110,11cyclictest24659-21seq20:56:283
446999118,2cyclictest8540-21tr20:25:162
446999118,2cyclictest8191-21sed21:35:172
446999118,2cyclictest3870-21ssh00:25:322
446999118,2cyclictest325472sleep221:50:092
446999118,2cyclictest26868-21ssh00:00:442
446999118,2cyclictest26698-21ssh21:26:462
446999118,2cyclictest25239-21ssh22:05:052
446999118,2cyclictest23666-21grep23:20:252
446999118,2cyclictest21088-21sort20:50:172
446999118,2cyclictest15232-21ssh00:32:482
446999118,2cyclictest15232-21ssh00:32:482
446999118,2cyclictest10970-21ssh22:35:102
446999115,2cyclictest225-21systemd-journal23:01:422
446999110,1cyclictest0-21swapper/220:30:452
446799119,2cyclictest26725-21cat21:00:271
446799119,1cyclictest4752-21aten2_r7power_e19:10:121
446799119,1cyclictest215992sleep119:45:091
446799119,1cyclictest19541-21cat20:45:251
446799119,1cyclictest17160-21grep20:40:241
446799119,1cyclictest12836-21wc19:25:201
446799119,1cyclictest12165-21cat20:30:251
446799119,1cyclictest10965-21head00:30:171
446799119,1cyclictest10965-21head00:30:161
446799118,2cyclictest6283-21ssh22:12:421
446799118,2cyclictest29840-21ssh23:25:021
446799110,8cyclictest0-21swapper/123:35:031
446799110,5cyclictest0-21swapper/120:25:151
446799110,5cyclictest0-21swapper/119:55:131
446799110,10cyclictest28912-21cut20:00:141
446399118,2cyclictest837-21sh22:10:000
446399118,2cyclictest7904-21cut23:50:140
446399118,2cyclictest7024-21ssh23:30:170
446399118,2cyclictest5322-21taskset20:16:060
446399118,2cyclictest31336-21grep20:05:140
446399118,2cyclictest31193-21ssh21:30:060
446399118,2cyclictest24758-21grep00:00:150
446399118,2cyclictest21511-21cat19:45:010
446399118,2cyclictest19793-21ssh22:01:010
446399118,2cyclictest13029-21ssh23:15:020
446399118,2cyclictest10173-21grep19:20:160
446399117,2cyclictest13277-21tr20:35:160
446399113,3cyclictest7075-21apache_accesses19:15:130
446399110,7cyclictest12562-21sshd20:32:180
446399110,1cyclictest0-21swapper/021:10:070
446399110,10cyclictest17607-21ssh23:55:240
446399110,10cyclictest13922-21taskset19:29:460
447299108,1cyclictest4961-21tr19:10:133
447299108,1cyclictest31545-21expr20:05:163
447299108,1cyclictest21009-21df20:50:163
446999108,1cyclictest5054-21expr19:10:132
446999108,1cyclictest30292-21awk20:00:262
446999108,1cyclictest25631-21cat19:50:262
446999108,1cyclictest25352-21hostname21:00:152
446999107,2cyclictest19650-21cat19:40:142
446999106,3cyclictest26676-21tr19:55:142
446799108,1cyclictest8615-21wc19:15:271
446799108,1cyclictest5249-21head20:15:271
446799108,1cyclictest22877-21awk20:55:141
446799108,1cyclictest21165-21tr20:50:191
446799108,1cyclictest186942sleep119:36:351
446799107,2cyclictest14385-21tr20:35:231
446799106,3cyclictest20204-21grep22:40:171
446799100,9cyclictest0-21swapper/119:20:121
446399108,1cyclictest29391-21chrt21:05:580
446399108,1cyclictest15979-21grep21:40:150
446399108,1cyclictest10075-21chrt20:25:510
446399105,3cyclictest1142-21runrttasks19:35:130
446399100,9cyclictest14227-21basename19:30:130
446399100,6cyclictest25848-21sshd19:51:090
44699997,1cyclictest23399-21chrt19:45:462
44679996,2cyclictest1768-21cut20:10:171
44639990,3cyclictest0-21swapper/019:45:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional