You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 14:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Jun 09, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
287929886,8sleep00-21swapper/019:08:500
226928864,19sleep20-21swapper/219:05:272
273428764,18sleep10-21swapper/119:06:581
291428673,9sleep30-21swapper/319:09:183
305899260,0cyclictest0-21swapper/122:17:001
305399260,1cyclictest0-21swapper/022:52:180
306399250,0cyclictest0-21swapper/322:30:053
305399250,25cyclictest0-21swapper/021:47:370
306399241,23cyclictest0-21swapper/323:19:173
306099240,21cyclictest0-21swapper/221:53:242
3063992321,1cyclictest0-21swapper/319:15:123
306399230,2cyclictest0-21swapper/300:33:393
3063992222,0cyclictest0-21swapper/323:01:093
3058992222,0cyclictest0-21swapper/100:00:131
305899220,22cyclictest0-21swapper/120:17:221
306099210,17cyclictest0-21swapper/222:00:512
3058992118,3cyclictest0-21swapper/122:30:031
3053992121,0cyclictest0-21swapper/021:41:370
3053992120,1cyclictest0-21swapper/023:10:400
306399200,3cyclictest0-21swapper/322:40:453
306399200,1cyclictest0-21swapper/322:22:143
306099200,16cyclictest0-21swapper/222:44:512
305899200,1cyclictest141rcu_preempt22:23:031
305899200,1cyclictest0-21swapper/121:30:441
305399209,9cyclictest15019-21ssh22:20:140
3063991917,2cyclictest0-21swapper/322:07:203
306399190,2cyclictest0-21swapper/322:58:333
306399190,1cyclictest0-21swapper/323:39:393
306399190,18cyclictest0-21swapper/321:17:303
3060991914,4cyclictest0-21swapper/221:43:022
306099190,2cyclictest141rcu_preempt21:47:562
305899192,1cyclictest141rcu_preempt00:12:221
3058991917,1cyclictest0-21swapper/123:20:141
3058991914,4cyclictest0-21swapper/123:35:011
305899191,1cyclictest28486-21diskmemload22:54:521
305899190,19cyclictest0-21swapper/119:20:121
305399190,1cyclictest0-21swapper/000:33:230
306399180,1cyclictest0-21swapper/323:48:043
306399180,1cyclictest0-21swapper/322:01:523
306099180,17cyclictest0-21swapper/223:42:542
306099180,17cyclictest0-21swapper/200:36:172
305899188,5cyclictest225-21systemd-journal23:04:591
305399187,6cyclictest22287-21munin-run19:50:000
3053991818,0cyclictest0-21swapper/021:16:590
305399180,1cyclictest0-21swapper/022:17:340
79412170,0sleep10-21swapper/123:33:531
3063991716,1cyclictest55750irq/128-enp2s0-00:37:453
306399170,0cyclictest0-21swapper/300:05:303
3060991714,3cyclictest0-21swapper/223:02:422
3060991713,4cyclictest0-21swapper/222:57:552
306099170,17cyclictest0-21swapper/200:34:372
305899174,12cyclictest29801-21aten2_r7power_e20:05:171
305399171,15cyclictest28486-21diskmemload22:37:490
3063991616,0cyclictest0-21swapper/322:52:383
306399160,3cyclictest0-21swapper/321:05:133
3060991613,3cyclictest0-21swapper/222:28:222
3060991610,5cyclictest6604-21cstates21:55:132
3060991610,5cyclictest6604-21cstates21:55:132
306099160,16cyclictest0-21swapper/223:05:532
306099160,12cyclictest0-21swapper/223:37:472
305899167,7cyclictest23994-21fschecks_time22:25:151
305899164,8cyclictest22171-21/usr/sbin/munin21:45:171
3058991615,1cyclictest27424-21kworker/u8:122:09:271
3058991615,1cyclictest0-21swapper/121:52:261
3058991614,1cyclictest25-21ksoftirqd/122:04:221
3058991614,1cyclictest25-21ksoftirqd/121:12:491
3058991614,1cyclictest0-21swapper/123:12:551
305899161,1cyclictest28486-21diskmemload21:36:181
306399150,15cyclictest0-21swapper/300:23:293
306099154,8cyclictest11553-21df_abs23:55:142
3060991514,1cyclictest0-21swapper/223:32:192
306099151,14cyclictest0-21swapper/219:48:492
3060991510,4cyclictest28486-21diskmemload21:15:132
306099150,3cyclictest25648-21tail19:55:172
306099150,14cyclictest12480-21fschecks_time20:35:162
3058991513,1cyclictest25-21ksoftirqd/123:51:461
3058991513,1cyclictest25-21ksoftirqd/122:57:031
3058991510,4cyclictest3833-21df19:10:161
305899150,15cyclictest0-21swapper/100:34:331
305399155,3cyclictest225-21systemd-journal20:15:150
305399150,5cyclictest0-21swapper/000:08:290
305399150,2cyclictest0-21swapper/023:34:410
305399150,2cyclictest0-21swapper/019:30:120
3063991412,1cyclictest0-21swapper/323:33:023
3063991411,2cyclictest5893-21wc20:20:233
306399141,11cyclictest0-21swapper/321:47:583
3063991410,3cyclictest4184-21cron20:20:013
3063991410,3cyclictest23239-21grep20:55:263
306099145,5cyclictest225-21systemd-journal21:40:002
306099141,4cyclictest141rcu_preempt23:22:442
306099141,1cyclictest7632-21ssh23:14:232
3060991411,3cyclictest0-21swapper/223:48:172
3060991411,3cyclictest0-21swapper/223:27:132
3060991411,3cyclictest0-21swapper/223:27:122
3060991410,3cyclictest30454-21grep21:10:242
305899146,8cyclictest38150irq/131-snd_hda21:27:331
305899140,1cyclictest0-21swapper/100:20:331
305399144,5cyclictest16335-21perf20:45:000
3053991413,1cyclictest0-21swapper/022:01:300
3053991411,2cyclictest13-21ksoftirqd/023:45:110
3053991411,2cyclictest13-21ksoftirqd/021:20:370
3053991410,2cyclictest17621-21wc23:20:150
305399140,4cyclictest0-21swapper/022:40:160
305399140,1cyclictest0-21swapper/022:48:170
305399140,0cyclictest0-21swapper/021:35:320
305399140,0cyclictest0-21swapper/000:35:480
305399140,0cyclictest0-21swapper/000:01:300
306399139,3cyclictest844-21tr23:10:143
306399139,3cyclictest13412-21cron21:40:003
3063991310,2cyclictest8517-21grep22:15:243
3063991310,2cyclictest25656-21tail23:05:183
3063991310,2cyclictest24801-21head22:45:173
3063991310,2cyclictest12694-21expr00:15:163
306399130,2cyclictest0-21swapper/300:00:393
306399130,1cyclictest0-21swapper/323:22:593
306399130,1cyclictest0-21swapper/300:25:483
306099139,3cyclictest22034-21sort21:25:152
306099139,3cyclictest16745-21cron23:20:002
306099139,2cyclictest439-21grep20:10:182
306099139,2cyclictest0-21swapper/221:22:042
306099131,4cyclictest0-21swapper/220:40:142
3060991310,3cyclictest0-21swapper/222:06:232
3060991310,2cyclictest670-21cut22:50:162
3060991310,2cyclictest30982-21ssh00:06:192
3060991310,2cyclictest12338-21grep00:15:152
305899135,7cyclictest25-21ksoftirqd/123:05:161
3058991311,1cyclictest25-21ksoftirqd/122:41:511
3058991310,2cyclictest26524-21cpuspeed_turbos21:05:141
3058991310,2cyclictest24162-21tr21:00:131
3058991310,2cyclictest23854-21wc21:00:001
3058991310,2cyclictest21245-21cron00:40:001
3058991310,2cyclictest1342-21grep20:10:271
305399139,3cyclictest11794-21wc00:14:590
305399132,9cyclictest32502-21basename23:10:000
305399131,6cyclictest18698-21munin-run20:50:000
305399131,11cyclictest31583-21apache_volume22:30:120
3053991310,2cyclictest28456-21grep20:00:190
3053991310,2cyclictest28456-21grep20:00:180
3053991310,2cyclictest22973-21tr22:05:130
3053991310,2cyclictest11325-21wc23:35:210
3053991310,1cyclictest0-21swapper/022:29:290
305399130,1cyclictest0-21swapper/022:11:510
305399130,0cyclictest0-21swapper/023:27:400
305399130,0cyclictest0-21swapper/023:27:400
306399129,2cyclictest3977-21grep23:50:213
306399129,2cyclictest30520-21grep21:30:173
306399129,2cyclictest27141-21awk23:25:223
306399129,2cyclictest27141-21awk23:25:223
306399129,2cyclictest25693-21cut19:55:173
306399129,2cyclictest25031-21cat21:00:193
306399129,2cyclictest22191-21sed21:25:163
306399129,2cyclictest17385-21cat19:40:003
306399129,2cyclictest15154-21perf22:39:593
306399129,2cyclictest14003-21cron22:00:003
306399129,2cyclictest14003-21cron22:00:003
306399129,2cyclictest13888-21head21:20:163
306399129,2cyclictest12078-21tr20:35:153
306399128,3cyclictest5863-21sed00:10:263
306399120,11cyclictest10667-21tr19:25:153
306099129,2cyclictest8074-21cut22:35:142
306099129,2cyclictest6408-21head19:15:162
306099129,2cyclictest5895-21grep00:10:262
306099129,2cyclictest31835-21sed22:10:162
306099129,2cyclictest25633-21sed21:00:262
306099129,2cyclictest18392-21sort19:40:162
306099129,2cyclictest17311-21systemctl19:40:002
306099129,2cyclictest16849-21grep20:45:142
306099129,2cyclictest11219-21head19:25:182
306099128,3cyclictest7522-21grep22:15:142
306099128,3cyclictest31931-21cstates22:30:142
306099123,5cyclictest23954-21aten2_r7power_c22:45:132
306099121,10cyclictest23791-21proc_pri19:50:202
3060991210,1cyclictest0-21swapper/223:50:122
306099120,3cyclictest0-21swapper/222:23:152
306099120,1cyclictest0-21swapper/221:32:142
306099120,0cyclictest0-21swapper/200:00:072
305899129,2cyclictest6710-21ls20:25:011
305899129,2cyclictest26481-21sed23:25:171
305899129,2cyclictest26481-21sed23:25:161
305899129,2cyclictest25053-21cut19:55:151
305899129,2cyclictest23770-21head19:50:211
305899129,2cyclictest19938-21head20:50:171
305899129,2cyclictest18821-21wc19:40:191
305899129,2cyclictest16883-21sed19:35:261
305899129,2cyclictest12827-21ssh00:15:161
305899128,3cyclictest9179-21turbostat20:30:011
305899121,11cyclictest28486-21diskmemload23:40:121
3058991211,1cyclictest25-21ksoftirqd/100:05:351
3058991211,1cyclictest0-21swapper/123:48:111
3058991210,1cyclictest25-21ksoftirqd/121:20:131
305899120,12cyclictest0-21swapper/123:19:271
305899120,12cyclictest0-21swapper/122:35:411
305899120,12cyclictest0-21swapper/100:26:031
305899120,10cyclictest0-21swapper/122:11:331
305899120,0cyclictest0-21swapper/122:49:311
305399129,2cyclictest3428-21head23:50:160
305399129,2cyclictest30449-21grep21:30:170
305399129,2cyclictest29817-21grep20:05:170
305399129,2cyclictest23728-21chrt00:21:170
305399129,2cyclictest23244-21grep21:25:250
305399129,2cyclictest19443-21sed19:40:260
305399129,2cyclictest17586-21chrt23:59:050
305399129,2cyclictest10174-21turbostat19:25:000
305399127,4cyclictest0-21swapper/020:30:010
305399122,8cyclictest5694-21cron21:55:000
3053991211,1cyclictest0-21swapper/021:12:220
3053991210,2cyclictest0-21swapper/000:28:120
305399120,2cyclictest0-21swapper/022:56:090
261292121,1sleep228486-21diskmemload00:23:112
306399119,1cyclictest29507-21sh21:10:153
306399118,2cyclictest31534-21df22:10:133
306399118,2cyclictest29483-21munin-plugin-st20:05:003
306399118,2cyclictest29483-21munin-plugin-st20:04:593
306399118,2cyclictest28091-21ssh22:27:373
306399118,2cyclictest20122-21chrt19:45:033
306399118,2cyclictest18031-21tr19:40:153
306399118,2cyclictest15258-21ssh23:57:143
306399118,2cyclictest14980-21head20:40:183
306399110,0cyclictest0-21swapper/323:40:173
306099118,2cyclictest6159-21head20:20:242
306099118,2cyclictest30457-21head20:05:202
306099118,2cyclictest29832-21sed00:25:172
306099118,2cyclictest28032-21cut20:00:162
306099118,2cyclictest28032-21cut20:00:162
306099118,2cyclictest26583-21tr21:05:142
306099118,2cyclictest21789-21grep20:55:142
306099115,2cyclictest225-21systemd-journal20:35:012
306099111,9cyclictest56550irq/124-enp0s3120:27:392
306099111,7cyclictest19657-21df_abs20:50:152
305899119,2cyclictest25-21ksoftirqd/123:56:501
305899118,2cyclictest11472-21taskset20:34:131
305899118,2cyclictest10576-21tr19:25:151
305899116,4cyclictest7211-21expr19:15:261
305899110,10cyclictest9118-21ssh21:56:101
305899110,10cyclictest9118-21ssh21:56:101
305399118,2cyclictest9171-21grep23:15:130
305399118,2cyclictest7511-21taskset21:55:200
305399118,2cyclictest7511-21taskset21:55:190
305399118,2cyclictest6772-21strings19:15:200
305399118,2cyclictest25661-21grep19:55:170
305399118,2cyclictest17960-21ssh23:00:220
305399118,2cyclictest14156-21sed00:15:270
305399118,2cyclictest13912-21cron20:40:010
305399115,3cyclictest225-21systemd-journal20:14:590
305399112,8cyclictest5409-21munin-run19:15:000
306399108,1cyclictest826-21ntpq20:10:223
306399108,1cyclictest7059-21apache_accesses20:25:113
306399108,1cyclictest683-21sh21:50:593
306399108,1cyclictest5072-21taskset19:10:593
306399108,1cyclictest31277-21cat20:05:293
306399108,1cyclictest19584-21expr20:50:153
306399108,1cyclictest14222-21grep21:40:143
306399108,1cyclictest13251-21cat19:30:153
306399108,1cyclictest10138-21gltestperf20:30:163
306399107,2cyclictest22802-21tr19:50:133
306399107,2cyclictest16819-21grep20:45:143
306399105,4cyclictest8067-21apt-get19:20:133
306099108,1cyclictest4075-21chrt20:18:402
306099108,1cyclictest3356-21wget19:10:142
306099108,1cyclictest13844-21cat19:30:192
306099107,2cyclictest8097-21grep19:20:132
305899108,1cyclictest220782sleep119:47:371
305899108,1cyclictest16721-21grep20:45:151
305899108,1cyclictest15090-21cat19:35:001
305899108,1cyclictest13815-21taskset20:38:481
305899107,2cyclictest29196-21taskset20:01:341
305899107,2cyclictest29196-21taskset20:01:341
305899107,2cyclictest16136-21ssh21:40:361
305899107,2cyclictest15006-21hostname20:40:181
305899102,2cyclictest5020-21tr21:15:141
305399108,1cyclictest28386-21chrt21:08:580
305399108,1cyclictest22443-21cat20:55:180
305399108,1cyclictest20121-21wc20:50:190
305399108,1cyclictest18033-21ssh23:40:040
305399108,1cyclictest12319-21sshd19:27:150
305399107,2cyclictest6280-21taskset20:21:140
305399107,2cyclictest25532-21date21:00:240
305399107,2cyclictest15807-21grep19:35:170
305399100,9cyclictest10519-21cat20:30:190
30539985,2cyclictest23552-21load19:50:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional