You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 20:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Apr 14, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
779121070,0sleep20-21swapper/211:38:472
2235729885,9sleep30-21swapper/307:06:003
2247029342,46sleep10-21swapper/107:07:261
2249429249,39sleep00-21swapper/007:07:450
2263728875,9sleep20-21swapper/207:09:372
77442580,1sleep0161rcuc/009:43:280
2275599240,24cyclictest0-21swapper/211:46:222
22752992423,1cyclictest0-21swapper/110:12:071
2274899230,20cyclictest0-21swapper/010:04:120
161822230,0sleep20-21swapper/210:26:092
22758992221,1cyclictest0-21swapper/312:10:133
22755992218,3cyclictest19873-21ssh10:48:022
2274899227,7cyclictest4676-21perf07:40:000
64432210,0sleep20-21swapper/212:35:162
2275899210,20cyclictest9545-21kworker/3:011:44:263
22752992120,1cyclictest0-21swapper/110:21:301
2275299210,1cyclictest0-21swapper/111:48:481
2274899210,17cyclictest0-21swapper/011:14:360
2275899200,20cyclictest9545-21kworker/3:011:53:483
2275599205,14cyclictest0-21swapper/210:23:102
2275299201,3cyclictest30467-21ssh12:30:171
22748992019,1cyclictest0-21swapper/010:31:290
2274899200,4cyclictest0-21swapper/010:55:250
2274899200,4cyclictest0-21swapper/010:55:240
2275599194,15cyclictest0-21swapper/207:41:592
2274899195,8cyclictest27482-21sendmail07:19:590
22748991910,4cyclictest12821-21ssh10:44:050
2275599189,9cyclictest0-21swapper/210:06:492
2275599189,9cyclictest0-21swapper/208:40:242
2275599180,14cyclictest32513-21apache_volume07:30:132
2275299188,4cyclictest0-21swapper/109:23:491
2275299181,13cyclictest4558-21apache_processe12:15:131
2275299180,1cyclictest0-21swapper/110:40:231
22748991813,3cyclictest13-21ksoftirqd/009:50:330
2275899171,13cyclictest10296-21ssh09:45:143
2275599179,7cyclictest0-21swapper/212:25:542
22755991717,0cyclictest0-21swapper/212:03:562
22755991716,1cyclictest0-21swapper/211:10:232
2275599170,17cyclictest0-21swapper/209:33:472
2275299171,13cyclictest15332-21diskmemload11:57:151
2275299170,1cyclictest0-21swapper/110:27:211
2274899174,6cyclictest22423-21awk10:50:000
22748991717,0cyclictest0-21swapper/009:26:330
2274899170,1cyclictest0-21swapper/010:12:180
2274899170,13cyclictest0-21swapper/011:16:510
2275899161,2cyclictest0-21swapper/310:32:023
2275899160,1cyclictest0-21swapper/311:35:523
2275599164,6cyclictest12046-21apt-get07:55:142
2275599164,11cyclictest56550irq/124-enp0s3108:37:072
22752991615,1cyclictest0-21swapper/110:34:321
22752991614,1cyclictest0-21swapper/109:39:411
22752991612,2cyclictest0-21swapper/110:55:221
22752991612,2cyclictest0-21swapper/110:55:221
2275299160,16cyclictest0-21swapper/111:13:101
2274899167,8cyclictest26084-21munin-run08:25:000
22748991616,0cyclictest0-21swapper/009:35:060
2274899160,16cyclictest0-21swapper/011:59:390
2274899160,15cyclictest0-21swapper/012:29:020
2274899160,15cyclictest0-21swapper/012:17:350
2275599159,6cyclictest0-21swapper/209:15:322
2275599150,15cyclictest0-21swapper/209:27:022
2275599150,14cyclictest0-21swapper/212:08:072
2275599150,12cyclictest0-21swapper/212:11:022
2274899152,7cyclictest19313-21munin-run12:05:000
22748991511,2cyclictest32115-21wc08:35:200
2274899150,13cyclictest0-21swapper/008:25:110
22758991413,1cyclictest0-21swapper/310:40:523
2275899141,1cyclictest15332-21diskmemload10:16:113
22758991411,2cyclictest7919-21uniq07:45:153
22758991410,2cyclictest30600-21ssh12:30:183
2275899140,14cyclictest0-21swapper/311:03:493
2275599141,9cyclictest0-21swapper/207:50:122
22755991412,2cyclictest0-21swapper/210:33:432
2275599141,13cyclictest0-21swapper/209:04:552
22755991411,2cyclictest4189-21grep07:35:262
22755991410,4cyclictest0-21swapper/211:32:222
22755991410,4cyclictest0-21swapper/209:57:032
22755991410,3cyclictest7481-21ls10:40:232
22755991410,3cyclictest17801-21ls08:05:182
2275599140,3cyclictest0-21swapper/207:20:142
2275299142,6cyclictest38150irq/131-snd_hda09:09:591
22752991411,2cyclictest0-21swapper/111:17:571
22752991410,3cyclictest8832-21ssh11:00:231
2274899141,7cyclictest8481-21munin-run09:25:000
22748991413,1cyclictest0-21swapper/011:54:200
22748991413,1cyclictest0-21swapper/011:44:540
2274899140,8cyclictest2222-21perf07:34:590
2274899140,2cyclictest0-21swapper/011:37:140
22758991312,1cyclictest55650irq/127-enp2s0-11:46:453
22758991310,2cyclictest30243-21cut07:25:143
22758991310,2cyclictest23754-21grep10:50:173
22758991310,2cyclictest22677-21uname12:25:193
22758991310,2cyclictest12536-21grep10:05:173
22758991310,2cyclictest10670-21tr09:00:133
2275599139,3cyclictest30278-21tr07:25:142
2275599139,3cyclictest24598-21sh11:29:472
2275599139,3cyclictest16324-21tr09:10:152
2275599139,3cyclictest10631-21cstates09:45:152
22755991310,2cyclictest16517-21head11:05:162
22755991310,2cyclictest16040-21grep08:00:272
22755991310,2cyclictest16040-21grep08:00:262
22755991310,2cyclictest15258-21sort11:05:012
22755991310,2cyclictest0-21swapper/207:15:102
2275599130,2cyclictest0-21swapper/209:42:322
2275299139,3cyclictest22422-21basename10:50:001
2275299139,3cyclictest11677-21tr12:00:141
22752991310,2cyclictest4906-21wc08:45:241
22752991310,2cyclictest10816-21df09:45:171
22752991310,2cyclictest10560-21sh09:25:241
2275299130,8cyclictest6937-21kworker/1:109:50:021
2275299130,2cyclictest0-21swapper/110:02:561
2275299130,1cyclictest0-21swapper/112:25:101
2275299130,1cyclictest0-21swapper/111:35:501
2275299130,12cyclictest25574-21sed08:20:261
2274899139,3cyclictest17297-21tr09:10:200
2274899139,2cyclictest8415-21tr07:45:200
2274899138,4cyclictest3570-21apache212:08:570
2274899137,2cyclictest30675-21basename10:55:010
2274899132,3cyclictest225-21systemd-journal11:24:550
22748991313,0cyclictest0-21swapper/010:29:250
2274899131,1cyclictest0-21swapper/008:50:140
22748991310,2cyclictest5179-21tr07:40:140
22748991310,2cyclictest30869-21grep10:35:170
22748991310,2cyclictest23181-21sed08:15:280
22748991310,2cyclictest19131-21wc09:30:240
313672120,1sleep10-21swapper/109:19:131
2275899129,2cyclictest8786-21seq09:44:173
2275899129,2cyclictest3704-21grep10:00:163
2275899129,2cyclictest31869-21sed08:35:193
2275899129,2cyclictest29947-21chrt09:37:053
2275899129,2cyclictest28719-21proc_pri09:55:213
2275899129,2cyclictest2705-21cut07:35:153
2275899129,2cyclictest24209-21grep07:10:203
2275899129,2cyclictest21179-21cat12:05:203
2275899129,2cyclictest13546-21tr10:25:163
2275899129,2cyclictest12117-21cut12:00:163
2275899129,2cyclictest10070-21ssh11:20:253
2275899128,2cyclictest25592-21grep07:15:113
2275899124,7cyclictest15810-21ssh09:10:033
22758991211,1cyclictest55650irq/127-enp2s0-09:34:383
2275599129,2cyclictest31004-21wc08:35:002
2275599129,2cyclictest26431-21df09:35:172
2275599129,2cyclictest11936-21sed11:40:282
2275599129,2cyclictest11257-21awk10:04:592
2275599129,2cyclictest10597-21grep11:20:292
2275599128,4cyclictest0-21swapper/212:19:522
2275599123,6cyclictest15332-21diskmemload10:53:172
2275599121,3cyclictest14991-21unixbench_singl12:20:272
2275599121,3cyclictest14991-21unixbench_singl12:20:272
2275599120,11cyclictest31974-21grep10:35:262
2275599120,11cyclictest20485-21tr10:10:142
2275299129,2cyclictest6400-21wc07:40:251
2275299129,2cyclictest4665-21awk07:40:001
2275299129,2cyclictest20193-21tr12:05:151
2275299129,2cyclictest1997-21basename09:40:151
2275299129,2cyclictest13006-21cat09:05:001
2275299129,2cyclictest11149-21wc07:50:241
2275299129,2cyclictest10514-21wc08:59:591
2275299128,3cyclictest19903-21grep08:10:171
2275299128,2cyclictest30379-21tr07:25:141
2275299127,4cyclictest32643-21cut07:30:151
2275299123,4cyclictest0-21swapper/108:05:141
2275299122,5cyclictest141rcu_preempt08:00:151
2275299122,5cyclictest141rcu_preempt08:00:141
22752991210,1cyclictest1807-21sed08:40:171
2274899129,2cyclictest29768-21wc08:30:210
2274899129,2cyclictest2813-21ssh10:00:010
2274899129,2cyclictest24776-21grep09:15:160
2274899129,2cyclictest23585-21cron11:10:000
2274899129,2cyclictest17917-21head08:05:200
2274899128,3cyclictest16539-21cron08:05:010
2274899128,3cyclictest16539-21cron08:05:000
2274899128,2cyclictest9807-21tr07:50:130
2274899128,2cyclictest28866-21cut10:15:130
2274899128,2cyclictest24553-21cat07:10:250
22748991211,1cyclictest0-21swapper/012:23:280
22748991211,1cyclictest0-21swapper/012:23:270
2274899120,1cyclictest0-21swapper/009:45:420
2275899119,1cyclictest7894-21grep12:35:283
2275899119,1cyclictest4843-21grep12:15:153
2275899119,1cyclictest3791-21ssh11:16:423
2275899118,2cyclictest9397-21ntpq08:55:213
2275899118,2cyclictest9059-21rm11:58:353
2275899118,2cyclictest8961-21sh09:25:123
2275899118,2cyclictest7144-21ssh10:20:403
2275899118,2cyclictest6064-21grep08:50:153
2275899118,2cyclictest5789-21rm09:22:573
2275899118,2cyclictest5025-21tr07:40:143
2275899118,2cyclictest31108-21uname10:35:203
2275899118,2cyclictest3064-21chrt08:43:523
2275899118,2cyclictest29056-21expr08:30:163
2275899118,2cyclictest28874-21ssh11:31:333
2275899118,2cyclictest28542-21cut08:30:013
2275899118,2cyclictest28422-21taskset11:12:143
2275899118,2cyclictest26134-21python09:15:253
2275899118,2cyclictest19724-21ssh10:47:563
2275899118,2cyclictest19442-21grep08:10:163
2275899118,2cyclictest19215-21expr09:50:153
2275899118,2cyclictest18829-21taskset08:09:013
2275899118,2cyclictest1804-21ssh10:56:153
2275899118,2cyclictest1804-21ssh10:56:143
2275899118,2cyclictest16825-21ssh11:05:193
2275899118,2cyclictest16535-21cron08:05:013
2275899118,2cyclictest16535-21cron08:05:003
2275899118,2cyclictest15957-21ssh12:21:113
2275899118,2cyclictest15957-21ssh12:21:103
2275899118,2cyclictest13465-21tr09:05:163
2275899111,1cyclictest0-21swapper/307:30:143
2275899110,10cyclictest23704-21ssh11:29:053
2275599118,2cyclictest736-21sed10:55:272
2275599118,2cyclictest736-21sed10:55:262
2275599118,2cyclictest4971-21ssh09:22:192
2275599118,2cyclictest3178-21grep11:55:132
2275599118,2cyclictest30658-21grep12:30:192
2275599118,2cyclictest29571-21grep10:15:172
2275599118,2cyclictest2921-21ssh11:16:042
2275599118,2cyclictest23972-21cat07:10:192
2275599118,2cyclictest19400-21expr09:50:152
2275299118,2cyclictest8477-21ssh11:20:071
2275299118,2cyclictest7838-21sed12:35:271
2275299118,2cyclictest7327-21grep07:45:141
2275299118,2cyclictest4675-21chrt10:39:191
2275299118,2cyclictest28549-21ssh10:15:051
2275299118,2cyclictest27944-21sed08:25:241
2275299118,2cyclictest27193-21wc11:30:241
2275299118,2cyclictest25414-21ssh10:50:501
2275299118,2cyclictest24934-21ssh09:34:451
2275299118,2cyclictest24826-21chrt07:11:431
2275299118,2cyclictest23284-21ssh11:28:451
2275299118,2cyclictest16809-21cut09:10:171
2275299118,2cyclictest15913-21ssh11:05:131
2275299118,2cyclictest13465-21ssh11:41:381
2275299118,2cyclictest12918-21cat10:05:191
2275299118,2cyclictest1285-21ssh09:58:521
2275299117,3cyclictest31792-21grep08:35:191
2275299117,3cyclictest12767-21tail07:55:171
2275299114,2cyclictest225-21systemd-journal07:16:591
2275299110,3cyclictest20236-21sshd12:24:301
2275299110,3cyclictest20236-21sshd12:24:291
2274899118,2cyclictest940-21ssh12:31:380
2274899118,2cyclictest6166-21ssh12:35:170
2274899118,2cyclictest5681-21ssh10:20:160
2274899118,2cyclictest5605-21apt08:49:590
2274899118,2cyclictest32053-21ssh12:11:380
2274899118,2cyclictest31840-21sed07:25:270
2274899118,2cyclictest28439-21cut07:20:170
2274899118,2cyclictest26663-21ssh11:30:180
2274899118,2cyclictest24658-21ssh11:29:500
2274899118,2cyclictest11395-21ssh11:02:060
2274899116,2cyclictest452-21systemd-logind08:12:030
2274899115,2cyclictest225-21systemd-journal09:05:020
2274899114,7cyclictest12986-21strings10:05:200
2274899110,10cyclictest21105-21chrt11:46:020
2275899108,1cyclictest3764-21cut08:45:153
2275899108,1cyclictest20011-21ssh10:10:063
2275899107,2cyclictest12793-21expr07:55:173
2275899107,2cyclictest11239-21ls07:50:263
2275899100,9cyclictest27722-21sshd07:20:053
2275599108,1cyclictest8764-21threads07:45:252
2275599108,1cyclictest8192-21taskset08:55:022
2275599108,1cyclictest4692-21cut08:45:212
2275599108,1cyclictest28145-21taskset08:26:192
2275599108,1cyclictest24076-21aten2_r7power_v08:20:142
2275599108,1cyclictest22912-21grep08:15:242
2275599108,1cyclictest20109-21uname08:10:192
2275599108,1cyclictest15159-21chrt09:07:592
2275599107,2cyclictest6584-21head08:50:182
2275599106,3cyclictest27982-21tr11:50:202
2275299108,1cyclictest29220-21tune2fs12:10:161
2275299108,1cyclictest28781-21ssh11:50:301
2275299108,1cyclictest23277-21sed08:15:281
2275299107,2cyclictest30404-21head08:30:281
2275299107,2cyclictest29204-21wc07:20:261
2275299103,3cyclictest225-21systemd-journal08:54:061
2274899108,1cyclictest8316-21awk08:55:150
2274899107,2cyclictest2162-21head08:40:200
2274899107,2cyclictest12339-21wc07:55:150
2274899100,9cyclictest12617-21taskset09:03:330
227589998,1cyclictest24282-21expr08:20:153
227589997,1cyclictest21906-21cat08:15:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional