You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-15 - 02:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot7s.osadl.org (updated Wed Oct 14, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3156029683,9sleep00-21swapper/007:06:490
3089929648,43sleep10-21swapper/107:05:231
3152728965,8sleep20-21swapper/207:06:242
3162028667,8sleep30-21swapper/307:07:383
280252630,0sleep00-21swapper/010:38:510
3189299290,25cyclictest0-21swapper/110:25:161
3189999272,24cyclictest25568-21kworker/3:011:04:133
31899992624,2cyclictest25568-21kworker/3:011:39:133
3189999261,24cyclictest25568-21kworker/3:011:51:433
3189099260,2cyclictest141rcu_preempt12:29:570
3189099260,1cyclictest141rcu_preempt10:04:400
31896992522,3cyclictest0-21swapper/210:18:532
3189299250,24cyclictest0-21swapper/110:09:411
31899992422,2cyclictest25568-21kworker/3:010:54:433
31896992423,1cyclictest0-21swapper/209:30:152
31896992423,1cyclictest0-21swapper/209:30:152
3189099240,22cyclictest0-21swapper/011:54:550
31892992320,2cyclictest1142-21runrttasks09:26:151
31892992320,2cyclictest1142-21runrttasks09:26:141
3189099230,21cyclictest0-21swapper/009:14:000
3189999222,19cyclictest0-21swapper/310:31:523
31899992220,2cyclictest25568-21kworker/3:011:10:433
31899992219,2cyclictest23227-21fschecks_count10:55:173
3189999220,21cyclictest25568-21kworker/3:009:54:133
3189299228,13cyclictest8953-21runrttasks11:05:291
31892992220,1cyclictest9853-21latency_hist12:05:001
3189299220,1cyclictest0-21swapper/110:40:131
31899992119,2cyclictest25568-21kworker/3:011:40:033
31899992119,1cyclictest0-21swapper/309:55:103
3189999210,20cyclictest25568-21kworker/3:012:36:133
3189999210,20cyclictest25568-21kworker/3:010:40:243
31896992120,1cyclictest0-21swapper/211:29:402
31896992119,2cyclictest0-21swapper/210:04:142
3189699210,1cyclictest0-21swapper/209:13:592
3189099210,20cyclictest0-21swapper/010:26:210
31899992017,3cyclictest25568-21kworker/3:012:17:133
3189699201,17cyclictest56550irq/124-enp0s3112:12:282
31892992016,3cyclictest24466-21fschecks_time08:00:151
266062200,0chrt0-21swapper/111:55:181
31899991917,2cyclictest25568-21kworker/3:010:15:333
31899991917,2cyclictest25568-21kworker/3:009:31:343
31899991917,2cyclictest25568-21kworker/3:009:31:333
31899991916,2cyclictest25568-21kworker/3:010:20:233
3189999191,18cyclictest0-21swapper/310:29:513
31896991917,2cyclictest0-21swapper/209:48:542
31896991916,3cyclictest0-21swapper/209:37:432
3189299192,15cyclictest19214-21diskmemload11:02:481
31899991816,2cyclictest25568-21kworker/3:010:49:543
31899991816,2cyclictest25568-21kworker/3:009:36:133
31899991816,2cyclictest25568-21kworker/3:009:27:033
31899991816,2cyclictest25568-21kworker/3:009:27:023
31899991816,1cyclictest26128-21/usr/sbin/munin09:15:163
31896991815,3cyclictest0-21swapper/212:03:032
3189699180,1cyclictest0-21swapper/211:16:012
3189699180,17cyclictest0-21swapper/211:58:182
3189099184,13cyclictest553-21sshd11:15:550
31890991812,3cyclictest1236-21cron12:00:000
3189099180,2cyclictest0-21swapper/010:24:480
3189099180,1cyclictest0-21swapper/011:29:400
3189999173,9cyclictest23501-21awk08:00:013
31899991715,2cyclictest25568-21kworker/3:011:15:533
31899991715,2cyclictest25568-21kworker/3:011:09:243
31899991715,2cyclictest25568-21kworker/3:010:04:333
31896991716,1cyclictest0-21swapper/210:20:042
31896991713,4cyclictest0-21swapper/210:32:042
3189699170,2cyclictest20680-21kworker/u8:211:23:022
3189699170,17cyclictest56550irq/124-enp0s3108:37:502
3189699170,15cyclictest0-21swapper/211:54:552
31892991715,1cyclictest1142-21runrttasks09:31:421
31892991715,1cyclictest1142-21runrttasks09:31:421
3189299170,1cyclictest0-21swapper/112:18:201
3189099173,10cyclictest31807-21apache_accesses11:20:110
3189099170,1cyclictest0-21swapper/009:29:050
3189099170,1cyclictest0-21swapper/009:29:050
3189099170,16cyclictest0-21swapper/011:43:320
3189999162,13cyclictest25058-21kworker/u8:112:26:253
3189699167,9cyclictest0-21swapper/208:44:382
31896991614,2cyclictest0-21swapper/211:07:432
31896991614,2cyclictest0-21swapper/210:45:042
31896991613,3cyclictest0-21swapper/210:50:232
31896991613,3cyclictest0-21swapper/209:50:532
31892991615,1cyclictest38150irq/131-snd_hda11:37:531
31892991614,1cyclictest18651-21/usr/sbin/munin12:10:211
31892991613,2cyclictest1142-21runrttasks10:23:111
3189099163,11cyclictest23553-21/usr/sbin/munin09:55:220
3189099160,9cyclictest0-21swapper/008:35:110
3189099160,2cyclictest0-21swapper/009:37:120
3189699154,7cyclictest14082-21apt-get08:55:092
3189699153,10cyclictest21536-21apache_processe09:55:082
3189699152,7cyclictest4961-21/usr/sbin/munin10:25:112
3189699151,13cyclictest0-21swapper/209:23:022
31892991512,2cyclictest22065-21cstates10:35:161
3189299151,13cyclictest0-21swapper/110:16:351
3189099155,8cyclictest7338-21cups-browsed07:10:220
31899991413,1cyclictest0-21swapper/312:00:133
31899991411,2cyclictest6986-21cstates09:45:113
31899991411,2cyclictest11342-21if_err_enp0s31f12:05:183
31899991410,2cyclictest4823-21ssh10:05:133
3189999140,6cyclictest13194-21cron08:49:593
3189699149,4cyclictest5422-21cron10:45:002
31896991413,1cyclictest0-21swapper/210:07:322
31896991412,2cyclictest0-21swapper/211:10:342
31896991412,1cyclictest0-21swapper/212:32:362
31896991411,3cyclictest0-21swapper/211:39:432
31896991411,3cyclictest0-21swapper/211:32:242
3189699140,1cyclictest0-21swapper/212:20:182
3189299149,4cyclictest3747-21latency_hist12:40:011
31892991413,1cyclictest0-21swapper/107:45:131
31892991413,0cyclictest0-21swapper/111:16:471
31892991412,1cyclictest0-21swapper/110:04:311
31892991411,2cyclictest6650-21cut07:25:011
31892991410,2cyclictest6862-21ls08:30:241
3189299140,1cyclictest0-21swapper/109:41:361
3189299140,13cyclictest9935-21grep07:30:161
3189099141,2cyclictest0-21swapper/011:09:020
31890991412,2cyclictest0-21swapper/010:44:000
3189099141,10cyclictest19214-21diskmemload10:15:140
3189099140,8cyclictest0-21swapper/009:51:580
3189099140,12cyclictest0-21swapper/011:15:000
3189999132,5cyclictest0-21swapper/308:39:183
3189999131,11cyclictest0-21swapper/312:22:413
31899991310,2cyclictest4362-21grep08:25:203
31899991310,2cyclictest24516-21grep08:00:163
3189699135,8cyclictest0-21swapper/208:08:142
3189699134,5cyclictest18752-21turbostat07:50:002
3189699131,1cyclictest7935-21memory07:25:182
31896991310,3cyclictest0-21swapper/210:55:042
31896991310,3cyclictest0-21swapper/209:26:232
31896991310,3cyclictest0-21swapper/209:26:222
31896991310,3cyclictest0-21swapper/209:18:532
3189299139,3cyclictest22736-21cut10:55:141
3189299139,3cyclictest14152-21cat07:40:001
3189299138,3cyclictest30239-21head08:10:261
31892991310,2cyclictest27007-21sed09:15:121
31892991310,2cyclictest20286-21sh10:15:001
3189299130,13cyclictest19214-21diskmemload10:45:491
3189299130,11cyclictest0-21swapper/109:11:501
3189099131,8cyclictest0-21swapper/012:15:520
31890991312,1cyclictest0-21swapper/012:31:590
31890991310,2cyclictest4554-21cut07:20:160
31890991310,2cyclictest15590-21wc07:40:190
3189099130,1cyclictest0-21swapper/009:15:480
3189999129,2cyclictest31889-21tr11:20:133
3189999129,2cyclictest26002-21grep11:55:153
3189999129,2cyclictest24509-21ssh12:13:253
3189699129,2cyclictest7401-21grep08:30:292
3189699129,2cyclictest5302-21basename07:20:202
3189699129,2cyclictest3292-21cut07:15:222
3189699129,2cyclictest19262-21awk12:30:002
3189699129,2cyclictest15734-21sh11:49:152
3189699129,2cyclictest1354-21expr08:20:172
3189699129,1cyclictest19214-21diskmemload09:43:262
3189699126,6cyclictest0-21swapper/207:40:102
3189699126,5cyclictest14140-21/usr/sbin/munin10:10:262
3189699124,4cyclictest225-21systemd-journal07:35:012
3189699122,5cyclictest0-21swapper/212:16:022
3189699121,1cyclictest1302-21unixbench-2d07:10:252
3189699121,11cyclictest1692-21ssh11:01:232
31896991210,1cyclictest23515-21perf08:00:012
3189299129,2cyclictest9764-21grep11:25:261
3189299129,2cyclictest9132-21wc08:35:161
3189299129,2cyclictest7685-21head07:25:161
3189299129,2cyclictest2603-21sed08:20:281
3189299129,2cyclictest22086-21perf10:55:001
3189299129,2cyclictest14557-21tr07:40:121
3189299128,3cyclictest28233-21taskset08:09:411
3189299128,3cyclictest24262-21ssh12:32:151
31892991211,1cyclictest25-21ksoftirqd/109:53:451
31892991210,2cyclictest38150irq/131-snd_hda09:39:371
3189099129,2cyclictest741-21cat08:19:590
3189099129,2cyclictest4000-21tail08:25:180
3189099129,2cyclictest3768-21head07:15:260
3189099129,2cyclictest3444-21sh12:20:150
3189099129,2cyclictest28429-21ssh11:36:560
3189099129,2cyclictest23059-21sed07:55:250
3189099129,2cyclictest22450-21ssh10:55:080
3189099129,2cyclictest20911-21chrt07:52:250
3189099129,2cyclictest19830-21ssh10:53:140
3189099129,2cyclictest19112-21expr12:10:150
3189099129,2cyclictest18692-21ssh11:30:500
3189099129,2cyclictest16210-21systemctl09:00:000
3189099129,2cyclictest14450-21sed10:10:270
3189099129,2cyclictest14334-21grep10:30:210
3189099124,7cyclictest19214-21diskmemload09:45:150
3189099124,5cyclictest5945-21diskmemload10:45:130
3189099122,1cyclictest19214-21diskmemload12:38:470
31890991211,1cyclictest7321-21apache210:07:170
3189099120,2cyclictest0-21swapper/012:07:290
3189099120,1cyclictest0-21swapper/011:47:500
3189099120,11cyclictest7563-21sort07:25:160
3189099120,11cyclictest5873-21cut08:30:170
3189999118,2cyclictest9558-21grep07:30:143
3189999118,2cyclictest8115-21tr11:25:143
3189999118,2cyclictest4564-21cut07:20:153
3189999118,2cyclictest3865-21ssh09:21:353
3189999118,2cyclictest31126-21ls08:15:163
3189999118,2cyclictest23571-21ssh12:31:433
3189999118,2cyclictest22869-21ssh10:35:193
3189999118,2cyclictest19790-21ssh09:10:163
3189999118,2cyclictest17577-21awk11:30:223
3189999118,2cyclictest16672-21cut07:45:133
3189999118,2cyclictest15734-21ssh10:11:253
3189999118,2cyclictest15356-21tail07:40:183
3189999118,2cyclictest1518-21taskset07:11:223
3189999118,2cyclictest1500-21sort08:20:173
3189999118,2cyclictest14752-21grep08:55:123
3189999118,2cyclictest13886-21cat08:55:003
3189999118,2cyclictest13103-21ssh11:47:123
3189999117,2cyclictest7110-21cut07:25:143
3189999111,7cyclictest28630-21cut08:10:143
3189999110,8cyclictest32086-21apt-get09:40:023
3189999110,10cyclictest17804-21expr09:05:133
3189699119,2cyclictest12838-21expr07:35:192
3189699119,1cyclictest28498-21taskset08:10:052
3189699119,1cyclictest24128-21ssh10:35:462
3189699119,1cyclictest17232-21wget09:05:112
3189699118,2cyclictest3547-21cut08:25:152
3189699118,2cyclictest24035-21tr08:00:132
3189699118,2cyclictest12358-21ssh12:05:252
3189699118,2cyclictest11871-21grep08:45:152
3189699116,4cyclictest28732-21expr12:35:152
3189699111,9cyclictest56550irq/124-enp0s3108:15:142
3189699110,8cyclictest0-21swapper/207:50:222
3189699110,11cyclictest0-21swapper/211:40:352
3189299118,2cyclictest8834-21ssh11:45:111
3189299118,2cyclictest8386-21sleep11:45:001
3189299118,2cyclictest3429-21ssh09:21:141
3189299118,2cyclictest20730-21grep07:50:251
3189299118,2cyclictest18449-21ssh11:11:271
3189299118,2cyclictest18444-21sh11:30:391
3189299118,2cyclictest15965-21ssh12:27:261
3189299118,2cyclictest14700-21ssh09:50:011
3189299118,2cyclictest10816-21cron08:44:591
3189299117,3cyclictest21672-21tr07:55:141
3189299112,7cyclictest0-21swapper/111:51:051
3189299111,9cyclictest5405-21ssh11:23:331
31892991111,0cyclictest0-21swapper/112:07:011
3189299110,11cyclictest0-21swapper/112:20:151
3189299110,10cyclictest0-21swapper/110:30:501
3189099119,2cyclictest1791-21sh12:00:050
3189099119,1cyclictest3502-21ssh09:21:170
3189099119,1cyclictest32224-21ssh09:40:080
3189099119,1cyclictest18352-21ps09:05:170
3189099119,1cyclictest13951-21date08:55:000
3189099119,1cyclictest136852sleep007:36:180
3189099119,1cyclictest11263-21chrt07:33:270
3189099119,1cyclictest10831-21sh08:44:590
3189099118,3cyclictest17838-21chrt09:30:430
3189099118,3cyclictest17838-21chrt09:30:420
3189099118,2cyclictest31550-21ssh11:00:160
3189099118,2cyclictest1865-21sort08:20:210
3189999108,1cyclictest28347-21cat08:10:013
3189999108,1cyclictest211002sleep307:54:523
3189999107,2cyclictest7699-21chrt08:33:443
3189999107,2cyclictest12479-21uniq07:35:163
3189999102,4cyclictest225-21systemd-journal08:43:153
3189999102,4cyclictest225-21systemd-journal07:17:573
3189699103,7cyclictest56550irq/124-enp0s3108:54:512
3189699102,8cyclictest56550irq/124-enp0s3109:00:272
3189299108,2cyclictest31832-21awk08:15:191
3189299108,1cyclictest32157-21wget07:10:121
3189299108,1cyclictest2718-21cut07:15:161
3189299108,1cyclictest236002sleep109:55:231
3189299108,1cyclictest16578-21taskset09:01:531
3189299108,1cyclictest14674-21fschecks_count08:55:121
3189299108,1cyclictest11327-21expr08:45:131
3189299107,2cyclictest13812-21cron08:54:591
3189299106,4cyclictest18415-21grep09:05:181
3189099109,1cyclictest29674-21ntpq08:10:210
3189099108,1cyclictest26200-21tr08:05:130
3189099108,1cyclictest16846-21grep07:45:150
3189099108,1cyclictest13197-21idleruntime-cro08:50:000
3189099100,7cyclictest0-21swapper/008:00:110
318929990,2cyclictest0-21swapper/108:25:141
318909990,5cyclictest16740-21sshd09:03:560
318999980,4cyclictest225-21systemd-journal09:05:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional