You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 16:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Fri Mar 27, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
81608729263,20sleep20-21swapper/207:05:212
81656327741,13sleep30-21swapper/307:06:183
81615727451,15sleep00-21swapper/007:05:220
81652127138,28sleep10-21swapper/107:05:441
8763402450,3sleep00-21swapper/009:55:120
81700699340,25cyclictest888203-21chrt10:27:190
81700699330,19cyclictest0-21swapper/012:35:210
81700699330,17cyclictest862126-21taskset09:15:030
81700699320,21cyclictest0-21swapper/011:19:410
81700699320,17cyclictest0-21swapper/012:35:000
9015942310,6sleep10-21swapper/111:05:191
81700699311,16cyclictest822422-21seq07:22:310
81700699310,21cyclictest920918-21ntp_states12:00:190
81700699310,19cyclictest889082-21/usr/sbin/munin10:30:150
81700699310,19cyclictest0-21swapper/011:38:220
81700699310,19cyclictest0-21swapper/011:38:220
81700699310,18cyclictest923321-21chrt12:09:290
81700699300,20cyclictest0-21swapper/008:08:540
81700699300,18cyclictest0-21swapper/007:17:460
81700699300,16cyclictest0-21swapper/008:37:090
81700699300,15cyclictest0-21swapper/010:40:150
81700699290,15cyclictest852682-21chrt08:46:320
81700699290,15cyclictest825491-21sensors_temp07:30:200
81700699290,15cyclictest0-21swapper/010:15:130
817012992813,5cyclictest34-21ksoftirqd/211:20:012
81700699280,15cyclictest845843-21chrt08:27:590
81700699280,15cyclictest833589-21chrt07:55:070
81700699280,15cyclictest0-21swapper/011:31:300
81700699280,15cyclictest0-21swapper/008:02:200
81700699280,14cyclictest918086-21chrt11:54:010
8664912270,4sleep33056-21PK-Backend09:25:203
81700699271,12cyclictest0-21swapper/007:45:120
81700699270,19cyclictest0-21swapper/007:10:190
81700699270,16cyclictest1095-21runrttasks07:37:140
81700699270,14cyclictest914274-21sed11:40:220
81700699270,14cyclictest1095-21runrttasks08:41:580
81700699270,14cyclictest1095-21runrttasks08:41:580
81700699270,14cyclictest0-21swapper/012:25:480
81700699270,14cyclictest0-21swapper/011:25:130
81700699270,14cyclictest0-21swapper/009:26:010
81700699270,14cyclictest0-21swapper/008:10:020
139099270,4rtkit-daemon1389-21rtkit-daemon11:50:332
139099270,4rtkit-daemon1389-21rtkit-daemon09:20:232
817008992514,9cyclictest2254-21gdbus08:18:021
81700699254,19cyclictest854858-21grep08:55:010
817006992511,12cyclictest857447-21ntpq09:00:190
81700699250,24cyclictest0-21swapper/008:21:040
81700699250,14cyclictest846261-21taskset08:30:080
81700699250,13cyclictest0-21swapper/010:05:020
81700699250,12cyclictest0-21swapper/011:10:180
817006992419,4cyclictest0-21swapper/009:36:410
817006992415,8cyclictest0-21swapper/009:05:110
817012992320,2cyclictest872569-21apt09:45:112
81700899230,21cyclictest878047-21cpu10:00:111
81700699239,12cyclictest919196-21ntpq11:55:190
81700699236,15cyclictest855277-21cat08:55:140
817006992318,4cyclictest0-21swapper/011:21:520
817006992311,10cyclictest1095-21runrttasks08:18:560
81700699230,22cyclictest0-21swapper/007:53:130
81700699230,11cyclictest0-21swapper/010:58:350
8944412220,3sleep33056-21PK-Backend10:45:163
8358312220,2sleep23056-21PK-Backend08:00:142
81701699225,15cyclictest931706-21taskset12:30:213
817016992219,2cyclictest0-21swapper/309:51:083
81701699220,21cyclictest3056-21PK-Backend11:42:213
81701699220,20cyclictest3056-21PK-Backend11:00:573
81701699220,20cyclictest3056-21PK-Backend07:29:413
81701699220,20cyclictest3056-21PK-Backend07:22:513
817012992220,1cyclictest3056-21PK-Backend09:55:262
817012992220,1cyclictest0-21swapper/211:40:572
817012992220,1cyclictest0-21swapper/211:14:072
817012992220,1cyclictest0-21swapper/210:45:152
817012992220,1cyclictest0-21swapper/210:05:402
817012992220,1cyclictest0-21swapper/210:00:332
817012992220,1cyclictest0-21swapper/208:51:442
817012992220,1cyclictest0-21swapper/208:49:432
817012992220,1cyclictest0-21swapper/208:30:232
817012992220,1cyclictest0-21swapper/207:15:062
81701299220,21cyclictest0-21swapper/209:28:502
81700899229,3cyclictest8940002tr10:45:111
817008992220,1cyclictest0-21swapper/112:25:111
817008992220,1cyclictest0-21swapper/109:55:171
817008992219,2cyclictest927782-21strings12:20:161
817008992218,2cyclictest881753-21apt10:10:121
81700899220,20cyclictest3056-21PK-Backend09:22:541
81700899220,20cyclictest0-21swapper/109:32:261
81700699229,11cyclictest894620-21ntp_kernel_pll_10:45:170
81700699226,14cyclictest867675-21cat09:30:140
81700699226,14cyclictest864381-21cut09:20:140
81700699225,15cyclictest881926-21cut10:10:130
817006992220,1cyclictest0-21swapper/009:10:520
817006992216,4cyclictest900581-21latency_hist11:05:010
817006992213,7cyclictest924018-21sshd12:10:130
81700699221,19cyclictest1122-21snapd09:48:350
81700699220,9cyclictest8292852smartctl07:40:210
81700699220,21cyclictest3636880-21snapd12:23:040
81700699220,20cyclictest885443-21cstates10:20:130
81700699220,19cyclictest2422-21snap-store09:54:010
139099220,19rtkit-daemon2637-21gmain12:02:523
817016992119,1cyclictest3056-21PK-Backend11:38:293
817016992119,1cyclictest3056-21PK-Backend11:38:283
817016992119,1cyclictest3056-21PK-Backend10:55:023
817016992119,1cyclictest3056-21PK-Backend09:31:213
817016992119,1cyclictest3056-21PK-Backend09:11:263
817016992119,1cyclictest3056-21PK-Backend09:01:063
817016992119,1cyclictest3056-21PK-Backend08:58:163
817016992119,1cyclictest3056-21PK-Backend08:05:503
817016992119,1cyclictest0-21swapper/312:07:153
817016992119,1cyclictest0-21swapper/309:21:483
817016992118,2cyclictest3056-21PK-Backend11:10:583
817016992118,2cyclictest3056-21PK-Backend08:12:053
817016992118,2cyclictest3056-21PK-Backend07:37:183
81701699210,20cyclictest3056-21PK-Backend11:52:213
81701699210,20cyclictest3056-21PK-Backend11:32:333
81701699210,20cyclictest3056-21PK-Backend08:24:363
81701699210,20cyclictest0-21swapper/309:48:043
81701699210,20cyclictest0-21swapper/309:43:583
81701699210,20cyclictest0-21swapper/309:43:573
81701699210,19cyclictest3056-21PK-Backend12:12:093
81701699210,19cyclictest3056-21PK-Backend08:50:453
817012992119,1cyclictest0-21swapper/212:34:452
817012992119,1cyclictest0-21swapper/212:25:462
817012992119,1cyclictest0-21swapper/212:21:412
817012992119,1cyclictest0-21swapper/212:04:162
817012992119,1cyclictest0-21swapper/211:45:042
817012992119,1cyclictest0-21swapper/211:35:502
817012992119,1cyclictest0-21swapper/211:35:502
817012992119,1cyclictest0-21swapper/210:39:532
817012992119,1cyclictest0-21swapper/209:51:212
817012992119,1cyclictest0-21swapper/209:10:182
817012992119,1cyclictest0-21swapper/208:55:372
817012992119,1cyclictest0-21swapper/208:20:502
817012992119,1cyclictest0-21swapper/208:16:102
817012992119,1cyclictest0-21swapper/208:06:032
817012992119,1cyclictest0-21swapper/207:41:152
817012992119,1cyclictest0-21swapper/207:31:562
81701299210,20cyclictest0-21swapper/212:07:202
81701299210,20cyclictest0-21swapper/210:54:272
81701299210,20cyclictest0-21swapper/210:11:482
81701299210,20cyclictest0-21swapper/209:08:152
81701299210,20cyclictest0-21swapper/209:03:212
81701299210,20cyclictest0-21swapper/208:37:052
81701299210,20cyclictest0-21swapper/208:13:192
81701299210,20cyclictest0-21swapper/207:55:272
817008992119,1cyclictest3056-21PK-Backend12:03:211
817008992119,1cyclictest3056-21PK-Backend07:48:581
817008992119,1cyclictest0-21swapper/112:18:141
817008992119,1cyclictest0-21swapper/111:28:401
817008992119,1cyclictest0-21swapper/110:31:281
817008992119,1cyclictest0-21swapper/109:26:451
817008992119,1cyclictest0-21swapper/108:01:071
817008992119,1cyclictest0-21swapper/107:19:031
81700899210,20cyclictest912701-21seq11:36:371
81700899210,20cyclictest912701-21seq11:36:371
81700899210,20cyclictest3056-21PK-Backend09:46:191
817006992119,1cyclictest0-21swapper/011:45:460
817006992119,1cyclictest0-21swapper/010:35:430
817006992119,1cyclictest0-21swapper/009:43:380
817006992119,1cyclictest0-21swapper/009:43:380
81700699211,19cyclictest901856-21wc11:05:210
81700699210,20cyclictest0-21swapper/007:28:250
139099210,18rtkit-daemon1317-21systemd09:15:272
139099210,18rtkit-daemon1317-21systemd07:35:282
817016992018,1cyclictest3056-21PK-Backend09:58:043
817016992018,1cyclictest3056-21PK-Backend09:16:063
817016992018,1cyclictest3056-21PK-Backend08:31:193
817016992018,1cyclictest3056-21PK-Backend08:01:103
817016992018,1cyclictest0-21swapper/310:35:023
817016992017,2cyclictest3056-21PK-Backend10:02:573
817016992017,2cyclictest3056-21PK-Backend08:18:203
817016992017,1cyclictest3056-21PK-Backend09:35:133
81701699200,1cyclictest3056-21PK-Backend12:17:023
81701699200,1cyclictest3056-21PK-Backend10:14:473
81701699200,1cyclictest3056-21PK-Backend08:47:553
81701699200,19cyclictest3056-21PK-Backend11:21:463
81701699200,19cyclictest3056-21PK-Backend10:53:003
81701699200,19cyclictest3056-21PK-Backend10:23:463
81701699200,19cyclictest3056-21PK-Backend10:17:523
81701699200,19cyclictest0-21swapper/308:25:373
81701699200,18cyclictest3056-21PK-Backend11:26:393
817012992018,1cyclictest0-21swapper/212:36:472
817012992018,1cyclictest0-21swapper/208:26:302
817012992018,1cyclictest0-21swapper/207:28:102
81701299200,1cyclictest0-21swapper/212:16:342
81701299200,1cyclictest0-21swapper/210:25:212
81701299200,19cyclictest0-21swapper/212:12:272
81701299200,19cyclictest0-21swapper/211:59:102
81701299200,19cyclictest0-21swapper/209:34:442
81701299200,19cyclictest0-21swapper/208:44:492
81701299200,19cyclictest0-21swapper/208:44:492
817008992018,1cyclictest0-21swapper/112:33:561
817008992018,1cyclictest0-21swapper/110:59:401
817008992018,1cyclictest0-21swapper/110:25:191
817008992018,1cyclictest0-21swapper/110:08:091
817008992018,1cyclictest0-21swapper/108:42:521
817008992018,1cyclictest0-21swapper/108:42:521
817008992018,1cyclictest0-21swapper/108:35:211
817008992018,1cyclictest0-21swapper/107:51:481
817008992017,2cyclictest0-21swapper/108:20:211
817008992012,6cyclictest904701-21awk11:15:141
817008992012,6cyclictest818060-21swap07:10:211
81700899200,19cyclictest3056-21PK-Backend12:05:241
81700899200,19cyclictest3056-21PK-Backend09:53:141
81700899200,19cyclictest0-21swapper/112:37:481
81700899200,19cyclictest0-21swapper/111:20:411
81700899200,19cyclictest0-21swapper/111:01:431
81700899200,19cyclictest0-21swapper/110:54:331
81700899200,19cyclictest0-21swapper/110:23:161
81700899200,19cyclictest0-21swapper/109:03:461
81700899200,19cyclictest0-21swapper/108:31:291
81700899200,19cyclictest0-21swapper/107:44:181
81700899200,19cyclictest0-21swapper/107:21:471
81700699206,12cyclictest895824-21grep10:50:120
81700699200,18cyclictest877800-21apt-config10:00:010
139099200,17rtkit-daemon1317-21systemd11:05:132
817016991917,1cyclictest3056-21PK-Backend12:38:393
817016991917,1cyclictest3056-21PK-Backend11:08:553
817016991917,1cyclictest3056-21PK-Backend10:26:503
817016991917,1cyclictest3056-21PK-Backend07:40:553
817016991917,1cyclictest3056-21PK-Backend07:32:523
817016991917,1cyclictest3056-21PK-Backend07:18:113
817016991917,1cyclictest3056-21PK-Backend07:11:353
817016991915,3cyclictest978-21snmpd10:33:253
81701699190,7cyclictest835092-21cut08:00:003
81701699190,1cyclictest3056-21PK-Backend11:58:153
81701699190,1cyclictest3056-21PK-Backend10:44:013
81701699190,1cyclictest0-21swapper/307:50:023
81701699190,18cyclictest3056-21PK-Backend12:26:483
817012991917,1cyclictest0-21swapper/211:30:442
817012991916,2cyclictest341ksoftirqd/211:03:052
817012991913,5cyclictest907328-21sed11:20:222
817012991910,7cyclictest884965-21perf10:20:012
81701299190,1cyclictest0-21swapper/207:51:352
817008991917,1cyclictest0-21swapper/111:31:441
817008991917,1cyclictest0-21swapper/110:15:181
817008991917,1cyclictest0-21swapper/109:38:211
817008991917,1cyclictest0-21swapper/108:28:391
817008991914,3cyclictest827901-21idleruntime-cro07:40:001
817008991912,5cyclictest861163-21seq09:10:181
817008991911,6cyclictest823283-21cat07:25:151
817008991911,6cyclictest436-20systemd-journal09:10:001
817008991911,5cyclictest1095-21runrttasks08:10:211
81700899190,1cyclictest3056-21PK-Backend09:44:161
81700899190,1cyclictest3056-21PK-Backend09:44:161
81700899190,1cyclictest0-21swapper/109:19:021
81700899190,18cyclictest3056-21PK-Backend11:45:351
81700899190,18cyclictest3056-21PK-Backend11:43:331
81700899190,18cyclictest0-21swapper/107:33:221
81700699190,16cyclictest925378-21apt12:15:120
139099190,16rtkit-daemon1317-21systemd09:40:522
139099190,16rtkit-daemon1317-21systemd09:40:512
817016991816,1cyclictest3056-21PK-Backend08:40:383
817016991816,1cyclictest3056-21PK-Backend08:40:373
817016991816,1cyclictest0-21swapper/311:45:253
817016991815,2cyclictest3056-21PK-Backend12:22:573
81701699180,1cyclictest3056-21PK-Backend09:05:463
81701299186,3cyclictest141rcu_preempt10:55:182
817012991816,1cyclictest0-21swapper/207:46:562
817012991816,1cyclictest0-21swapper/207:13:442
817012991811,5cyclictest870075-21wc09:35:212
817008991816,1cyclictest0-21swapper/111:12:441
817008991816,1cyclictest0-21swapper/108:56:161
817008991816,1cyclictest0-21swapper/107:56:281
817008991811,5cyclictest854410-21sshd08:51:291
81700899180,1cyclictest0-21swapper/110:44:311
8307272170,4sleep33056-21PK-Backend07:45:153
817016991715,1cyclictest3056-21PK-Backend11:17:543
817016991715,1cyclictest3056-21PK-Backend10:08:523
817012991715,1cyclictest0-21swapper/210:20:142
817012991713,2cyclictest34-21ksoftirqd/210:40:102
817012991712,2cyclictest34-21ksoftirqd/207:20:182
817012991710,5cyclictest908971-21cat11:25:212
817008991715,1cyclictest3056-21PK-Backend10:35:331
817008991715,1cyclictest0-21swapper/111:52:331
817008991715,1cyclictest0-21swapper/108:46:441
817008991710,5cyclictest838060-21ntpq08:05:171
817008991614,1cyclictest0-21swapper/112:13:201
817008991614,1cyclictest0-21swapper/111:58:271
817012991511,2cyclictest34-21ksoftirqd/210:30:132
81701699140,12cyclictest8491752sleep308:36:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional