You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 16:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Mon May 11, 2026 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
124331321070,3sleep23056-21PK-Backend12:35:172
123253821070,3sleep20-21swapper/212:05:142
11700862990,4sleep23056-21PK-Backend09:05:222
139099850,73rtkit-daemon0-21swapper/207:07:412
112746027956,14sleep30-21swapper/307:06:583
112758727051,13sleep10-21swapper/107:08:351
112662726644,13sleep00-21swapper/007:05:170
12206162460,4sleep10-21swapper/111:30:191
112783599371,24cyclictest1233298-21runrttasks12:05:550
112783599360,21cyclictest1240535-21chrt12:26:280
112783599351,19cyclictest1242361-21chrt12:32:300
112783599350,22cyclictest1221063-21seq11:33:110
112783599350,20cyclictest1145815-21taskset07:55:380
112783599350,20cyclictest1133552-21taskset07:23:020
112783599344,17cyclictest1156416-21taskset08:28:180
112783599341,18cyclictest2156-21gnome-shell11:53:090
112783599340,20cyclictest1222811-21chrt11:38:120
112783599340,20cyclictest1212412-21taskset11:08:290
112783599340,19cyclictest1191470-21taskset10:06:380
112783599340,19cyclictest1164498-21ntp_states08:50:180
112783599339,16cyclictest1095-21runrttasks12:17:590
112783599334,16cyclictest1142357-21taskset07:45:440
112783599334,16cyclictest0-21swapper/012:36:550
112783599331,19cyclictest1894998-21snapd10:49:140
112783599331,18cyclictest1219245-21taskset11:27:070
112783599331,17cyclictest1224556-21taskset11:43:180
112783599330,20cyclictest1180855-21chrt09:37:140
112783599330,20cyclictest1163401-21taskset08:49:000
112783599330,20cyclictest1161569-21taskset08:42:410
112783599330,19cyclictest1095-21runrttasks10:23:590
112783599330,19cyclictest1095-21runrttasks09:31:530
112783599330,18cyclictest1095-21runrttasks09:52:270
112783599330,17cyclictest1235083-21cat12:10:220
112783599321,19cyclictest1152958-21taskset08:18:240
112783599320,30cyclictest1189406-21grep10:00:210
112783599320,20cyclictest1214209-21chrt11:14:180
112783599320,20cyclictest1210616-21chrt11:02:400
112783599320,20cyclictest1168523-21timerwakeupswit09:00:220
112783599320,19cyclictest1188160-21taskset09:58:360
112783599320,19cyclictest1144163-21chrt07:51:410
112783599320,19cyclictest1129447-21chrt07:14:310
112783599320,18cyclictest1158141-21taskset08:33:120
112783599320,18cyclictest1095-21runrttasks10:51:220
112783599310,19cyclictest1231607-21chrt12:01:260
112783599310,19cyclictest1182740-21taskset09:44:130
112783599310,19cyclictest1159685-21chrt08:35:430
112783599310,19cyclictest1135223-21taskset07:27:150
112783599310,18cyclictest1217410-21taskset11:20:530
112783599310,18cyclictest1194881-21taskset10:15:570
112783599310,18cyclictest1173907-21chrt09:17:020
112783599310,18cyclictest1166897-21taskset08:56:010
112783599310,17cyclictest883-21dbus-daemon10:44:590
112783599302,16cyclictest1151230-21seq08:13:300
112783599300,18cyclictest1170925-21chrt09:10:090
112783599300,17cyclictest1177297-21taskset09:26:040
112783599280,18cyclictest1138987-21chrt07:37:410
112783599280,17cyclictest1146408-21taskset08:00:070
11848412270,16sleep234-21ksoftirqd/209:50:002
1127837992718,7cyclictest1152335-21/usr/sbin/munin08:15:161
12216172260,9sleep126-21ksoftirqd/111:35:131
11393722260,4sleep30-21swapper/307:40:003
112783599262,22cyclictest1214757-21cstates11:15:150
11747962250,10sleep342-21ksoftirqd/309:20:143
1127837992518,5cyclictest1193509-21munin-run10:15:011
112783799250,23cyclictest1190651-21df_inode10:05:141
112783599258,15cyclictest1139651-21tr07:40:120
112783599251,21cyclictest1129910-21munin-node07:15:010
1127845992412,10cyclictest42-21ksoftirqd/311:00:203
1127842992410,12cyclictest1209112-21unin-run11:00:012
112783599241,21cyclictest1197644-21/usr/sbin/munin10:25:160
112783599241,19cyclictest1135953-21apt-get07:30:040
112783599240,23cyclictest0-21swapper/010:12:340
112784599231,19cyclictest0-21swapper/307:20:123
1127842992321,1cyclictest0-21swapper/209:39:322
112783799230,21cyclictest11443242sleep107:53:471
112783599234,17cyclictest1169315-21ls09:05:120
112783599233,18cyclictest1207840-21expr10:55:150
112783599231,21cyclictest1226559-21latency_hist11:50:010
112783599231,20cyclictest1174569-21cpuspeed_turbos09:20:130
1127835992310,11cyclictest1149046-21swap08:05:210
112783599230,22cyclictest1894998-21snapd12:24:140
112783599230,19cyclictest3056-21PK-Backend09:49:160
112784599220,21cyclictest3056-21PK-Backend07:43:453
112784599220,20cyclictest3056-21PK-Backend08:46:053
112784599220,20cyclictest3056-21PK-Backend07:45:203
112784299225,15cyclictest1235613-21cron12:15:002
1127842992220,1cyclictest0-21swapper/211:48:362
1127842992220,1cyclictest0-21swapper/211:16:462
1127842992220,1cyclictest0-21swapper/209:11:482
1127842992220,1cyclictest0-21swapper/207:28:252
1127842992219,2cyclictest0-21swapper/208:33:222
1127842992213,7cyclictest1224090-21sensors_temp11:40:202
1127842992213,7cyclictest1200418-21cron10:35:002
1127837992214,6cyclictest1166621-21python308:55:211
112783599229,11cyclictest1199160-21cstates10:30:130
112783599221,18cyclictest1229566-21python311:55:210
112783599221,18cyclictest1153344-21/usr/sbin/munin08:20:110
1127845992113,6cyclictest1215891-21sshd11:18:363
112784599210,19cyclictest3056-21PK-Backend09:33:153
112784599210,19cyclictest3056-21PK-Backend08:28:003
112784599210,19cyclictest3056-21PK-Backend08:13:193
1127842992119,1cyclictest0-21swapper/212:00:272
1127842992119,1cyclictest0-21swapper/211:35:462
1127842992119,1cyclictest0-21swapper/211:33:442
1127842992119,1cyclictest0-21swapper/211:26:462
1127842992119,1cyclictest0-21swapper/211:01:192
1127842992119,1cyclictest0-21swapper/209:40:342
1127842992119,1cyclictest0-21swapper/209:17:152
1127842992119,1cyclictest0-21swapper/208:48:082
1127842992119,1cyclictest0-21swapper/208:27:132
1127842992119,1cyclictest0-21swapper/208:21:522
1127842992119,1cyclictest0-21swapper/208:06:322
1127842992118,2cyclictest3056-21PK-Backend08:55:472
1127842992115,4cyclictest886-21NetworkManager10:10:012
112784299210,20cyclictest3056-21PK-Backend10:21:102
112784299210,20cyclictest0-21swapper/207:51:242
112784299210,20cyclictest0-21swapper/207:39:422
1127837992112,6cyclictest436-20systemd-journal07:38:351
112783799210,20cyclictest1159236-21strings08:35:171
112783799210,19cyclictest3056-21PK-Backend10:40:021
112783599219,11cyclictest1201786-21sshd10:35:350
112784599209,9cyclictest42-21ksoftirqd/309:45:213
112784599207,11cyclictest1230171-21sh12:00:013
1127845992018,1cyclictest3056-21PK-Backend12:24:183
1127845992018,1cyclictest3056-21PK-Backend09:11:393
1127845992017,2cyclictest3056-21PK-Backend11:41:453
1127845992017,2cyclictest3056-21PK-Backend08:24:153
1127845992017,2cyclictest3056-21PK-Backend07:57:383
1127845992012,6cyclictest1225267-21expr11:45:133
112784599200,7cyclictest1200674-21awk10:35:113
112784599200,1cyclictest3056-21PK-Backend12:35:533
112784599200,1cyclictest3056-21PK-Backend12:32:013
112784599200,19cyclictest3056-21PK-Backend11:11:033
112784599200,19cyclictest3056-21PK-Backend09:44:103
112784599200,19cyclictest3056-21PK-Backend09:27:493
112784599200,19cyclictest3056-21PK-Backend07:11:343
112784599200,19cyclictest0-21swapper/309:50:403
112784599200,18cyclictest3056-21PK-Backend10:12:433
112784599200,18cyclictest3056-21PK-Backend08:32:333
1127842992018,1cyclictest0-21swapper/212:24:192
1127842992018,1cyclictest0-21swapper/209:04:592
1127842992018,1cyclictest0-21swapper/208:35:462
1127842992018,1cyclictest0-21swapper/208:17:532
1127842992018,1cyclictest0-21swapper/207:21:572
1127842992016,2cyclictest3056-21PK-Backend11:10:032
1127842992013,5cyclictest1201126-21cat10:35:162
1127842992011,2cyclictest34-21ksoftirqd/207:55:112
112784299200,1cyclictest0-21swapper/212:15:202
112784299200,1cyclictest0-21swapper/210:41:452
112784299200,1cyclictest0-21swapper/209:25:342
112784299200,19cyclictest0-21swapper/212:32:172
112784299200,19cyclictest0-21swapper/211:08:022
112784299200,19cyclictest0-21swapper/210:51:312
112784299200,19cyclictest0-21swapper/210:00:022
112784299200,19cyclictest0-21swapper/208:52:212
112784299200,19cyclictest0-21swapper/207:49:002
112784299200,19cyclictest0-21swapper/207:41:172
112784299200,19cyclictest0-21swapper/207:12:112
1127837992018,1cyclictest0-21swapper/112:01:471
1127837992018,1cyclictest0-21swapper/111:26:591
1127837992018,1cyclictest0-21swapper/109:53:301
1127837992018,1cyclictest0-21swapper/109:36:081
1127837992018,1cyclictest0-21swapper/108:45:041
1127837992013,5cyclictest436-20systemd-journal10:20:011
1127837992013,5cyclictest1209110-21systemd-run11:00:001
112783799200,1cyclictest0-21swapper/112:15:271
112783799200,1cyclictest0-21swapper/112:10:331
112783799200,1cyclictest0-21swapper/111:55:041
112783799200,1cyclictest0-21swapper/108:40:041
112783799200,1cyclictest0-21swapper/107:29:391
112783799200,19cyclictest3056-21PK-Backend10:03:251
112783799200,19cyclictest0-21swapper/112:21:201
112783799200,19cyclictest0-21swapper/111:47:211
112783799200,19cyclictest0-21swapper/111:10:311
112783799200,19cyclictest0-21swapper/110:28:251
112783799200,19cyclictest0-21swapper/109:55:201
112783799200,19cyclictest0-21swapper/109:09:041
112783799200,19cyclictest0-21swapper/108:25:231
112783799200,19cyclictest0-21swapper/107:30:271
112783799200,19cyclictest0-21swapper/107:20:281
12169062190,2sleep10-21swapper/111:20:171
11833302190,4sleep10-21swapper/109:45:091
1127845991917,1cyclictest3056-21PK-Backend11:08:123
1127845991917,1cyclictest3056-21PK-Backend09:05:513
1127845991917,1cyclictest0-21swapper/310:44:263
1127845991917,1cyclictest0-21swapper/310:07:023
1127845991916,2cyclictest3056-21PK-Backend11:35:023
1127845991916,2cyclictest3056-21PK-Backend10:16:363
1127845991912,5cyclictest1174304-21date09:20:013
112784599190,1cyclictest3056-21PK-Backend12:13:293
112784599190,1cyclictest3056-21PK-Backend12:08:353
112784599190,1cyclictest3056-21PK-Backend11:30:103
112784599190,1cyclictest3056-21PK-Backend11:22:263
112784599190,1cyclictest3056-21PK-Backend10:52:113
112784599190,1cyclictest3056-21PK-Backend10:31:493
112784599190,1cyclictest3056-21PK-Backend09:58:443
112784599190,1cyclictest3056-21PK-Backend09:00:033
112784599190,1cyclictest3056-21PK-Backend08:37:053
112784599190,1cyclictest3056-21PK-Backend08:09:203
112784599190,1cyclictest3056-21PK-Backend07:28:113
112784599190,1cyclictest0-21swapper/310:45:283
112784599190,18cyclictest3056-21PK-Backend12:28:103
112784599190,18cyclictest3056-21PK-Backend12:18:223
112784599190,18cyclictest3056-21PK-Backend07:50:423
112784599190,18cyclictest0-21swapper/310:02:223
1127842991917,1cyclictest3056-21PK-Backend10:47:392
1127842991917,1cyclictest0-21swapper/211:55:342
1127842991917,1cyclictest0-21swapper/211:50:392
1127842991917,1cyclictest0-21swapper/210:12:242
1127842991917,1cyclictest0-21swapper/209:53:332
1127842991917,1cyclictest0-21swapper/209:30:142
1127842991917,1cyclictest0-21swapper/207:32:122
1127842991916,2cyclictest0-21swapper/208:43:572
112784299190,1cyclictest0-21swapper/210:16:162
112784299190,18cyclictest0-21swapper/211:23:422
112783799199,8cyclictest918-21rs:main1
112783799196,11cyclictest1151396-21unin-run08:15:011
1127837991917,1cyclictest0-21swapper/112:33:101
1127837991917,1cyclictest0-21swapper/111:43:291
1127837991917,1cyclictest0-21swapper/111:16:121
1127837991917,1cyclictest0-21swapper/108:30:431
1127837991913,4cyclictest1242924-21cpuspeed12:35:141
1127837991912,5cyclictest1227133-21grep11:50:111
1127837991912,5cyclictest1167671-21grep09:00:121
1127837991911,6cyclictest1195474-21awk10:20:111
112783799190,1cyclictest3056-21PK-Backend10:45:431
112783799190,1cyclictest0-21swapper/108:52:271
112783799190,1cyclictest0-21swapper/108:23:481
112783799190,1cyclictest0-21swapper/107:56:091
112783799190,18cyclictest3056-21PK-Backend10:39:011
112783799190,18cyclictest0-21swapper/111:09:291
112783799190,18cyclictest0-21swapper/111:01:581
112783799190,18cyclictest0-21swapper/110:32:181
112783799190,18cyclictest0-21swapper/109:33:171
112783799190,18cyclictest0-21swapper/109:16:541
112783799190,18cyclictest0-21swapper/109:13:161
1127845991816,1cyclictest3056-21PK-Backend10:28:593
1127845991816,1cyclictest3056-21PK-Backend08:53:283
1127845991811,5cyclictest1228579-21sed11:55:013
1127842991815,2cyclictest3056-21PK-Backend12:27:242
1127837991816,1cyclictest0-21swapper/107:40:561
1127837991810,6cyclictest436-20systemd-journal08:10:001
1127837991810,6cyclictest1239352-21awk12:25:111
1127845991715,1cyclictest3056-21PK-Backend12:02:543
1127845991715,1cyclictest3056-21PK-Backend11:27:203
1127845991715,1cyclictest3056-21PK-Backend10:56:033
1127842991715,1cyclictest0-21swapper/210:28:072
1127842991715,1cyclictest0-21swapper/209:20:072
1127842991715,1cyclictest0-21swapper/208:00:102
1127842991712,3cyclictest34-21ksoftirqd/207:15:022
112783799179,6cyclictest436-20systemd-journal07:20:001
112783799179,6cyclictest1182424-21users09:40:211
112783799179,6cyclictest1129126-21awk07:10:241
1127837991715,1cyclictest0-21swapper/110:50:361
1127837991710,5cyclictest1176486-21expr09:25:131
112784599167,3cyclictest0-21swapper/308:05:013
1127845991614,1cyclictest3056-21PK-Backend10:22:173
1127845991614,1cyclictest3056-21PK-Backend09:36:063
112784299169,5cyclictest1150629-21awk08:10:192
1127842991613,2cyclictest0-21swapper/209:56:232
112783799168,6cyclictest1146799-21df08:00:131
1127837991614,1cyclictest0-21swapper/107:46:171
1127845991513,1cyclictest3056-21PK-Backend08:59:163
1127837991512,2cyclictest1175888-21latency_hist09:25:001
112784599141,12cyclictest1160971-21needreboot08:40:173
1127845991410,3cyclictest1131990-21latency_hist07:20:003
112784599134,7cyclictest42-21ksoftirqd/308:15:143
1127845991210,1cyclictest3056-21PK-Backend07:32:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional