You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 08:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Jul 17, 2026 00:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
106054724170,7sleep00-21swapper/019:05:220
106816721460,4sleep00-21swapper/019:25:200
114375621260,4sleep10-21swapper/123:00:221
114142521040,3sleep10-21swapper/122:55:131
139099930,59rtkit-daemon0-21swapper/119:05:221
106098327639,14sleep30-21swapper/319:06:383
106102526940,22sleep20-21swapper/219:07:102
11210352650,6sleep10-21swapper/121:55:191
10645182510,2sleep13056-21PK-Backend19:15:181
11538322490,3sleep10-21swapper/123:30:171
10655572470,1sleep30-21swapper/319:20:103
10933082370,3sleep21093307-21gpg-connect-age20:40:002
1061383992710,15cyclictest1120125-21unin-run21:55:000
11547812250,4sleep21154779-21awk23:35:002
1061387992515,7cyclictest883-21dbus-daemon19:52:321
11554002240,3sleep0161rcuc/023:35:140
1061387992414,8cyclictest436-20systemd-journal00:00:011
106138799240,20cyclictest1101980-21basename21:05:001
1061383992414,8cyclictest1143755-21python323:00:220
1061390992315,4cyclictest1144356-21idleruntime-cro23:05:002
1061387992313,6cyclictest1107230-21nm-dispatcher21:17:361
1061383992321,1cyclictest0-21swapper/020:04:420
106138399231,3cyclictest1173525-21diskstats00:25:130
106138399230,21cyclictest1118134-21munin-run21:50:000
139099220,19rtkit-daemon26-21ksoftirqd/120:04:591
11115442220,4chrt3056-21PK-Backend21:30:143
106139299226,14cyclictest1103698-21apt-get21:10:003
106139299220,20cyclictest3056-21PK-Backend19:41:423
1061390992220,1cyclictest0-21swapper/223:24:502
1061390992220,1cyclictest0-21swapper/222:36:102
1061390992220,1cyclictest0-21swapper/222:21:162
1061390992220,1cyclictest0-21swapper/221:35:192
1061390992220,1cyclictest0-21swapper/221:27:562
1061390992220,1cyclictest0-21swapper/221:14:102
1061390992219,2cyclictest0-21swapper/220:52:352
106139099220,21cyclictest0-21swapper/221:19:042
106139099220,21cyclictest0-21swapper/220:57:482
106139099220,20cyclictest3056-21PK-Backend20:40:242
106139099220,20cyclictest1061960-21diskstats19:10:132
106138799228,12cyclictest1112639-21sh21:35:001
1061387992215,5cyclictest1158981-21cut23:45:171
1061387992212,8cyclictest957-21gdbus00:37:371
1061387992212,7cyclictest1100212-21cron21:00:011
106138799220,20cyclictest3056-21PK-Backend19:22:421
1061383992220,1cyclictest0-21swapper/023:27:270
1061383992220,1cyclictest0-21swapper/023:17:470
1061383992220,1cyclictest0-21swapper/022:49:420
1061383992220,1cyclictest0-21swapper/022:32:530
1061383992220,1cyclictest0-21swapper/021:56:090
1061383992220,1cyclictest0-21swapper/020:13:340
1061383992220,1cyclictest0-21swapper/019:34:070
1061383992214,6cyclictest1111899-21ntp_kernel_err21:30:330
106138399220,21cyclictest1166422-21if_err_enp2s000:05:190
106138399220,21cyclictest0-21swapper/023:43:360
106138399220,20cyclictest1153872-21memory23:30:180
11609732210,3sleep20-21swapper/223:50:192
1061392992119,1cyclictest3056-21PK-Backend20:07:303
106139299210,20cyclictest3056-21PK-Backend20:34:413
106139299210,19cyclictest3056-21PK-Backend23:55:173
106139299210,19cyclictest3056-21PK-Backend20:50:163
106139299210,19cyclictest3056-21PK-Backend20:43:203
106139299210,19cyclictest3056-21PK-Backend19:53:243
106139299210,19cyclictest3056-21PK-Backend00:20:453
1061390992119,1cyclictest0-21swapper/223:35:532
1061390992119,1cyclictest0-21swapper/222:12:242
1061390992119,1cyclictest0-21swapper/221:30:472
1061390992119,1cyclictest0-21swapper/221:07:142
1061390992119,1cyclictest0-21swapper/221:04:032
1061390992119,1cyclictest0-21swapper/200:31:222
1061390992119,1cyclictest0-21swapper/200:26:552
1061390992119,1cyclictest0-21swapper/200:02:232
1061390992114,5cyclictest1172630-21munin-plugin-st00:25:002
1061390992113,6cyclictest1148862-21awk23:15:202
106139099210,20cyclictest0-21swapper/222:42:332
106139099210,20cyclictest0-21swapper/221:56:422
106139099210,20cyclictest0-21swapper/220:13:062
106139099210,20cyclictest0-21swapper/219:35:422
106139099210,20cyclictest0-21swapper/200:35:412
106139099210,20cyclictest0-21swapper/200:16:412
106139099210,20cyclictest0-21swapper/200:10:532
1061387992119,1cyclictest3056-21PK-Backend19:12:281
1061387992119,1cyclictest0-21swapper/123:50:371
1061387992119,1cyclictest0-21swapper/123:36:521
1061387992119,1cyclictest0-21swapper/122:40:281
1061387992119,1cyclictest0-21swapper/122:06:071
1061387992119,1cyclictest0-21swapper/120:45:301
1061387992119,1cyclictest0-21swapper/120:37:591
1061387992119,1cyclictest0-21swapper/120:30:441
1061387992119,1cyclictest0-21swapper/120:05:351
1061387992119,1cyclictest0-21swapper/119:36:211
1061387992112,6cyclictest2175-21JS10:27:481
106138799210,20cyclictest0-21swapper/122:37:171
106138799210,20cyclictest0-21swapper/120:26:311
106138799210,20cyclictest0-21swapper/100:30:321
106138799210,1cyclictest0-21swapper/100:05:041
106138799210,19cyclictest3056-21PK-Backend19:27:151
1061383992119,1cyclictest0-21swapper/023:50:540
1061383992119,1cyclictest0-21swapper/021:41:290
1061383992119,1cyclictest0-21swapper/021:13:450
1061383992119,1cyclictest0-21swapper/021:03:160
1061383992119,1cyclictest0-21swapper/020:42:550
1061383992119,1cyclictest0-21swapper/020:37:210
1061383992114,5cyclictest13-21ksoftirqd/022:50:130
106138399210,20cyclictest0-21swapper/023:47:150
106138399210,20cyclictest0-21swapper/023:21:460
106138399210,20cyclictest0-21swapper/023:10:580
106138399210,20cyclictest0-21swapper/022:29:410
106138399210,20cyclictest0-21swapper/020:51:470
106138399210,20cyclictest0-21swapper/019:18:320
106138399210,20cyclictest0-21swapper/000:32:100
1061392992018,1cyclictest0-21swapper/323:03:133
1061392992017,2cyclictest3056-21PK-Backend22:17:083
1061392992013,5cyclictest1107451-21cron21:20:003
106139299200,1cyclictest3056-21PK-Backend21:14:293
106139299200,1cyclictest3056-21PK-Backend20:58:343
106139299200,1cyclictest3056-21PK-Backend19:55:473
106139299200,19cyclictest3056-21PK-Backend22:56:503
106139299200,19cyclictest3056-21PK-Backend22:25:263
106139299200,19cyclictest3056-21PK-Backend21:52:483
106139299200,19cyclictest3056-21PK-Backend21:39:163
106139299200,19cyclictest3056-21PK-Backend21:23:073
106139299200,19cyclictest3056-21PK-Backend20:18:593
106139299200,19cyclictest3056-21PK-Backend20:12:183
106139299200,18cyclictest3056-21PK-Backend22:22:293
106139299200,18cyclictest3056-21PK-Backend00:31:193
106139299200,18cyclictest1069484-21irqstats19:30:153
1061390992018,1cyclictest0-21swapper/222:58:072
1061390992018,1cyclictest0-21swapper/222:50:362
1061390992018,1cyclictest0-21swapper/221:45:542
1061390992018,1cyclictest0-21swapper/220:46:462
1061390992018,1cyclictest0-21swapper/220:23:082
1061390992018,1cyclictest0-21swapper/219:54:012
1061390992018,1cyclictest0-21swapper/219:27:582
1061390992012,6cyclictest1064328-21ls19:15:172
106139099200,1cyclictest0-21swapper/223:49:032
106139099200,1cyclictest0-21swapper/223:11:322
106139099200,1cyclictest0-21swapper/222:07:442
106139099200,19cyclictest0-21swapper/223:59:382
106139099200,19cyclictest0-21swapper/223:42:212
106139099200,19cyclictest0-21swapper/222:46:172
106139099200,19cyclictest0-21swapper/221:40:202
106139099200,19cyclictest0-21swapper/221:20:132
106139099200,19cyclictest0-21swapper/220:05:362
106139099200,19cyclictest0-21swapper/200:05:392
1061387992018,1cyclictest3056-21PK-Backend21:27:201
1061387992018,1cyclictest0-21swapper/123:43:211
1061387992018,1cyclictest0-21swapper/122:32:031
1061387992018,1cyclictest0-21swapper/121:52:421
1061387992018,1cyclictest0-21swapper/121:11:121
1061387992018,1cyclictest0-21swapper/120:17:171
1061387992011,7cyclictest1165433-21cron00:05:001
1061387992010,7cyclictest1146091-21perf23:10:001
106138799200,19cyclictest0-21swapper/121:21:531
106138799200,19cyclictest0-21swapper/121:05:241
106138799200,19cyclictest0-21swapper/119:41:421
106138799200,19cyclictest0-21swapper/100:12:481
1061383992018,1cyclictest0-21swapper/022:38:060
1061383992018,1cyclictest0-21swapper/021:37:030
1061383992018,1cyclictest0-21swapper/020:30:040
1061383992018,1cyclictest0-21swapper/020:25:170
1061383992018,1cyclictest0-21swapper/020:22:470
1061383992018,1cyclictest0-21swapper/019:40:160
1061383992018,1cyclictest0-21swapper/000:01:270
1061383992014,4cyclictest2156-21gnome-shell21:29:090
1061383992013,6cyclictest1124114-21expr22:05:140
1061383992011,7cyclictest1095650-21df_inode20:45:130
106138399200,1cyclictest0-21swapper/023:55:050
106138399200,1cyclictest0-21swapper/019:21:290
106138399200,19cyclictest0-21swapper/022:02:170
106138399200,19cyclictest0-21swapper/020:57:350
106138399200,19cyclictest0-21swapper/019:37:530
106138399200,19cyclictest0-21swapper/000:38:040
106138399200,19cyclictest0-21swapper/000:14:590
11485032190,2sleep30-21swapper/323:15:163
11207922190,4chrt3056-21PK-Backend21:55:163
1061392991917,1cyclictest3056-21PK-Backend20:37:113
1061392991917,1cyclictest3056-21PK-Backend00:37:013
1061392991917,1cyclictest0-21swapper/323:12:183
1061392991917,1cyclictest0-21swapper/320:21:293
1061392991917,1cyclictest0-21swapper/319:10:523
1061392991916,2cyclictest3056-21PK-Backend22:10:533
1061392991916,2cyclictest3056-21PK-Backend00:05:313
1061392991914,3cyclictest1146082-21dump-pmu-power23:10:003
106139299190,1cyclictest3056-21PK-Backend23:34:283
106139299190,1cyclictest3056-21PK-Backend22:44:053
106139299190,1cyclictest3056-21PK-Backend22:36:163
106139299190,1cyclictest3056-21PK-Backend22:05:403
106139299190,1cyclictest3056-21PK-Backend22:03:023
106139299190,1cyclictest3056-21PK-Backend21:04:013
106139299190,1cyclictest3056-21PK-Backend20:48:203
106139299190,1cyclictest3056-21PK-Backend00:25:453
106139299190,1cyclictest3056-21PK-Backend00:11:533
106139299190,1cyclictest0-21swapper/319:28:373
106139299190,18cyclictest3056-21PK-Backend21:47:143
106139299190,18cyclictest3056-21PK-Backend00:04:093
1061390991916,2cyclictest3056-21PK-Backend20:25:032
1061390991916,2cyclictest3056-21PK-Backend20:15:032
1061390991916,2cyclictest0-21swapper/219:30:212
1061390991914,3cyclictest1065734-21cstates19:20:112
1061390991912,5cyclictest1145068-21cat23:05:162
106139099190,1cyclictest0-21swapper/222:01:562
106139099190,1cyclictest0-21swapper/220:00:362
1061387991917,1cyclictest0-21swapper/123:20:221
1061387991913,5cyclictest1129875-21rt-features22:20:211
1061387991912,5cyclictest1171604-21cat00:20:161
1061387991911,6cyclictest1131683-21cat22:25:221
1061387991910,7cyclictest2638-21gdbus20:12:321
1061387991910,6cyclictest1127029-21latency_hist22:15:001
106138799190,1cyclictest0-21swapper/122:16:071
106138799190,1cyclictest0-21swapper/122:00:531
106138799190,1cyclictest0-21swapper/120:43:341
106138799190,1cyclictest0-21swapper/119:59:471
106138799190,1cyclictest0-21swapper/119:48:381
106138799190,18cyclictest0-21swapper/100:25:321
1061383991917,1cyclictest0-21swapper/021:05:540
1061383991917,1cyclictest0-21swapper/020:08:490
1061383991917,1cyclictest0-21swapper/019:45:360
1061383991913,5cyclictest883-21dbus-daemon21:17:360
1061383991912,5cyclictest1127153-21date22:15:010
1061383991912,5cyclictest1063051-21sh19:15:010
106138399190,1cyclictest0-21swapper/000:20:470
139099180,16rtkit-daemon13-21ksoftirqd/019:50:160
1061392991816,1cyclictest3056-21PK-Backend23:42:393
1061392991816,1cyclictest3056-21PK-Backend22:53:393
1061392991816,1cyclictest3056-21PK-Backend00:18:153
1061392991811,5cyclictest436-20systemd-journal19:16:313
1061392991811,5cyclictest1071136-21which19:35:143
1061392991810,6cyclictest1153016-21cron23:29:593
106139299180,1cyclictest3056-21PK-Backend23:39:013
106139299180,1cyclictest3056-21PK-Backend20:02:573
1061390991816,1cyclictest0-21swapper/222:32:592
1061390991816,1cyclictest0-21swapper/222:27:452
1061390991816,1cyclictest0-21swapper/222:15:222
1061390991814,3cyclictest978-21snmpd19:46:162
106138799184,12cyclictest1147823-21sh23:15:001
1061387991816,1cyclictest0-21swapper/120:22:521
1061387991811,4cyclictest1118330-21apt-key21:50:001
106138799181,12cyclictest1114643-21apt21:40:111
1061387991810,2cyclictest26-21ksoftirqd/123:30:001
106138799180,1cyclictest3056-21PK-Backend19:31:491
106138399189,8cyclictest1079177-21unin-run20:00:000
106138399180,16cyclictest1169993-21sed00:15:170
106139299171,15cyclictest1109800-21df21:25:123
1061390991715,1cyclictest0-21swapper/220:32:202
1061390991712,3cyclictest1079311-21latency_hist20:00:002
1061387991715,1cyclictest0-21swapper/121:38:361
1061387991712,3cyclictest1170151-21ntp_offset00:15:191
1061387991710,5cyclictest1097569-21cat20:50:171
1061383991715,1cyclictest0-21swapper/023:05:170
1061383991712,3cyclictest1128201-21switchtime22:15:240
1061383991711,4cyclictest2156-21gnome-shell20:19:090
1061383991711,4cyclictest1136983-21sed22:40:230
1061392991614,1cyclictest3056-21PK-Backend23:46:183
1061392991614,1cyclictest3056-21PK-Backend22:45:143
1061392991614,1cyclictest3056-21PK-Backend20:29:543
1061387991610,4cyclictest1-21systemd23:20:021
106138399161,11cyclictest1141135-21/usr/sbin/munin22:55:240
139099150,4rtkit-daemon3056-21PK-Backend21:40:493
1061392991513,1cyclictest3056-21PK-Backend23:50:313
1061392991513,1cyclictest3056-21PK-Backend22:33:383
1061392991510,3cyclictest1075668-21latency_hist19:50:013
1061390991513,1cyclictest3056-21PK-Backend19:40:282
1061390991511,3cyclictest886-21NetworkManager23:27:362
1061390991511,3cyclictest1119212-21missed_timers21:50:182
1061387991511,2cyclictest26-21ksoftirqd/122:45:131
1061383991513,1cyclictest0-21swapper/021:22:220
1061383991510,3cyclictest1-21systemd22:25:010
1061392991310,2cyclictest3056-21PK-Backend23:25:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional