You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 05:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Apr 18, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40630822530,9sleep20-21swapper/219:05:262
40574822500,16sleep013-21ksoftirqd/019:05:150
45353521260,3sleep30-21swapper/321:20:173
42913921240,3sleep13056-21PK-Backend20:10:181
51742621150,3sleep342-21ksoftirqd/300:20:213
40607127455,11sleep10-21swapper/119:05:211
139099740,61rtkit-daemon0-21swapper/319:08:533
5121452430,8sleep2331rcuc/200:05:212
4922542330,3sleep03056-21PK-Backend23:10:180
4518812270,4sleep13056-21PK-Backend21:15:181
4162252270,4sleep13056-21PK-Backend19:35:011
139099270,4rtkit-daemon1389-21rtkit-daemon23:20:222
4440182250,3sleep0444021-21sh20:55:000
40696499250,23cyclictest522859-21chrt00:37:202
406967992414,9cyclictest978-21snmpd00:16:163
406967992412,10cyclictest978-21snmpd00:11:153
406967992315,6cyclictest452080-21python321:15:213
40696799231,19cyclictest475206-21apt-get22:25:003
40696499230,3cyclictest435392-21munin-run20:30:002
4654782220,6sleep30-21swapper/321:55:133
40696799222,18cyclictest478619-21anacron22:31:043
406967992220,1cyclictest0-21swapper/322:41:113
40696799221,8cyclictest436492-21python320:30:203
406967992212,7cyclictest503123-21python323:40:213
40696799220,20cyclictest886-21NetworkManager20:39:163
406964992220,1cyclictest0-21swapper/221:56:052
406962992220,1cyclictest3056-21PK-Backend21:50:371
406962992219,2cyclictest3056-21PK-Backend20:52:161
40696299220,20cyclictest3056-21PK-Backend00:18:331
40695799220,2cyclictest494803-21munin-run23:20:000
139099220,19rtkit-daemon1788643-21eadedCompositor00:03:013
406967992119,1cyclictest0-21swapper/322:05:143
406967992119,1cyclictest0-21swapper/320:53:373
406967992119,1cyclictest0-21swapper/320:43:423
406967992119,1cyclictest0-21swapper/320:26:253
406967992119,1cyclictest0-21swapper/320:23:013
406967992119,1cyclictest0-21swapper/320:13:003
406967992119,1cyclictest0-21swapper/319:58:143
406967992119,1cyclictest0-21swapper/319:53:203
406967992119,1cyclictest0-21swapper/319:14:533
406967992119,1cyclictest0-21swapper/300:08:583
406967992118,2cyclictest3056-21PK-Backend23:24:233
406967992118,2cyclictest0-21swapper/322:10:153
40696799211,4cyclictest411102-21apt19:20:103
406967992114,5cyclictest410534-21sshd19:16:233
406967992113,6cyclictest427282-21memory20:05:163
406967992111,7cyclictest501430-21switchtime23:35:213
40696799210,20cyclictest3056-21PK-Backend21:32:223
40696799210,20cyclictest3056-21PK-Backend19:41:573
40696799210,20cyclictest0-21swapper/320:45:183
40696799210,20cyclictest0-21swapper/300:39:473
40696799210,19cyclictest3056-21PK-Backend23:33:293
406964992119,1cyclictest0-21swapper/223:40:082
406964992119,1cyclictest0-21swapper/223:35:292
406964992119,1cyclictest0-21swapper/223:08:182
406964992119,1cyclictest0-21swapper/222:52:582
406964992119,1cyclictest0-21swapper/222:36:002
406964992119,1cyclictest0-21swapper/222:22:572
406964992119,1cyclictest0-21swapper/222:19:312
406964992119,1cyclictest0-21swapper/221:29:032
406964992119,1cyclictest0-21swapper/220:32:382
406964992119,1cyclictest0-21swapper/220:02:172
406964992119,1cyclictest0-21swapper/219:53:172
406964992119,1cyclictest0-21swapper/219:22:142
406964992119,1cyclictest0-21swapper/200:32:412
406964992119,1cyclictest0-21swapper/200:29:502
406964992119,1cyclictest0-21swapper/200:20:312
406964992119,1cyclictest0-21swapper/200:11:592
406964992118,2cyclictest0-21swapper/222:10:182
406964992118,2cyclictest0-21swapper/220:18:592
406964992117,3cyclictest34-21ksoftirqd/219:55:192
406964992113,6cyclictest488272-21expr23:00:122
40696499210,20cyclictest0-21swapper/222:06:062
40696499210,20cyclictest0-21swapper/221:40:242
40696499210,20cyclictest0-21swapper/220:23:112
40696499210,20cyclictest0-21swapper/200:00:372
406962992119,1cyclictest3056-21PK-Backend22:42:011
406962992119,1cyclictest3056-21PK-Backend19:57:351
406962992119,1cyclictest3056-21PK-Backend00:35:091
406962992119,1cyclictest3056-21PK-Backend00:21:241
406962992119,1cyclictest3056-21PK-Backend00:13:061
406962992119,1cyclictest3056-21PK-Backend00:01:091
406962992119,1cyclictest0-21swapper/123:33:531
406962992118,2cyclictest3056-21PK-Backend22:08:211
40696299210,20cyclictest3056-21PK-Backend20:35:331
40696299210,20cyclictest0-21swapper/119:38:421
40696299210,19cyclictest3056-21PK-Backend23:41:091
40696299210,19cyclictest3056-21PK-Backend22:36:471
406957992119,1cyclictest0-21swapper/023:45:150
406957992119,1cyclictest0-21swapper/022:50:190
406957992119,1cyclictest0-21swapper/022:46:210
406957992119,1cyclictest0-21swapper/022:07:340
406957992119,1cyclictest0-21swapper/020:44:330
406957992119,1cyclictest0-21swapper/020:24:320
406957992119,1cyclictest0-21swapper/020:06:210
406957992119,1cyclictest0-21swapper/000:23:000
406957992113,6cyclictest498514-21cat23:30:110
40695799211,18cyclictest457885-21munin-run21:35:000
40695799210,20cyclictest0-21swapper/022:39:590
40695799210,20cyclictest0-21swapper/019:29:440
406967992018,1cyclictest3056-21PK-Backend19:30:013
406967992018,1cyclictest0-21swapper/322:01:023
406967992018,1cyclictest0-21swapper/320:00:163
406967992017,2cyclictest0-21swapper/320:55:113
406967992014,4cyclictest42-21ksoftirqd/323:55:003
406967992014,4cyclictest42-21ksoftirqd/323:55:003
406967992013,6cyclictest454790-21cat21:25:123
406967992013,5cyclictest505488-21pluginstate23:50:003
40696799201,17cyclictest472014-21apt22:15:103
40696799200,19cyclictest3056-21PK-Backend23:06:593
40696799200,19cyclictest3056-21PK-Backend19:26:243
40696799200,19cyclictest0-21swapper/323:01:323
40696799200,19cyclictest0-21swapper/321:40:203
406964992018,1cyclictest0-21swapper/223:46:372
406964992018,1cyclictest0-21swapper/222:42:162
406964992018,1cyclictest0-21swapper/221:05:432
406964992018,1cyclictest0-21swapper/221:03:202
406964992018,1cyclictest0-21swapper/220:53:122
406964992018,1cyclictest0-21swapper/220:10:472
406964992018,1cyclictest0-21swapper/220:06:212
406964992018,1cyclictest0-21swapper/219:38:512
406964992018,1cyclictest0-21swapper/219:32:222
406964992012,6cyclictest464826-21cron21:55:002
406964992011,7cyclictest34-21ksoftirqd/220:40:122
40696499200,1cyclictest0-21swapper/221:20:502
40696499200,19cyclictest0-21swapper/222:32:222
40696499200,19cyclictest0-21swapper/222:01:532
40696499200,19cyclictest0-21swapper/221:46:322
40696499200,19cyclictest0-21swapper/221:38:482
40696499200,19cyclictest0-21swapper/219:48:232
406962992018,1cyclictest3056-21PK-Backend23:03:451
406962992018,1cyclictest3056-21PK-Backend22:52:281
406962992018,1cyclictest3056-21PK-Backend22:47:141
406962992018,1cyclictest3056-21PK-Backend22:31:341
406962992018,1cyclictest3056-21PK-Backend22:16:321
406962992018,1cyclictest3056-21PK-Backend20:46:221
406962992018,1cyclictest3056-21PK-Backend20:41:351
406962992018,1cyclictest0-21swapper/123:50:141
406962992018,1cyclictest0-21swapper/123:50:141
406962992018,1cyclictest0-21swapper/119:48:011
406962992017,2cyclictest3056-21PK-Backend21:58:341
40696299200,1cyclictest3056-21PK-Backend23:11:001
40696299200,1cyclictest3056-21PK-Backend22:55:051
40696299200,1cyclictest3056-21PK-Backend21:47:161
40696299200,1cyclictest3056-21PK-Backend21:01:221
40696299200,1cyclictest0-21swapper/123:29:121
40696299200,19cyclictest3056-21PK-Backend23:37:301
40696299200,19cyclictest3056-21PK-Backend23:20:061
40696299200,19cyclictest3056-21PK-Backend21:36:591
40696299200,19cyclictest3056-21PK-Backend20:28:441
40696299200,19cyclictest0-21swapper/121:34:461
40696299200,18cyclictest3056-21PK-Backend20:33:141
406957992018,1cyclictest0-21swapper/023:25:140
406957992018,1cyclictest0-21swapper/022:59:390
406957992018,1cyclictest0-21swapper/022:28:160
406957992018,1cyclictest0-21swapper/021:46:320
406957992018,1cyclictest0-21swapper/021:24:550
406957992018,1cyclictest0-21swapper/019:53:030
406957992018,1cyclictest0-21swapper/019:41:470
40695799200,2cyclictest4489612chrt21:06:500
40695799200,19cyclictest3056-21PK-Backend21:35:230
40695799200,19cyclictest0-21swapper/023:54:000
40695799200,19cyclictest0-21swapper/023:54:000
40695799200,19cyclictest0-21swapper/023:05:130
40695799200,19cyclictest0-21swapper/019:34:440
4469262190,3sleep00-21swapper/021:00:210
40696799197,9cyclictest448220-21/usr/sbin/munin21:05:153
40696799195,13cyclictest4494621/usr/sbin/munin21:10:153
406967991917,1cyclictest0-21swapper/323:58:373
406967991917,1cyclictest0-21swapper/321:51:153
406967991917,1cyclictest0-21swapper/320:18:483
406967991917,1cyclictest0-21swapper/300:33:183
406967991915,3cyclictest494065-21ntp_states23:15:173
406967991915,3cyclictest42-21ksoftirqd/323:15:003
406967991915,2cyclictest42-21ksoftirqd/321:00:153
406967991912,5cyclictest416751-21awk19:35:153
406967991910,7cyclictest42-21ksoftirqd/322:25:073
40696799190,6cyclictest421455-21latency_hist19:50:003
40696799190,1cyclictest3056-21PK-Backend23:28:013
40696799190,1cyclictest0-21swapper/322:50:023
40696799190,1cyclictest0-21swapper/322:37:323
40696799190,1cyclictest0-21swapper/321:47:173
40696499192,13cyclictest456423-21apt21:30:112
406964991917,1cyclictest0-21swapper/223:25:082
406964991917,1cyclictest0-21swapper/222:26:202
406964991917,1cyclictest0-21swapper/221:15:162
406964991917,1cyclictest0-21swapper/200:19:292
406964991915,3cyclictest978-21snmpd23:31:472
406964991915,3cyclictest978-21snmpd23:17:402
406964991912,5cyclictest438220-21sensors_temp20:35:202
40696499190,1cyclictest0-21swapper/219:25:522
40696499190,1cyclictest0-21swapper/219:18:362
40696499190,18cyclictest0-21swapper/222:46:422
40696499190,18cyclictest0-21swapper/220:59:222
406962991917,1cyclictest3056-21PK-Backend23:48:251
406962991917,1cyclictest3056-21PK-Backend22:29:441
406962991917,1cyclictest3056-21PK-Backend21:11:301
406962991917,1cyclictest3056-21PK-Backend21:09:071
406962991917,1cyclictest3056-21PK-Backend20:21:211
406962991917,1cyclictest0-21swapper/119:41:341
406962991916,2cyclictest3056-21PK-Backend23:18:171
406962991912,5cyclictest407618-21awk19:10:171
406962991911,2cyclictest26-21ksoftirqd/122:25:001
40696299190,1cyclictest3056-21PK-Backend21:40:431
40696299190,1cyclictest3056-21PK-Backend19:50:521
40696299190,1cyclictest3056-21PK-Backend00:28:411
40696299190,1cyclictest3056-21PK-Backend00:08:261
40696299190,1cyclictest0-21swapper/120:07:211
40696299190,18cyclictest0-21swapper/120:03:061
406957991917,1cyclictest0-21swapper/023:59:000
406957991917,1cyclictest0-21swapper/023:36:160
406957991917,1cyclictest0-21swapper/021:15:290
406957991917,1cyclictest0-21swapper/020:19:590
406957991917,1cyclictest0-21swapper/019:46:130
40695799190,1cyclictest0-21swapper/023:24:130
40695799190,1cyclictest0-21swapper/000:19:350
139099190,15rtkit-daemon406948-21cyclictest19:41:132
406967991816,1cyclictest0-21swapper/321:39:323
40696799181,14cyclictest978-21snmpd22:55:123
40696799180,16cyclictest518525-21df_inode00:25:143
40696499182,12cyclictest491599-21apt23:10:112
406964991816,1cyclictest0-21swapper/223:51:172
406964991816,1cyclictest0-21swapper/223:51:172
406964991811,5cyclictest34-21ksoftirqd/221:10:112
406962991816,1cyclictest3056-21PK-Backend19:19:571
406962991811,5cyclictest470869-21expr22:10:161
406962991810,6cyclictest26-21ksoftirqd/121:00:001
40696299180,1cyclictest3056-21PK-Backend21:24:481
406957991816,1cyclictest0-21swapper/019:59:040
406957991816,1cyclictest0-21swapper/019:20:460
406957991813,3cyclictest470423-21cpuspeed_turbos22:10:120
406957991812,5cyclictest409082-21sed19:15:020
406957991811,5cyclictest520690-21cut00:30:200
40696799171,8cyclictest978-21snmpd22:45:113
406964991715,1cyclictest0-21swapper/223:57:462
406964991715,1cyclictest0-21swapper/220:48:392
406964991711,4cyclictest486534-21cut22:55:132
40696299179,6cyclictest26-21ksoftirqd/119:25:121
406962991715,1cyclictest3056-21PK-Backend19:22:331
406962991715,1cyclictest3056-21PK-Backend00:31:311
406962991715,1cyclictest0-21swapper/123:09:111
406962991712,3cyclictest467562-21open_files22:00:171
40696299170,1cyclictest141rcu_preempt20:17:261
40695799173,13cyclictest437091-21cron20:35:000
406957991715,1cyclictest0-21swapper/020:45:220
406957991713,3cyclictest444467-21tr20:55:110
406957991711,5cyclictest522684-21cat00:35:220
406957991711,4cyclictest465757-21cut21:55:160
406957991711,4cyclictest449630-21cpuspeed_turbos21:10:110
406957991710,5cyclictest503192-21sed23:40:210
40695799170,16cyclictest0-21swapper/021:43:010
406962991614,1cyclictest0-21swapper/123:55:421
40695799169,5cyclictest518468-21expr00:25:130
40695799168,6cyclictest883-21dbus-daemon00:10:000
40695799168,6cyclictest407386-21expr19:10:120
406957991614,1cyclictest0-21swapper/022:22:210
406957991614,1cyclictest0-21swapper/022:18:360
406957991614,1cyclictest0-21swapper/021:50:030
406957991614,1cyclictest0-21swapper/020:11:070
406957991614,1cyclictest0-21swapper/019:36:330
406957991611,3cyclictest488299-21cpuspeed_turbos23:00:120
406957991610,5cyclictest510220-21grep00:00:220
406957991610,4cyclictest425514-21cat20:00:160
40695799158,5cyclictest513002-21sort00:10:100
406957991513,1cyclictest3056-21PK-Backend21:25:430
406957991513,1cyclictest0-21swapper/022:30:390
406957991510,4cyclictest13-21ksoftirqd/022:05:000
406957991510,3cyclictest433972-21cpuspeed20:25:100
406957991510,3cyclictest13-21ksoftirqd/022:45:000
40695799150,1cyclictest3056-21PK-Backend20:36:490
40696499149,3cyclictest1095-21runrttasks19:14:592
40696299146,2cyclictest26-21ksoftirqd/121:28:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional