You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 14:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Jun 21, 2026 00:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41742719912023,5cyclictest131rcu_preempt19:30:023
4174261998379,2cyclictest0-21swapper/023:25:020
417426899738,46cyclictest40645-21kworker/u8:100:10:032
4174261997268,2cyclictest0-21swapper/000:05:030
4174261997129,18cyclictest36365-21ssh23:05:000
4174261997010,6cyclictest131rcu_preempt20:40:000
417426199697,9cyclictest12-21ksoftirqd/021:09:590
417426199693,53cyclictest77947-21systemctl00:40:000
4174261996919,44cyclictest3467602-21systemd-journal23:30:000
417426199686,8cyclictest121ksoftirqd/022:30:010
417426199664,31cyclictest131rcu_preempt21:33:370
417426199655,18cyclictest4174721-21cut19:20:000
4174261996439,18cyclictest60687-21sed00:00:030
4174261996257,2cyclictest0-21swapper/021:40:020
4174261996231,3cyclictest150-21pr/ttyS222:50:030
417426499613,23cyclictest131rcu_preempt23:30:011
417426199616,6cyclictest121ksoftirqd/023:32:400
4174261996127,31cyclictest151rcuc/021:15:030
4174261996127,31cyclictest151rcuc/021:15:020
417426899608,20cyclictest58474-21ssh23:55:032
417426199604,9cyclictest12-21ksoftirqd/022:43:510
417426199598,4cyclictest131rcu_preempt00:20:510
417426199598,4cyclictest131rcu_preempt00:20:500
4174261995950,6cyclictest16950irq/224-800000021:20:000
417426199594,6cyclictest131rcu_preempt00:30:020
4174261995933,17cyclictest275-21dbus-daemon19:50:000
417426199592,5cyclictest131rcu_preempt22:07:120
4174268995842,7cyclictest56134-21kworker/2:200:30:012
417426199583,8cyclictest12-21ksoftirqd/000:15:010
4174261995822,32cyclictest12-21ksoftirqd/022:16:230
4174261995736,14cyclictest16950irq/224-800000021:30:000
417426199572,11cyclictest12-21ksoftirqd/000:20:010
4174261995718,8cyclictest131rcu_preempt21:50:010
4174261995713,6cyclictest131rcu_preempt22:05:020
417426199571,14cyclictest12-21ksoftirqd/023:15:010
417426199565,36cyclictest131rcu_preempt22:49:080
417426199565,36cyclictest131rcu_preempt22:49:070
4174261995641,11cyclictest12-21ksoftirqd/021:41:190
4174261995641,11cyclictest12-21ksoftirqd/021:41:190
4174261995628,13cyclictest23328-21ssh22:34:590
4174261995624,16cyclictest4185566-21ssh21:21:340
4174261995622,10cyclictest12-21ksoftirqd/022:12:180
4174261995614,5cyclictest131rcu_preempt23:08:330
4174271995536,14cyclictest1-21systemd21:15:023
4174271995536,14cyclictest1-21systemd21:15:013
417426199558,28cyclictest131rcu_preempt23:16:300
417426199555,7cyclictest12-21ksoftirqd/023:36:190
4174261995534,15cyclictest4176770-21cron20:00:010
4174261995526,13cyclictest57249-21ssh23:52:120
4174261995522,29cyclictest4176023-21latency_hist19:45:030
417426199552,19cyclictest53961-21munin-run23:45:010
4174268995438,5cyclictest4164334-21kworker/2:321:12:542
4174268995438,5cyclictest4164334-21kworker/2:321:12:532
417426899542,5cyclictest131rcu_preempt00:25:012
417426899542,5cyclictest131rcu_preempt00:25:002
4174264995414,18cyclictest23-21ksoftirqd/122:25:011
4174261995423,26cyclictest56116-21latency_hist23:50:020
4174268995345,5cyclictest56134-21kworker/2:200:39:482
4174268995344,5cyclictest4178416-21kworker/2:021:17:092
417426199538,39cyclictest131rcu_preempt22:38:320
417426199538,39cyclictest131rcu_preempt22:38:310
417426199536,11cyclictest12-21ksoftirqd/019:40:000
4174261995328,18cyclictest32593-21rm22:55:480
4174261995328,18cyclictest32593-21rm22:55:470
4174268995243,6cyclictest56134-21kworker/2:200:17:372
417426899523,3cyclictest131rcu_preempt00:00:012
4174264995239,9cyclictest3467602-21systemd-journal23:39:591
4174264995239,6cyclictest23-21ksoftirqd/121:35:021
417426199527,6cyclictest131rcu_preempt00:00:390
4174261995237,11cyclictest8189-21ssh22:00:010
4174268995143,5cyclictest4178416-21kworker/2:021:32:092
4174268995143,5cyclictest21162-21kworker/2:123:05:022
4174268995143,5cyclictest21162-21kworker/2:122:43:232
4174268995135,6cyclictest4164334-21kworker/2:321:05:002
4174268995115,7cyclictest4178416-21kworker/2:022:25:022
417426199515,8cyclictest12-21ksoftirqd/022:23:240
4174268995042,5cyclictest5980-21kworker/2:322:35:162
4174268995042,5cyclictest5980-21kworker/2:322:35:152
4174268995042,5cyclictest56134-21kworker/2:200:10:202
4174268995042,5cyclictest4178416-21kworker/2:021:40:442
4174268995042,5cyclictest4178416-21kworker/2:021:40:432
4174268995042,5cyclictest21162-21kworker/2:122:53:412
4174268995041,5cyclictest4178416-21kworker/2:021:39:582
4174268995038,6cyclictest21162-21kworker/2:100:00:152
4174261995017,6cyclictest131rcu_preempt21:55:010
4174261995017,6cyclictest131rcu_preempt21:55:010
417427199493,23cyclictest60529-40gzip00:00:043
4174268994941,5cyclictest4178416-21kworker/2:022:11:172
4174268994941,5cyclictest36407-21kworker/2:323:32:142
4174268994941,5cyclictest36407-21kworker/2:323:28:362
4174268994941,5cyclictest21162-21kworker/2:123:42:272
417426199495,27cyclictest364-21snmpd19:15:010
4174261994935,10cyclictest309-21in:imuxsock20:50:010
417426199491,42cyclictest151rcuc/020:05:000
4174268994840,5cyclictest4178416-21kworker/2:022:15:382
4174268994840,5cyclictest4178416-21kworker/2:021:58:022
4174268994840,5cyclictest4178416-21kworker/2:021:27:272
4174268994840,5cyclictest21162-21kworker/2:123:37:072
4174268994839,6cyclictest56134-21kworker/2:200:30:172
4174268994839,6cyclictest4164334-21kworker/2:320:40:012
4174268994829,14cyclictest275-21dbus-daemon23:00:012
4174268994829,14cyclictest275-21dbus-daemon23:00:002
417426499483,5cyclictest131rcu_preempt00:19:221
4174264994833,12cyclictest3467602-21systemd-journal22:48:241
4174264994833,12cyclictest3467602-21systemd-journal22:48:241
417426199487,7cyclictest12-21ksoftirqd/019:25:010
417426199485,11cyclictest12-21ksoftirqd/020:45:000
4174261994827,14cyclictest3467602-21systemd-journal21:05:010
4174261994819,19cyclictest275-21dbus-daemon19:30:020
4174261994816,25cyclictest12-21ksoftirqd/020:13:150
417427199474,3cyclictest131rcu_preempt23:03:333
417426899479,5cyclictest131rcu_preempt23:48:242
4174268994739,5cyclictest4182690-21kworker/2:121:47:342
417426499477,6cyclictest131rcu_preempt22:40:011
417426499477,6cyclictest131rcu_preempt22:40:001
417426499474,4cyclictest131rcu_preempt00:00:001
4174264994740,4cyclictest4177061-21munin-plugin-st20:05:011
4174261994738,6cyclictest19350irq/34-mmc120:30:010
417426199473,6cyclictest131rcu_preempt19:10:020
417426199473,39cyclictest4177877-21sendmail20:20:010
417426199472,4cyclictest131rcu_preempt00:33:190
417427199463,3cyclictest131rcu_preempt22:24:163
4174271994632,11cyclictest3467602-21systemd-journal21:20:023
4174271994626,9cyclictest25690-21latency_hist22:40:033
4174271994626,9cyclictest25690-21latency_hist22:40:033
4174271994626,15cyclictest275-21dbus-daemon20:20:013
417426899468,6cyclictest131rcu_preempt22:08:492
4174268994638,5cyclictest5980-21kworker/2:322:28:592
4174268994638,5cyclictest4178416-21kworker/2:022:03:542
4174268994638,5cyclictest4178416-21kworker/2:021:24:392
4174268994637,5cyclictest36407-21kworker/2:323:20:082
4174264994632,10cyclictest23-21ksoftirqd/121:10:021
4174264994631,11cyclictest3467602-21systemd-journal19:57:101
4174264994629,12cyclictest23445-21cut22:35:031
417426199468,9cyclictest12-21ksoftirqd/021:00:010
417427199453,4cyclictest131rcu_preempt23:30:513
417427199452,4cyclictest131rcu_preempt23:25:393
417427199452,3cyclictest131rcu_preempt23:17:223
4174271994515,8cyclictest37-21ksoftirqd/323:43:403
417427199451,3cyclictest131rcu_preempt23:23:363
417427199451,25cyclictest0-21swapper/323:13:153
4174268994538,4cyclictest4164334-21kworker/2:320:20:022
4174268994537,5cyclictest4164334-21kworker/2:320:47:592
4174268994529,12cyclictest30-21ksoftirqd/223:10:222
4174268994511,6cyclictest131rcu_preempt23:15:472
417426499453,3cyclictest131rcu_preempt00:25:031
4174264994532,10cyclictest275-21dbus-daemon23:10:011
4174264994524,6cyclictest131rcu_preempt00:02:591
417426499451,3cyclictest131rcu_preempt21:55:021
417426499451,3cyclictest131rcu_preempt21:55:011
4174261994527,5cyclictest131rcu_preempt20:06:240
4174261994527,5cyclictest131rcu_preempt20:06:230
417427199444,3cyclictest150-21pr/ttyS220:55:003
417427199443,3cyclictest150-21pr/ttyS223:45:093
4174271994430,10cyclictest3467602-21systemd-journal00:32:233
417427199442,9cyclictest48-21kcompactd022:30:153
4174271994424,14cyclictest1-21systemd22:50:013
4174271994424,14cyclictest1-21systemd22:50:013
417427199441,4cyclictest131rcu_preempt21:44:163
417427199441,4cyclictest131rcu_preempt21:44:163
417427199441,3cyclictest131rcu_preempt22:54:073
417427199441,3cyclictest131rcu_preempt21:32:433
417427199441,3cyclictest131rcu_preempt00:39:443
4174268994436,5cyclictest21162-21kworker/2:123:06:402
4174268994436,5cyclictest21162-21kworker/2:122:45:032
4174268994436,5cyclictest21162-21kworker/2:122:45:032
4174268994428,12cyclictest275-21dbus-daemon21:00:012
417426899441,13cyclictest22986-21sh22:34:032
417426499444,3cyclictest131rcu_preempt21:48:171
417426499443,7cyclictest23-21ksoftirqd/123:45:001
417427199433,4cyclictest131rcu_preempt00:28:213
417427199433,3cyclictest131rcu_preempt21:48:593
417427199431,6cyclictest131rcu_preempt21:55:003
417427199431,6cyclictest131rcu_preempt21:55:003
417426499435,8cyclictest23-21ksoftirqd/123:00:011
417426499435,8cyclictest23-21ksoftirqd/123:00:011
417426199432,8cyclictest12-21ksoftirqd/019:35:000
4174261994326,12cyclictest275-21dbus-daemon20:55:010
4174261994312,6cyclictest131rcu_preempt20:35:020
417427199424,4cyclictest131rcu_preempt23:50:343
417427199423,4cyclictest131rcu_preempt21:16:413
4174271994229,10cyclictest275-21dbus-daemon22:30:013
4174271994228,11cyclictest37-21ksoftirqd/320:13:163
4174271994228,10cyclictest275-21dbus-daemon20:50:013
417427199422,10cyclictest67470-21ssh00:15:423
417427199421,3cyclictest131rcu_preempt22:16:273
417427199421,3cyclictest131rcu_preempt22:03:173
4174268994234,5cyclictest4164334-21kworker/2:320:41:192
4174268994234,5cyclictest4164334-21kworker/2:320:31:172
4174268994234,5cyclictest4164334-21kworker/2:320:15:022
4174268994228,10cyclictest275-21dbus-daemon19:20:002
4174268994224,12cyclictest275-21dbus-daemon19:15:022
417426499424,3cyclictest131rcu_preempt20:35:001
4174264994231,8cyclictest45258-21date23:25:021
4174264994228,11cyclictest78097-21cat00:40:031
4174264994218,7cyclictest131rcu_preempt21:55:221
417426499421,3cyclictest131rcu_preempt22:19:461
417426199425,8cyclictest12-21ksoftirqd/019:55:010
417427199415,4cyclictest131rcu_preempt00:12:173
417427199414,3cyclictest131rcu_preempt21:56:393
417427199414,3cyclictest131rcu_preempt21:27:023
4174271994127,10cyclictest1-21systemd23:00:013
4174271994127,10cyclictest1-21systemd23:00:003
417427199412,4cyclictest131rcu_preempt22:07:593
417427199412,3cyclictest131rcu_preempt00:05:503
417427199411,3cyclictest150-21pr/ttyS200:00:363
417427199411,3cyclictest131rcu_preempt21:40:003
417426899416,5cyclictest131rcu_preempt20:00:002
4174268994126,11cyclictest3467602-21systemd-journal19:40:022
4174268994125,12cyclictest3467602-21systemd-journal20:05:032
4174268994125,12cyclictest3467602-21systemd-journal20:05:022
417426499416,10cyclictest23-21ksoftirqd/121:30:011
417426499413,6cyclictest131rcu_preempt22:10:011
417426499413,3cyclictest131rcu_preempt00:25:581
4174264994132,6cyclictest23-21ksoftirqd/121:23:541
4174264994132,5cyclictest309-21in:imuxsock19:20:001
4174264994124,12cyclictest3467602-21systemd-journal19:32:071
4174264994117,7cyclictest131rcu_preempt00:21:291
4174264994117,20cyclictest23-21ksoftirqd/121:36:151
417426499411,4cyclictest131rcu_preempt23:13:451
417426499411,4cyclictest131rcu_preempt22:00:311
417426199417,9cyclictest12-21ksoftirqd/020:22:430
4174271994026,11cyclictest4179188-21sh20:45:003
4174271994025,11cyclictest1-21systemd19:45:013
4174271994023,11cyclictest275-21dbus-daemon19:50:003
417426899406,3cyclictest131rcu_preempt20:05:002
4174268994031,5cyclictest4175482-21cron19:35:002
4174268994021,15cyclictest3467602-21systemd-journal19:30:002
417426499405,5cyclictest131rcu_preempt21:20:031
417426499404,3cyclictest131rcu_preempt00:07:221
4174264994029,8cyclictest23-21ksoftirqd/123:20:041
4174264994028,9cyclictest23-21ksoftirqd/121:44:321
4174264994028,9cyclictest23-21ksoftirqd/121:44:311
4174264994025,11cyclictest23-21ksoftirqd/121:05:011
4174264994025,11cyclictest1-21systemd23:50:011
4174264994016,7cyclictest0-21swapper/121:13:201
4174264994016,7cyclictest0-21swapper/121:13:191
417427199395,3cyclictest131rcu_preempt19:25:003
417427199394,4cyclictest131rcu_preempt23:37:313
417427199394,11cyclictest131rcu_preempt20:30:013
4174271993926,9cyclictest37-21ksoftirqd/323:09:223
4174271993926,10cyclictest37-21ksoftirqd/322:43:343
4174271993923,12cyclictest3467602-21systemd-journal20:40:003
4174271993920,12cyclictest3467602-21systemd-journal20:25:013
4174271993914,6cyclictest131rcu_preempt19:40:003
417426899393,4cyclictest131rcu_preempt19:50:002
417426499393,3cyclictest131rcu_preempt23:00:031
4174264993932,4cyclictest4177892-21sendmail20:20:001
4174264993930,6cyclictest4179186-21sh20:45:001
4174264993929,7cyclictest1-21systemd23:35:011
4174264993927,9cyclictest23-21ksoftirqd/120:50:001
4174264993927,7cyclictest309-21in:imuxsock00:15:011
4174264993926,9cyclictest3467602-21systemd-journal22:52:221
4174264993924,12cyclictest3467602-21systemd-journal20:05:241
4174264993924,12cyclictest3467602-21systemd-journal20:05:231
4174264993921,11cyclictest275-21dbus-daemon20:15:001
417427199387,3cyclictest131rcu_preempt20:35:013
417427199384,3cyclictest131rcu_preempt21:09:593
4174271993825,10cyclictest1-21systemd21:05:023
417427199382,3cyclictest131rcu_preempt22:11:243
417427199381,20cyclictest0-21swapper/300:21:213
417427199381,20cyclictest0-21swapper/300:21:213
417426899384,3cyclictest131rcu_preempt21:52:072
417426899384,3cyclictest131rcu_preempt21:52:062
4174268993829,5cyclictest4164334-21kworker/2:320:15:342
417426499387,7cyclictest131rcu_preempt22:30:031
417426499386,4cyclictest131rcu_preempt22:11:021
4174264993825,10cyclictest23-21ksoftirqd/120:30:021
4174264993824,11cyclictest23-21ksoftirqd/120:40:021
417427199373,3cyclictest131rcu_preempt21:00:023
4174268993729,5cyclictest4164334-21kworker/2:319:37:122
4174268993728,6cyclictest4180683-21latency_hist21:10:022
417426499378,7cyclictest131rcu_preempt19:55:001
4174264993722,12cyclictest23-21ksoftirqd/122:41:331
4174264993720,13cyclictest275-21dbus-daemon20:55:021
417426499371,3cyclictest131rcu_preempt23:52:391
417426499371,3cyclictest131rcu_preempt19:45:021
4174264993711,5cyclictest131rcu_preempt00:30:241
4174271993628,5cyclictest1-21systemd19:55:013
417427199362,3cyclictest131rcu_preempt20:05:003
417427199361,20cyclictest0-21swapper/319:15:523
417426899365,3cyclictest131rcu_preempt20:30:012
417426899364,4cyclictest131rcu_preempt19:25:012
417426499362,6cyclictest131rcu_preempt19:25:001
4174264993623,10cyclictest23-21ksoftirqd/120:57:041
4174264993619,12cyclictest1-21systemd19:50:011
4174264993614,6cyclictest131rcu_preempt19:40:011
4174264993614,4cyclictest23-21ksoftirqd/119:15:021
417427199352,3cyclictest131rcu_preempt20:10:033
417427199352,3cyclictest131rcu_preempt20:10:023
4174271993523,9cyclictest4174429-21sshd19:13:233
4174271993518,8cyclictest37-21ksoftirqd/320:00:033
4174268993527,5cyclictest4164334-21kworker/2:319:53:402
417426499358,4cyclictest4178190-21latency_hist20:25:031
417426899342,6cyclictest131rcu_preempt20:55:002
417426499344,3cyclictest131rcu_preempt19:29:221
417427199332,3cyclictest131rcu_preempt19:10:013
417427199332,10cyclictest131rcu_preempt19:35:003
4174268993220,5cyclictest30-21ksoftirqd/219:10:022
417426499324,3cyclictest131rcu_preempt19:10:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional