You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 14:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Tue Aug 25, 2026 00:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4166470999610,82cyclictest0-21swapper/023:05:030
4166474998040,17cyclictest3467602-21systemd-journal22:15:001
416647699752,54cyclictest4189563-21sh21:45:022
4166470997514,25cyclictest4192461-21rm21:51:040
416647099741,40cyclictest0-21swapper/022:35:020
416647099722,40cyclictest3467602-21systemd-journal23:16:540
4166470997167,2cyclictest0-21swapper/020:10:020
4166470997167,2cyclictest0-21swapper/020:10:020
4166470997119,33cyclictest0-21swapper/021:15:450
4166470997059,3cyclictest12-21ksoftirqd/023:45:010
416647099692,36cyclictest5094-21latency_hist22:05:010
4166470996916,13cyclictest4167615-21latency_hist19:30:020
4166470996861,5cyclictest12-21ksoftirqd/020:35:010
4166470996823,9cyclictest16837-21latency_hist22:30:010
416647099681,37cyclictest0-21swapper/022:05:440
4166479996629,12cyclictest37-21ksoftirqd/322:20:003
416647999635,10cyclictest37-21ksoftirqd/300:40:013
4166479996316,11cyclictest37-21ksoftirqd/323:10:023
4166470996342,18cyclictest0-21swapper/022:41:490
4166470996337,21cyclictest275-21dbus-daemon21:39:590
4166474996242,15cyclictest3467602-21systemd-journal19:25:001
4166470996225,33cyclictest33528-21latency_hist23:05:020
416647099621,27cyclictest61914-21latency_hist00:05:010
416647699613,8cyclictest131rcu_preempt20:10:002
416647699613,8cyclictest131rcu_preempt20:10:002
4166470996120,32cyclictest16850irq/134-800000023:27:570
416647099611,26cyclictest50149-21ls23:40:020
4166470996028,15cyclictest64682-21ssh00:10:350
4166470996015,11cyclictest275-21dbus-daemon22:15:000
416647099601,32cyclictest0-21swapper/023:58:090
4166470995955,2cyclictest0-21swapper/023:50:020
4166470995922,30cyclictest16850irq/134-800000000:17:200
416647999585,10cyclictest37-21ksoftirqd/319:40:013
416647999581,24cyclictest0-21swapper/323:14:393
4166470995818,5cyclictest16850irq/134-800000023:12:370
4166470995816,33cyclictest16850irq/134-800000021:57:380
416647999574,9cyclictest37-21ksoftirqd/300:00:033
416647999572,9cyclictest37-21ksoftirqd/322:30:013
416647999572,7cyclictest37-21ksoftirqd/323:20:013
4166479995713,6cyclictest131rcu_preempt19:17:003
416647499578,45cyclictest59491-40mandb00:00:031
4166470995735,5cyclictest16850irq/134-800000022:50:010
416647099571,26cyclictest4172677-21diskmemload22:50:530
416647999565,12cyclictest37-21ksoftirqd/321:22:593
4166479995616,8cyclictest37-21ksoftirqd/321:56:133
4166476995647,5cyclictest14501-21kworker/2:200:36:192
4166476995646,6cyclictest4158139-21kworker/2:023:39:092
4166470995627,14cyclictest56078-21ssh23:52:320
4166470995621,18cyclictest16850irq/134-800000021:46:000
4166470995621,18cyclictest16850irq/134-800000021:45:590
4166470995619,32cyclictest16850irq/134-800000023:21:570
416647099561,31cyclictest0-21swapper/022:16:310
4166479995543,8cyclictest37-21ksoftirqd/320:05:003
4166476995547,5cyclictest4158139-21kworker/2:023:12:382
4166476995547,5cyclictest4158139-21kworker/2:022:15:392
4166476995547,5cyclictest4158139-21kworker/2:021:35:572
4166476995547,5cyclictest4158139-21kworker/2:021:23:112
4166476995546,6cyclictest4158139-21kworker/2:020:05:022
4166476995546,5cyclictest4189472-21kworker/2:321:55:012
4166470995523,14cyclictest31104-21ssh23:00:000
4166470995521,17cyclictest16850irq/134-800000000:37:110
416647999545,8cyclictest37-21ksoftirqd/321:35:363
4166479995412,5cyclictest131rcu_preempt23:39:423
4166476995446,5cyclictest4189472-21kworker/2:322:00:192
4166476995446,5cyclictest14501-21kworker/2:200:24:432
4166476995445,5cyclictest4158139-21kworker/2:023:54:242
4166476995445,5cyclictest4158139-21kworker/2:022:37:232
4166476995444,6cyclictest4158139-21kworker/2:023:26:282
4166476995444,6cyclictest4158139-21kworker/2:021:18:062
4166476995427,8cyclictest4158139-21kworker/2:023:08:202
416647099543,7cyclictest1-21systemd21:30:010
4166470995428,10cyclictest4183549-21ssh21:32:250
416647999538,4cyclictest131rcu_preempt21:44:043
416647999533,12cyclictest37-21ksoftirqd/322:59:323
4166479995310,6cyclictest131rcu_preempt19:21:093
416647699538,26cyclictest4158139-21kworker/2:019:10:012
4166476995345,5cyclictest4158139-21kworker/2:021:13:332
4166476995345,5cyclictest4158139-21kworker/2:000:01:432
4166476995344,6cyclictest4158139-21kworker/2:023:45:062
4166476995344,6cyclictest4158139-21kworker/2:023:03:212
4166476995344,6cyclictest4158139-21kworker/2:022:52:332
4166476995344,6cyclictest4158139-21kworker/2:022:33:442
4166476995344,6cyclictest4158139-21kworker/2:021:45:552
4166476995344,6cyclictest4158139-21kworker/2:021:45:542
4166476995344,6cyclictest4158139-21kworker/2:021:30:112
4166476995344,5cyclictest4158139-21kworker/2:022:43:082
4166476995318,5cyclictest0-21swapper/221:56:572
4166474995335,14cyclictest1-21systemd00:10:001
4166470995345,5cyclictest12-21ksoftirqd/019:40:030
4166470995317,27cyclictest16850irq/134-800000022:22:270
4166470995315,29cyclictest16850irq/134-800000022:38:330
4166470995312,32cyclictest16850irq/134-800000000:32:530
416647999525,6cyclictest4965-21ssh22:05:003
416647999525,11cyclictest37-21ksoftirqd/322:45:013
4166479995214,5cyclictest131rcu_preempt00:32:303
4166476995244,5cyclictest4189472-21kworker/2:322:13:062
4166476995244,5cyclictest4158139-21kworker/2:023:58:132
4166476995244,5cyclictest4158139-21kworker/2:023:21:062
4166476995244,5cyclictest4158139-21kworker/2:022:46:212
4166476995244,5cyclictest4158139-21kworker/2:020:35:022
4166476995243,5cyclictest4189472-21kworker/2:322:05:392
4166476995243,5cyclictest4158139-21kworker/2:023:41:592
4166470995228,9cyclictest45881-21ssh23:30:510
4166470995225,11cyclictest62413-21ssh00:05:460
416647999518,4cyclictest131rcu_preempt00:22:453
416647999516,12cyclictest37-21ksoftirqd/323:50:023
416647999514,7cyclictest37-21ksoftirqd/322:10:023
416647999514,7cyclictest37-21ksoftirqd/319:34:593
4166479995129,17cyclictest275-21dbus-daemon20:40:013
4166479995114,6cyclictest131rcu_preempt22:39:353
4166479995113,10cyclictest37-21ksoftirqd/300:15:283
4166479995110,5cyclictest131rcu_preempt00:12:383
4166476995143,5cyclictest4158139-21kworker/2:022:21:262
4166476995141,6cyclictest4158139-21kworker/2:022:59:142
4166474995137,10cyclictest1-21systemd19:45:021
416647099511,46cyclictest0-21swapper/019:16:090
416647999509,6cyclictest131rcu_preempt22:51:313
416647999508,5cyclictest131rcu_preempt20:00:003
416647999508,5cyclictest131rcu_preempt00:02:233
416647999506,9cyclictest37-21ksoftirqd/300:29:153
4166479995040,7cyclictest37-21ksoftirqd/319:30:013
416647999502,8cyclictest37-21ksoftirqd/321:30:003
4166476995042,5cyclictest4158139-21kworker/2:020:55:022
4166476995042,5cyclictest4158139-21kworker/2:000:05:092
4166476995042,5cyclictest14501-21kworker/2:200:31:192
4166476995040,6cyclictest4158139-21kworker/2:023:30:392
4166476995040,6cyclictest4158139-21kworker/2:000:12:372
4166470995045,3cyclictest0-21swapper/020:05:020
416647099504,39cyclictest0-21swapper/020:40:010
4166470995022,13cyclictest4188297-21ssh21:42:300
416647999498,9cyclictest37-21ksoftirqd/323:51:263
416647999497,10cyclictest37-21ksoftirqd/321:32:353
416647999496,4cyclictest131rcu_preempt21:13:303
416647999494,8cyclictest37-21ksoftirqd/321:55:013
416647999494,10cyclictest37-21ksoftirqd/322:50:023
4166479994933,12cyclictest3467602-21systemd-journal23:25:033
4166479994911,5cyclictest131rcu_preempt00:06:143
4166476994941,5cyclictest4158139-21kworker/2:023:18:132
4166476994941,5cyclictest4158139-21kworker/2:022:25:422
4166476994941,5cyclictest4158139-21kworker/2:000:15:302
4166476994940,6cyclictest14501-21kworker/2:200:25:532
416647099493,3cyclictest131rcu_preempt21:25:020
416647099491,5cyclictest4170350-21latency_hist20:25:030
416647099491,5cyclictest4170350-21latency_hist20:25:020
416647999489,6cyclictest131rcu_preempt23:31:263
416647999488,11cyclictest37-21ksoftirqd/322:25:033
416647999485,8cyclictest37-21ksoftirqd/322:13:453
4166479994836,8cyclictest37-21ksoftirqd/322:35:013
4166479994818,20cyclictest37-21ksoftirqd/323:43:053
416647999481,6cyclictest0-21swapper/323:03:023
4166479994816,10cyclictest37-21ksoftirqd/321:45:593
4166479994816,10cyclictest37-21ksoftirqd/321:45:583
4166479994815,5cyclictest131rcu_preempt21:17:033
4166476994840,5cyclictest4158139-21kworker/2:021:25:182
4166470994825,7cyclictest4173585-21ssh21:11:340
4166479994716,6cyclictest131rcu_preempt23:22:063
4166476994733,10cyclictest3467602-21systemd-journal20:17:302
4166476994733,10cyclictest1-21systemd21:10:002
4166474994735,9cyclictest4187004-21cron21:40:001
4166474994731,11cyclictest3467602-21systemd-journal23:50:001
416647499471,4cyclictest131rcu_preempt00:20:051
416647099474,3cyclictest131rcu_preempt21:10:000
4166470994731,13cyclictest71733-21ssh00:25:320
416647999463,9cyclictest37-21ksoftirqd/320:45:013
4166479994636,7cyclictest37-21ksoftirqd/320:25:033
4166479994636,7cyclictest37-21ksoftirqd/320:25:023
416647999462,8cyclictest37-21ksoftirqd/321:05:013
4166479994622,17cyclictest275-21dbus-daemon20:20:013
4166479994612,25cyclictest37-21ksoftirqd/320:55:033
4166476994638,5cyclictest4158139-21kworker/2:020:42:382
4166476994638,5cyclictest4157088-21kworker/2:319:57:172
4166476994638,5cyclictest4157088-21kworker/2:319:44:382
4166476994638,5cyclictest4157088-21kworker/2:319:21:392
4166474994635,6cyclictest309-21in:imuxsock22:05:001
416647099464,10cyclictest4170061-21cron20:20:000
416647099462,25cyclictest19350irq/34-mmc119:10:020
416647099461,41cyclictest0-21swapper/021:05:020
4166470994613,8cyclictest12-21ksoftirqd/019:35:000
416647999456,9cyclictest37-21ksoftirqd/321:10:003
416647999455,8cyclictest37-21ksoftirqd/319:10:013
4166479994533,7cyclictest37-21ksoftirqd/320:10:013
4166479994533,7cyclictest37-21ksoftirqd/320:10:003
416647999451,6cyclictest0-21swapper/320:30:013
4166476994537,5cyclictest4157088-21kworker/2:319:47:542
4166476994537,5cyclictest4157088-21kworker/2:319:26:202
4166476994537,4cyclictest4157088-21kworker/2:319:10:342
4166474994531,10cyclictest3467602-21systemd-journal22:20:001
4166474994531,10cyclictest3467602-21systemd-journal19:59:101
4166474994528,13cyclictest275-21dbus-daemon23:00:021
4166474994528,12cyclictest3467602-21systemd-journal20:05:001
4166474994527,13cyclictest23-21ksoftirqd/123:01:071
4166474994525,15cyclictest61934-21unin-run00:05:011
416647499451,4cyclictest131rcu_preempt21:14:011
4166470994530,9cyclictest4172171-21sendmail-msp20:59:590
416647999449,5cyclictest131rcu_preempt19:55:013
416647999446,6cyclictest37-21ksoftirqd/320:31:093
416647999441,10cyclictest4166701-21munin-plugin-st19:15:013
416647499444,3cyclictest131rcu_preempt21:57:151
416647099449,14cyclictest2129045-1kworker/0:2H20:00:020
416647999431,7cyclictest37-21ksoftirqd/319:42:093
4166476994335,5cyclictest4157088-21kworker/2:319:38:482
4166474994329,11cyclictest23-21ksoftirqd/122:25:091
4166474994328,11cyclictest57124-21cron23:55:001
416647499432,3cyclictest131rcu_preempt22:41:471
416647099431,38cyclictest0-21swapper/000:21:040
4166476994236,4cyclictest4158139-21kworker/2:020:59:142
4166476994227,11cyclictest30-21ksoftirqd/219:55:012
4166474994233,5cyclictest21624-21sh22:40:021
4166474994229,10cyclictest38480-21sshd23:15:091
4166474994227,11cyclictest23-21ksoftirqd/123:14:041
4166470994225,13cyclictest275-21dbus-daemon19:45:010
4166476994130,7cyclictest4167733-21cron19:35:002
4166474994129,8cyclictest23-21ksoftirqd/100:20:021
4166474994126,10cyclictest309-21in:imuxsock00:30:011
416647499411,5cyclictest131rcu_preempt20:30:001
416647499411,36cyclictest23-21ksoftirqd/100:38:191
4166470994129,7cyclictest309-21in:imuxsock19:25:010
4166470994125,12cyclictest4168789-21systemd-run19:55:010
416647999405,7cyclictest37-21ksoftirqd/319:49:593
416647999401,8cyclictest37-21ksoftirqd/320:50:003
4166476994022,13cyclictest3467602-21systemd-journal19:20:002
4166474994028,9cyclictest4170833-21sshd20:32:331
4166474994026,11cyclictest23-21ksoftirqd/121:28:091
4166474994025,12cyclictest23-21ksoftirqd/123:05:201
4166474994025,11cyclictest23-21ksoftirqd/121:46:011
4166474994025,11cyclictest23-21ksoftirqd/121:46:011
4166474994013,5cyclictest131rcu_preempt23:31:431
4166474994012,6cyclictest131rcu_preempt00:10:231
416647999396,8cyclictest37-21ksoftirqd/320:11:213
416647999396,10cyclictest37-21ksoftirqd/320:55:103
4166476993932,4cyclictest4158139-21kworker/2:020:36:432
4166476993931,5cyclictest4158139-21kworker/2:020:10:232
416647499399,7cyclictest131rcu_preempt20:14:591
4166474993931,5cyclictest4172677-21diskmemload21:25:021
4166474993930,6cyclictest4172680-21latency_hist21:10:001
4166474993929,8cyclictest23-21ksoftirqd/122:45:061
4166474993921,13cyclictest3467602-21systemd-journal20:16:101
416647099393,5cyclictest4166653-21latency_hist19:10:030
4166470993931,5cyclictest4170597-21modprobe20:30:000
4166470993929,7cyclictest12-21ksoftirqd/020:15:020
4166476993823,10cyclictest1-21systemd21:05:012
4166476993820,14cyclictest4170809-21date20:30:012
416647499383,3cyclictest131rcu_preempt21:19:591
4166474993830,6cyclictest23-21ksoftirqd/123:25:021
4166474993829,7cyclictest23-21ksoftirqd/121:42:131
4166474993825,9cyclictest4171148-21systemctl20:40:001
4166474993823,11cyclictest23-21ksoftirqd/123:35:371
4166474993822,12cyclictest4170473-21cat20:25:031
4166474993822,12cyclictest4170473-21cat20:25:021
416647499381,4cyclictest131rcu_preempt21:34:491
416647099383,9cyclictest1-21systemd20:42:090
4166470993828,7cyclictest12-21ksoftirqd/020:50:020
4166476993729,5cyclictest4158139-21kworker/2:020:45:412
4166474993729,6cyclictest23-21ksoftirqd/121:00:021
4166474993728,5cyclictest4172429-21sh21:05:001
4166474993727,7cyclictest23-21ksoftirqd/121:52:521
4166474993726,9cyclictest23-21ksoftirqd/123:28:011
4166474993725,9cyclictest23-21ksoftirqd/122:23:541
4166474993725,9cyclictest23-21ksoftirqd/119:40:001
4166474993722,12cyclictest7433-21cat22:10:011
4166474993711,5cyclictest131rcu_preempt22:33:511
4166474993626,7cyclictest52494-21sed23:45:021
4166474993625,8cyclictest23-21ksoftirqd/119:30:021
4166474993624,10cyclictest23-21ksoftirqd/122:55:011
4166474993622,11cyclictest23-21ksoftirqd/119:55:011
4166474993622,10cyclictest23-21ksoftirqd/119:35:011
4166470993629,5cyclictest0-21swapper/020:54:100
4166470993624,9cyclictest12-21ksoftirqd/019:50:000
4166476993522,10cyclictest30-21ksoftirqd/220:25:022
4166476993522,10cyclictest30-21ksoftirqd/220:25:012
4166474993526,7cyclictest23-21ksoftirqd/100:32:571
4166474993524,9cyclictest23-21ksoftirqd/120:09:091
4166474993524,9cyclictest23-21ksoftirqd/120:09:091
4166474993424,7cyclictest23-21ksoftirqd/120:53:431
4166474993423,8cyclictest4168619-21sed19:50:021
416647499334,3cyclictest131rcu_preempt19:19:591
4166474993319,8cyclictest131rcu_preempt19:15:011
416647499322,3cyclictest131rcu_preempt19:10:011
416647499322,22cyclictest4171409-21cat20:45:021
4166474993119,10cyclictest275-21dbus-daemon20:50:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional