You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 15:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sun May 10, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3424845991181,109cyclictest0-21swapper/010:50:030
34248459911099,3cyclictest0-21swapper/011:05:020
34248459911099,3cyclictest0-21swapper/011:05:020
34248459910497,3cyclictest0-21swapper/010:05:020
34248459910497,3cyclictest0-21swapper/010:05:010
342485799831,8cyclictest131rcu_preempt08:25:023
3424857997770,5cyclictest37-21ksoftirqd/310:25:023
342484599741,69cyclictest0-21swapper/012:25:030
342484599741,69cyclictest0-21swapper/012:25:020
3424845997268,2cyclictest0-21swapper/011:15:020
3424845997218,17cyclictest275-21dbus-daemon10:05:010
3424857997136,30cyclictest3425305-21sendmail07:20:003
3424845997166,3cyclictest0-21swapper/008:40:030
342484599707,28cyclictest16850irq/134-800000009:20:020
3424857996831,12cyclictest3445209-21cron09:40:003
3424845996753,8cyclictest16850irq/134-800000012:35:000
3424845996715,12cyclictest12-21ksoftirqd/010:40:000
3424857996627,7cyclictest3516328-21sshd12:09:263
3424857996532,13cyclictest37-21ksoftirqd/310:35:013
3424857996529,5cyclictest3505150-21sh11:45:223
3424857996525,13cyclictest275-21dbus-daemon12:40:013
3424845996529,14cyclictest3464309-21sendmail10:20:000
3424845996423,5cyclictest3427156-21latency_hist07:55:020
3424857996330,12cyclictest37-21ksoftirqd/309:23:563
3424857996328,6cyclictest3514244-21sh12:05:013
3424857996324,10cyclictest37-21ksoftirqd/311:09:373
3424845996340,11cyclictest12-21ksoftirqd/011:45:000
3424845996340,11cyclictest12-21ksoftirqd/011:45:000
3424845996324,35cyclictest3497750-21latency_hist11:30:010
3424855996132,9cyclictest3415397-21kworker/2:311:37:422
3424845996014,8cyclictest12-21ksoftirqd/008:20:010
3424857995927,5cyclictest3445907-21sh09:41:093
3424857995926,8cyclictest37-21ksoftirqd/311:59:063
3424845995929,15cyclictest3492871-21ssh11:19:590
3424845995919,6cyclictest3530781-21sendmail12:40:000
3424845995915,10cyclictest12-21ksoftirqd/007:35:010
342484599591,21cyclictest3519016-21cat12:15:020
3424845995825,4cyclictest0-21swapper/009:10:020
342484599582,37cyclictest3516641-21latency_hist12:10:030
3424845995814,9cyclictest12-21ksoftirqd/011:30:360
342484599581,21cyclictest3486921-21ssh11:07:170
3424845995738,14cyclictest3429469-21systemctl08:40:010
3424845995733,21cyclictest3479412-21ssh10:51:220
3424845995729,13cyclictest3501410-21ssh11:37:460
3424845995727,14cyclictest3470460-21ssh10:32:520
3424845995726,14cyclictest3449963-21ssh09:50:010
3424855995648,5cyclictest3440715-21kworker/2:010:31:272
3424855995646,6cyclictest3478668-21kworker/2:011:35:012
3424855995641,7cyclictest3440715-21kworker/2:010:13:542
3424855995638,13cyclictest1-21systemd12:00:002
3424845995627,14cyclictest3460282-21ssh10:11:170
3424855995547,5cyclictest3415397-21kworker/2:312:16:542
3424855995547,5cyclictest3415397-21kworker/2:312:16:542
3424851995546,6cyclictest23-21ksoftirqd/110:55:021
3424845995527,12cyclictest3443204-21ssh09:35:290
3424845995518,33cyclictest3451921-21ssh09:54:040
3424845995513,8cyclictest12-21ksoftirqd/009:30:010
3424857995431,17cyclictest275-21dbus-daemon10:50:013
342485799542,47cyclictest1-21systemd07:30:003
3424855995445,5cyclictest3415397-21kworker/2:309:56:242
3424855995445,5cyclictest3415397-21kworker/2:309:56:242
3424855995440,6cyclictest3478668-21kworker/2:011:21:392
3424855995440,6cyclictest3478668-21kworker/2:011:21:392
342484599546,10cyclictest3507167-21latency_hist11:50:010
342484599543,33cyclictest12-21ksoftirqd/011:23:220
342484599543,33cyclictest12-21ksoftirqd/011:23:220
3424855995345,5cyclictest3511913-21kworker/2:012:35:422
3424855995345,5cyclictest3424221-21kworker/2:109:23:532
3424855995345,5cyclictest3415397-21kworker/2:309:27:132
3424855995344,6cyclictest3415397-21kworker/2:310:41:512
3424855995343,6cyclictest3415397-21kworker/2:308:50:022
3424855995343,5cyclictest3440715-21kworker/2:009:39:402
342485199532,5cyclictest131rcu_preempt11:50:001
3424845995344,5cyclictest16950irq/224-800000011:00:030
3424845995330,19cyclictest3520695-21ssh12:18:440
3424845995330,19cyclictest3520695-21ssh12:18:440
3424845995327,11cyclictest3475222-21ssh10:42:450
3424845995326,12cyclictest3508974-21ssh11:53:480
3424845995318,31cyclictest16850irq/134-800000009:58:100
3424845995318,31cyclictest16850irq/134-800000009:58:100
3424855995244,5cyclictest3478668-21kworker/2:011:25:382
3424855995244,5cyclictest3478668-21kworker/2:011:14:372
3424855995244,5cyclictest3415397-21kworker/2:312:21:402
3424855995244,5cyclictest3415397-21kworker/2:310:04:252
3424855995244,5cyclictest3415397-21kworker/2:309:48:132
3424855995244,5cyclictest3415397-21kworker/2:309:33:362
3424855995244,5cyclictest3415397-21kworker/2:308:20:042
3424855995242,6cyclictest3511913-21kworker/2:012:26:242
3424855995242,6cyclictest3511913-21kworker/2:012:26:242
3424855995210,9cyclictest131rcu_preempt08:00:002
3424851995237,12cyclictest1-21systemd11:40:011
342484599528,9cyclictest3436177-21ssh09:20:390
342484599527,31cyclictest16950irq/224-800000007:10:020
3424845995232,13cyclictest12-21ksoftirqd/010:30:010
342484599523,11cyclictest3511867-21cron12:00:000
3424845995216,7cyclictest12-21ksoftirqd/012:04:230
3424857995134,13cyclictest275-21dbus-daemon12:30:013
3424857995134,13cyclictest275-21dbus-daemon12:30:013
342485599517,7cyclictest131rcu_preempt12:15:022
3424855995144,4cyclictest3415397-21kworker/2:310:45:352
3424855995143,5cyclictest3478668-21kworker/2:011:07:332
3424855995143,5cyclictest3478668-21kworker/2:011:04:272
3424855995143,5cyclictest3478668-21kworker/2:011:04:272
3424855995143,5cyclictest3415397-21kworker/2:311:54:462
3424855995143,5cyclictest3415397-21kworker/2:309:50:332
3424855995143,5cyclictest3415397-21kworker/2:309:11:272
3424855995143,5cyclictest3415397-21kworker/2:309:11:272
3424855995142,6cyclictest3440715-21kworker/2:010:25:352
3424855995135,11cyclictest30-21ksoftirqd/210:05:132
3424855995135,11cyclictest30-21ksoftirqd/210:05:132
3424855995133,13cyclictest1-21systemd09:05:012
3424855995133,13cyclictest1-21systemd09:05:002
342485599511,5cyclictest131rcu_preempt12:05:022
3424851995134,13cyclictest275-21dbus-daemon12:30:011
3424851995134,13cyclictest275-21dbus-daemon12:30:011
3424845995140,5cyclictest16950irq/224-800000008:00:000
3424845995122,13cyclictest3431359-21ssh09:10:360
3424845995122,13cyclictest3431359-21ssh09:10:360
3424857995030,15cyclictest3467602-21systemd-journal09:30:003
3424855995043,4cyclictest3415397-21kworker/2:308:08:252
3424855995042,5cyclictest3511913-21kworker/2:012:30:462
3424855995042,5cyclictest3478668-21kworker/2:010:56:372
3424855995042,5cyclictest3415397-21kworker/2:312:06:002
3424855995042,5cyclictest3415397-21kworker/2:311:49:112
342485199504,7cyclictest131rcu_preempt08:10:011
3424851995030,14cyclictest275-21dbus-daemon08:15:011
3424851995011,6cyclictest131rcu_preempt12:35:001
3424851995011,6cyclictest131rcu_preempt12:20:001
3424851995011,6cyclictest131rcu_preempt12:20:001
3424845995031,13cyclictest3467602-21systemd-journal08:25:000
3424845995023,12cyclictest3447504-21ssh09:44:450
3424845995018,15cyclictest0-21swapper/008:25:020
3424855994941,5cyclictest3478668-21kworker/2:011:17:072
3424855994941,5cyclictest3440715-21kworker/2:010:22:592
3424855994941,5cyclictest3415397-21kworker/2:311:41:132
3424855994941,5cyclictest3415397-21kworker/2:311:41:132
3424855994941,5cyclictest3415397-21kworker/2:310:52:252
3424855994941,5cyclictest3415397-21kworker/2:310:35:102
3424855994941,5cyclictest3415397-21kworker/2:309:43:362
3424855994940,5cyclictest3440715-21kworker/2:010:19:262
342485199496,6cyclictest131rcu_preempt11:20:001
3424851994917,22cyclictest0-21swapper/111:40:421
3424851994917,22cyclictest0-21swapper/111:40:421
342484599498,10cyclictest12-21ksoftirqd/009:00:020
3424857994834,10cyclictest1-21systemd10:55:013
3424855994840,5cyclictest3424221-21kworker/2:109:19:392
3424855994840,5cyclictest3415397-21kworker/2:307:25:012
3424855994839,5cyclictest3415397-21kworker/2:307:35:412
3424851994833,11cyclictest23-21ksoftirqd/112:22:531
3424845994819,14cyclictest3441949-21ssh09:33:000
342484599481,10cyclictest12-21ksoftirqd/008:02:250
3424855994740,5cyclictest3415397-21kworker/2:308:53:342
342485199478,6cyclictest131rcu_preempt09:46:261
342485199476,8cyclictest131rcu_preempt09:20:021
3424851994731,12cyclictest1-21systemd09:00:001
342485199471,12cyclictest3518886-21ssh12:15:021
3424845994732,10cyclictest3523708-21cat12:25:010
3424857994631,12cyclictest0-21swapper/311:05:013
3424857994631,12cyclictest0-21swapper/311:05:013
3424857994629,14cyclictest3467602-21systemd-journal10:02:253
3424857994629,12cyclictest275-21dbus-daemon12:35:003
3424855994639,4cyclictest3415397-21kworker/2:308:30:092
3424855994639,4cyclictest3415397-21kworker/2:307:15:452
3424855994638,5cyclictest3415397-21kworker/2:308:57:352
3424855994638,5cyclictest3415397-21kworker/2:308:37:512
3424855994638,5cyclictest3415397-21kworker/2:307:54:372
342485199466,21cyclictest3507350-21ssh11:50:021
3424851994622,20cyclictest23-21ksoftirqd/112:35:301
342485199461,7cyclictest131rcu_preempt10:49:361
3424851994611,6cyclictest131rcu_preempt11:57:291
342484599461,13cyclictest3430248-21systemd-run08:55:000
342485799454,3cyclictest131rcu_preempt11:29:123
3424855994539,4cyclictest3415397-21kworker/2:308:25:282
3424855994537,5cyclictest3415397-21kworker/2:307:30:132
342485199455,5cyclictest131rcu_preempt10:35:021
3424851994530,12cyclictest23-21ksoftirqd/107:48:261
3424851994517,23cyclictest131rcu_preempt11:33:521
342484599454,7cyclictest12-21ksoftirqd/009:05:010
342484599454,7cyclictest12-21ksoftirqd/009:05:010
3424845994538,5cyclictest12-21ksoftirqd/010:20:390
3424857994430,10cyclictest3467602-21systemd-journal07:55:253
3424855994436,5cyclictest3415397-21kworker/2:309:09:372
3424855994436,5cyclictest3415397-21kworker/2:308:40:072
3424855994436,5cyclictest3415397-21kworker/2:308:12:162
3424855994436,5cyclictest3415397-21kworker/2:307:43:312
3424855994436,5cyclictest3415397-21kworker/2:307:14:342
3424855994411,5cyclictest131rcu_preempt07:30:012
342485199444,5cyclictest131rcu_preempt12:10:021
3424851994428,11cyclictest275-21dbus-daemon08:05:011
342485199442,6cyclictest131rcu_preempt07:20:001
342485199442,5cyclictest131rcu_preempt09:10:011
342485199441,16cyclictest131rcu_preempt11:30:001
3424857994328,12cyclictest37-21ksoftirqd/311:40:563
3424857994328,12cyclictest37-21ksoftirqd/311:40:563
3424857994327,12cyclictest37-21ksoftirqd/309:35:013
3424857994326,12cyclictest275-21dbus-daemon11:25:013
3424857994326,12cyclictest275-21dbus-daemon11:25:013
3424857994326,12cyclictest275-21dbus-daemon10:15:003
342485599435,6cyclictest131rcu_preempt07:10:022
3424855994327,11cyclictest1-21systemd08:05:012
342485199438,5cyclictest131rcu_preempt08:20:001
342485199434,4cyclictest131rcu_preempt10:20:001
3424851994327,12cyclictest275-21dbus-daemon10:55:491
3424851994327,12cyclictest23-21ksoftirqd/111:06:151
3424851994327,12cyclictest23-21ksoftirqd/110:08:081
3424851994327,12cyclictest23-21ksoftirqd/110:08:081
3424851994327,12cyclictest1-21systemd07:35:011
3424851994317,6cyclictest23-21ksoftirqd/109:39:181
3424845994330,7cyclictest309-21in:imuxsock07:30:010
3424845994329,11cyclictest12-21ksoftirqd/008:50:000
3424845994328,11cyclictest3467602-21systemd-journal08:30:020
3424845994327,11cyclictest275-21dbus-daemon07:20:000
342485599428,5cyclictest131rcu_preempt08:24:452
342485199423,7cyclictest23-21ksoftirqd/109:25:021
3424851994230,10cyclictest23-21ksoftirqd/109:51:541
3424851994226,4cyclictest23-21ksoftirqd/110:21:311
3424851994218,20cyclictest23-21ksoftirqd/109:28:141
342485199421,6cyclictest131rcu_preempt10:14:381
3424857994129,9cyclictest37-21ksoftirqd/308:00:243
3424857994121,15cyclictest1-21systemd09:50:023
3424855994134,4cyclictest3415397-21kworker/2:307:45:452
342485199417,6cyclictest131rcu_preempt07:55:021
3424851994128,9cyclictest3457163-21ssh10:05:011
342485199411,5cyclictest131rcu_preempt10:45:011
342485199411,32cyclictest3452531-21sh09:55:021
342485199411,32cyclictest3452531-21sh09:55:021
3424845994125,12cyclictest275-21dbus-daemon08:10:010
342484599411,36cyclictest3426407-21latency_hist07:40:020
342485799406,5cyclictest131rcu_preempt11:52:293
3424857994027,9cyclictest309-21in:imuxsock11:10:023
3424857994027,10cyclictest3452286-21cron09:55:003
3424857994025,11cyclictest37-21ksoftirqd/312:20:013
3424857994025,11cyclictest37-21ksoftirqd/312:20:013
3424857994025,11cyclictest275-21dbus-daemon07:35:013
3424857994020,16cyclictest275-21dbus-daemon11:40:013
342485799401,11cyclictest346304250ssh10:17:133
342485199405,6cyclictest131rcu_preempt09:44:591
3424851994027,10cyclictest3432132-21sshd09:12:261
3424851994027,10cyclictest3432132-21sshd09:12:261
342485199401,5cyclictest131rcu_preempt09:35:011
342485199401,5cyclictest131rcu_preempt07:15:001
342485199401,4cyclictest131rcu_preempt12:05:011
3424851994013,6cyclictest131rcu_preempt11:05:021
3424851994013,6cyclictest131rcu_preempt11:05:021
342485799393,4cyclictest131rcu_preempt11:20:003
3424857993926,10cyclictest3428608-21sed08:20:043
3424857993925,9cyclictest309-21in:imuxsock12:24:263
3424857993925,11cyclictest1-21systemd08:45:003
342485799391,3cyclictest131rcu_preempt08:35:013
3424851993929,8cyclictest23-21ksoftirqd/111:12:311
3424851993929,7cyclictest23-21ksoftirqd/107:38:511
342485199392,5cyclictest131rcu_preempt11:23:161
342485199392,5cyclictest131rcu_preempt11:23:161
342485199392,5cyclictest131rcu_preempt10:40:001
3424851993916,5cyclictest23-21ksoftirqd/107:20:241
342485199391,5cyclictest131rcu_preempt10:27:451
342485199391,5cyclictest131rcu_preempt08:00:021
3424845993930,7cyclictest12-21ksoftirqd/007:50:020
342485799387,6cyclictest131rcu_preempt09:05:013
342485799387,6cyclictest131rcu_preempt09:05:013
342485799382,8cyclictest0-21swapper/311:34:203
3424857993827,8cyclictest37-21ksoftirqd/309:56:303
3424857993827,8cyclictest37-21ksoftirqd/309:56:303
3424857993823,12cyclictest37-21ksoftirqd/310:41:193
3424857993822,12cyclictest1-21systemd08:50:013
3424857993820,14cyclictest275-21dbus-daemon09:00:013
342485799381,6cyclictest131rcu_preempt09:20:013
3424851993829,7cyclictest0-21swapper/107:10:021
342485199382,7cyclictest23-21ksoftirqd/108:55:001
3424851993826,9cyclictest3429754-21wc08:45:011
3424851993826,9cyclictest3428959-21systemd-run08:30:011
3424857993727,7cyclictest3483443-21cat11:00:023
3424857993727,6cyclictest3431147-21latency_hist09:10:023
342485799372,6cyclictest0-21swapper/309:11:063
342485799372,6cyclictest0-21swapper/309:11:063
3424857993726,6cyclictest3429442-21cron08:40:013
3424857993721,13cyclictest1-21systemd07:50:003
342485199376,6cyclictest131rcu_preempt08:33:251
3424851993727,8cyclictest23-21ksoftirqd/108:25:001
3424851993713,19cyclictest131rcu_preempt09:04:201
3424851993713,19cyclictest131rcu_preempt09:04:201
342484599371,5cyclictest12-21ksoftirqd/008:11:250
342484599371,33cyclictest0-21swapper/007:15:010
342485799364,5cyclictest131rcu_preempt08:55:003
3424857993625,9cyclictest37-21ksoftirqd/310:09:013
3424857993625,9cyclictest37-21ksoftirqd/310:09:013
3424857993624,10cyclictest37-21ksoftirqd/307:55:013
342485799362,3cyclictest131rcu_preempt10:35:033
342485199363,3cyclictest131rcu_preempt08:45:021
3424851993625,7cyclictest3429456-21sh08:40:011
3424851993622,10cyclictest3467602-21systemd-journal07:30:001
342485799354,4cyclictest131rcu_preempt12:11:273
3424857993527,5cyclictest3467372-21sh10:26:103
3424857993517,15cyclictest3467602-21systemd-journal07:11:253
342484599354,8cyclictest0-21swapper/007:42:200
3424845993522,9cyclictest12-21ksoftirqd/007:20:310
3424857993423,8cyclictest3427898-21cron08:10:003
3424851993423,8cyclictest1-21systemd07:45:001
342485799336,3cyclictest131rcu_preempt08:30:013
342485799332,5cyclictest131rcu_preempt07:45:003
342485799332,3cyclictest131rcu_preempt07:40:013
342485799326,7cyclictest37-21ksoftirqd/308:15:013
342485799313,24cyclictest3425644-21grep07:25:013
342485799272,3cyclictest131rcu_preempt07:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional