You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 21:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Tue Sep 22, 2026 12:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
113485599753,9cyclictest12-21ksoftirqd/012:00:010
113485599753,42cyclictest12-21ksoftirqd/011:15:030
113485599734,38cyclictest12-21ksoftirqd/009:30:010
1134855997114,29cyclictest131rcu_preempt10:08:240
1134868996757,7cyclictest37-21ksoftirqd/311:55:033
113485599653,11cyclictest121ksoftirqd/009:44:510
1134863996314,12cyclictest30-21ksoftirqd/211:55:002
1134863996310,5cyclictest131rcu_preempt11:19:592
113485599639,5cyclictest131rcu_preempt08:15:000
113485599636,30cyclictest12-21ksoftirqd/011:00:000
1134863996252,7cyclictest30-21ksoftirqd/210:15:032
1134863996252,7cyclictest30-21ksoftirqd/210:15:022
113485599623,6cyclictest121ksoftirqd/009:50:570
113485599623,6cyclictest121ksoftirqd/009:50:570
1134855996221,5cyclictest131rcu_preempt10:43:560
1134855996221,5cyclictest131rcu_preempt10:43:550
1134855996216,10cyclictest12-21ksoftirqd/009:20:000
1134855996214,6cyclictest131rcu_preempt07:20:020
113486399617,7cyclictest30-21ksoftirqd/209:20:022
1134863996110,5cyclictest131rcu_preempt10:55:022
1134863996110,5cyclictest131rcu_preempt10:55:012
113485599618,21cyclictest12-21ksoftirqd/009:47:370
113485599615,11cyclictest12-21ksoftirqd/008:10:010
113485599613,30cyclictest131rcu_preempt11:45:030
1134868996033,15cyclictest37-21ksoftirqd/309:45:393
113485599603,20cyclictest1164403-21unin-run10:00:010
1134855996031,13cyclictest1208938-21latency_hist11:35:020
1134855996031,13cyclictest1208938-21latency_hist11:35:020
1134855996027,29cyclictest151rcuc/010:19:530
1134855996027,29cyclictest151rcuc/010:19:530
113485599602,44cyclictest12-21ksoftirqd/010:02:380
1134855996012,5cyclictest131rcu_preempt11:40:010
1134863995912,6cyclictest131rcu_preempt09:24:322
1134863995911,6cyclictest131rcu_preempt08:49:592
113485599595,27cyclictest131rcu_preempt11:08:010
1134855995949,6cyclictest16950irq/224-800000009:10:030
1134855995949,6cyclictest16950irq/224-800000009:10:030
113485599593,5cyclictest131rcu_preempt12:25:380
1134855995932,12cyclictest1218067-21ssh11:54:220
113485599591,8cyclictest1181083-21latency_hist10:35:020
1134868995825,23cyclictest37-21ksoftirqd/309:32:243
1134868995811,6cyclictest131rcu_preempt12:40:023
1134863995850,5cyclictest30-21ksoftirqd/212:05:012
1134863995850,5cyclictest30-21ksoftirqd/212:05:012
1134863995834,7cyclictest1216050-21kworker/2:012:19:012
113485599586,29cyclictest12-21ksoftirqd/011:27:220
113485599583,9cyclictest12-21ksoftirqd/012:40:000
1134855995822,32cyclictest1190360-21sed10:55:020
1134855995822,32cyclictest1190360-21sed10:55:010
1134855995810,6cyclictest131rcu_preempt10:48:220
1134868995716,6cyclictest131rcu_preempt10:45:013
1134868995716,6cyclictest131rcu_preempt10:45:013
113486899571,5cyclictest131rcu_preempt10:55:013
113486899571,5cyclictest131rcu_preempt10:55:013
113486399576,6cyclictest131rcu_preempt10:00:002
113485599578,18cyclictest131rcu_preempt07:50:020
113485599578,18cyclictest131rcu_preempt07:50:010
113485599577,22cyclictest131rcu_preempt10:25:380
113485599572,11cyclictest12-21ksoftirqd/008:20:010
113486899568,10cyclictest37-21ksoftirqd/308:45:013
113486899568,10cyclictest37-21ksoftirqd/308:45:003
1134868995627,9cyclictest37-21ksoftirqd/311:20:013
113486899562,6cyclictest131rcu_preempt11:40:013
113486399568,5cyclictest131rcu_preempt07:10:022
1134863995611,6cyclictest131rcu_preempt09:30:012
1134863995611,5cyclictest131rcu_preempt09:37:032
113485599567,11cyclictest12-21ksoftirqd/012:05:020
113485599567,11cyclictest12-21ksoftirqd/012:05:020
113485599565,15cyclictest131rcu_preempt11:04:050
113485599564,13cyclictest309-21in:imuxsock07:40:000
113485599563,16cyclictest12-21ksoftirqd/008:49:590
1134855995625,15cyclictest1151664-21ssh09:32:370
1134855995621,31cyclictest12-21ksoftirqd/012:08:250
1134855995616,6cyclictest131rcu_preempt08:00:010
1134855995615,37cyclictest12-21ksoftirqd/012:33:310
113485599561,14cyclictest120-21kswapd012:20:000
1134868995531,8cyclictest37-21ksoftirqd/311:32:113
1134868995531,8cyclictest37-21ksoftirqd/311:32:103
113486899552,4cyclictest131rcu_preempt10:40:013
1134868995515,11cyclictest37-21ksoftirqd/310:30:013
113486399557,6cyclictest131rcu_preempt12:40:002
113486399555,7cyclictest131rcu_preempt09:40:052
1134863995511,9cyclictest131rcu_preempt11:15:012
1134863995510,6cyclictest131rcu_preempt11:00:002
1134855995514,5cyclictest131rcu_preempt12:20:540
1134855995514,36cyclictest131rcu_preempt09:36:340
113486899548,6cyclictest131rcu_preempt11:50:003
113486899544,10cyclictest37-21ksoftirqd/310:04:013
1134868995424,21cyclictest37-21ksoftirqd/311:03:543
113486399548,6cyclictest131rcu_preempt10:40:012
113486399548,4cyclictest131rcu_preempt11:03:452
113486399546,11cyclictest30-21ksoftirqd/211:32:502
113486399546,11cyclictest30-21ksoftirqd/211:32:502
1134863995422,6cyclictest131rcu_preempt12:06:072
113486399541,7cyclictest1211344-21systemctl11:40:002
1134863995414,6cyclictest131rcu_preempt08:30:002
113486399541,14cyclictest30-21ksoftirqd/209:14:182
1134858995413,24cyclictest0-21swapper/111:35:031
1134858995413,24cyclictest0-21swapper/111:35:021
113485599548,26cyclictest131rcu_preempt11:23:270
1134855995417,5cyclictest131rcu_preempt10:35:330
113486899536,7cyclictest37-21ksoftirqd/309:45:023
113486899533,7cyclictest37-21ksoftirqd/310:05:083
113486399539,5cyclictest131rcu_preempt10:10:002
113486399537,7cyclictest30-21ksoftirqd/209:45:102
113486399536,8cyclictest30-21ksoftirqd/208:20:032
113486399535,6cyclictest131rcu_preempt11:25:012
1134863995344,6cyclictest1138744-21kworker/2:111:25:112
1134863995340,6cyclictest1162085-21kworker/2:210:24:002
1134863995310,6cyclictest131rcu_preempt09:00:012
1134858995329,4cyclictest23-21ksoftirqd/112:40:021
113485599538,5cyclictest131rcu_preempt11:46:550
113485599538,5cyclictest131rcu_preempt07:55:010
113485599536,11cyclictest12-21ksoftirqd/009:14:590
113485599534,11cyclictest12-21ksoftirqd/007:30:000
1134855995326,12cyclictest1170377-21ssh10:12:320
1134855995318,31cyclictest16850irq/134-800000012:11:550
113486899529,12cyclictest37-21ksoftirqd/310:20:023
113486899529,12cyclictest37-21ksoftirqd/310:20:023
1134868995236,11cyclictest275-21dbus-daemon09:55:023
1134868995236,11cyclictest275-21dbus-daemon09:55:013
1134868995228,15cyclictest37-21ksoftirqd/309:14:333
1134868995219,10cyclictest37-21ksoftirqd/312:17:383
1134868995217,25cyclictest37-21ksoftirqd/311:28:553
113486399529,5cyclictest131rcu_preempt12:24:452
113486399527,5cyclictest131rcu_preempt07:40:022
113486399526,9cyclictest30-21ksoftirqd/211:50:012
1134863995238,6cyclictest1138744-21kworker/2:109:31:562
1134863995211,5cyclictest131rcu_preempt09:55:012
1134863995211,5cyclictest131rcu_preempt09:55:012
113485599527,26cyclictest19350irq/34-mmc108:25:020
113485599525,5cyclictest131rcu_preempt11:15:530
1134855995218,17cyclictest16850irq/134-800000009:20:260
1134855995217,17cyclictest16850irq/134-800000010:23:030
113486899518,6cyclictest131rcu_preempt08:05:003
113486899518,4cyclictest131rcu_preempt12:25:233
113486899515,6cyclictest131rcu_preempt09:40:013
1134868995120,17cyclictest37-21ksoftirqd/312:08:333
113486899511,8cyclictest37-21ksoftirqd/307:20:003
1134868995118,24cyclictest37-21ksoftirqd/310:46:103
1134868995113,6cyclictest131rcu_preempt08:40:013
1134868995111,6cyclictest131rcu_preempt07:15:013
1134868995111,6cyclictest131rcu_preempt07:15:003
113486399518,6cyclictest131rcu_preempt07:20:022
113486399518,5cyclictest131rcu_preempt10:14:382
113486399517,6cyclictest131rcu_preempt12:30:042
1134863995143,5cyclictest1162085-21kworker/2:210:28:062
1134863995143,5cyclictest1138744-21kworker/2:110:02:122
1134863995142,6cyclictest1128467-21kworker/2:007:23:012
1134863995115,6cyclictest131rcu_preempt10:41:122
1134863995115,6cyclictest131rcu_preempt10:41:112
1134863995113,6cyclictest131rcu_preempt08:00:002
1134858995130,16cyclictest3467602-21systemd-journal08:50:001
1134855995112,5cyclictest131rcu_preempt07:45:030
113485599511,10cyclictest12-21ksoftirqd/007:10:020
113486899505,6cyclictest131rcu_preempt09:30:003
113486899504,6cyclictest131rcu_preempt12:14:353
113486899504,4cyclictest131rcu_preempt08:50:013
1134868995010,5cyclictest131rcu_preempt12:04:163
1134868995010,5cyclictest131rcu_preempt12:04:153
113486399504,7cyclictest30-21ksoftirqd/211:05:542
113486399504,5cyclictest131rcu_preempt10:31:232
1134863995042,5cyclictest1138744-21kworker/2:111:58:232
1134863995036,5cyclictest1138744-21kworker/2:111:42:422
113485899506,36cyclictest1227767-21ssh12:15:021
1134858995033,12cyclictest3467602-21systemd-journal07:38:371
113485599506,12cyclictest12-21ksoftirqd/008:40:000
113485599505,8cyclictest12-21ksoftirqd/008:35:020
113486899497,4cyclictest131rcu_preempt10:10:473
113486899494,7cyclictest1192543-21ssh10:59:593
1134868994931,13cyclictest275-21dbus-daemon11:15:013
1134868994925,14cyclictest37-21ksoftirqd/311:58:523
1134868994922,18cyclictest0-21swapper/311:42:443
113486899491,18cyclictest0-21swapper/309:57:123
113486399499,5cyclictest131rcu_preempt10:45:092
113486399498,6cyclictest131rcu_preempt07:51:222
113486399497,5cyclictest131rcu_preempt08:10:002
113486399496,5cyclictest131rcu_preempt08:25:012
113486399495,6cyclictest131rcu_preempt08:45:032
113486399495,6cyclictest131rcu_preempt08:45:032
1134863994941,5cyclictest1216050-21kworker/2:012:10:152
1134863994941,5cyclictest1138744-21kworker/2:112:28:392
1134863994941,5cyclictest1135609-21kworker/2:308:00:382
1134858994932,13cyclictest275-21dbus-daemon08:35:011
113485599496,11cyclictest12-21ksoftirqd/007:25:020
1134855994912,10cyclictest12-21ksoftirqd/008:55:020
113486899485,6cyclictest131rcu_preempt08:55:013
1134868994811,6cyclictest131rcu_preempt11:22:153
1134868994811,5cyclictest131rcu_preempt08:00:003
113486399488,6cyclictest131rcu_preempt09:05:022
1134863994840,5cyclictest1135609-21kworker/2:307:43:082
113486399482,5cyclictest131rcu_preempt09:10:022
113486399482,5cyclictest131rcu_preempt09:10:012
113486399482,4cyclictest131rcu_preempt07:15:022
113486399482,4cyclictest131rcu_preempt07:15:012
1134858994829,5cyclictest23-21ksoftirqd/108:55:021
1134855994813,5cyclictest131rcu_preempt09:05:010
113486899478,4cyclictest131rcu_preempt10:20:373
113486899477,4cyclictest131rcu_preempt12:23:153
113486899477,10cyclictest37-21ksoftirqd/310:35:013
113486899474,9cyclictest37-21ksoftirqd/307:35:013
113486899474,6cyclictest131rcu_preempt11:10:013
113486899474,6cyclictest131rcu_preempt09:22:253
113486899473,7cyclictest37-21ksoftirqd/309:18:043
113486899471,17cyclictest0-21swapper/312:31:343
1134868994710,6cyclictest131rcu_preempt09:00:013
1134863994740,4cyclictest1138744-21kworker/2:108:53:332
1134863994739,5cyclictest1135609-21kworker/2:308:33:012
1134863994739,5cyclictest1128467-21kworker/2:007:31:142
1134863994739,5cyclictest1128467-21kworker/2:007:25:122
1134863994738,5cyclictest1135609-21kworker/2:307:46:162
1134863994738,5cyclictest1135609-21kworker/2:307:46:152
1134863994729,10cyclictest30-21ksoftirqd/208:40:012
113485599471,7cyclictest0-21swapper/008:45:010
113485599471,7cyclictest0-21swapper/008:45:000
113486899464,6cyclictest131rcu_preempt08:15:003
113486899463,5cyclictest131rcu_preempt08:25:023
113486899462,4cyclictest131rcu_preempt08:29:383
1134863994638,5cyclictest1135609-21kworker/2:308:13:172
1134858994628,12cyclictest23-21ksoftirqd/110:00:021
1134858994611,6cyclictest131rcu_preempt11:15:001
1134855994629,12cyclictest275-21dbus-daemon09:00:010
113486899451,5cyclictest131rcu_preempt08:20:013
1134858994533,7cyclictest1145649-21cron09:20:001
113485599457,5cyclictest131rcu_preempt07:35:020
1134855994527,13cyclictest1-21systemd08:30:000
113485599451,5cyclictest131rcu_preempt07:15:000
113485599451,5cyclictest131rcu_preempt07:15:000
113486899446,6cyclictest131rcu_preempt09:05:003
113486899445,5cyclictest131rcu_preempt08:35:003
1134868994411,5cyclictest131rcu_preempt08:10:043
1134868994410,5cyclictest131rcu_preempt07:41:363
1134858994431,9cyclictest23-21ksoftirqd/107:50:011
1134858994431,9cyclictest23-21ksoftirqd/107:50:011
1134858994414,21cyclictest0-21swapper/111:20:021
113486899439,7cyclictest131rcu_preempt07:55:013
113486899431,8cyclictest0-21swapper/307:25:023
113486899431,4cyclictest131rcu_preempt09:10:013
113486899431,4cyclictest131rcu_preempt09:10:013
113485899433,3cyclictest131rcu_preempt09:41:091
113485899432,4cyclictest131rcu_preempt12:04:091
113485899432,4cyclictest131rcu_preempt12:04:091
113485899431,5cyclictest131rcu_preempt07:25:021
1134858994232,7cyclictest1-21systemd08:20:001
1134858994231,8cyclictest23-21ksoftirqd/111:39:171
1134858994230,9cyclictest1136761-21sed07:45:011
1134858994226,13cyclictest275-21dbus-daemon11:30:001
113485899421,3cyclictest131rcu_preempt10:17:301
113485899421,3cyclictest131rcu_preempt10:17:291
113486899417,5cyclictest131rcu_preempt07:29:593
1134868994112,19cyclictest131rcu_preempt07:40:033
113485899413,3cyclictest131rcu_preempt10:12:071
113485899413,27cyclictest1171367-21kworker/u8:010:35:021
1134858994127,11cyclictest3467602-21systemd-journal07:11:021
1134858994127,11cyclictest3467602-21systemd-journal07:11:011
1134858994127,11cyclictest23-21ksoftirqd/111:56:191
113485899412,5cyclictest131rcu_preempt11:54:311
113485899412,3cyclictest131rcu_preempt10:26:141
1134858994123,13cyclictest275-21dbus-daemon09:25:011
113485899411,8cyclictest1140515-21munin-run09:00:011
1134858994118,19cyclictest23-21ksoftirqd/107:18:361
113485899411,5cyclictest131rcu_preempt09:30:021
113486899404,5cyclictest131rcu_preempt07:10:023
1134868994010,5cyclictest131rcu_preempt07:49:183
1134868994010,5cyclictest131rcu_preempt07:49:173
113485899403,3cyclictest131rcu_preempt11:00:211
1134858994032,5cyclictest1136100-21munin-plugin-st07:35:011
1134858994030,7cyclictest1154026-21ssh09:37:371
1134858994028,10cyclictest23-21ksoftirqd/110:48:371
1134858994027,10cyclictest23-21ksoftirqd/109:10:021
1134858994027,10cyclictest23-21ksoftirqd/109:10:011
1134858994027,10cyclictest1225362-21sed12:10:011
1134858994026,8cyclictest309-21in:imuxsock07:50:021
1134858994026,11cyclictest23-21ksoftirqd/108:22:381
113485899402,5cyclictest131rcu_preempt09:34:421
1134858994019,6cyclictest131rcu_preempt10:36:151
1134858994018,4cyclictest23-21ksoftirqd/111:41:261
113485899401,5cyclictest131rcu_preempt12:16:371
113485599405,8cyclictest12-21ksoftirqd/008:05:000
113485899399,6cyclictest131rcu_preempt10:45:001
113485899399,6cyclictest131rcu_preempt10:45:001
113485899395,28cyclictest3467602-21systemd-journal08:00:001
113485899394,6cyclictest131rcu_preempt09:50:581
113485899394,6cyclictest131rcu_preempt09:50:581
1134858993926,10cyclictest23-21ksoftirqd/109:05:001
113485899392,4cyclictest131rcu_preempt10:55:491
1134858993918,5cyclictest131rcu_preempt10:01:561
1134858993914,17cyclictest23-21ksoftirqd/112:31:331
1134858993912,5cyclictest131rcu_preempt12:25:011
113485899386,5cyclictest131rcu_preempt09:12:531
113485899383,5cyclictest131rcu_preempt09:45:441
113485899381,19cyclictest0-21swapper/110:06:561
1134858993810,5cyclictest131rcu_preempt11:49:061
1134858993810,5cyclictest131rcu_preempt08:09:391
1134858993727,8cyclictest23-21ksoftirqd/111:21:471
1134858993726,7cyclictest3467602-21systemd-journal08:40:001
113485899371,9cyclictest0-21swapper/111:09:161
1134858993719,14cyclictest1190192-21munin-run10:55:011
1134858993719,14cyclictest1190192-21munin-run10:55:011
113485899371,4cyclictest131rcu_preempt10:20:071
1134858993624,9cyclictest23-21ksoftirqd/108:41:371
1134858993624,9cyclictest23-21ksoftirqd/108:41:361
113485899362,3cyclictest131rcu_preempt12:25:571
113485899354,5cyclictest131rcu_preempt08:30:011
113485899351,9cyclictest1138181-21true08:15:011
1134858993517,6cyclictest131rcu_preempt08:05:021
1134858993423,8cyclictest1135995-21sed07:30:011
113485899311,5cyclictest0-21swapper/107:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional