You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 16:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Jun 24, 2026 12:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1097269996926,5cyclictest309-21in:imuxsock10:00:012
109725999683,44cyclictest131rcu_preempt11:08:420
1097269996623,12cyclictest3467602-21systemd-journal10:40:002
1097259996330,21cyclictest131rcu_preempt12:15:010
1097269996227,12cyclictest30-21ksoftirqd/211:06:212
1097269996223,8cyclictest1100557-21sshd08:13:422
1097264996225,13cyclictest1-21systemd08:00:011
109725999624,40cyclictest1196309-21munin-plugin-st12:25:000
1097259996233,4cyclictest12-21ksoftirqd/008:50:020
1097259996221,37cyclictest1165375-21latency_hist11:20:010
1097269996128,11cyclictest30-21ksoftirqd/210:21:522
1097259996130,26cyclictest1-21systemd09:50:030
1097269996025,6cyclictest1185766-21sh12:02:392
109725999608,15cyclictest131rcu_preempt12:40:000
109726499591,53cyclictest1-21systemd08:45:011
109726499591,53cyclictest1-21systemd08:45:011
1097259995925,29cyclictest1-21systemd09:55:000
1097259995916,38cyclictest12-21ksoftirqd/011:30:010
1097269995828,9cyclictest30-21ksoftirqd/212:40:022
109726999573,50cyclictest1141794-21ssh10:30:032
1097269995727,10cyclictest30-21ksoftirqd/211:26:302
1097269995725,8cyclictest30-21ksoftirqd/212:26:072
1097269995725,8cyclictest30-21ksoftirqd/212:26:072
1097259995729,12cyclictest1112304-21ssh09:28:230
1097259995719,6cyclictest16850irq/134-800000010:23:490
1097259995718,7cyclictest16850irq/134-800000010:15:010
1097269995626,8cyclictest30-21ksoftirqd/208:25:012
109725999567,12cyclictest1155867-21munin-plugin-st11:00:010
1097259995628,13cyclictest1199386-21ssh12:31:090
1097259995627,13cyclictest1113671-21ssh09:30:580
1097259995619,6cyclictest16850irq/134-800000011:14:200
1097259995619,32cyclictest16850irq/134-800000010:06:170
1097259995617,7cyclictest16850irq/134-800000009:19:220
1097269995527,7cyclictest30-21ksoftirqd/210:13:212
1097259995520,31cyclictest16850irq/134-800000011:53:400
1097259995519,32cyclictest16850irq/134-800000012:09:330
1097259995519,32cyclictest16850irq/134-800000012:09:330
1097259995518,32cyclictest16850irq/134-800000010:36:150
1097259995517,6cyclictest16850irq/134-800000009:38:420
1097259995517,34cyclictest16850irq/134-800000011:04:040
1097259995517,34cyclictest16850irq/134-800000011:04:040
1097259995517,34cyclictest16850irq/134-800000010:29:420
109725999544,16cyclictest1103523-21wc09:10:020
1097259995419,31cyclictest16850irq/134-800000010:41:380
1097259995418,7cyclictest16850irq/134-800000011:20:290
1097259995418,7cyclictest16850irq/134-800000011:20:290
1097259995418,32cyclictest16850irq/134-800000009:24:460
1097259995418,32cyclictest16850irq/134-800000009:11:350
1097259995418,31cyclictest16850irq/134-800000012:25:280
1097259995418,31cyclictest16850irq/134-800000012:25:280
1097259995418,31cyclictest16850irq/134-800000011:55:420
1097259995418,31cyclictest16850irq/134-800000011:32:190
1097259995415,35cyclictest16850irq/134-800000009:56:240
1097259995415,34cyclictest16850irq/134-800000010:45:260
1097259995414,35cyclictest16850irq/134-800000010:03:330
1097274995337,12cyclictest275-21dbus-daemon07:40:013
1097269995335,9cyclictest1139265-21kworker/2:110:51:352
1097269995326,9cyclictest30-21ksoftirqd/211:38:282
1097259995327,11cyclictest1178421-21ssh11:47:060
1097259995319,30cyclictest16850irq/134-800000011:38:270
1097259995317,32cyclictest16850irq/134-800000011:43:150
1097259995316,33cyclictest16850irq/134-800000012:01:420
1097259995316,32cyclictest16850irq/134-800000009:43:490
1097259995315,34cyclictest16850irq/134-800000010:34:310
1097269995244,5cyclictest1160895-21kworker/2:011:46:472
1097264995225,6cyclictest23-21ksoftirqd/111:30:001
1097259995224,13cyclictest1135171-21ssh10:16:100
1097259995216,32cyclictest16850irq/134-800000012:15:150
1097259995216,32cyclictest16850irq/134-800000012:15:150
1097259995213,35cyclictest16850irq/134-800000010:52:100
1097269995142,5cyclictest1120192-21kworker/2:210:40:512
1097269995042,5cyclictest1160895-21kworker/2:012:18:262
1097269995042,5cyclictest1160895-21kworker/2:012:18:262
1097264995031,15cyclictest3467602-21systemd-journal07:45:001
109725999501,15cyclictest1100879-21munin-run08:20:020
1097274994931,14cyclictest275-21dbus-daemon10:45:013
1097274994918,4cyclictest150-21pr/ttyS211:00:073
1097274994918,4cyclictest150-21pr/ttyS211:00:073
1097269994940,5cyclictest1139265-21kworker/2:110:48:272
109727499483,4cyclictest131rcu_preempt11:52:553
109727499481,5cyclictest131rcu_preempt11:30:013
1097274994811,5cyclictest131rcu_preempt10:52:523
1097269994840,5cyclictest1139265-21kworker/2:111:20:472
1097269994840,5cyclictest1139265-21kworker/2:111:20:472
1097269994840,5cyclictest1100602-21kworker/2:109:19:492
1097269994839,6cyclictest1108342-21kworker/2:310:07:452
1097269994839,6cyclictest1108342-21kworker/2:309:53:312
1097269994835,10cyclictest275-21dbus-daemon08:40:002
1097264994835,9cyclictest1-21systemd09:40:011
1097264994829,14cyclictest1-21systemd07:30:011
109725999483,10cyclictest1102913-21munin-run09:00:000
109725999483,10cyclictest1102913-21munin-run09:00:000
109727499478,7cyclictest37-21ksoftirqd/310:06:573
109727499474,3cyclictest131rcu_preempt10:33:033
109727499472,4cyclictest131rcu_preempt12:04:183
1097269994740,5cyclictest1120192-21kworker/2:210:04:352
109726499475,7cyclictest23-21ksoftirqd/109:30:051
1097264994731,13cyclictest23-21ksoftirqd/110:25:021
109725999477,7cyclictest12-21ksoftirqd/008:40:000
109725999472,41cyclictest1099755-21sshd07:55:420
109727499465,6cyclictest131rcu_preempt09:47:173
109727499464,3cyclictest131rcu_preempt12:40:023
109727499464,3cyclictest131rcu_preempt12:10:013
109727499464,3cyclictest131rcu_preempt12:10:013
109727499463,4cyclictest131rcu_preempt09:14:183
109727499462,4cyclictest131rcu_preempt11:39:093
109727499461,5cyclictest131rcu_preempt10:05:023
1097274994611,5cyclictest131rcu_preempt09:44:263
1097269994638,5cyclictest1177589-21kworker/2:212:10:432
1097269994638,5cyclictest1120192-21kworker/2:210:32:162
1097269994637,6cyclictest1139265-21kworker/2:111:11:552
1097269994632,10cyclictest275-21dbus-daemon11:45:022
1097269994630,11cyclictest1-21systemd12:10:012
1097269994630,11cyclictest1-21systemd12:10:012
1097269994618,18cyclictest30-21ksoftirqd/212:22:132
109726499466,6cyclictest131rcu_preempt12:25:021
1097264994633,9cyclictest3467602-21systemd-journal11:00:011
1097264994633,10cyclictest1174874-21cron11:40:001
1097264994612,6cyclictest131rcu_preempt10:33:021
1097259994628,14cyclictest275-21dbus-daemon07:40:010
109727499454,4cyclictest131rcu_preempt09:58:233
109727499454,3cyclictest131rcu_preempt09:24:113
1097274994533,9cyclictest37-21ksoftirqd/312:00:023
1097274994530,12cyclictest275-21dbus-daemon10:40:013
109727499452,3cyclictest131rcu_preempt08:40:003
109727499452,13cyclictest1150668-21ssh10:48:583
109727499451,8cyclictest1138101-21ssh10:22:233
109727499451,3cyclictest131rcu_preempt12:20:033
109727499451,3cyclictest131rcu_preempt12:20:023
109727499451,3cyclictest131rcu_preempt09:40:013
1097274994512,4cyclictest1103441-21diskmemload12:12:123
1097269994537,5cyclictest1177589-21kworker/2:212:33:492
1097269994537,5cyclictest1120192-21kworker/2:211:00:112
1097269994537,5cyclictest1120192-21kworker/2:211:00:102
1097269994537,5cyclictest1099830-21kworker/2:009:12:042
1097269994537,5cyclictest1099830-21kworker/2:009:06:402
1097269994535,6cyclictest1139265-21kworker/2:111:30:562
1097269994531,11cyclictest1-21systemd07:40:012
1097269994529,11cyclictest275-21dbus-daemon09:35:002
1097269994519,17cyclictest30-21ksoftirqd/211:55:332
109726499453,7cyclictest23-21ksoftirqd/112:11:271
109726499452,6cyclictest1103441-21diskmemload10:54:591
1097264994518,12cyclictest275-21dbus-daemon11:20:001
1097259994520,22cyclictest1101569-21latency_hist08:30:030
109727499444,3cyclictest131rcu_preempt11:24:513
109727499444,3cyclictest131rcu_preempt11:24:513
109727499444,3cyclictest131rcu_preempt09:25:253
109727499443,4cyclictest131rcu_preempt08:45:023
109727499443,4cyclictest131rcu_preempt08:45:023
109727499443,3cyclictest131rcu_preempt12:33:433
109727499443,3cyclictest131rcu_preempt07:15:003
1097274994432,9cyclictest37-21ksoftirqd/310:15:003
109727499442,11cyclictest1163771-21ssh11:16:273
1097269994436,5cyclictest1160895-21kworker/2:011:18:492
1097269994436,5cyclictest1108342-21kworker/2:309:25:272
1097269994435,6cyclictest1100864-21sendmail-msp08:20:012
1097269994430,10cyclictest1134951-21rm10:15:332
109726499444,4cyclictest131rcu_preempt11:25:011
109726499444,4cyclictest131rcu_preempt11:25:001
109726499443,3cyclictest131rcu_preempt10:05:011
1097264994431,10cyclictest1111580-21sshd09:26:421
1097264994427,12cyclictest275-21dbus-daemon09:25:011
109726499442,3cyclictest131rcu_preempt12:09:351
109726499442,3cyclictest131rcu_preempt12:09:341
109726499441,3cyclictest131rcu_preempt12:40:001
109726499441,3cyclictest131rcu_preempt09:15:001
1097259994433,7cyclictest1-21systemd07:15:000
109725999443,37cyclictest1097731-21munin-run07:20:000
109727499434,4cyclictest131rcu_preempt11:42:233
109727499434,4cyclictest131rcu_preempt09:16:213
109727499433,4cyclictest131rcu_preempt11:33:443
1097274994327,12cyclictest275-21dbus-daemon07:35:013
1097274994316,4cyclictest37-21ksoftirqd/307:45:013
1097274994316,23cyclictest361rcuc/311:05:363
109727499431,5cyclictest131rcu_preempt10:59:043
109727499431,3cyclictest131rcu_preempt07:50:023
1097269994335,5cyclictest1120192-21kworker/2:210:56:422
1097269994329,11cyclictest1-21systemd11:55:012
109726499435,5cyclictest131rcu_preempt12:17:431
109726499435,5cyclictest131rcu_preempt12:17:421
109726499433,3cyclictest131rcu_preempt09:50:031
1097264994327,11cyclictest275-21dbus-daemon11:55:021
1097264994326,13cyclictest1-21systemd08:05:011
109726499432,3cyclictest131rcu_preempt11:35:021
109726499431,3cyclictest131rcu_preempt09:50:001
1097264994312,4cyclictest150-21pr/ttyS210:10:131
109725999433,8cyclictest3467602-21systemd-journal09:05:000
1097259994329,9cyclictest1099540-21cut07:55:010
1097259994327,11cyclictest1100563-21cron08:15:000
109727499423,3cyclictest131rcu_preempt12:29:423
109727499423,3cyclictest131rcu_preempt12:29:413
109727499423,3cyclictest0-21swapper/309:54:283
1097274994226,12cyclictest275-21dbus-daemon07:20:013
109727499421,4cyclictest131rcu_preempt10:26:543
1097269994234,5cyclictest1099830-21kworker/2:009:04:422
1097269994230,9cyclictest3467602-21systemd-journal09:43:102
1097269994230,8cyclictest1099292-21systemd-run07:50:012
1097269994228,11cyclictest1098159-21sed07:25:032
1097269994227,12cyclictest3467602-21systemd-journal08:43:442
1097269994227,12cyclictest3467602-21systemd-journal08:43:432
1097269994227,11cyclictest1099658-21cat07:55:022
1097269994226,12cyclictest1122744-21sed09:50:032
109726499427,6cyclictest131rcu_preempt09:15:441
109726499427,5cyclictest131rcu_preempt10:46:561
109726499426,4cyclictest131rcu_preempt12:28:021
109726499426,4cyclictest131rcu_preempt12:28:011
109726499423,4cyclictest131rcu_preempt11:02:401
109726499423,4cyclictest131rcu_preempt11:02:391
1097264994228,11cyclictest23-21ksoftirqd/107:25:021
109726499422,4cyclictest131rcu_preempt11:08:431
109726499421,3cyclictest131rcu_preempt10:10:021
109726499421,3cyclictest131rcu_preempt08:35:011
1097259994227,12cyclictest275-21dbus-daemon07:35:010
109727499417,4cyclictest131rcu_preempt08:35:003
109727499413,3cyclictest131rcu_preempt11:13:213
109727499412,6cyclictest0-21swapper/308:15:013
1097274994125,12cyclictest37-21ksoftirqd/312:24:063
1097274994116,4cyclictest37-21ksoftirqd/307:55:023
109727499411,4cyclictest131rcu_preempt08:30:003
109727499411,4cyclictest131rcu_preempt08:30:003
109726999415,8cyclictest30-21ksoftirqd/209:23:002
1097269994128,10cyclictest275-21dbus-daemon08:55:012
1097269994128,10cyclictest1102385-21sshd08:49:422
1097269994126,11cyclictest1099824-21systemd-run08:00:002
109726499419,3cyclictest150-21pr/ttyS210:29:231
109726499418,3cyclictest131rcu_preempt11:14:231
109726499414,3cyclictest131rcu_preempt09:45:001
1097264994128,10cyclictest23-21ksoftirqd/111:50:011
109726499412,6cyclictest131rcu_preempt12:01:471
109726499412,3cyclictest131rcu_preempt10:18:511
109726499412,10cyclictest1143481-21kworker/u8:310:35:201
109726499411,9cyclictest1-21systemd08:50:001
109726499411,4cyclictest131rcu_preempt11:40:431
1097259994125,9cyclictest0-21swapper/007:50:020
1097259994123,14cyclictest1098234-21sshd07:29:420
109725999411,36cyclictest1101333-21sshd08:25:420
109727499404,4cyclictest131rcu_preempt11:49:163
109727499404,4cyclictest131rcu_preempt08:55:013
109727499404,3cyclictest131rcu_preempt09:30:213
109727499403,4cyclictest131rcu_preempt10:18:163
1097269994032,5cyclictest1108342-21kworker/2:309:38:402
109726499403,3cyclictest131rcu_preempt11:57:031
1097259994029,7cyclictest12-21ksoftirqd/008:10:010
109727499394,3cyclictest131rcu_preempt08:16:423
1097274993920,13cyclictest3467602-21systemd-journal08:25:013
109726499393,4cyclictest131rcu_preempt09:59:501
1097264993921,13cyclictest3467602-21systemd-journal08:10:001
109726499391,3cyclictest131rcu_preempt12:35:001
109726499391,3cyclictest131rcu_preempt10:42:341
109725999391,34cyclictest1101593-21sshd08:32:420
1097274993824,11cyclictest275-21dbus-daemon09:05:003
109727499381,3cyclictest131rcu_preempt08:02:483
1097269993831,5cyclictest1085439-21kworker/2:207:13:512
1097269993828,7cyclictest1100294-21sshd08:07:422
1097269993823,11cyclictest1-21systemd07:35:012
109726999381,4cyclictest131rcu_preempt07:20:002
109726499382,3cyclictest131rcu_preempt07:20:021
1097264993821,12cyclictest275-21dbus-daemon09:05:001
109726499381,4cyclictest131rcu_preempt08:20:021
109726499381,4cyclictest131rcu_preempt07:15:011
109726499381,33cyclictest1098841-21wc07:40:021
1097259993834,2cyclictest0-21swapper/008:25:020
109725999381,33cyclictest1099012-21sshd07:42:460
1097274993724,9cyclictest275-21dbus-daemon07:30:003
109727499371,3cyclictest131rcu_preempt08:10:013
109727499371,3cyclictest131rcu_preempt08:00:003
1097274993713,17cyclictest1102903-21modprobe09:00:003
1097274993713,17cyclictest1102903-21modprobe09:00:003
109726999371,3cyclictest131rcu_preempt07:10:022
109726999371,3cyclictest131rcu_preempt07:10:022
109726499371,5cyclictest131rcu_preempt09:10:021
109726499371,31cyclictest1102618-21sshd08:50:421
1097259993725,10cyclictest12-21ksoftirqd/007:10:020
1097259993725,10cyclictest12-21ksoftirqd/007:10:020
1097259993721,7cyclictest309-21in:imuxsock08:45:000
1097259993721,7cyclictest309-21in:imuxsock08:45:000
109726999368,5cyclictest131rcu_preempt08:30:382
1097269993617,13cyclictest3467602-21systemd-journal08:30:002
1097269993617,13cyclictest3467602-21systemd-journal08:30:002
109726999361,25cyclictest0-21swapper/207:30:022
109726499364,3cyclictest131rcu_preempt07:10:021
109726499364,3cyclictest131rcu_preempt07:10:021
109726499361,6cyclictest131rcu_preempt08:40:001
109726499361,3cyclictest131rcu_preempt09:00:011
109726499361,3cyclictest131rcu_preempt09:00:011
109726499361,3cyclictest131rcu_preempt08:15:011
109725999366,22cyclictest1100071-21munin-plugin-st08:05:000
109725999365,3cyclictest131rcu_preempt08:53:420
1097259993621,11cyclictest3467718-21lldpd07:21:410
109727499353,3cyclictest131rcu_preempt09:09:423
109727499352,3cyclictest131rcu_preempt08:48:003
109727499352,3cyclictest131rcu_preempt07:23:423
109727499351,4cyclictest131rcu_preempt07:10:023
109727499351,4cyclictest131rcu_preempt07:10:023
1097269993521,10cyclictest275-21dbus-daemon08:05:012
1097264993522,10cyclictest1-21systemd08:25:481
1097264993522,10cyclictest1-21systemd08:25:481
1097269993416,13cyclictest1-21systemd09:00:012
1097269993416,13cyclictest1-21systemd09:00:012
1097264993426,4cyclictest1098491-21cron07:35:001
1097264993318,11cyclictest1099697-21cat07:55:031
109726499331,3cyclictest131rcu_preempt07:50:011
1097269993222,8cyclictest30-21ksoftirqd/207:45:012
109726499322,3cyclictest131rcu_preempt08:25:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional