You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 06:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Mon May 25, 2026 00:45:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4018130991091,11cyclictest0-21swapper/023:50:020
4018130991091,11cyclictest0-21swapper/023:50:020
401814399861,81cyclictest4041067-21ssh21:45:013
4018130996940,14cyclictest12-21ksoftirqd/021:43:190
4018130996714,47cyclictest12-21ksoftirqd/021:50:030
4018138996644,9cyclictest4023819-21kworker/2:322:26:482
4018130996617,29cyclictest12-21ksoftirqd/000:20:030
4018130996514,4cyclictest12-21ksoftirqd/023:40:010
4018130996413,47cyclictest12-21ksoftirqd/023:31:280
4018130996328,14cyclictest131rcu_preempt22:05:020
4018130996328,14cyclictest131rcu_preempt22:05:010
401813099632,38cyclictest131rcu_preempt22:46:210
4018130996239,15cyclictest12-21ksoftirqd/022:06:240
401813099613,42cyclictest12-21ksoftirqd/022:52:530
4018130996031,14cyclictest4063080-21ssh22:31:470
4018130996031,14cyclictest4063080-21ssh22:31:460
4018130996030,26cyclictest4026601-21ssh21:14:440
4018138995944,7cyclictest4023819-21kworker/2:321:17:142
4018138995940,8cyclictest4067540-21kworker/2:023:12:432
4018130995937,5cyclictest12-21ksoftirqd/021:59:520
401813099575,4cyclictest131rcu_preempt00:10:020
4018130995714,8cyclictest12-21ksoftirqd/021:20:010
4018130995711,4cyclictest12-21ksoftirqd/000:38:120
4018130995711,4cyclictest12-21ksoftirqd/000:38:110
4018138995648,5cyclictest4067540-21kworker/2:022:46:022
4018138995638,9cyclictest4023819-21kworker/2:322:02:192
4018138995638,9cyclictest4023819-21kworker/2:322:02:182
401813099562,39cyclictest4067490-21ssh22:40:550
401813099562,37cyclictest4117560-21ssh00:26:030
401813099562,37cyclictest4117560-21ssh00:26:020
4018130995619,32cyclictest16850irq/134-800000023:02:250
4018138995546,6cyclictest4007725-21kworker/2:220:00:022
4018138995546,5cyclictest4023819-21kworker/2:321:48:292
4018138995545,5cyclictest4023819-21kworker/2:321:54:142
4018130995528,12cyclictest4033574-21ssh21:29:400
4018130995520,6cyclictest16850irq/134-800000023:54:570
4018130995520,31cyclictest16850irq/134-800000021:32:520
4018130995519,6cyclictest16850irq/134-800000023:25:300
4018130995519,6cyclictest16850irq/134-800000023:25:300
4018130995519,31cyclictest16850irq/134-800000000:00:540
4018130995519,31cyclictest16850irq/134-800000000:00:530
401813099551,39cyclictest4121003-21ssh00:33:370
401813099551,39cyclictest4056928-21ssh22:18:550
4018138995436,7cyclictest4055051-21kworker/2:223:35:022
4018133995414,9cyclictest131rcu_preempt21:45:011
401813099543,5cyclictest131rcu_preempt22:40:010
4018130995426,13cyclictest4080762-21ssh23:09:110
4018130995419,31cyclictest16850irq/134-800000022:55:180
4018130995419,31cyclictest12-21ksoftirqd/000:05:450
4018130995419,30cyclictest16850irq/134-800000022:21:540
4018130995419,30cyclictest16850irq/134-800000022:21:530
4018130995418,32cyclictest16850irq/134-800000023:19:580
4018130995418,31cyclictest16850irq/134-800000023:21:050
4018130995417,6cyclictest16850irq/134-800000023:55:300
4018138995345,5cyclictest4055051-21kworker/2:223:28:262
4018138995345,5cyclictest4055051-21kworker/2:223:28:262
4018138995345,5cyclictest4007725-21kworker/2:219:50:002
4018138995344,6cyclictest4067540-21kworker/2:023:45:052
4018138995344,6cyclictest4067540-21kworker/2:023:45:042
4018138995344,6cyclictest4055051-21kworker/2:200:24:042
4018138995344,5cyclictest4055051-21kworker/2:200:11:092
4018138995344,5cyclictest4023819-21kworker/2:322:11:332
4018138995343,6cyclictest4067540-21kworker/2:022:55:012
4018138995329,9cyclictest4067540-21kworker/2:023:38:072
4018133995315,25cyclictest4085777-21modprobe23:19:591
4018130995326,12cyclictest4081481-21ssh23:10:110
4018130995324,13cyclictest405494750ssh22:14:490
4018130995318,31cyclictest16850irq/134-800000021:21:390
4018130995318,31cyclictest16850irq/134-800000000:21:500
4018130995317,32cyclictest16850irq/134-800000023:41:330
4018138995244,5cyclictest4067540-21kworker/2:023:20:022
4018138995244,5cyclictest4023819-21kworker/2:321:21:422
4018138995244,5cyclictest4007725-21kworker/2:220:45:022
4018138995242,6cyclictest4023819-21kworker/2:322:16:122
4018138995242,6cyclictest4023819-21kworker/2:321:55:262
4018138995238,7cyclictest4055051-21kworker/2:200:38:122
4018138995238,7cyclictest4055051-21kworker/2:200:38:122
4018138995229,8cyclictest4007725-21kworker/2:221:10:012
4018138995227,8cyclictest4055051-21kworker/2:200:05:592
4018133995225,20cyclictest275-21dbus-daemon21:55:011
4018130995228,9cyclictest4036535-21ssh21:35:250
4018130995216,32cyclictest16850irq/134-800000022:28:570
4018138995144,5cyclictest4067540-21kworker/2:023:23:072
4018138995143,5cyclictest4055051-21kworker/2:222:31:332
4018138995143,5cyclictest4055051-21kworker/2:222:31:332
4018138995143,5cyclictest4023819-21kworker/2:322:10:012
4018138995143,4cyclictest4023819-21kworker/2:321:37:242
4018138995142,5cyclictest4067540-21kworker/2:023:07:152
4018138995137,6cyclictest4055051-21kworker/2:200:19:332
4018138995129,6cyclictest4007725-21kworker/2:221:12:082
4018130995118,15cyclictest16850irq/134-800000021:10:010
4018130995116,6cyclictest16850irq/134-800000021:52:180
4018138995042,5cyclictest4067540-21kworker/2:023:40:222
4018138995042,5cyclictest4067540-21kworker/2:022:56:332
4018138995042,5cyclictest4055051-21kworker/2:222:40:132
4018138995042,5cyclictest4055051-21kworker/2:222:35:072
4018138995042,5cyclictest4055051-21kworker/2:200:02:382
4018138995042,5cyclictest4055051-21kworker/2:200:02:372
4018133995033,12cyclictest3467602-21systemd-journal00:05:011
4018133995033,12cyclictest3467602-21systemd-journal00:05:011
4018133995011,6cyclictest131rcu_preempt00:35:001
4018138994941,5cyclictest4055051-21kworker/2:223:55:032
4018138994941,5cyclictest4055051-21kworker/2:200:35:002
4018138994941,5cyclictest4055051-21kworker/2:200:27:102
4018138994941,5cyclictest4055051-21kworker/2:200:27:092
4018138994941,5cyclictest4023819-21kworker/2:322:22:072
4018138994941,5cyclictest4023819-21kworker/2:322:22:062
4018138994939,7cyclictest4007725-21kworker/2:219:20:002
4018133994935,11cyclictest1-21systemd23:00:011
401813099494,3cyclictest131rcu_preempt20:40:010
401813099494,3cyclictest131rcu_preempt20:40:000
4018143994835,10cyclictest275-21dbus-daemon00:05:013
4018143994835,10cyclictest275-21dbus-daemon00:05:003
4018138994840,5cyclictest4067540-21kworker/2:023:00:572
4018138994840,5cyclictest4055051-21kworker/2:223:53:432
4018138994840,5cyclictest4023819-21kworker/2:321:44:142
4018138994838,6cyclictest4023819-21kworker/2:321:31:532
401813399487,9cyclictest23-21ksoftirqd/119:14:591
4018133994830,14cyclictest275-21dbus-daemon22:00:001
401813399482,6cyclictest131rcu_preempt00:00:021
4018133994819,25cyclictest23-21ksoftirqd/121:12:191
401813399481,5cyclictest131rcu_preempt21:50:011
4018138994739,5cyclictest4007725-21kworker/2:220:18:422
4018138994738,5cyclictest4007725-21kworker/2:220:50:002
4018138994738,5cyclictest4007725-21kworker/2:220:50:002
4018138994737,5cyclictest4007725-21kworker/2:221:05:012
401813399476,7cyclictest131rcu_preempt22:14:131
4018133994717,8cyclictest23-21ksoftirqd/122:50:001
401813099474,3cyclictest131rcu_preempt20:20:020
4018130994740,4cyclictest16950irq/224-800000021:00:010
4018130994740,4cyclictest16950irq/224-800000021:00:000
4018130994723,7cyclictest4020422-21kworker/0:020:10:010
401814399464,3cyclictest131rcu_preempt22:23:403
401814399464,3cyclictest131rcu_preempt22:23:403
4018143994622,20cyclictest37-21ksoftirqd/320:10:013
401814399461,3cyclictest131rcu_preempt00:22:453
401813899467,8cyclictest30-21ksoftirqd/220:35:072
401813899467,8cyclictest30-21ksoftirqd/220:35:072
4018138994638,5cyclictest4007725-21kworker/2:221:00:012
4018138994638,5cyclictest4007725-21kworker/2:221:00:012
4018138994638,5cyclictest4007725-21kworker/2:220:25:012
4018138994636,5cyclictest4007725-21kworker/2:220:30:012
4018138994636,5cyclictest4007725-21kworker/2:220:30:002
4018138994634,9cyclictest1-21systemd20:10:012
4018133994632,11cyclictest23-21ksoftirqd/122:54:071
4018133994627,13cyclictest309-21in:imuxsock23:25:001
4018133994625,13cyclictest4092853-21sh23:35:001
4018130994629,12cyclictest275-21dbus-daemon19:50:010
4018143994538,4cyclictest4105299-21kworker/u8:500:05:033
4018143994532,10cyclictest4104740-40install23:59:593
4018143994529,12cyclictest1-21systemd21:30:013
4018143994527,13cyclictest275-21dbus-daemon19:25:023
4018143994520,16cyclictest0-21swapper/323:15:083
4018138994538,5cyclictest4007725-21kworker/2:220:05:012
4018138994536,6cyclictest4007725-21kworker/2:219:45:012
4018138994525,7cyclictest4023819-21kworker/2:321:28:502
401813899451,5cyclictest131rcu_preempt19:35:022
4018133994533,8cyclictest309-21in:imuxsock22:21:071
4018133994533,8cyclictest309-21in:imuxsock22:21:061
4018133994529,12cyclictest275-21dbus-daemon19:20:011
4018133994529,11cyclictest3467602-21systemd-journal21:10:011
4018133994527,12cyclictest309-21in:imuxsock20:35:021
4018133994527,12cyclictest309-21in:imuxsock20:35:011
4018133994522,17cyclictest4069325-21systemd-run22:45:021
4018133994521,5cyclictest23-21ksoftirqd/100:15:411
4018133994517,23cyclictest23-21ksoftirqd/120:30:021
4018133994517,23cyclictest23-21ksoftirqd/120:30:011
4018130994534,5cyclictest12-21ksoftirqd/019:15:020
401814399443,3cyclictest131rcu_preempt22:36:493
401814399443,3cyclictest131rcu_preempt21:15:253
4018143994430,10cyclictest4024556-21cut21:10:033
4018143994429,11cyclictest4038510-21cron21:40:013
4018138994436,5cyclictest4007725-21kworker/2:220:35:022
4018138994436,5cyclictest4007725-21kworker/2:219:15:012
4018138994436,4cyclictest4007725-21kworker/2:220:10:162
4018133994433,9cyclictest23-21ksoftirqd/121:27:281
4018133994431,9cyclictest4029093-21systemctl21:20:001
4018133994431,10cyclictest23-21ksoftirqd/123:05:021
401813399442,6cyclictest131rcu_preempt00:09:521
4018133994426,13cyclictest23-21ksoftirqd/119:25:011
4018133994412,8cyclictest131rcu_preempt23:50:081
4018130994425,12cyclictest12-21ksoftirqd/020:00:000
4018143994334,5cyclictest37-21ksoftirqd/323:25:013
4018143994328,11cyclictest275-21dbus-daemon21:25:013
401814399431,5cyclictest309-21in:imuxsock20:50:013
401814399431,5cyclictest309-21in:imuxsock20:50:013
401814399431,3cyclictest131rcu_preempt00:12:503
4018133994335,6cyclictest23-21ksoftirqd/120:45:081
4018133994335,6cyclictest23-21ksoftirqd/120:45:071
4018133994329,10cyclictest275-21dbus-daemon20:25:011
4018133994325,4cyclictest23-21ksoftirqd/123:27:391
4018133994325,4cyclictest23-21ksoftirqd/123:27:391
4018133994324,9cyclictest309-21in:imuxsock20:10:021
401813099439,5cyclictest12-21ksoftirqd/020:15:010
401813099437,8cyclictest12-21ksoftirqd/020:30:020
401813099437,8cyclictest12-21ksoftirqd/020:30:020
4018143994228,10cyclictest37-21ksoftirqd/323:05:003
401814399422,6cyclictest37-21ksoftirqd/323:06:123
401814399422,3cyclictest131rcu_preempt22:46:043
401814399421,3cyclictest131rcu_preempt21:11:213
4018138994235,4cyclictest4007725-21kworker/2:220:54:062
4018133994232,7cyclictest4124287-21cat00:40:031
4018133994232,7cyclictest4124287-21cat00:40:021
4018133994227,10cyclictest275-21dbus-daemon00:30:011
4018133994227,10cyclictest275-21dbus-daemon00:30:001
4018133994226,4cyclictest23-21ksoftirqd/100:21:331
4018133994226,12cyclictest275-21dbus-daemon23:10:001
4018133994224,11cyclictest4038518-21modprobe21:40:011
4018133994223,4cyclictest23-21ksoftirqd/122:28:271
4018133994216,18cyclictest23-21ksoftirqd/120:00:011
4018133994213,19cyclictest4057452-21modprobe22:20:001
4018130994224,13cyclictest4018625-21modprobe19:20:010
4018130994223,15cyclictest3467602-21systemd-journal20:50:010
4018130994223,15cyclictest3467602-21systemd-journal20:50:000
4018130994216,4cyclictest12-21ksoftirqd/019:35:020
401813099421,3cyclictest131rcu_preempt19:55:010
401814399419,5cyclictest131rcu_preempt00:16:203
401814399415,6cyclictest131rcu_preempt00:40:023
401814399415,6cyclictest131rcu_preempt00:40:023
401814399414,4cyclictest131rcu_preempt23:51:283
4018143994129,8cyclictest4022742-21cron20:40:013
4018143994129,8cyclictest4022742-21cron20:40:013
4018143994128,11cyclictest37-21ksoftirqd/321:35:023
401813899412,3cyclictest131rcu_preempt19:26:572
401813399416,25cyclictest4020405-21modprobe19:55:011
4018133994131,7cyclictest23-21ksoftirqd/121:01:401
4018133994127,11cyclictest3467602-21systemd-journal19:40:011
401813399412,3cyclictest131rcu_preempt23:48:481
401813399412,3cyclictest131rcu_preempt23:48:471
4018133994121,15cyclictest275-21dbus-daemon21:00:021
4018133994121,15cyclictest275-21dbus-daemon21:00:011
4018133994116,4cyclictest23-21ksoftirqd/123:43:481
4018133994115,18cyclictest221rcuc/123:38:561
401813399411,3cyclictest131rcu_preempt23:11:451
4018133994112,6cyclictest131rcu_preempt22:38:401
4018133994110,9cyclictest131rcu_preempt22:05:071
401813099414,7cyclictest12-21ksoftirqd/019:26:080
401813099414,3cyclictest131rcu_preempt20:45:010
4018130994132,6cyclictest19350irq/34-mmc120:02:470
401814399403,3cyclictest131rcu_preempt23:34:333
4018143994029,7cyclictest4019111-21cron19:29:593
401814399402,3cyclictest131rcu_preempt21:56:443
401814399401,4cyclictest131rcu_preempt22:51:553
4018143994012,6cyclictest131rcu_preempt20:01:083
401814399401,21cyclictest0-21swapper/323:35:083
4018138994032,5cyclictest4007725-21kworker/2:219:40:012
4018138994032,4cyclictest4007725-21kworker/2:219:23:162
401813399409,6cyclictest131rcu_preempt19:30:011
401813399405,28cyclictest4020163-21munin-plugin-st19:50:011
4018133994026,10cyclictest1-21systemd20:45:001
4018133994025,11cyclictest23-21ksoftirqd/121:34:061
4018133994015,4cyclictest23-21ksoftirqd/122:31:121
4018133994015,4cyclictest23-21ksoftirqd/122:31:121
401813099404,7cyclictest4019659-21modprobe19:40:000
401813099404,6cyclictest131rcu_preempt19:25:000
4018130994018,4cyclictest12-21ksoftirqd/020:55:010
401814399399,6cyclictest131rcu_preempt21:45:093
4018143993929,7cyclictest37-21ksoftirqd/320:30:033
4018143993929,7cyclictest37-21ksoftirqd/320:30:033
4018143993927,10cyclictest37-21ksoftirqd/320:20:023
4018143993926,10cyclictest4097788-21cut23:45:013
4018143993925,11cyclictest275-21dbus-daemon19:50:013
401814399392,3cyclictest131rcu_preempt22:12:253
4018143993923,13cyclictest37-21ksoftirqd/322:17:563
401814399391,3cyclictest131rcu_preempt23:45:543
401814399391,3cyclictest131rcu_preempt23:45:533
401814399391,3cyclictest131rcu_preempt22:27:463
401814399391,34cyclictest1-21systemd22:45:023
4018138993931,5cyclictest4007725-21kworker/2:219:55:002
401813399394,26cyclictest3467602-21systemd-journal20:05:001
4018133993921,14cyclictest275-21dbus-daemon00:15:011
4018133993913,4cyclictest23-21ksoftirqd/122:03:081
4018133993913,4cyclictest23-21ksoftirqd/122:03:081
4018130993914,6cyclictest12-21ksoftirqd/020:24:590
401814399387,5cyclictest131rcu_preempt22:31:123
401814399387,5cyclictest131rcu_preempt22:31:123
401814399384,3cyclictest131rcu_preempt19:20:013
4018143993827,8cyclictest1-21systemd19:35:013
4018143993825,10cyclictest37-21ksoftirqd/300:35:023
401814399382,3cyclictest131rcu_preempt22:05:493
401814399381,3cyclictest131rcu_preempt19:45:013
401813399385,24cyclictest1-21systemd20:55:001
4018133993826,10cyclictest23-21ksoftirqd/120:35:011
4018133993823,11cyclictest275-21dbus-daemon20:20:021
4018130993817,5cyclictest12-21ksoftirqd/019:45:010
401813099381,3cyclictest131rcu_preempt20:35:000
401814399373,3cyclictest131rcu_preempt22:00:583
401814399373,3cyclictest131rcu_preempt22:00:573
4018143993728,7cyclictest37-21ksoftirqd/323:29:303
4018143993728,7cyclictest37-21ksoftirqd/323:29:303
4018143993728,5cyclictest4081867-21sh23:11:253
4018143993725,9cyclictest4023141-21sed20:45:013
4018143993725,10cyclictest37-21ksoftirqd/322:56:283
401813899372,3cyclictest131rcu_preempt19:10:022
401813399371,5cyclictest131rcu_preempt21:25:021
4018130993728,6cyclictest19350irq/34-mmc121:04:250
4018143993623,9cyclictest37-21ksoftirqd/300:28:113
4018143993623,9cyclictest37-21ksoftirqd/300:28:103
4018143993622,11cyclictest37-21ksoftirqd/321:53:063
4018143993616,6cyclictest131rcu_preempt20:15:003
401813399365,25cyclictest4021448-21munin-plugin-st20:14:591
4018130993627,6cyclictest16950irq/224-800000019:10:020
401814399358,20cyclictest0-21swapper/321:05:003
4018143993524,9cyclictest37-21ksoftirqd/320:35:003
4018143993518,14cyclictest275-21dbus-daemon19:40:013
401814399351,7cyclictest4020700-21true20:00:003
401814399351,13cyclictest4018332-21munin-plugin-st19:14:593
4018143993421,9cyclictest4023764-21cron21:00:013
4018143993421,9cyclictest4023764-21cron21:00:003
401813399345,23cyclictest4019898-21modprobe19:44:591
401813399333,24cyclictest4019515-21wc19:35:011
4018143993223,5cyclictest309-21in:imuxsock20:55:003
4018143993219,10cyclictest4020510-21wc19:55:023
401813399321,8cyclictest131rcu_preempt19:10:021
401814399301,5cyclictest4022000-21/usr/sbin/munin20:25:003
401814399291,6cyclictest4018231-21date19:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional