You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 22:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Sep 23, 2026 12:45:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1915741997215,27cyclictest12-21ksoftirqd/011:15:330
1915741996741,5cyclictest12-21ksoftirqd/010:55:010
1915741996525,3cyclictest131rcu_preempt11:58:570
191574199647,7cyclictest131rcu_preempt07:30:000
1915741996418,5cyclictest12-21ksoftirqd/012:12:450
1915750996338,7cyclictest1989505-21kworker/2:112:05:022
1915750996338,7cyclictest1989505-21kworker/2:112:05:022
1915750996238,8cyclictest1676221-21kworker/2:307:36:422
1915741996220,27cyclictest12-21ksoftirqd/009:49:210
1915741996219,27cyclictest12-21ksoftirqd/011:35:590
1915741996219,27cyclictest12-21ksoftirqd/011:35:590
1915750996125,17cyclictest131rcu_preempt08:20:012
1915741996116,27cyclictest12-21ksoftirqd/009:52:250
1915750996040,7cyclictest1989505-21kworker/2:112:23:002
191574199591,44cyclictest20187141ssh12:37:310
1915741995914,41cyclictest12-21ksoftirqd/011:26:310
1915741995913,36cyclictest16850irq/134-800000011:00:020
1915750995838,14cyclictest275-21dbus-daemon10:25:012
1915750995835,8cyclictest1989505-21kworker/2:111:53:002
191574199586,30cyclictest131rcu_preempt11:33:440
191574199584,30cyclictest131rcu_preempt10:23:540
1915741995824,5cyclictest131rcu_preempt12:19:180
1915741995819,19cyclictest16850irq/134-800000009:15:220
1915741995815,34cyclictest16850irq/134-800000009:24:150
1915741995814,7cyclictest16850irq/134-800000012:30:000
1915741995814,7cyclictest16850irq/134-800000012:30:000
191575099579,6cyclictest131rcu_preempt12:20:012
1915750995712,7cyclictest131rcu_preempt10:50:012
1915741995728,14cyclictest1951816-21ssh10:14:070
191574199571,7cyclictest121ksoftirqd/009:26:500
1915741995710,41cyclictest131rcu_preempt10:26:300
1915750995647,6cyclictest1989505-21kworker/2:112:38:012
1915741995626,15cyclictest1974715-21ssh11:02:580
1915741995616,35cyclictest16850irq/134-800000009:10:110
1915741995615,21cyclictest16850irq/134-800000010:02:250
1915750995546,6cyclictest1676221-21kworker/2:310:19:262
1915750995537,6cyclictest1676221-21kworker/2:309:18:392
1915750995511,6cyclictest131rcu_preempt09:50:002
1915741995529,23cyclictest16850irq/134-800000010:10:020
1915741995528,12cyclictest1997802-21ssh11:52:400
1915741995522,13cyclictest12-21ksoftirqd/009:43:260
1915741995522,13cyclictest12-21ksoftirqd/009:43:260
1915741995519,32cyclictest16850irq/134-800000010:46:350
1915741995516,34cyclictest16850irq/134-800000009:57:190
1915741995512,7cyclictest12-21ksoftirqd/009:34:520
1915750995446,5cyclictest1931330-21kworker/2:011:18:072
1915750995444,6cyclictest1931330-21kworker/2:011:25:022
1915750995431,8cyclictest1676221-21kworker/2:309:31:522
1915750995429,15cyclictest1931330-21kworker/2:010:43:422
1915741995434,4cyclictest12-21ksoftirqd/011:43:400
191574199542,48cyclictest1934472-21sshd09:36:410
1915741995415,35cyclictest16850irq/134-800000012:32:530
1915741995415,34cyclictest16850irq/134-800000012:06:420
1915741995415,34cyclictest16850irq/134-800000012:06:420
1915741995415,19cyclictest16850irq/134-800000010:17:070
1915741995414,14cyclictest131rcu_preempt11:07:130
1915750995345,5cyclictest1968635-21kworker/2:211:03:372
1915750995345,5cyclictest1931330-21kworker/2:011:27:592
1915750995345,5cyclictest1676221-21kworker/2:310:07:012
1915750995343,6cyclictest1676221-21kworker/2:309:27:292
1915750995313,6cyclictest131rcu_preempt10:55:012
191574199535,10cyclictest12-21ksoftirqd/012:05:010
1915741995316,33cyclictest16850irq/134-800000010:43:340
1915741995315,34cyclictest16850irq/134-800000010:39:230
1915741995315,19cyclictest16850irq/134-800000011:20:470
1915750995244,5cyclictest1989505-21kworker/2:111:59:472
1915750995244,5cyclictest1977915-21kworker/2:312:30:232
1915750995244,5cyclictest1977915-21kworker/2:311:35:232
1915750995244,5cyclictest1977915-21kworker/2:311:35:232
1915750995244,5cyclictest1931330-21kworker/2:011:30:362
1915750995244,5cyclictest1676221-21kworker/2:310:27:482
1915750995244,5cyclictest1676221-21kworker/2:309:12:012
1915750995243,6cyclictest1931330-21kworker/2:010:35:472
1915750995243,5cyclictest1920903-21kworker/2:109:20:092
1915750995233,7cyclictest1989505-21kworker/2:112:12:502
1915746995233,15cyclictest23-21ksoftirqd/108:55:021
1915746995233,15cyclictest23-21ksoftirqd/108:55:021
191574199524,37cyclictest1915744-21latency_hist07:10:010
1915741995236,11cyclictest3467602-21systemd-journal08:40:000
1915741995236,11cyclictest3467602-21systemd-journal08:40:000
1915741995234,13cyclictest3467602-21systemd-journal09:10:000
1915741995224,17cyclictest12-21ksoftirqd/011:47:100
191574199522,16cyclictest1916207-21grep07:20:000
1915741995216,32cyclictest16850irq/134-800000012:23:160
1915741995215,33cyclictest16850irq/134-800000010:30:230
191575099517,6cyclictest131rcu_preempt09:10:002
1915750995143,5cyclictest1989505-21kworker/2:111:44:372
1915750995143,5cyclictest1676221-21kworker/2:310:01:342
1915750995143,5cyclictest1676221-21kworker/2:309:38:412
1915750995110,6cyclictest131rcu_preempt09:05:022
1915746995130,16cyclictest275-21dbus-daemon09:40:011
191574199516,8cyclictest131rcu_preempt07:55:000
1915741995135,13cyclictest12-21ksoftirqd/007:45:010
191574199512,15cyclictest1980243-21unin-run11:15:010
1915741995111,6cyclictest131rcu_preempt08:15:020
1915741995111,6cyclictest131rcu_preempt08:15:020
191575599501,31cyclictest1918761-21kworker/u8:208:20:023
1915750995042,5cyclictest1989505-21kworker/2:112:28:192
1915750995042,5cyclictest1989505-21kworker/2:112:28:192
1915750995042,5cyclictest1989505-21kworker/2:111:48:202
1915750995042,5cyclictest1931330-21kworker/2:011:07:032
1915750995042,5cyclictest1676221-21kworker/2:310:30:202
1915750995042,5cyclictest1676221-21kworker/2:309:55:452
1915750995042,5cyclictest1676221-21kworker/2:309:54:262
1915750995031,15cyclictest1-21systemd09:00:002
1915750995029,7cyclictest1989505-21kworker/2:112:03:502
1915750995012,10cyclictest131rcu_preempt09:45:032
191574199503,40cyclictest1917222-21modprobe07:40:010
191575599491,5cyclictest131rcu_preempt11:24:223
1915750994941,5cyclictest1977915-21kworker/2:311:10:172
1915750994941,5cyclictest1968635-21kworker/2:210:55:092
1915750994941,5cyclictest1676221-21kworker/2:309:42:102
1915746994933,11cyclictest275-21dbus-daemon10:55:021
191575099489,7cyclictest131rcu_preempt08:55:002
191575099489,7cyclictest131rcu_preempt08:55:002
1915750994840,5cyclictest1918810-21kworker/2:208:33:112
1915750994840,5cyclictest1676221-21kworker/2:310:10:212
1915750994840,5cyclictest1676221-21kworker/2:307:58:022
1915746994829,13cyclictest1-21systemd08:45:001
1915741994834,10cyclictest275-21dbus-daemon08:25:010
191575599473,4cyclictest131rcu_preempt12:04:293
1915755994734,9cyclictest1-21systemd12:40:013
191575099477,5cyclictest131rcu_preempt08:50:012
1915750994739,5cyclictest1918810-21kworker/2:208:40:572
1915750994739,5cyclictest1918810-21kworker/2:208:20:032
1915750994739,5cyclictest1676221-21kworker/2:307:43:522
1915750994739,5cyclictest1676221-21kworker/2:307:34:102
1915746994732,11cyclictest275-21dbus-daemon09:50:011
191574699472,3cyclictest131rcu_preempt07:15:011
191574699472,3cyclictest131rcu_preempt07:15:011
1915741994732,10cyclictest1919320-21munin-plugin-st08:20:000
1915755994633,10cyclictest1-21systemd12:35:003
1915755994632,10cyclictest1-21systemd12:20:013
1915750994639,4cyclictest1918810-21kworker/2:208:28:112
1915750994639,4cyclictest1918810-21kworker/2:208:28:112
1915750994639,4cyclictest1676221-21kworker/2:307:54:252
1915750994638,5cyclictest1918810-21kworker/2:208:36:522
1915750994638,5cyclictest1918810-21kworker/2:208:36:512
1915750994638,5cyclictest1918810-21kworker/2:208:14:082
1915750994638,5cyclictest1918810-21kworker/2:208:14:082
1915750994638,5cyclictest1676221-21kworker/2:307:47:082
1915750994638,5cyclictest1676221-21kworker/2:307:47:082
1915750994637,6cyclictest1676221-21kworker/2:307:27:132
1915750994630,12cyclictest1-21systemd07:15:012
1915750994630,12cyclictest1-21systemd07:15:002
191575599453,9cyclictest0-21swapper/312:30:023
191575599453,9cyclictest0-21swapper/312:30:023
1915755994533,8cyclictest1-21systemd10:15:013
1915750994537,5cyclictest1676221-21kworker/2:308:08:172
1915750994537,5cyclictest1676221-21kworker/2:307:17:142
1915750994511,5cyclictest131rcu_preempt08:05:022
191574699453,5cyclictest131rcu_preempt10:30:001
191574699453,3cyclictest131rcu_preempt11:09:201
191574699453,3cyclictest131rcu_preempt09:19:511
1915746994527,13cyclictest3467602-21systemd-journal11:30:001
191574199453,3cyclictest131rcu_preempt08:35:010
1915741994530,10cyclictest3467602-21systemd-journal08:59:590
191574199452,6cyclictest131rcu_preempt08:50:000
1915741994519,22cyclictest12-21ksoftirqd/008:05:010
1915755994435,7cyclictest37-21ksoftirqd/310:05:033
191575599443,3cyclictest131rcu_preempt10:07:323
1915755994431,11cyclictest37-21ksoftirqd/311:15:023
1915755994429,11cyclictest1921370-21cron09:00:003
1915755994428,12cyclictest1938191-21cron09:45:003
1915755994428,12cyclictest1938191-21cron09:45:003
1915755994414,13cyclictest37-21ksoftirqd/311:10:013
191574699443,3cyclictest131rcu_preempt10:33:541
1915746994432,9cyclictest23-21ksoftirqd/108:40:021
1915746994432,9cyclictest23-21ksoftirqd/108:40:021
1915746994428,12cyclictest275-21dbus-daemon11:45:001
1915746994428,11cyclictest23-21ksoftirqd/110:10:011
1915746994427,12cyclictest275-21dbus-daemon07:30:011
191574699442,3cyclictest131rcu_preempt09:40:251
191574699442,3cyclictest131rcu_preempt09:40:251
1915741994437,4cyclictest16950irq/224-800000008:45:000
1915755994334,6cyclictest37-21ksoftirqd/311:32:373
191575599433,3cyclictest131rcu_preempt10:18:163
191575599431,3cyclictest131rcu_preempt12:22:233
191575599431,3cyclictest131rcu_preempt10:26:553
1915750994335,5cyclictest1676221-21kworker/2:307:21:072
191574699434,3cyclictest131rcu_preempt09:30:251
191574699433,3cyclictest131rcu_preempt10:05:011
1915746994331,7cyclictest274-21cron08:00:001
191574199432,7cyclictest0-21swapper/007:35:010
1915741994326,13cyclictest1918234-21sshd07:56:410
1915755994230,10cyclictest1943372-21sshd09:55:423
1915755994227,11cyclictest275-21dbus-daemon11:30:013
191575599421,4cyclictest131rcu_preempt11:50:513
191575599421,3cyclictest131rcu_preempt11:15:083
191575599421,3cyclictest131rcu_preempt10:49:413
1915755994211,21cyclictest120-21kswapd011:00:003
191574699427,8cyclictest23-21ksoftirqd/110:47:321
1915746994234,5cyclictest1984888-21sh11:25:001
1915746994226,13cyclictest3467602-21systemd-journal08:04:591
191574699422,3cyclictest131rcu_preempt12:35:361
191574699422,3cyclictest131rcu_preempt09:29:351
191574699421,3cyclictest131rcu_preempt12:19:021
1915746994211,14cyclictest23-21ksoftirqd/111:17:461
1915755994128,9cyclictest1966422-21wc10:45:033
191574699418,18cyclictest23-21ksoftirqd/109:20:281
191574699414,3cyclictest131rcu_preempt12:05:491
191574699414,3cyclictest131rcu_preempt12:05:491
1915746994129,9cyclictest1989260-21rm11:34:261
1915746994124,4cyclictest23-21ksoftirqd/112:31:281
1915746994121,10cyclictest23-21ksoftirqd/110:57:201
1915746994112,14cyclictest23-21ksoftirqd/112:23:091
1915741994124,12cyclictest1-21systemd07:50:010
1915741994124,12cyclictest1-21systemd07:50:010
1915741994124,11cyclictest275-21dbus-daemon07:25:000
1915755994030,8cyclictest37-21ksoftirqd/312:10:003
1915755994030,8cyclictest37-21ksoftirqd/312:10:003
1915755994028,9cyclictest275-21dbus-daemon09:20:013
1915755994027,11cyclictest37-21ksoftirqd/311:02:273
1915755994027,11cyclictest37-21ksoftirqd/309:11:513
1915755994026,9cyclictest309-21in:imuxsock08:40:013
1915755994026,9cyclictest309-21in:imuxsock08:40:013
191575599402,4cyclictest131rcu_preempt09:21:323
191575599402,3cyclictest131rcu_preempt09:27:423
1915755994022,9cyclictest0-21swapper/311:46:433
191574699404,3cyclictest131rcu_preempt09:58:171
1915746994029,7cyclictest23-21ksoftirqd/108:25:021
1915746994020,16cyclictest0-21swapper/109:00:001
191574699401,3cyclictest131rcu_preempt11:37:041
191574699401,3cyclictest131rcu_preempt11:37:041
1915746994010,14cyclictest23-21ksoftirqd/111:13:391
191574199406,6cyclictest12-21ksoftirqd/008:55:020
191574199406,6cyclictest12-21ksoftirqd/008:55:020
191574199404,3cyclictest131rcu_preempt09:05:000
191575599394,3cyclictest131rcu_preempt09:50:483
1915755993929,8cyclictest37-21ksoftirqd/309:48:043
1915755993925,11cyclictest37-21ksoftirqd/310:50:523
1915755993924,11cyclictest3467602-21systemd-journal10:25:023
191574699396,7cyclictest23-21ksoftirqd/108:15:021
191574699396,7cyclictest23-21ksoftirqd/108:15:011
191574699394,9cyclictest23-21ksoftirqd/109:10:251
191574699393,3cyclictest131rcu_preempt11:50:501
191574699393,3cyclictest131rcu_preempt11:48:151
191574699393,24cyclictest1994741-21kworker/u8:312:10:031
1915746993930,5cyclictest309-21in:imuxsock07:55:001
191574699392,7cyclictest0-21swapper/110:35:581
1915746993927,9cyclictest2002959-21sshd12:03:511
1915746993924,11cyclictest23-21ksoftirqd/111:00:571
1915746993920,8cyclictest23-21ksoftirqd/109:03:441
191574699391,35cyclictest1957139-21sed10:25:021
1915741993935,2cyclictest0-21swapper/008:30:020
1915741993935,2cyclictest0-21swapper/008:30:020
1915755993827,9cyclictest37-21ksoftirqd/311:37:193
1915755993827,9cyclictest37-21ksoftirqd/311:37:193
1915755993824,10cyclictest275-21dbus-daemon08:45:003
191575599381,3cyclictest131rcu_preempt11:56:433
191575599381,3cyclictest131rcu_preempt10:38:583
191575099381,7cyclictest0-21swapper/207:10:022
1915746993828,8cyclictest23-21ksoftirqd/110:44:171
1915746993825,10cyclictest23-21ksoftirqd/110:10:411
1915746993823,11cyclictest3467602-21systemd-journal11:55:421
191574699381,4cyclictest131rcu_preempt09:52:211
191574699381,3cyclictest131rcu_preempt10:19:131
1915741993829,6cyclictest1918812-21unin-run08:10:010
1915755993727,8cyclictest37-21ksoftirqd/310:30:043
1915755993727,8cyclictest37-21ksoftirqd/307:30:023
191575599372,5cyclictest1932312-21ssh09:31:533
1915755993721,13cyclictest275-21dbus-daemon07:55:003
1915755993721,13cyclictest1917848-21sed07:50:023
1915755993721,13cyclictest1917848-21sed07:50:013
1915755993718,8cyclictest37-21ksoftirqd/308:30:033
1915755993712,15cyclictest37-21ksoftirqd/311:40:553
1915755993711,6cyclictest131rcu_preempt07:40:013
191574699371,7cyclictest0-21swapper/107:48:421
191574699371,7cyclictest0-21swapper/107:48:421
191574699371,3cyclictest131rcu_preempt08:30:031
191574199371,5cyclictest131rcu_preempt07:15:020
191574199371,5cyclictest131rcu_preempt07:15:010
1915755993625,9cyclictest37-21ksoftirqd/309:39:313
1915755993624,8cyclictest2008103-21systemd-run12:15:013
1915755993623,10cyclictest37-21ksoftirqd/307:15:033
1915755993623,10cyclictest37-21ksoftirqd/307:15:023
191575599362,15cyclictest1916489-21unin-run07:25:013
1915755993621,11cyclictest1-21systemd08:20:003
1915755993615,16cyclictest275-21dbus-daemon09:10:013
1915746993623,9cyclictest1918764-21sshd08:05:421
1915746993619,12cyclictest275-21dbus-daemon07:40:011
1915746993618,14cyclictest275-21dbus-daemon09:10:011
1915755993522,8cyclictest37-21ksoftirqd/308:05:003
191575599351,26cyclictest0-21swapper/308:00:013
191574699355,19cyclictest23-21ksoftirqd/112:28:231
191574699355,19cyclictest23-21ksoftirqd/112:28:231
1915746993525,7cyclictest1919282-21sshd08:17:421
1915746993524,8cyclictest3467602-21systemd-journal07:15:031
1915746993521,11cyclictest1917049-21sed07:35:021
191575599344,4cyclictest131rcu_preempt08:35:023
1915755993425,6cyclictest1921650-21cron09:05:003
1915755993423,9cyclictest37-21ksoftirqd/308:55:023
1915755993423,9cyclictest37-21ksoftirqd/308:55:013
1915755993419,12cyclictest37-21ksoftirqd/307:16:423
1915746993427,4cyclictest1921046-21latency_hist08:50:021
1915746993426,5cyclictest1920093-21latency_hist08:35:001
191574699342,4cyclictest131rcu_preempt07:25:011
1915755993326,4cyclictest1918796-21latency_hist08:10:003
191575599332,5cyclictest1919918-21latency_hist08:30:023
1915755993321,8cyclictest3467602-21systemd-journal07:45:003
1915746993319,10cyclictest3467602-21systemd-journal08:30:011
1915746993317,13cyclictest275-21dbus-daemon07:45:011
191575599325,3cyclictest131rcu_preempt07:10:023
1915755993223,7cyclictest37-21ksoftirqd/308:15:023
1915755993223,7cyclictest37-21ksoftirqd/308:15:023
1915755993223,7cyclictest37-21ksoftirqd/307:26:393
1915746992917,9cyclictest1915808-21sed07:10:021
1915755992815,10cyclictest37-21ksoftirqd/308:46:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional