You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 14:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Fri Aug 28, 2026 00:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2305277998118,40cyclictest12-21ksoftirqd/000:12:090
2305277998118,40cyclictest12-21ksoftirqd/000:12:080
2305277997614,40cyclictest12-21ksoftirqd/023:59:590
2305286997522,12cyclictest2400114-21sshd00:18:102
2305277997571,2cyclictest0-21swapper/019:15:010
230527799728,42cyclictest12-21ksoftirqd/022:38:340
2305286996928,10cyclictest30-21ksoftirqd/221:23:222
2305286996928,10cyclictest30-21ksoftirqd/221:23:222
230527799694,23cyclictest2306497-21modprobe19:35:010
2305277996910,38cyclictest12-21ksoftirqd/021:23:570
2305277996910,38cyclictest12-21ksoftirqd/021:23:560
2305277996718,39cyclictest12-21ksoftirqd/022:32:210
2305277996617,40cyclictest12-21ksoftirqd/022:48:190
2305286996531,14cyclictest2405817-21latency_hist00:30:022
2305277996518,43cyclictest12-21ksoftirqd/000:37:290
2305286996441,14cyclictest0-21swapper/200:15:022
2305286996441,14cyclictest0-21swapper/200:15:012
230527799647,53cyclictest2374995-21sh23:25:010
2305286996327,9cyclictest30-21ksoftirqd/222:55:032
2305277996227,3cyclictest23192741ssh21:26:170
2305286996125,12cyclictest30-21ksoftirqd/222:42:412
2305286996114,5cyclictest131rcu_preempt22:47:362
230527799619,40cyclictest12-21ksoftirqd/023:34:330
2305277996115,37cyclictest12-21ksoftirqd/022:15:300
230527799595,12cyclictest2325568-21unin-run21:40:010
230527799595,12cyclictest2325568-21unin-run21:40:000
2305277995939,15cyclictest2306754-21modprobe19:39:590
230527799583,19cyclictest2334966-21cron22:00:010
230527799583,19cyclictest2334966-21cron22:00:000
2305277995826,19cyclictest131rcu_preempt00:04:160
2305277995810,38cyclictest12-21ksoftirqd/023:10:440
2305277995728,13cyclictest120-21kswapd022:55:010
230527799572,19cyclictest1-21systemd23:00:000
2305277995721,32cyclictest12-21ksoftirqd/023:27:060
2305277995620,32cyclictest12-21ksoftirqd/022:42:490
2305286995525,10cyclictest275-21dbus-daemon19:35:022
2305277995534,14cyclictest2310914-21systemctl20:59:590
2305277995530,6cyclictest131rcu_preempt00:29:250
2305282995416,27cyclictest2363468-21ssh23:00:021
2305277995424,15cyclictest2335919-21ssh22:01:360
2305277995419,30cyclictest12-21ksoftirqd/022:28:210
230527799541,48cyclictest2370609-21ssh23:15:120
2305277995414,6cyclictest131rcu_preempt21:54:590
2305277995329,9cyclictest2366736-21ssh23:07:140
2305277995326,12cyclictest2383076-21ssh23:41:500
2305277995326,11cyclictest2386583-21ssh23:49:310
2305277995323,4cyclictest131rcu_preempt00:24:110
2305277995322,26cyclictest275-21dbus-daemon20:25:010
2305286995244,5cyclictest2310162-21kworker/2:223:40:222
2305286995243,5cyclictest2335554-21kworker/2:100:25:022
2305286995243,5cyclictest2309381-21kworker/2:321:11:542
2305286995241,6cyclictest2335554-21kworker/2:100:08:332
230527799523,14cyclictest2311601-21cat21:10:030
2305277995225,12cyclictest2363770-21ssh23:00:380
2305277995224,13cyclictest2394593-21ssh00:06:150
2305277995220,28cyclictest12-21ksoftirqd/021:10:480
230527799512,44cyclictest2327947-21date21:45:020
2305286995042,5cyclictest2310162-21kworker/2:223:15:012
2305286995042,5cyclictest2310162-21kworker/2:200:35:152
2305286995041,5cyclictest2310162-21kworker/2:221:59:172
2305286995041,5cyclictest2310162-21kworker/2:221:59:162
2305286995041,5cyclictest2219619-21kworker/2:120:10:022
2305286995040,6cyclictest2310162-21kworker/2:222:18:532
2305286995032,13cyclictest275-21dbus-daemon21:35:012
230528299504,3cyclictest131rcu_preempt23:52:311
230528299503,5cyclictest131rcu_preempt23:50:001
230528299502,13cyclictest2305706-21cron19:20:001
230527799504,38cyclictest0-21swapper/022:15:030
2305277995043,4cyclictest16850irq/134-800000023:55:010
230528899496,8cyclictest37-21ksoftirqd/322:25:023
230528899494,4cyclictest131rcu_preempt21:15:013
2305288994935,10cyclictest275-21dbus-daemon23:55:013
2305286994941,5cyclictest2310162-21kworker/2:223:55:282
2305286994941,5cyclictest2310162-21kworker/2:223:38:462
2305286994941,5cyclictest2310162-21kworker/2:223:34:582
2305286994941,5cyclictest2310162-21kworker/2:223:16:082
2305286994941,5cyclictest2310162-21kworker/2:221:29:292
2305286994940,6cyclictest2335554-21kworker/2:123:07:022
230528699491,10cyclictest2306265-21munin-run19:30:012
230528299495,3cyclictest131rcu_preempt21:50:381
230528299494,4cyclictest131rcu_preempt00:40:001
230528299492,8cyclictest0-21swapper/121:30:021
2305277994940,6cyclictest19350irq/34-mmc119:10:020
2305277994940,6cyclictest19350irq/34-mmc119:10:020
2305277994938,6cyclictest16850irq/134-800000021:19:590
2305277994936,10cyclictest16950irq/224-800000021:45:070
2305277994936,10cyclictest16950irq/224-800000021:45:060
230527799493,5cyclictest131rcu_preempt21:32:170
2305277994931,14cyclictest2346963-21grep22:25:020
2305277994925,4cyclictest131rcu_preempt22:10:000
2305288994831,12cyclictest1-21systemd00:05:013
2305286994842,4cyclictest30-21ksoftirqd/222:35:032
2305286994840,5cyclictest2335554-21kworker/2:122:59:262
2305286994840,5cyclictest2310162-21kworker/2:223:45:052
2305286994840,5cyclictest2310162-21kworker/2:223:28:242
2305286994840,5cyclictest2219619-21kworker/2:119:15:012
230528299486,8cyclictest23-21ksoftirqd/121:13:381
230528299485,3cyclictest131rcu_preempt22:15:551
230528299484,4cyclictest131rcu_preempt23:33:241
230528299484,3cyclictest131rcu_preempt00:11:321
230528299484,3cyclictest131rcu_preempt00:11:311
230528299483,3cyclictest131rcu_preempt22:46:161
230528299483,3cyclictest131rcu_preempt21:48:251
230528299483,3cyclictest131rcu_preempt21:48:241
2305282994823,5cyclictest23-21ksoftirqd/121:15:041
2305277994840,5cyclictest16850irq/134-800000000:32:010
230527799482,12cyclictest2307801-21munin-run20:00:010
2305277994820,12cyclictest2400252-21ssh00:18:270
230527799481,14cyclictest2309906-21true20:40:010
230528899471,3cyclictest131rcu_preempt23:45:003
230528899471,3cyclictest131rcu_preempt20:00:003
2305286994739,5cyclictest2335554-21kworker/2:122:14:092
2305286994739,5cyclictest2313881-21kworker/2:021:42:502
2305286994739,5cyclictest2310162-21kworker/2:222:27:432
2305286994739,5cyclictest2219619-21kworker/2:120:40:022
2305286994738,6cyclictest3467602-21systemd-journal20:55:012
2305286994738,6cyclictest3467602-21systemd-journal20:55:002
2305286994738,6cyclictest2313881-21kworker/2:021:49:002
2305286994738,6cyclictest2313881-21kworker/2:021:49:002
2305286994738,5cyclictest2309381-21kworker/2:321:02:092
2305286994733,11cyclictest1-21systemd19:50:002
230528299474,3cyclictest131rcu_preempt22:06:141
230528299473,3cyclictest131rcu_preempt20:05:001
2305282994728,13cyclictest3467602-21systemd-journal00:25:021
230528299471,5cyclictest131rcu_preempt00:33:471
230528299471,41cyclictest275-21dbus-daemon23:30:001
230527799477,36cyclictest131rcu_preempt19:50:010
2305277994720,23cyclictest12-21ksoftirqd/023:37:390
230528899466,3cyclictest131rcu_preempt23:10:303
230528899464,3cyclictest131rcu_preempt21:35:013
230528899463,3cyclictest131rcu_preempt21:43:513
230528899462,3cyclictest131rcu_preempt22:50:103
230528899461,3cyclictest131rcu_preempt22:15:003
230528899461,3cyclictest131rcu_preempt21:25:013
230528899461,3cyclictest131rcu_preempt21:25:003
2305286994640,3cyclictest2219619-21kworker/2:120:00:032
2305286994639,5cyclictest2310162-21kworker/2:200:31:002
2305286994639,4cyclictest2058883-21kworker/2:019:45:022
2305286994638,5cyclictest2335554-21kworker/2:122:06:352
2305286994638,5cyclictest2335554-21kworker/2:100:00:182
2305286994638,5cyclictest2219619-21kworker/2:119:23:302
2305286994637,6cyclictest2310162-21kworker/2:222:01:242
2305286994637,5cyclictest2310162-21kworker/2:222:22:012
2305286994636,6cyclictest2219619-21kworker/2:119:40:002
230528299466,3cyclictest131rcu_preempt20:50:031
230528299466,3cyclictest131rcu_preempt20:50:021
230528299465,4cyclictest131rcu_preempt22:50:341
230528299464,4cyclictest131rcu_preempt22:35:141
230528299464,3cyclictest131rcu_preempt23:16:041
230528299464,3cyclictest131rcu_preempt23:10:011
230528299463,4cyclictest131rcu_preempt22:13:251
230528299463,3cyclictest131rcu_preempt21:31:281
230528299462,7cyclictest23-21ksoftirqd/121:44:131
230528299462,3cyclictest131rcu_preempt23:43:431
230527799465,32cyclictest12-21ksoftirqd/020:06:120
2305277994636,5cyclictest16950irq/224-800000019:20:010
2305277994625,14cyclictest1-21systemd20:45:010
2305277994625,14cyclictest1-21systemd20:45:010
230528899453,3cyclictest131rcu_preempt22:15:523
230528899453,3cyclictest131rcu_preempt22:07:043
230528899453,3cyclictest131rcu_preempt21:39:593
230528899453,3cyclictest131rcu_preempt21:39:593
2305288994530,11cyclictest2375163-21sed23:25:023
230528899452,3cyclictest131rcu_preempt22:56:333
2305288994515,22cyclictest0-21swapper/300:16:103
230528899451,3cyclictest131rcu_preempt20:20:013
230528899451,3cyclictest131rcu_preempt00:00:023
2305286994527,12cyclictest309-21in:imuxsock20:45:012
2305286994527,12cyclictest309-21in:imuxsock20:45:012
230528299456,3cyclictest131rcu_preempt20:33:091
230528299455,5cyclictest131rcu_preempt23:36:091
230528299455,3cyclictest131rcu_preempt21:21:531
230528299455,3cyclictest131rcu_preempt21:21:521
230528299455,3cyclictest131rcu_preempt00:16:331
230528299454,4cyclictest131rcu_preempt23:01:021
230528299454,3cyclictest131rcu_preempt21:55:281
230528299454,3cyclictest131rcu_preempt21:55:281
230528299454,3cyclictest131rcu_preempt19:35:001
2305282994530,9cyclictest309-21in:imuxsock21:04:071
230528299452,4cyclictest131rcu_preempt22:41:491
230528299451,11cyclictest1-21systemd22:35:021
230527799456,32cyclictest131rcu_preempt20:50:010
230527799456,32cyclictest131rcu_preempt20:50:000
230527799453,29cyclictest2308823-21cron20:19:590
230528899443,3cyclictest131rcu_preempt22:47:183
230528899443,3cyclictest131rcu_preempt22:32:413
230528899443,3cyclictest131rcu_preempt22:00:023
230528899443,3cyclictest131rcu_preempt22:00:023
230528899443,3cyclictest131rcu_preempt21:45:263
230528899443,3cyclictest131rcu_preempt21:45:263
230528899443,3cyclictest131rcu_preempt21:28:123
230528899443,3cyclictest131rcu_preempt00:34:073
230528899442,3cyclictest131rcu_preempt00:10:443
230528899442,3cyclictest131rcu_preempt00:10:443
230528899441,4cyclictest131rcu_preempt22:43:133
230528899441,3cyclictest131rcu_preempt23:36:363
230528899441,3cyclictest131rcu_preempt20:29:083
230528899441,3cyclictest131rcu_preempt00:35:263
2305286994437,4cyclictest2310162-21kworker/2:222:30:202
2305286994436,5cyclictest2335554-21kworker/2:123:00:192
2305286994436,5cyclictest2313881-21kworker/2:021:50:222
2305286994435,5cyclictest2307538-21munin-plugin-st19:55:002
230528299444,4cyclictest131rcu_preempt19:14:081
230528299444,4cyclictest131rcu_preempt00:00:011
230528299444,3cyclictest131rcu_preempt22:25:241
230528299444,3cyclictest131rcu_preempt21:38:461
230528299444,3cyclictest131rcu_preempt21:38:451
230528299444,3cyclictest131rcu_preempt00:27:551
230528299443,4cyclictest131rcu_preempt23:24:531
230528899435,3cyclictest131rcu_preempt21:50:263
230528899434,3cyclictest131rcu_preempt23:07:033
230528899434,3cyclictest131rcu_preempt19:40:023
230528899434,3cyclictest131rcu_preempt00:23:343
2305288994335,6cyclictest37-21ksoftirqd/319:45:023
230528899433,4cyclictest131rcu_preempt23:45:233
230528899433,3cyclictest131rcu_preempt21:15:283
230528899432,3cyclictest131rcu_preempt00:25:353
230528899431,3cyclictest131rcu_preempt23:31:283
230528899431,3cyclictest131rcu_preempt23:25:143
230528899431,3cyclictest131rcu_preempt23:02:043
230528899431,3cyclictest131rcu_preempt22:37:073
230528899431,3cyclictest131rcu_preempt22:01:583
2305286994335,5cyclictest2313881-21kworker/2:021:20:022
2305286994335,5cyclictest2310162-21kworker/2:223:54:212
2305286994330,10cyclictest2308875-21systemd-run20:20:002
230528299439,6cyclictest131rcu_preempt20:38:471
230528299434,3cyclictest131rcu_preempt20:50:071
230528299434,3cyclictest131rcu_preempt20:50:071
230528299434,3cyclictest131rcu_preempt00:08:531
2305282994333,6cyclictest2310910-21munin-plugin-st21:00:001
230528299431,3cyclictest131rcu_preempt22:20:281
2305277994311,6cyclictest131rcu_preempt21:05:020
230528899423,3cyclictest131rcu_preempt22:28:053
2305288994229,9cyclictest37-21ksoftirqd/320:15:043
2305288994227,12cyclictest2310688-21cat20:55:023
2305288994227,12cyclictest2310688-21cat20:55:013
230528899422,3cyclictest131rcu_preempt20:45:023
230528899422,3cyclictest131rcu_preempt20:45:013
230528899422,3cyclictest131rcu_preempt19:18:083
230528899421,3cyclictest131rcu_preempt00:06:243
230528299425,3cyclictest131rcu_preempt19:27:131
230528299424,3cyclictest131rcu_preempt23:11:331
230528299424,3cyclictest131rcu_preempt22:05:011
230528299423,4cyclictest131rcu_preempt20:19:331
2305282994231,8cyclictest3467602-21systemd-journal20:29:021
230527799421,36cyclictest2310715-21grep20:55:020
230527799421,36cyclictest2310715-21grep20:55:010
230528899416,6cyclictest131rcu_preempt21:00:003
2305288994126,11cyclictest37-21ksoftirqd/323:15:203
230528899411,3cyclictest131rcu_preempt20:25:023
230528899411,3cyclictest131rcu_preempt19:55:013
230528699417,7cyclictest30-21ksoftirqd/221:35:462
230528699417,7cyclictest30-21ksoftirqd/221:35:452
2305286994129,9cyclictest30-21ksoftirqd/220:15:042
230528699411,3cyclictest131rcu_preempt23:24:592
230528299415,3cyclictest131rcu_preempt00:01:401
2305282994127,11cyclictest23-21ksoftirqd/119:55:011
2305282994126,11cyclictest275-21dbus-daemon19:50:001
2305282994125,13cyclictest1-21systemd21:10:011
2305277994121,13cyclictest12-21ksoftirqd/019:55:010
230528899405,5cyclictest131rcu_preempt19:30:013
230528899403,3cyclictest131rcu_preempt20:01:123
2305288994030,6cyclictest1-21systemd21:10:023
2305288994028,8cyclictest275-21dbus-daemon21:05:013
230528899402,3cyclictest131rcu_preempt20:50:033
230528899402,3cyclictest131rcu_preempt20:50:023
2305286994031,5cyclictest2309381-21kworker/2:321:10:022
2305286994030,6cyclictest2310390-21cron20:50:002
2305286994030,6cyclictest2310390-21cron20:49:592
2305286994027,9cyclictest275-21dbus-daemon21:00:002
2305282994024,12cyclictest275-21dbus-daemon20:00:001
2305277994026,11cyclictest12-21ksoftirqd/019:45:020
2305277994026,10cyclictest275-21dbus-daemon20:05:000
2305277994011,5cyclictest131rcu_preempt20:15:010
230528899394,3cyclictest131rcu_preempt19:21:033
230528899392,3cyclictest131rcu_preempt19:14:083
2305288993920,14cyclictest275-21dbus-daemon19:50:013
230528299393,3cyclictest131rcu_preempt20:42:541
230528299393,3cyclictest131rcu_preempt20:42:531
230528299392,6cyclictest131rcu_preempt20:14:591
230527799393,5cyclictest2309632-21munin-run20:35:010
230528899381,3cyclictest131rcu_preempt20:06:123
2305286993830,5cyclictest2058883-21kworker/2:020:21:162
2305286993828,6cyclictest2305706-21cron19:20:012
2305286993824,11cyclictest30-21ksoftirqd/220:30:012
230528299387,3cyclictest131rcu_preempt19:37:091
230527799386,6cyclictest131rcu_preempt19:25:020
230528899373,3cyclictest131rcu_preempt20:33:083
230528899371,8cyclictest0-21swapper/320:36:183
2305286993727,6cyclictest309-21in:imuxsock20:30:022
230528299363,3cyclictest131rcu_preempt20:06:071
2305282993624,9cyclictest23-21ksoftirqd/119:25:021
230527799368,6cyclictest131rcu_preempt19:30:010
230527799361,33cyclictest0-21swapper/020:30:010
230528899352,3cyclictest131rcu_preempt19:10:033
230528899352,3cyclictest131rcu_preempt19:10:023
230528699352,3cyclictest131rcu_preempt19:10:022
230528699352,3cyclictest131rcu_preempt19:10:022
2305288993424,6cyclictest2306501-21latency_hist19:35:003
230528299344,3cyclictest131rcu_preempt20:23:081
230528299344,3cyclictest131rcu_preempt19:10:031
230528299344,3cyclictest131rcu_preempt19:10:021
230528299343,4cyclictest131rcu_preempt19:45:011
2305286993314,15cyclictest3467602-21systemd-journal20:05:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional