You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 05:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 29, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2204521100,10sleep2301ktimersoftd/206:52:332
24118991076,74cyclictest0-21swapper/005:46:050
3237521060,5sleep02411899cyclictest09:01:460
241189910452,24cyclictest0-21swapper/008:36:040
90222900,2sleep00-21swapper/009:51:470
64282870,2sleep30-21swapper/307:51:403
24118998429,4cyclictest7950irq/56-dwc_otg_06:42:460
24118998429,4cyclictest7950irq/56-dwc_otg_06:42:460
2411899801,25cyclictest2501-21/usr/sbin/munin09:06:090
309582790,1sleep10-21swapper/109:01:071
24118997618,4cyclictest5392-21cron08:30:500
2411899761,24cyclictest335-21uname06:21:150
264282750,4sleep126431-21latency_hist08:55:521
2411899751,20cyclictest0-21swapper/008:11:520
270462740,2sleep30-21swapper/307:37:473
270462740,2sleep30-21swapper/307:37:473
2411899742,21cyclictest0-21swapper/009:21:210
24118997318,5cyclictest6777-21ssh08:31:190
24118997214,17cyclictest0-21swapper/005:08:110
24118997214,17cyclictest0-21swapper/005:08:110
2411899721,23cyclictest9500-21unixbench_singl05:16:190
24118997211,4cyclictest0-21swapper/008:51:030
2411899712,18cyclictest0-21swapper/009:36:200
24118997120,17cyclictest0-21swapper/004:46:170
28862700,2sleep30-21swapper/307:47:453
24118997017,5cyclictest21306-21/usr/sbin/munin07:31:100
2411899701,29cyclictest0-21swapper/005:51:020
2411899692,20cyclictest0-21swapper/009:40:550
24118996914,4cyclictest0-21swapper/007:20:220
24118996912,12cyclictest0-21swapper/006:53:280
2411899682,19cyclictest0-21swapper/009:26:580
24118996815,12cyclictest0-21swapper/007:14:580
24118996815,12cyclictest0-21swapper/007:14:580
2411899681,4cyclictest21168-21apt-get08:10:510
24118996814,51cyclictest908-21runrttasks09:57:360
2411899681,35cyclictest0-21swapper/008:03:470
24118996812,6cyclictest27589-21sh08:56:160
2411899681,25cyclictest24499-21cut06:56:070
24118996715,13cyclictest0-21swapper/006:48:070
2411899671,27cyclictest17951-21awk08:45:500
2411899671,19cyclictest0-21swapper/007:01:290
24118996615,12cyclictest0-21swapper/008:50:400
2411899661,24cyclictest0-21swapper/007:06:520
24118996610,5cyclictest2990-21diskmemload08:25:510
2411899652,21cyclictest0-21swapper/006:29:190
226572650,3sleep123-21ksoftirqd/106:53:411
2411899648,28cyclictest0-21swapper/007:51:090
24118996424,6cyclictest15582-21latency_hist05:35:520
24118996420,4cyclictest7850irq/56-dwc_otg_07:26:070
24118996420,26cyclictest7841-21grep09:11:190
2411899641,33cyclictest0-21swapper/007:46:110
2411899641,18cyclictest0-21swapper/009:16:070
2411899641,18cyclictest0-21swapper/007:36:150
2411899641,18cyclictest0-21swapper/007:36:150
2412199636,5cyclictest7850irq/56-dwc_otg_09:21:033
2411899637,20cyclictest0-21swapper/005:38:230
2411899633,14cyclictest7588-21/usr/sbin/munin06:36:100
2411899622,24cyclictest0-21swapper/005:10:530
24118996216,5cyclictest7850irq/56-dwc_otg_09:32:560
2411899616,15cyclictest0-21swapper/007:22:290
245822600,2sleep30-21swapper/309:31:253
2411899601,20cyclictest0-21swapper/004:40:420
2411899591,45cyclictest2996-21cut05:01:080
2411899576,41cyclictest0-21swapper/007:56:040
2411899573,22cyclictest0-21swapper/007:41:020
24118995523,16cyclictest32569-21wc04:51:180
2411899552,27cyclictest12127-21aten_rbpower_en05:26:040
2411899551,32cyclictest29554-21runrttasks08:20:510
18092550,2sleep20-21swapper/209:43:432
130982550,2sleep20-21swapper/208:00:532
2411999541,11cyclictest11-21rcu_preempt06:56:011
2411899542,33cyclictest0-21swapper/004:56:140
2411899536,18cyclictest0-21swapper/006:06:050
2411899533,28cyclictest0-21swapper/006:00:510
24118995220,15cyclictest7850irq/56-dwc_otg_04:31:200
2411899521,27cyclictest4538-21unixbench-2d06:31:200
2411999515,14cyclictest7750irq/56-dwc_otg06:46:041
2411999513,5cyclictest0-21swapper/109:06:031
2411999512,12cyclictest7750irq/56-dwc_otg09:41:021
2411899512,30cyclictest0-21swapper/005:41:140
2412099502,3cyclictest7950irq/56-dwc_otg_06:46:032
24118995020,13cyclictest7850irq/56-dwc_otg_04:41:190
112372500,17sleep27950irq/56-dwc_otg_08:36:242
24119994918,13cyclictest7950irq/56-dwc_otg_06:36:031
2411899497,26cyclictest0-21swapper/006:01:050
24118994922,4cyclictest28755-21fschecks_time06:11:080
2411899481,25cyclictest0-21swapper/005:21:100
2411899477,3cyclictest0-21swapper/009:46:020
2412199461,36cyclictest0-21swapper/305:21:083
2412099467,30cyclictest3369-21rm06:31:062
24121994530,5cyclictest917-21snmpd06:57:413
2412099457,23cyclictest0-21swapper/208:21:042
2411999456,22cyclictest0-21swapper/106:05:571
24119994526,17cyclictest2423550wget04:31:061
24119994520,14cyclictest0-21swapper/104:36:041
235422453,36sleep00-21swapper/004:26:580
2412199441,6cyclictest25741-21ssh08:16:023
2412199436,23cyclictest0-21swapper/307:31:043
2412199431,4cyclictest5334-21ls07:11:203
2412199431,4cyclictest5334-21ls07:11:203
2412099436,22cyclictest0-21swapper/209:06:042
24120994317,6cyclictest908-21runrttasks04:48:362
24119994323,5cyclictest8061-21ssh08:33:101
2412199427,25cyclictest0-21swapper/306:01:053
24121994219,4cyclictest28428-21aten_rbpower_en07:01:053
2412199420,6cyclictest7850irq/56-dwc_otg_08:11:043
2412099426,24cyclictest0-21swapper/206:01:052
2411999426,7cyclictest7850irq/56-dwc_otg_06:21:021
24118994221,4cyclictest0-21swapper/006:15:550
2412199407,24cyclictest0-21swapper/305:56:043
24121994020,12cyclictest7950irq/56-dwc_otg_09:41:043
2412199401,10cyclictest0-21swapper/307:06:173
2412099407,23cyclictest0-21swapper/205:26:042
24120994017,14cyclictest0-21swapper/209:16:042
24119994017,6cyclictest16582-21seq08:05:061
2411999401,4cyclictest7950irq/56-dwc_otg_04:41:031
2411999401,12cyclictest1643-21cut06:26:081
201592400,1sleep30-21swapper/308:47:203
28392390,5sleep30-21swapper/309:06:083
24121993922,4cyclictest8163-21latency_hist05:15:513
2412199392,13cyclictest0-21swapper/310:00:523
2412099397,25cyclictest0-21swapper/205:41:042
24120993911,6cyclictest30550-21latency_hist09:00:512
2411999398,6cyclictest7750irq/56-dwc_otg08:37:281
24119993932,5cyclictest21-21rcuc/105:41:021
2411999392,3cyclictest7950irq/56-dwc_otg_07:26:031
2411999392,25cyclictest25531-21ssh08:15:511
24119993918,5cyclictest21-21rcuc/105:46:091
179662390,3sleep117964-21ssh08:06:111
156942390,5sleep22412099cyclictest07:24:462
2412199382,3cyclictest0-21swapper/309:16:023
2412099389,11cyclictest7950irq/56-dwc_otg_05:20:512
24120993812,5cyclictest765650wget06:36:062
2411999382,7cyclictest0-21swapper/105:16:041
2411999382,3cyclictest0-21swapper/109:46:021
24119993813,16cyclictest0-21swapper/108:24:121
64172370,4sleep123-21ksoftirqd/105:10:521
64172370,4sleep123-21ksoftirqd/105:10:521
2412199373,23cyclictest29913-21ssh08:21:043
2412199371,4cyclictest7850irq/56-dwc_otg_09:51:053
2412199371,3cyclictest7950irq/56-dwc_otg_07:56:033
2412199371,17cyclictest0-21swapper/306:48:133
2412099377,22cyclictest0-21swapper/204:36:042
24120993711,12cyclictest28492-21timerwakeupswit09:36:202
2411999378,11cyclictest32325-21cat07:05:511
2411999377,25cyclictest0-21swapper/108:56:051
2411999376,17cyclictest0-21swapper/109:31:051
2411999375,14cyclictest7850irq/56-dwc_otg_09:15:511
2411999374,5cyclictest7950irq/56-dwc_otg_08:30:291
2411999374,19cyclictest7950irq/56-dwc_otg_04:50:561
2411999371,3cyclictest7850irq/56-dwc_otg_05:15:511
89862360,1sleep20-21swapper/207:16:122
24121993617,6cyclictest22915-21fschecks_time08:51:093
2412199361,4cyclictest6699-21aten_rbpower_cu09:11:043
2412199361,23cyclictest0-21swapper/308:31:063
24121993611,7cyclictest7921-21fschecks_count06:36:093
2412099368,17cyclictest0-21swapper/208:10:592
2411999367,17cyclictest12927-21ssh06:41:191
2411999367,17cyclictest12927-21ssh06:41:191
2411999366,18cyclictest7950irq/56-dwc_otg_07:16:201
2411999365,15cyclictest7950irq/56-dwc_otg_06:05:511
2411999363,6cyclictest7950irq/56-dwc_otg_07:51:031
2411999361,8cyclictest15391-21idleruntime-cro10:00:511
2411999361,24cyclictest30026-21ssh07:41:171
2412199356,21cyclictest39-21ksoftirqd/306:51:213
24121993524,8cyclictest1269-21latency_hist06:25:513
24121993517,8cyclictest7950irq/56-dwc_otg_08:09:553
2412199351,25cyclictest0-21swapper/308:40:563
2412099359,13cyclictest0-21swapper/207:51:042
2412099359,12cyclictest0-21swapper/209:24:342
2412099357,22cyclictest0-21swapper/205:01:042
2412099357,19cyclictest0-21swapper/208:41:022
2412099355,9cyclictest7750irq/56-dwc_otg08:26:022
24120993520,7cyclictest7950irq/56-dwc_otg_08:52:382
24120993517,6cyclictest18431-21aten_rbpower_vo08:46:072
24120993510,12cyclictest0-21swapper/207:11:152
24120993510,12cyclictest0-21swapper/207:11:152
2411999357,25cyclictest0-21swapper/105:36:031
2411999352,20cyclictest7950irq/56-dwc_otg_08:41:171
24119993515,14cyclictest0-21swapper/109:54:511
234142354,19sleep223481-21vmstat04:26:202
204032350,1sleep30-21swapper/307:30:463
76932340,5sleep22412099cyclictest08:32:292
2412199348,6cyclictest0-21swapper/309:36:193
24121993425,5cyclictest381ktimersoftd/304:31:053
2412199342,13cyclictest7850irq/56-dwc_otg_09:26:203
2412199341,7cyclictest0-21swapper/306:16:043
2412099349,5cyclictest7510-21aten_rbpower_cu09:51:062
2412099349,14cyclictest0-21swapper/207:36:132
2412099349,14cyclictest0-21swapper/207:36:132
2412099348,13cyclictest0-21swapper/209:31:092
2412099344,4cyclictest6092-21systemd-cgroups05:09:252
2412099344,4cyclictest6092-21systemd-cgroups05:09:252
2412099343,3cyclictest7950irq/56-dwc_otg_05:36:022
2412099341,8cyclictest25446-21unixbench_multi06:56:192
2412099341,6cyclictest3464-21grep09:46:062
2412099341,5cyclictest22897-21modprobe09:30:502
2411999346,3cyclictest0-21swapper/108:17:001
2411999345,3cyclictest7950irq/56-dwc_otg_07:36:191
2411999345,3cyclictest7950irq/56-dwc_otg_07:36:191
2411999345,20cyclictest9080-21ssh07:56:041
2411999342,6cyclictest21-21rcuc/107:31:151
2411999341,3cyclictest0-21swapper/109:20:511
2411999341,3cyclictest0-21swapper/105:56:011
24119993413,4cyclictest0-21swapper/108:50:511
24119993411,13cyclictest0-21swapper/109:30:451
2412199333,15cyclictest7950irq/56-dwc_otg_07:20:503
2412199332,5cyclictest7750irq/56-dwc_otg04:40:523
2412199332,3cyclictest7950irq/56-dwc_otg_06:31:043
2412199332,21cyclictest31961-21timerwakeupswit09:01:203
2412199331,5cyclictest4028-21ls09:46:133
2412199331,14cyclictest0-21swapper/307:41:033
2412099339,11cyclictest0-21swapper/206:44:132
2412099339,11cyclictest0-21swapper/206:44:132
2412099332,7cyclictest20293-21grep05:46:182
2412099332,6cyclictest7750irq/56-dwc_otg07:10:512
2412099332,20cyclictest0-21swapper/207:26:152
2411999336,5cyclictest23-21ksoftirqd/107:25:511
2411999335,6cyclictest7750irq/56-dwc_otg09:21:061
2411999332,9cyclictest2682-21latency_hist05:00:521
2412199323,6cyclictest7850irq/56-dwc_otg_08:26:183
2412199322,12cyclictest0-21swapper/308:59:563
2412199321,8cyclictest0-21swapper/305:01:173
2412199321,21cyclictest14195-21fschecks_time05:31:083
2412099329,5cyclictest0-21swapper/209:56:232
2412099327,15cyclictest0-21swapper/205:56:042
2412099324,16cyclictest7950irq/56-dwc_otg_06:11:192
2412099323,19cyclictest4186-1kworker/2:0H05:11:112
24120993219,4cyclictest24847-21ntp_states04:31:152
24120993218,5cyclictest32335-21ps04:51:152
2412099321,6cyclictest28684-21fschecks_count07:01:092
2412099321,25cyclictest29555-21latency_hist08:20:512
24120993210,4cyclictest0-21swapper/209:11:022
24119993214,9cyclictest21112-21aten_rbpower_cu05:51:041
24119993210,4cyclictest5662-21ssh06:33:051
2412199319,3cyclictest0-21swapper/308:37:433
2412199318,7cyclictest13014-21sed06:41:203
2412199318,7cyclictest13014-21sed06:41:203
2412199318,13cyclictest7950irq/56-dwc_otg_04:40:523
2412199312,18cyclictest0-21swapper/306:30:513
2412199312,13cyclictest7950irq/56-dwc_otg_05:26:183
24121993111,4cyclictest18508-21sed05:41:183
2412099315,3cyclictest0-21swapper/205:51:122
2412099312,5cyclictest0-21swapper/206:26:162
24120993116,11cyclictest0-21swapper/207:56:042
24120993112,11cyclictest4562-21ssh07:50:482
24120993111,4cyclictest0-21swapper/208:06:072
2411999317,14cyclictest0-21swapper/107:47:241
2411999312,4cyclictest0-21swapper/107:07:081
2411999311,15cyclictest7950irq/56-dwc_otg_05:26:101
2412199309,9cyclictest7950irq/56-dwc_otg_07:21:033
2412199306,14cyclictest0-21swapper/304:50:573
24121993025,3cyclictest0-21swapper/304:50:513
2412199301,21cyclictest9283-21timerandwakeup05:16:173
2412199301,19cyclictest0-21swapper/308:04:253
2412099307,21cyclictest0-21swapper/204:56:062
2412099307,11cyclictest0-21swapper/207:41:182
2412099305,7cyclictest7950irq/56-dwc_otg_07:31:032
2412099303,5cyclictest0-21swapper/206:05:572
2412099302,19cyclictest0-21swapper/209:01:122
2411999304,13cyclictest7850irq/56-dwc_otg_09:38:131
2411999301,9cyclictest0-21swapper/105:31:081
24119993016,5cyclictest908-21runrttasks05:21:461
225072305,8sleep10-21swapper/104:26:061
2412199296,5cyclictest7950irq/56-dwc_otg_05:06:093
2412199296,5cyclictest7950irq/56-dwc_otg_05:06:093
24121992912,6cyclictest19888-21missed_timers05:46:123
2412199291,19cyclictest0-21swapper/305:36:193
2412099297,19cyclictest0-21swapper/205:21:042
2412099292,15cyclictest32033-21/usr/sbin/munin06:21:152
2411999293,5cyclictest7750irq/56-dwc_otg06:11:111
24119992914,13cyclictest0-21swapper/105:01:041
2411999291,2cyclictest0-21swapper/107:11:001
2411999291,2cyclictest0-21swapper/107:11:001
228952293,16sleep322969-21/usr/sbin/munin04:26:133
2412199285,5cyclictest7950irq/56-dwc_otg_06:11:153
2412199285,14cyclictest1609-21ntp_kernel_err04:56:143
2412199281,15cyclictest0-21swapper/305:51:043
2412099284,5cyclictest7850irq/56-dwc_otg_06:16:062
2411999282,18cyclictest0-21swapper/104:46:201
2412199271,15cyclictest0-21swapper/306:06:143
2412099271,5cyclictest7850irq/56-dwc_otg_05:31:112
2412099271,10cyclictest0-21swapper/204:40:532
2411999272,17cyclictest0-21swapper/106:16:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional