You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 06:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 20, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
209992890,2sleep10-21swapper/108:15:111
1936599863,39cyclictest0-21swapper/008:20:580
56822850,3sleep35680-21sh07:16:393
56822850,3sleep35680-21sh07:16:393
19365998510,43cyclictest9-21ksoftirqd/008:10:580
187372800,3sleep018735-21grep09:30:470
19365997434,10cyclictest29926-21ssh08:25:580
1936599709,12cyclictest0-21swapper/005:00:570
1936799699,37cyclictest7850irq/56-dwc_otg_04:46:012
276902680,1sleep20-21swapper/207:05:472
56302660,2sleep10-21swapper/107:56:061
19367996513,37cyclictest22787-21latency_hist09:35:472
19367996513,37cyclictest22787-21latency_hist09:35:472
19366996524,9cyclictest221ktimersoftd/106:35:571
19366996524,9cyclictest221ktimersoftd/106:35:571
19365996530,17cyclictest7850irq/56-dwc_otg_09:06:140
1936799646,8cyclictest0-21swapper/208:15:572
1936799641,6cyclictest17914-21timerwakeupswit07:31:142
19365996422,11cyclictest0-21swapper/008:30:580
310402620,38sleep21936799cyclictest05:01:122
1936799627,3cyclictest0-21swapper/206:43:122
1936799626,5cyclictest7950irq/56-dwc_otg_06:46:152
1936599627,35cyclictest0-21swapper/006:21:010
1936599626,25cyclictest26856-21fschecks_count09:01:030
1936599626,25cyclictest26856-21fschecks_count09:01:030
1936599625,23cyclictest7750irq/56-dwc_otg04:40:570
19365996227,10cyclictest0-21swapper/008:05:580
1936899604,8cyclictest7750irq/56-dwc_otg08:30:573
1936799602,43cyclictest0-21swapper/208:46:352
1936799601,7cyclictest27080-21tune2fs06:21:052
1936599604,18cyclictest908-21runrttasks09:15:570
1936599602,32cyclictest20960-21wget07:36:000
1936599601,50cyclictest0-21swapper/008:01:050
1936799593,4cyclictest0-21swapper/207:56:012
19365995941,5cyclictest7850irq/56-dwc_otg_07:00:570
19365995931,6cyclictest22651-21grep08:56:040
19365995919,23cyclictest908-21runrttasks09:36:050
1936799584,4cyclictest7950irq/56-dwc_otg_08:31:062
1936799584,4cyclictest7750irq/56-dwc_otg09:13:092
19367995836,19cyclictest30156-21diskmemload09:05:572
19365995810,10cyclictest0-21swapper/007:25:560
1936799572,9cyclictest7750irq/56-dwc_otg08:11:152
1936799571,12cyclictest9192-21rm09:57:312
1936699577,20cyclictest0-21swapper/104:46:001
19365995729,13cyclictest7950irq/56-dwc_otg_08:41:150
1936599571,40cyclictest4304-21sed09:51:160
1936799567,4cyclictest0-21swapper/209:38:502
1936799567,46cyclictest29206-21sed07:06:152
1936799563,5cyclictest0-21swapper/208:26:102
1936799562,41cyclictest7850irq/56-dwc_otg_06:18:462
1936799561,3cyclictest886-21ssh07:50:592
1936799561,10cyclictest0-21swapper/206:52:432
1936799556,4cyclictest7950irq/56-dwc_otg_09:16:082
1936799554,39cyclictest23242-21sensors08:56:122
1936799551,3cyclictest7750irq/56-dwc_otg09:23:242
1936799551,3cyclictest0-21swapper/209:46:042
19367995512,39cyclictest6777-21timerandwakeup08:36:132
1936799551,11cyclictest0-21swapper/207:37:202
1936799551,10cyclictest28642-21ssh09:41:262
19365995531,5cyclictest3537-21fschecks_count05:16:050
19365995527,6cyclictest32136-21tr09:46:120
19365995522,21cyclictest6618-21ssh06:41:000
1936599551,29cyclictest32254-21apt-get07:10:530
103642550,2sleep00-21swapper/007:22:230
1936799547,3cyclictest0-21swapper/209:54:202
1936799545,36cyclictest7950irq/56-dwc_otg_08:52:142
1936799545,36cyclictest7950irq/56-dwc_otg_08:52:142
1936799542,4cyclictest15911-21sshd07:29:492
1936799541,9cyclictest0-21swapper/209:03:002
1936799541,9cyclictest0-21swapper/209:03:002
1936799541,5cyclictest9383-21df08:01:022
1936799541,10cyclictest0-21swapper/208:22:182
1936599546,27cyclictest13-21rcuc/005:51:010
1936599544,30cyclictest30156-21diskmemload06:56:000
79882530,2sleep17989-21ssh06:41:221
1936799532,8cyclictest7950irq/56-dwc_otg_06:36:272
1936799532,8cyclictest7950irq/56-dwc_otg_06:36:272
1936799532,4cyclictest3888-21missed_timers05:16:092
19365995323,4cyclictest18177-21taskset08:50:580
19365995323,4cyclictest18177-21taskset08:50:580
19365995323,15cyclictest0-21swapper/006:26:120
1936599532,27cyclictest0-21swapper/005:30:460
1936599532,26cyclictest0-21swapper/009:55:590
19365995322,16cyclictest0-21swapper/005:55:590
19365995322,16cyclictest0-21swapper/005:55:590
19365995313,22cyclictest21465-21fschecks_count04:36:030
1936799525,8cyclictest7950irq/56-dwc_otg_07:45:192
1936799521,8cyclictest15758-21latency_hist05:50:472
1936799521,8cyclictest12415-21latency_hist07:25:462
1936799521,4cyclictest22692-21ssh06:59:382
1936799521,3cyclictest0-21swapper/208:09:202
1936599527,30cyclictest9-21ksoftirqd/007:30:570
19365995230,5cyclictest7750irq/56-dwc_otg09:45:460
1936599522,27cyclictest0-21swapper/007:16:160
1936599522,27cyclictest0-21swapper/007:16:160
1936599521,28cyclictest23500-21fschecks_time06:11:040
19365995212,23cyclictest0-21swapper/008:36:050
189042523,41sleep00-21swapper/004:28:170
1936799512,9cyclictest7950irq/56-dwc_otg_08:43:582
19367995121,28cyclictest19833-21fschecks_time06:01:032
1936799511,3cyclictest0-21swapper/205:41:162
1936599515,9cyclictest0-21swapper/006:45:590
19365995121,3cyclictest0-21swapper/009:20:590
1936599511,15cyclictest15499-21grep06:51:040
61632500,2chrt6162-21sed05:21:140
19368995018,17cyclictest39-21ksoftirqd/307:45:593
19368995018,17cyclictest39-21ksoftirqd/307:45:593
1936799505,4cyclictest0-21swapper/209:27:282
1936799504,5cyclictest31058-21grep06:31:072
1936799502,8cyclictest7850irq/56-dwc_otg_07:11:442
1936799502,7cyclictest12267-21cat05:40:472
1936799502,6cyclictest4232-21ssh07:16:012
1936799502,6cyclictest4232-21ssh07:16:012
1936799502,6cyclictest0-21swapper/204:59:302
1936799501,8cyclictest5165-21aten_rbpower_en05:21:012
1936799501,4cyclictest30441-21unixbench_multi07:46:152
1936799501,4cyclictest30441-21unixbench_multi07:46:152
19367995012,35cyclictest22283-21sed04:36:132
19365995024,5cyclictest22523-21ssh08:16:120
1936599502,30cyclictest29951-21ntp_kernel_pll_07:46:090
1936599502,30cyclictest29951-21ntp_kernel_pll_07:46:090
19365995017,5cyclictest7750irq/56-dwc_otg06:05:570
1936599501,30cyclictest25395-21grep07:41:050
1936799492,7cyclictest7469-21cat05:26:072
1936799492,6cyclictest7850irq/56-dwc_otg_05:10:432
1936799491,8cyclictest27572-21grep04:51:152
1936799491,7cyclictest21694-21cut06:06:042
1936799491,3cyclictest7950irq/56-dwc_otg_04:41:002
1936799491,10cyclictest0-21swapper/206:29:452
1936599493,5cyclictest0-21swapper/006:35:580
1936599493,5cyclictest0-21swapper/006:35:580
1936599492,26cyclictest0-21swapper/007:56:150
19365994917,15cyclictest0-21swapper/004:31:100
19365994917,15cyclictest0-21swapper/004:31:100
1936599491,5cyclictest0-21swapper/009:30:580
1936599491,5cyclictest0-21swapper/009:30:580
1936799486,3cyclictest0-21swapper/205:11:122
1936799482,6cyclictest16257-21fschecks_time05:51:042
1936799481,8cyclictest7850irq/56-dwc_otg_04:32:512
1936799481,8cyclictest7850irq/56-dwc_otg_04:32:512
1936699487,27cyclictest0-21swapper/104:55:581
1936699481,9cyclictest7750irq/56-dwc_otg09:56:011
1936599484,10cyclictest7750irq/56-dwc_otg05:45:580
19365994824,12cyclictest7950irq/56-dwc_otg_04:56:020
19365994821,7cyclictest11603-21grep05:36:150
171512480,2sleep10-21swapper/106:52:251
1936799474,7cyclictest25577-21systemd-journal05:35:462
1936599476,7cyclictest30526-21ssh06:31:000
19365994725,4cyclictest28857-21/usr/sbin/munin07:06:120
19365994724,7cyclictest0-21swapper/009:10:580
19365994724,5cyclictest917-21snmpd05:41:090
19365994717,4cyclictest4358-21sh07:55:120
1936799463,5cyclictest0-21swapper/206:15:462
1936799461,6cyclictest18203-21uname05:56:072
1936799461,6cyclictest18203-21uname05:56:072
1936899457,24cyclictest0-21swapper/305:31:003
1936899457,24cyclictest0-21swapper/305:10:513
19365994520,6cyclictest7750irq/56-dwc_otg08:46:060
19365994513,22cyclictest0-21swapper/004:51:090
1936599451,23cyclictest0-21swapper/005:10:570
1936599451,16cyclictest8694-21/usr/sbin/munin05:31:150
1936599451,13cyclictest0-21swapper/005:06:000
1936699448,23cyclictest0-21swapper/105:36:001
1936599445,36cyclictest0-21swapper/004:46:150
19365994416,19cyclictest0-21swapper/006:04:120
1936899436,19cyclictest7950irq/56-dwc_otg_09:36:043
19366994323,7cyclictest0-21swapper/108:35:561
19365994315,17cyclictest0-21swapper/006:16:190
178872430,4sleep217890-21fschecks_time04:26:032
171872430,4sleep1221ktimersoftd/109:28:141
1936699427,22cyclictest0-21swapper/105:31:001
1936699422,4cyclictest26100-21kworker/u8:009:00:571
1936699422,4cyclictest26100-21kworker/u8:009:00:571
1936899417,22cyclictest0-21swapper/309:00:593
1936899417,22cyclictest0-21swapper/309:00:593
1936899416,25cyclictest27609-21kworker/u8:407:10:533
1936699413,3cyclictest0-21swapper/108:56:001
19366994120,5cyclictest1111-21sh07:11:161
19366994119,4cyclictest21459-21/usr/sbin/munin07:36:061
79442400,2sleep11936699cyclictest09:16:271
304532390,2sleep10-21swapper/105:01:031
1936899396,24cyclictest14060-21ssh08:46:003
1936899392,7cyclictest7950irq/56-dwc_otg_08:10:593
1936699392,4cyclictest0-21swapper/108:41:001
1936899382,5cyclictest0-21swapper/306:16:003
1936699384,27cyclictest24741-21sh08:19:281
19366993813,7cyclictest11-21rcu_preempt06:46:001
1936699381,26cyclictest3455-21ls09:11:131
19366993812,5cyclictest7750irq/56-dwc_otg08:51:141
19366993812,5cyclictest7750irq/56-dwc_otg08:51:141
1936899379,5cyclictest0-21swapper/307:31:043
1936899375,6cyclictest7950irq/56-dwc_otg_04:35:583
1936699374,4cyclictest7750irq/56-dwc_otg09:21:051
19366993718,10cyclictest56612sleep107:16:371
19366993718,10cyclictest56612sleep107:16:371
230982360,3sleep123-21ksoftirqd/106:10:461
1936899369,5cyclictest2412-21aten_rbpower_po06:36:003
1936899369,5cyclictest2412-21aten_rbpower_po06:36:003
1936899366,14cyclictest7950irq/56-dwc_otg_06:05:463
19368993620,5cyclictest7850irq/56-dwc_otg_08:20:583
19368993614,4cyclictest14666-21head08:06:163
1936899361,3cyclictest7950irq/56-dwc_otg_04:45:593
19368993613,5cyclictest31341-21/usr/sbin/munin09:06:073
19368993613,13cyclictest0-21swapper/307:25:593
1936899361,22cyclictest7066-21cpuspeed05:26:013
1936699367,13cyclictest7950irq/56-dwc_otg_09:06:041
19366993622,8cyclictest221ktimersoftd/107:30:571
286382351,3sleep30-21swapper/304:56:023
1936899354,7cyclictest7950irq/56-dwc_otg_05:16:003
1936899354,3cyclictest7950irq/56-dwc_otg_09:30:473
1936899351,19cyclictest0-21swapper/305:56:123
1936899351,19cyclictest0-21swapper/305:56:123
1936899351,16cyclictest7950irq/56-dwc_otg_05:41:053
1936699358,5cyclictest7850irq/56-dwc_otg_09:51:011
1936699351,23cyclictest24573-21latency_hist04:45:461
19366993511,9cyclictest19691-21ntp_kernel_pll_09:31:091
19366993511,9cyclictest19691-21ntp_kernel_pll_09:31:091
1936899346,5cyclictest7750irq/56-dwc_otg07:25:463
1936899343,6cyclictest19903-21sh06:56:053
1936899342,3cyclictest7950irq/56-dwc_otg_05:35:583
1936899341,4cyclictest7950irq/56-dwc_otg_06:21:003
19368993413,5cyclictest14902-21latency_hist06:50:463
1936899341,22cyclictest2579-21ls09:11:013
1936699345,6cyclictest7750irq/56-dwc_otg09:46:001
1936699343,3cyclictest0-21swapper/106:33:501
19366993414,13cyclictest221ktimersoftd/107:06:161
1936699341,22cyclictest30156-21diskmemload07:26:121
19366993411,4cyclictest0-21swapper/109:41:081
19366993410,4cyclictest0-21swapper/107:25:161
219552330,1sleep10-21swapper/106:58:211
1936899337,14cyclictest18396-21df08:51:013
1936899337,14cyclictest18396-21df08:51:013
19368993314,9cyclictest7950irq/56-dwc_otg_08:04:493
19368993313,14cyclictest7850irq/56-dwc_otg_06:41:223
1936899331,20cyclictest19151-21ls09:31:023
1936899331,20cyclictest19151-21ls09:31:023
1936699339,5cyclictest29756-21cut07:46:071
1936699339,5cyclictest29756-21cut07:46:071
1936699339,4cyclictest24020-21/usr/sbin/munin09:36:121
1936699338,15cyclictest0-21swapper/105:26:001
1936699338,14cyclictest0-21swapper/108:09:331
1936699337,16cyclictest0-21swapper/105:16:011
19366993323,7cyclictest17045-21grep05:51:151
1936699331,14cyclictest7750irq/56-dwc_otg08:21:141
175572336,14sleep10-21swapper/104:25:591
237022320,2sleep10-21swapper/106:11:071
1936899328,5cyclictest7850irq/56-dwc_otg_09:20:463
1936899327,4cyclictest0-21swapper/309:46:113
1936899326,15cyclictest0-21swapper/307:55:593
1936899322,17cyclictest7750irq/56-dwc_otg09:42:113
19368993215,4cyclictest30066-21cpu08:26:013
1936899321,22cyclictest16917-21timerwakeupswit05:51:143
19368993210,6cyclictest0-21swapper/307:02:393
1936699326,10cyclictest7850irq/56-dwc_otg_07:50:581
1936699325,5cyclictest7750irq/56-dwc_otg07:05:461
19366993224,5cyclictest221ktimersoftd/104:30:561
19366993224,5cyclictest221ktimersoftd/104:30:561
1936699322,14cyclictest7850irq/56-dwc_otg_05:06:071
19366993215,5cyclictest908-21runrttasks08:29:401
1936699321,19cyclictest26936-21ls06:21:031
1936899317,4cyclictest7750irq/56-dwc_otg06:51:153
1936899314,4cyclictest7950irq/56-dwc_otg_08:36:043
1936899314,18cyclictest39-21ksoftirqd/305:00:573
1936899313,15cyclictest7850irq/56-dwc_otg_07:06:033
1936899312,20cyclictest0-21swapper/306:29:553
19368993118,10cyclictest0-21swapper/308:56:123
1936899311,6cyclictest320450wget09:51:023
1936899311,13cyclictest7850irq/56-dwc_otg_08:16:133
1936699315,6cyclictest0-21swapper/108:49:591
1936699314,4cyclictest28583-21apt-get06:25:581
19366993113,5cyclictest25248-21fschecks_time06:16:041
19366993111,4cyclictest21-21rcuc/105:56:061
19366993111,4cyclictest21-21rcuc/105:56:061
1936699311,14cyclictest12417-21wget05:41:021
1936899308,13cyclictest0-21swapper/307:45:203
1936899306,6cyclictest7850irq/56-dwc_otg_05:05:593
1936899305,4cyclictest7950irq/56-dwc_otg_07:40:463
1936899304,5cyclictest7750irq/56-dwc_otg08:41:093
1936899304,4cyclictest25577-21systemd-journal06:07:233
1936899301,18cyclictest0-21swapper/307:54:223
19368993010,3cyclictest0-21swapper/309:57:383
1936699308,3cyclictest0-21swapper/108:31:151
1936699307,21cyclictest0-21swapper/104:51:001
1936699306,4cyclictest7750irq/56-dwc_otg04:36:141
1936699306,15cyclictest0-21swapper/108:01:011
1936699302,15cyclictest7850irq/56-dwc_otg_05:11:001
1936699301,6cyclictest20661-21tail06:01:141
1936899298,5cyclictest0-21swapper/306:31:033
1936899293,6cyclictest7850irq/56-dwc_otg_06:11:043
1936899293,12cyclictest7950irq/56-dwc_otg_05:21:123
1936899292,19cyclictest0-21swapper/304:43:163
19368992913,5cyclictest7850irq/56-dwc_otg_04:35:473
19368992913,5cyclictest7850irq/56-dwc_otg_04:35:473
1936699297,6cyclictest7950irq/56-dwc_otg_07:40:481
1936699294,12cyclictest7850irq/56-dwc_otg_05:22:561
1936899286,4cyclictest0-21swapper/309:20:553
1936699283,8cyclictest15835-21latency_hist05:50:471
1936899274,5cyclictest7950irq/56-dwc_otg_05:46:013
188282273,14sleep30-21swapper/304:27:273
1936899261,15cyclictest0-21swapper/304:51:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional