You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 15:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 29, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1566822480,3sleep20-21swapper/216:26:112
1702199967,81cyclictest0-21swapper/018:01:040
121732890,2sleep00-21swapper/018:50:470
185632860,5sleep31702499cyclictest20:16:053
17021998436,5cyclictest7850irq/56-dwc_otg_17:56:030
262522820,4sleep326256-21unixbench_singl19:06:203
17021998029,14cyclictest908-21runrttasks19:36:450
17021997928,14cyclictest32353-21aten_rbpower_cu21:51:040
1702199791,55cyclictest0-21swapper/021:56:030
67832780,2sleep10-21swapper/117:31:101
12942780,5sleep31702499cyclictest17:16:103
147012760,2sleep20-21swapper/218:52:502
75342730,2sleep30-21swapper/319:23:353
75342730,2sleep30-21swapper/319:23:353
190842710,1sleep20-21swapper/216:36:082
1702499703,7cyclictest19187-21ssh20:56:043
1702199706,14cyclictest0-21swapper/016:41:030
61852690,4sleep0101ktimersoftd/020:40:260
1702499672,47cyclictest7950irq/56-dwc_otg_17:06:073
1702499658,41cyclictest0-21swapper/320:23:563
1702499658,41cyclictest0-21swapper/320:23:563
17021996520,4cyclictest101ktimersoftd/020:16:160
1702199642,32cyclictest15022-21ssh20:51:020
1702199641,48cyclictest11226-21modprobe21:25:500
250562630,4sleep225055-21awk21:41:192
1702499633,56cyclictest28005-21diskmemload21:16:033
1702499633,56cyclictest28005-21diskmemload21:16:033
1702199631,44cyclictest14683-21fschecks_count20:11:100
313342620,2sleep231336-21unixbench_singl19:51:202
1702499628,41cyclictest0-21swapper/320:03:323
1702499626,44cyclictest7950irq/56-dwc_otg_19:45:063
17024996210,32cyclictest0-21swapper/320:50:073
1702199621,43cyclictest3175-21latency_hist21:15:520
1702499613,5cyclictest7950irq/56-dwc_otg_19:51:063
1702499612,3cyclictest13815-21apt-get17:51:023
17021996142,4cyclictest101ktimersoftd/018:11:020
17021996133,7cyclictest7850irq/56-dwc_otg_20:00:520
105932610,2sleep10-21swapper/118:47:541
1702499602,5cyclictest0-21swapper/318:36:193
1702199601,46cyclictest0-21swapper/020:46:010
28642590,1sleep20-21swapper/219:17:382
183282590,1sleep20-21swapper/220:15:522
1702499597,4cyclictest7850irq/56-dwc_otg_18:11:063
1702199591,44cyclictest24547-21grep21:01:190
1702499586,9cyclictest7750irq/56-dwc_otg19:31:043
1702499583,8cyclictest29138-21ssh19:11:033
1702499582,41cyclictest0-21swapper/321:22:553
1702199581,38cyclictest5032-21aten_rbpower_cu19:21:050
1702199581,38cyclictest5032-21aten_rbpower_cu19:21:050
1702499574,43cyclictest26852-21aten_rbpower_en20:26:053
1702499572,10cyclictest7850irq/56-dwc_otg_19:59:303
1702499571,4cyclictest32424-21sed17:11:163
1702199576,39cyclictest0-21swapper/019:16:040
17021995735,6cyclictest25415-21unixbench_singl16:51:170
1702199571,47cyclictest0-21swapper/019:41:200
1702199571,38cyclictest26908-21wget20:26:060
1702499565,36cyclictest0-21swapper/320:11:573
1702499564,40cyclictest7850irq/56-dwc_otg_18:33:563
17021995634,7cyclictest27137-21grep19:46:190
17021995633,11cyclictest0-21swapper/017:16:030
17021995630,15cyclictest0-21swapper/017:10:510
1702499552,5cyclictest29552-21rm21:07:473
1702499551,5cyclictest18284-21switchtime18:01:163
1702499551,5cyclictest10291-21cut20:06:043
17024995514,30cyclictest28005-21diskmemload20:32:053
17021995529,4cyclictest28195-21ssh21:46:050
17021995529,4cyclictest28195-21ssh21:46:050
1702199552,40cyclictest0-21swapper/019:35:350
1702199552,37cyclictest32244-21idleruntime-cro18:35:500
1702199551,50cyclictest0-21swapper/017:45:560
1702199551,41cyclictest29462-21fschecks_time19:11:090
1702499546,39cyclictest0-21swapper/320:38:333
1702499541,4cyclictest16502-21ssh20:51:373
1702499541,3cyclictest7750irq/56-dwc_otg16:56:073
17021995439,5cyclictest7850irq/56-dwc_otg_20:31:150
1702199542,34cyclictest23756-21grep18:16:180
1702199541,40cyclictest19565-21grep18:06:080
1702199541,39cyclictest6638-21fschecks_count17:31:080
1702199541,33cyclictest9781-21apt20:05:510
1702499532,38cyclictest0-21swapper/322:00:513
17024995315,35cyclictest11700-21aten_rbpower_vo21:26:053
1702499531,4cyclictest0-21swapper/318:50:383
1702199532,42cyclictest0-21swapper/021:16:310
1702199532,42cyclictest0-21swapper/021:16:310
1702199532,37cyclictest0-21swapper/016:58:540
1702199531,35cyclictest28779-21ssh19:10:510
116272530,1sleep00-21swapper/019:28:190
1702499524,7cyclictest7950irq/56-dwc_otg_21:37:063
1702499522,38cyclictest7950irq/56-dwc_otg_18:25:243
1702499521,5cyclictest1018-21ls21:51:183
1702499521,4cyclictest10313-21ssh20:45:263
1702499521,43cyclictest0-21swapper/319:36:383
1702499521,42cyclictest0-21swapper/318:17:193
1702499521,42cyclictest0-21swapper/317:43:023
1702499521,3cyclictest0-21swapper/321:01:533
17024995211,37cyclictest5632-21grep17:26:203
1702199523,41cyclictest15376-21idleruntime-cro17:55:500
17021995231,5cyclictest12693-21aten_rbpower_vo18:51:060
1702199522,43cyclictest0-21swapper/021:36:020
1702199521,44cyclictest0-21swapper/020:06:030
1702199521,43cyclictest31063-21ssh19:51:180
1702199521,40cyclictest15509-21ssh21:30:510
17021995210,26cyclictest0-21swapper/017:01:160
1702499511,9cyclictest19535-21fschecks_time18:06:083
1702499511,6cyclictest28019-21awk18:30:513
1702499511,5cyclictest908-21runrttasks21:14:223
1702499511,4cyclictest0-21swapper/319:03:373
1702499511,3cyclictest0-21swapper/319:49:343
17024995112,36cyclictest143842chrt18:52:153
1702499511,12cyclictest0-21swapper/317:35:513
1702199515,38cyclictest0-21swapper/017:41:010
1702199515,22cyclictest0-21swapper/021:06:090
17021995113,21cyclictest0-21swapper/020:56:070
164412513,11sleep00-21swapper/016:26:550
1702499509,38cyclictest17961-21/usr/sbin/munin16:31:163
1702499505,4cyclictest7750irq/56-dwc_otg19:16:553
1702499502,8cyclictest21809-21grep16:41:183
1702499502,8cyclictest19184-21fschecks_time16:36:093
1702499502,8cyclictest17324-21grep18:56:113
1702499502,8cyclictest17324-21grep18:56:113
17024995016,31cyclictest31411-21sh21:49:523
17024995016,31cyclictest31411-21sh21:49:523
1702499501,3cyclictest7950irq/56-dwc_otg_16:51:023
1702399505,9cyclictest0-21swapper/220:01:022
1702199501,40cyclictest32685-21ssh18:36:050
1702199501,39cyclictest0-21swapper/021:41:180
1702499496,4cyclictest7850irq/56-dwc_otg_21:34:163
1702499491,7cyclictest10424-21unixbench-2d19:26:203
1702299497,28cyclictest0-21swapper/118:01:041
1702199498,29cyclictest0-21swapper/017:11:070
17021994931,10cyclictest7950irq/56-dwc_otg_17:21:140
1702199492,38cyclictest0-21swapper/021:34:060
1702199491,34cyclictest5067-21ntp_offset18:41:140
1702499481,8cyclictest8547-21sort17:36:073
1702499481,8cyclictest4231-21munin-run17:25:513
1702499481,3cyclictest7850irq/56-dwc_otg_17:01:163
17024994813,32cyclictest7950irq/56-dwc_otg_18:42:283
1702399482,22cyclictest28137-21cut17:01:082
1702299487,27cyclictest0-21swapper/117:45:571
17021994817,15cyclictest20346-21seq16:40:050
1702199481,38cyclictest0-21swapper/017:26:070
1702499474,4cyclictest7950irq/56-dwc_otg_17:56:013
1702499473,3cyclictest7950irq/56-dwc_otg_16:46:043
1702499472,42cyclictest26225-21rm21:42:573
1702499471,8cyclictest13568-21cron17:50:503
17022994720,6cyclictest3454-21proc_pri20:36:161
1702199471,38cyclictest0-21swapper/019:01:200
1702199471,38cyclictest0-21swapper/018:56:080
1702199471,38cyclictest0-21swapper/018:56:080
1702399467,24cyclictest0-21swapper/217:16:052
1702399464,5cyclictest0-21swapper/220:31:012
1702199461,33cyclictest0-21swapper/020:21:120
1702199461,33cyclictest0-21swapper/020:21:120
1702299457,26cyclictest0-21swapper/116:41:031
17021994525,4cyclictest8572-21fschecks_time17:36:080
1702199451,6cyclictest17390-21cut16:31:080
1702199451,35cyclictest0-21swapper/018:29:560
1702199451,29cyclictest0-21swapper/020:41:060
17023994411,20cyclictest11-21rcu_preempt18:36:022
1702199441,4cyclictest22755-21fschecks_time16:46:100
1702399434,14cyclictest0-21swapper/216:55:592
1702199432,3cyclictest0-21swapper/018:21:080
75712420,2sleep123-21ksoftirqd/120:01:501
1702399427,25cyclictest0-21swapper/217:51:042
1702299422,25cyclictest0-21swapper/121:01:171
17022994221,7cyclictest26486-21latency_hist20:25:521
17022994221,7cyclictest26486-21latency_hist20:25:521
17022994212,20cyclictest0-21swapper/118:56:021
17022994212,20cyclictest0-21swapper/118:56:021
120282420,1sleep20-21swapper/220:07:142
1702399417,25cyclictest0-21swapper/219:06:052
1702299419,13cyclictest30902-21ssh20:31:021
17022994127,5cyclictest22586-21ssh19:02:051
17022994019,6cyclictest221ktimersoftd/119:09:101
1702399397,12cyclictest7447-21ssh21:21:042
1702399392,11cyclictest7750irq/56-dwc_otg21:36:022
1702399391,6cyclictest11-21rcu_preempt19:56:012
1702399391,27cyclictest18638-21latency_hist16:35:512
17022993922,10cyclictest29259-21ssh18:31:161
1702399386,15cyclictest7950irq/56-dwc_otg_18:56:082
1702399386,15cyclictest7950irq/56-dwc_otg_18:56:082
1702399382,26cyclictest0-21swapper/218:06:042
1702299388,5cyclictest28785-21kworker/u8:019:21:191
1702299388,5cyclictest28785-21kworker/u8:019:21:191
1702299383,18cyclictest0-21swapper/121:43:381
1702299382,18cyclictest0-21swapper/117:16:051
17022993818,11cyclictest27650-21idleruntime-cro17:00:511
17022993812,19cyclictest0-21swapper/117:51:191
17022993811,12cyclictest21-21rcuc/119:51:211
270062370,2sleep20-21swapper/220:26:062
1702399376,4cyclictest7750irq/56-dwc_otg20:41:052
1702399373,8cyclictest7750irq/56-dwc_otg21:56:032
1702399372,20cyclictest0-21swapper/218:32:392
17023993712,5cyclictest5520-21unixbench-2d17:26:202
1702399371,25cyclictest27844-21ls21:06:092
1702299374,6cyclictest0-21swapper/119:11:031
17022993716,6cyclictest221ktimersoftd/120:13:491
17022993715,7cyclictest7950irq/56-dwc_otg_20:06:011
1702399368,5cyclictest9450irq/48-DMA09:12:252
1702399365,20cyclictest20872-21ssh19:01:052
1702399364,5cyclictest7950irq/56-dwc_otg_21:26:022
17023993612,8cyclictest8081-21idleruntime-cro18:45:512
1702399361,25cyclictest10679-21ls17:41:102
1702299368,24cyclictest0-21swapper/116:51:021
1702299363,26cyclictest7950irq/56-dwc_otg_21:11:021
1702299362,12cyclictest7950irq/56-dwc_otg_21:36:051
17022993611,4cyclictest0-21swapper/122:00:511
1702399354,15cyclictest7850irq/56-dwc_otg_20:21:072
1702399354,15cyclictest7850irq/56-dwc_otg_20:21:072
1702399353,5cyclictest7950irq/56-dwc_otg_21:01:042
17023993521,8cyclictest7950irq/56-dwc_otg_20:56:092
1702399351,24cyclictest19091-21ls18:05:512
17023993510,3cyclictest0-21swapper/220:36:032
1702299357,16cyclictest0-21swapper/117:41:031
1702299353,13cyclictest7950irq/56-dwc_otg_19:31:091
17022993520,12cyclictest23-21ksoftirqd/120:56:041
1702299351,6cyclictest7850irq/56-dwc_otg_21:51:061
17022993515,5cyclictest6723-21/usr/sbin/munin20:41:171
1702299351,27cyclictest19081-21cut16:36:081
17022993512,11cyclictest98582chrt21:23:221
1702299351,20cyclictest27056-21uniq20:26:081
1702299351,13cyclictest7950irq/56-dwc_otg_21:16:091
1702299351,13cyclictest7950irq/56-dwc_otg_21:16:091
17022993510,23cyclictest0-21swapper/119:25:571
1702399349,14cyclictest0-21swapper/220:45:532
1702399347,15cyclictest0-21swapper/217:11:032
1702399346,5cyclictest7850irq/56-dwc_otg_19:40:512
17023993410,12cyclictest7750irq/56-dwc_otg21:11:132
1702299346,13cyclictest7850irq/56-dwc_otg_20:46:051
1702299343,4cyclictest7950irq/56-dwc_otg_21:06:351
17022993422,6cyclictest15282-21/usr/sbin/munin20:51:071
17022993417,5cyclictest21-21rcuc/118:36:101
17022993411,10cyclictest0-21swapper/121:46:031
17022993411,10cyclictest0-21swapper/121:46:031
1702399339,5cyclictest7950irq/56-dwc_otg_16:45:512
1702399332,4cyclictest7750irq/56-dwc_otg18:16:162
1702399331,6cyclictest3342-21wget21:16:052
1702399331,6cyclictest3342-21wget21:16:052
1702299339,10cyclictest0-21swapper/120:18:241
1702299337,5cyclictest7850irq/56-dwc_otg_19:40:511
1702299333,4cyclictest7950irq/56-dwc_otg_18:41:081
1702299331,5cyclictest21769-21sh19:41:031
1702299331,25cyclictest22707-21cut16:46:101
1702399328,3cyclictest0-21swapper/219:41:082
1702399322,8cyclictest7750irq/56-dwc_otg19:46:012
1702399322,6cyclictest4750irq/17-3f00b88019:27:202
17023993213,12cyclictest6426-21/usr/sbin/munin17:31:052
17023993212,11cyclictest29884-21/usr/sbin/munin17:06:052
1702399321,20cyclictest32703-21ls21:51:092
1702399321,12cyclictest7750irq/56-dwc_otg19:21:092
1702399321,12cyclictest7750irq/56-dwc_otg19:21:092
1702299329,4cyclictest23977-21grep18:16:201
1702299327,12cyclictest0-21swapper/119:57:491
1702299326,4cyclictest7950irq/56-dwc_otg_18:51:091
1702299324,14cyclictest7850irq/56-dwc_otg_19:16:071
17022993218,10cyclictest0-21swapper/118:21:191
17022993211,9cyclictest0-21swapper/121:29:501
17022993211,4cyclictest29668-21/usr/sbin/munin17:06:091
17022993210,4cyclictest27169-21unixbench_singl19:46:191
1702399319,4cyclictest13654-21latency_hist17:50:512
1702399317,21cyclictest0-21swapper/221:46:062
1702399317,21cyclictest0-21swapper/221:46:062
1702399316,5cyclictest7750irq/56-dwc_otg18:46:102
1702399314,6cyclictest7950irq/56-dwc_otg_21:31:082
17023993114,5cyclictest2989-21fschecks_time17:21:082
17023993114,4cyclictest22348-1kworker/u9:119:13:542
1702399311,27cyclictest18804-21cut20:16:082
17023993110,10cyclictest7950irq/56-dwc_otg_19:31:162
1702299314,12cyclictest7850irq/56-dwc_otg_16:31:121
1702299313,11cyclictest7950irq/56-dwc_otg_17:21:481
1702299312,13cyclictest7850irq/56-dwc_otg_18:11:191
17022993118,5cyclictest15596-21/usr/sbin/munin17:56:041
1702299311,16cyclictest16077-21cut21:31:081
1702399302,9cyclictest25760-21unixbench_singl18:21:192
17023993014,9cyclictest22546-21aten_rbpower_po16:46:072
17023993012,4cyclictest16180-21/usr/sbin/munin17:56:112
1702299304,6cyclictest7950irq/56-dwc_otg_18:26:041
17022993012,5cyclictest21-21rcuc/117:36:131
1702399298,13cyclictest7950irq/56-dwc_otg_18:11:052
1702399292,20cyclictest0-21swapper/217:38:092
17023992914,11cyclictest0-21swapper/216:51:162
1702399291,16cyclictest0-21swapper/218:25:562
1702299293,4cyclictest7850irq/56-dwc_otg_18:06:101
17022992915,5cyclictest31840-21idleruntime17:11:081
156042296,9sleep10-21swapper/116:26:111
1702299284,5cyclictest29013-21grep17:01:191
1702299271,3cyclictest7950irq/56-dwc_otg_17:26:131
164762273,16sleep30-21swapper/316:27:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional