You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 14:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat May 02, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1310421300,2sleep113105-21/usr/sbin/munin20:36:081
1310421300,2sleep113105-21/usr/sbin/munin20:36:081
249089910535,36cyclictest0-21swapper/017:56:060
325872840,2sleep00-21swapper/021:38:230
325872840,2sleep00-21swapper/021:38:230
127732720,2sleep10-21swapper/119:19:181
290732710,1sleep00-21swapper/021:34:330
206492660,2sleep10-21swapper/121:24:151
2490899631,23cyclictest0-21swapper/016:56:070
2490899631,23cyclictest0-21swapper/016:56:070
116742610,5sleep02490899cyclictest21:13:070
24908995946,3cyclictest0-21swapper/021:21:170
24908995730,13cyclictest7850irq/56-dwc_otg_18:41:110
258842560,2sleep30-21swapper/321:30:533
2490899567,12cyclictest0-21swapper/019:45:530
24908995638,3cyclictest0-21swapper/022:00:530
24908995618,8cyclictest0-21swapper/020:50:520
104472560,1sleep20-21swapper/219:54:502
104472560,1sleep20-21swapper/219:54:502
24908995526,17cyclictest30805-21latency_hist19:40:520
2490899551,39cyclictest0-21swapper/018:16:030
24908995431,7cyclictest7850irq/56-dwc_otg_18:46:100
24908995425,3cyclictest0-21swapper/021:01:050
24908995337,5cyclictest7850irq/56-dwc_otg_21:00:530
24908995316,26cyclictest0-21swapper/018:26:120
2490899531,48cyclictest18130-21latency_hist19:25:530
246232532,44sleep00-21swapper/016:30:400
246232532,44sleep00-21swapper/016:30:400
24908995233,10cyclictest0-21swapper/017:36:100
24908995225,7cyclictest3780-21sort17:01:110
24908995225,5cyclictest26879-21aten_rbpower_cu16:36:060
2490899522,30cyclictest0-21swapper/020:31:080
24908995220,15cyclictest0-21swapper/017:31:110
24908995220,15cyclictest0-21swapper/017:31:110
2490899521,40cyclictest0-21swapper/016:55:540
2490899521,30cyclictest7282-21grep21:46:220
2490899521,25cyclictest25374-21timerwakeupswit20:11:200
24908995139,4cyclictest29108-21modprobe19:00:530
24908995130,4cyclictest28207-21grep18:06:200
24908995127,6cyclictest0-21swapper/020:06:050
24908995123,17cyclictest28870-21taskset20:55:270
24908995119,15cyclictest7750irq/56-dwc_otg19:06:040
2490899511,34cyclictest0-21swapper/019:30:530
24908995039,3cyclictest0-21swapper/018:36:080
24908995028,12cyclictest0-21swapper/017:25:530
24908995027,8cyclictest7850irq/56-dwc_otg_20:26:090
24908995020,16cyclictest30457-21ls18:15:530
24908994933,5cyclictest13918-21idleruntime-cro21:55:520
24908994930,6cyclictest7950irq/56-dwc_otg_21:06:080
24908994930,6cyclictest7950irq/56-dwc_otg_21:06:080
24908994928,6cyclictest29544-21aten_rbpower_po19:01:060
24908994923,6cyclictest8726-21idleruntime-cro17:15:520
2490899492,22cyclictest2712-21proc_pri21:41:170
24908994920,14cyclictest0-21swapper/018:31:190
24908994910,32cyclictest0-21swapper/018:51:070
261752480,4sleep32491499cyclictest20:12:223
2490899487,21cyclictest0-21swapper/021:15:540
24908994834,8cyclictest11421-21kworker/0:119:20:080
24908994828,4cyclictest61-21kswapd019:55:590
24908994827,5cyclictest19620-21ls17:45:520
24908994827,4cyclictest0-21swapper/017:30:530
24908994825,5cyclictest23111-21grep21:26:210
24908994824,9cyclictest16014-21fschecks_count20:01:080
24908994823,9cyclictest340-21ssh20:21:070
24908994815,26cyclictest0-21swapper/020:36:210
24908994815,26cyclictest0-21swapper/020:36:210
2490899481,13cyclictest101ktimersoftd/017:06:110
2490899471,6cyclictest9579-21ssh19:15:530
2490899471,34cyclictest0-21swapper/019:50:240
2490899471,29cyclictest0-21swapper/019:31:290
2490899471,29cyclictest0-21swapper/019:31:290
2490899471,14cyclictest7786-21/usr/sbin/munin19:51:140
2490899471,14cyclictest7786-21/usr/sbin/munin19:51:140
24908994711,23cyclictest9-21ksoftirqd/020:41:210
24914994623,5cyclictest29102-21sh19:00:523
2491399467,24cyclictest0-21swapper/217:05:572
2491399462,21cyclictest0-21swapper/217:26:072
2490899461,8cyclictest0-21swapper/017:16:180
2491299456,5cyclictest7850irq/56-dwc_otg_17:11:201
2490899451,25cyclictest11-21rcu_preempt17:48:320
2490899441,25cyclictest0-21swapper/020:16:460
24914994314,6cyclictest10578-21ls17:20:523
2491399437,26cyclictest0-21swapper/218:40:592
24913994317,5cyclictest303450wget19:46:082
2491399431,15cyclictest23797-21fschecks_time17:56:102
2491299437,5cyclictest7750irq/56-dwc_otg18:41:491
24912994320,4cyclictest27323-21fschecks_count18:06:091
2490899438,26cyclictest26575-21apt-config16:35:520
2490899432,24cyclictest0-21swapper/018:00:540
24908994318,15cyclictest0-21swapper/016:41:180
118712430,4sleep3381ktimersoftd/319:17:433
2491299424,28cyclictest7950irq/56-dwc_otg_16:56:201
2491299424,28cyclictest7950irq/56-dwc_otg_16:56:201
24912994218,5cyclictest30366-21sh21:36:081
24912994218,5cyclictest30366-21sh21:36:081
24908994230,7cyclictest101ktimersoftd/016:50:300
24908994221,4cyclictest7850irq/56-dwc_otg_17:51:160
205842420,3sleep337-21rcuc/320:45:283
24914994121,5cyclictest39-21ksoftirqd/321:01:093
2491499411,25cyclictest9450irq/48-DMA09:12:253
2491499411,25cyclictest9450irq/48-DMA09:12:253
2491399411,24cyclictest1161-21ssh21:01:072
24912994120,12cyclictest17493-21if_eth020:41:111
24908994123,6cyclictest7850irq/56-dwc_otg_18:21:170
24908994123,6cyclictest7850irq/56-dwc_otg_18:21:170
24914994024,6cyclictest16067-21ssh21:57:173
24914994015,15cyclictest37-21rcuc/319:25:353
2491399401,10cyclictest6581-21latency21:46:142
2491299405,7cyclictest5072-21ssh18:31:221
2491299403,4cyclictest7950irq/56-dwc_otg_19:36:071
2491299403,22cyclictest411-21ssh20:21:071
24912994022,5cyclictest29563-21timerandwakeup20:16:181
24912994020,7cyclictest7950irq/56-dwc_otg_20:11:151
231722400,1sleep30-21swapper/320:47:373
206632400,1sleep20-21swapper/219:28:132
141312400,5sleep27850irq/56-dwc_otg_20:36:212
141312400,5sleep27850irq/56-dwc_otg_20:36:212
260442390,1sleep20-21swapper/219:35:092
260442390,1sleep20-21swapper/219:35:092
2491399395,21cyclictest30705-1kworker/2:1H21:15:592
2491399391,28cyclictest7508-21ls17:11:102
2491399391,24cyclictest32439-21ls18:20:542
2491299399,4cyclictest0-21swapper/121:47:071
2491299392,13cyclictest0-21swapper/121:16:491
24912993917,5cyclictest8428-21grep19:51:221
24912993917,5cyclictest8428-21grep19:51:221
24912993914,6cyclictest25577-21systemd-journal18:25:531
24912993914,6cyclictest25577-21systemd-journal18:25:531
2491299391,3cyclictest7750irq/56-dwc_otg19:11:041
24913993818,12cyclictest1778450ssh18:46:212
24913993817,4cyclictest24608-21fschecks_time20:11:102
24913993815,16cyclictest0-21swapper/221:45:182
2491399381,4cyclictest0-21swapper/219:22:262
24913993810,15cyclictest0-21swapper/219:36:092
2491299388,4cyclictest0-21swapper/119:21:121
2491299383,18cyclictest0-21swapper/121:31:161
24912993813,8cyclictest29092-1kworker/u9:119:41:161
235922380,2sleep10-21swapper/120:48:241
127552380,2sleep30-21swapper/320:35:543
2491499379,5cyclictest7750irq/56-dwc_otg17:41:073
2491499373,15cyclictest7750irq/56-dwc_otg17:06:073
2491399375,19cyclictest26167-21ls18:01:182
24913993722,7cyclictest301ktimersoftd/217:36:002
2491299375,15cyclictest7950irq/56-dwc_otg_18:36:201
2491299374,6cyclictest25661-21fschecks_time20:51:101
2491299372,14cyclictest0-21swapper/120:28:521
24912993717,5cyclictest16758-21wget18:46:071
2491299371,5cyclictest1303-21aten_rbpower_cu19:06:061
2491299371,4cyclictest0-21swapper/121:41:251
24914993615,5cyclictest39-21ksoftirqd/320:55:523
24914993613,5cyclictest23927-21sh20:10:533
2491399366,20cyclictest26033-21ssh21:31:062
2491399364,5cyclictest7850irq/56-dwc_otg_20:41:172
24913993620,4cyclictest4893-21fschecks_count20:26:112
24913993617,8cyclictest520-21cron18:55:522
24913993613,4cyclictest22119-21irqstats17:51:122
2491399361,18cyclictest7950irq/56-dwc_otg_18:31:242
2491299369,4cyclictest0-21swapper/121:57:191
2491299368,4cyclictest0-21swapper/120:00:311
2491299362,13cyclictest23116-21grep21:26:211
2491299362,12cyclictest0-21swapper/120:06:271
24912993619,5cyclictest23-21ksoftirqd/117:51:151
24912993619,10cyclictest29090-21cron19:00:531
24912993618,5cyclictest32367-21wget16:51:071
24912993617,4cyclictest2145-21fschecks_count18:26:111
2491299361,5cyclictest0-21swapper/121:12:221
2491299361,3cyclictest0-21swapper/117:30:541
24912993612,3cyclictest0-21swapper/119:46:171
182652360,1sleep20-21swapper/217:41:092
313232350,6sleep12491299cyclictest18:16:141
2491499353,25cyclictest0-21swapper/321:51:013
24914993530,3cyclictest381ktimersoftd/318:16:043
24914993514,5cyclictest27329-21fschecks_count18:06:093
24914993510,3cyclictest7950irq/56-dwc_otg_21:31:043
2491399358,13cyclictest0-21swapper/220:16:152
2491399356,8cyclictest0-21swapper/217:21:102
2491399356,19cyclictest7950irq/56-dwc_otg_20:46:062
2491399356,10cyclictest22191-21sort21:26:102
2491399351,9cyclictest0-21swapper/219:02:142
2491399351,5cyclictest7750irq/56-dwc_otg20:35:532
2491399351,22cyclictest12051-21apt-config18:40:532
2491399351,13cyclictest10103-21ssh21:11:142
2491299356,13cyclictest7750irq/56-dwc_otg20:01:151
2491299353,11cyclictest7750irq/56-dwc_otg17:09:311
2491299352,6cyclictest11972-21grep17:21:211
2491299351,13cyclictest0-21swapper/121:04:371
2491499345,13cyclictest39-21ksoftirqd/319:06:123
2491499344,17cyclictest39-21ksoftirqd/318:41:193
2491499343,4cyclictest11787-21rm21:13:203
2491499342,19cyclictest0-21swapper/319:03:463
24914993419,4cyclictest7950irq/56-dwc_otg_20:37:353
24914993419,4cyclictest7950irq/56-dwc_otg_20:37:353
24914993414,4cyclictest9958-21sh18:37:233
2491399344,23cyclictest0-21swapper/220:51:222
2491399344,19cyclictest0-21swapper/219:09:452
2491399342,12cyclictest376-21wget20:21:072
24913993416,9cyclictest301ktimersoftd/216:57:012
24913993416,9cyclictest301ktimersoftd/216:57:012
24913993415,5cyclictest12088-21sh21:52:312
2491299348,10cyclictest7950irq/56-dwc_otg_18:11:171
2491299347,4cyclictest0-21swapper/120:56:041
2491299343,9cyclictest7750irq/56-dwc_otg20:32:431
24912993418,5cyclictest23746-21fschecks_time17:56:101
2491299341,3cyclictest7750irq/56-dwc_otg17:41:061
2491299341,10cyclictest22601-21ssh18:51:581
170362340,4sleep317040-21fschecks_time18:46:113
2491499339,4cyclictest7750irq/56-dwc_otg21:20:533
2491499339,3cyclictest0-21swapper/318:53:323
2491499338,11cyclictest0-21swapper/318:31:053
2491499337,12cyclictest7950irq/56-dwc_otg_16:40:523
2491499336,12cyclictest7850irq/56-dwc_otg_16:31:203
2491499333,14cyclictest7950irq/56-dwc_otg_20:56:103
2491499333,12cyclictest7850irq/56-dwc_otg_21:21:153
24914993317,6cyclictest39-21ksoftirqd/321:45:523
2491399336,5cyclictest413-21ls16:51:152
2491399332,15cyclictest0-21swapper/221:07:532
2491399332,15cyclictest0-21swapper/221:07:532
24913993319,4cyclictest30733-21uptime20:56:212
2491399331,6cyclictest17828-21ssh21:21:072
2491399331,5cyclictest29527-21grep16:41:192
2491399331,5cyclictest20521-21ntp_kernel_err17:46:152
2491399331,4cyclictest0-21swapper/218:30:532
2491399331,11cyclictest1733-21latency_hist18:25:532
2491399331,11cyclictest1733-21latency_hist18:25:532
24913993310,6cyclictest31-21ksoftirqd/219:00:532
2491299335,10cyclictest7750irq/56-dwc_otg21:52:391
2491299333,3cyclictest0-21swapper/119:34:411
2491299333,3cyclictest0-21swapper/119:34:411
2491299333,23cyclictest7950irq/56-dwc_otg_17:46:061
2491499329,6cyclictest22758-21grep17:51:203
2491499327,15cyclictest7950irq/56-dwc_otg_20:00:393
2491499327,12cyclictest7850irq/56-dwc_otg_18:11:173
2491499324,7cyclictest7950irq/56-dwc_otg_19:31:103
2491499324,7cyclictest7950irq/56-dwc_otg_19:31:103
2491499322,7cyclictest8224-21sed17:11:203
24914993216,4cyclictest37-21rcuc/318:26:293
24914993214,6cyclictest3441-21diskmemload21:48:393
24914993212,4cyclictest0-21swapper/318:01:203
24914993211,8cyclictest7950irq/56-dwc_otg_17:01:103
2491399325,15cyclictest0-21swapper/216:36:062
2491399324,4cyclictest7950irq/56-dwc_otg_20:10:532
2491399322,16cyclictest0-21swapper/219:56:352
24913993217,7cyclictest7750irq/56-dwc_otg19:41:012
2491399321,4cyclictest0-21swapper/221:40:002
2491399321,4cyclictest0-21swapper/221:40:002
2491399321,25cyclictest10751-21ls19:16:182
2491299326,4cyclictest7950irq/56-dwc_otg_19:01:091
2491299325,5cyclictest7950irq/56-dwc_otg_17:01:171
2491299324,8cyclictest7950irq/56-dwc_otg_16:48:281
2491299323,9cyclictest5608-21ssh21:06:111
2491299323,9cyclictest5608-21ssh21:06:111
24912993217,4cyclictest23-21ksoftirqd/117:40:521
2491499316,6cyclictest39-21ksoftirqd/319:39:303
24914993123,6cyclictest32011-21ssh21:37:223
24914993123,6cyclictest32011-21ssh21:37:223
24914993115,4cyclictest4342-21sh19:46:343
24914993113,9cyclictest37-21rcuc/319:14:463
24914993110,4cyclictest0-21swapper/320:19:053
2491399314,20cyclictest7950irq/56-dwc_otg_19:11:042
2491399311,4cyclictest7750irq/56-dwc_otg18:11:142
2491399311,3cyclictest7850irq/56-dwc_otg_20:01:042
24913993112,11cyclictest158312chrt17:34:042
24913993112,11cyclictest158312chrt17:34:042
2491299319,3cyclictest7750irq/56-dwc_otg19:26:061
2491299313,3cyclictest7850irq/56-dwc_otg_17:16:041
2491299313,17cyclictest0-21swapper/116:41:101
2491299313,13cyclictest14196-21kworker/u8:316:35:581
2491499306,5cyclictest7750irq/56-dwc_otg20:26:163
2491499301,5cyclictest16415-21wc20:01:123
2491499301,14cyclictest37-21rcuc/319:41:083
2491399307,8cyclictest7850irq/56-dwc_otg_21:56:152
2491399301,15cyclictest0-21swapper/216:31:172
2491299305,11cyclictest0-21swapper/117:32:491
2491299305,11cyclictest0-21swapper/117:32:491
2491299304,5cyclictest7750irq/56-dwc_otg18:05:521
238042307,13sleep30-21swapper/316:26:173
238042307,13sleep30-21swapper/316:26:173
2491499295,13cyclictest0-21swapper/319:30:303
2491499293,6cyclictest39-21ksoftirqd/320:22:093
24914992911,5cyclictest521-21taskset16:51:173
2491499291,12cyclictest32587-21aten_rbpower_vo18:21:073
2491499291,12cyclictest32587-21aten_rbpower_vo18:21:073
24914992910,8cyclictest7850irq/56-dwc_otg_21:08:143
24914992910,8cyclictest7850irq/56-dwc_otg_21:08:143
24914992910,4cyclictest37-21rcuc/316:58:093
24914992910,4cyclictest37-21rcuc/316:58:093
2491399296,14cyclictest0-21swapper/217:17:112
2491299292,4cyclictest26577-21latency_hist16:35:521
244482293,17sleep10-21swapper/116:28:221
244482293,17sleep10-21swapper/116:28:221
2491499283,13cyclictest37-21rcuc/317:35:343
2491499283,13cyclictest37-21rcuc/317:35:343
2491499281,24cyclictest12821-21ls17:26:103
2491399281,5cyclictest27416-21grep18:06:102
2491399281,14cyclictest0-21swapper/217:04:552
240182285,12sleep20-21swapper/216:26:202
240182285,12sleep20-21swapper/216:26:202
2491499276,5cyclictest7850irq/56-dwc_otg_17:36:173
2491499272,8cyclictest7950irq/56-dwc_otg_17:46:073
2491499271,4cyclictest0-21swapper/316:46:083
2491399275,8cyclictest7950irq/56-dwc_otg_16:48:282
2491499262,6cyclictest23962-21sort17:56:123
24914992621,3cyclictest0-21swapper/316:41:083
24914992611,5cyclictest37-21rcuc/317:24:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional