You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 20:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue May 05, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3217521030,2sleep30-21swapper/320:25:503
2541499944,59cyclictest0-21swapper/021:16:060
38032920,5sleep22541699cyclictest21:08:572
77272800,5sleep02541499cyclictest19:15:540
326512790,6sleep22541699cyclictest19:46:242
25414997951,5cyclictest7950irq/56-dwc_otg_21:11:070
25417997716,37cyclictest15809-21latency_hist20:05:553
126382770,2sleep20-21swapper/220:01:222
2541499748,14cyclictest0-21swapper/017:26:090
25414997446,14cyclictest0-21swapper/020:36:010
78542730,2sleep20-21swapper/217:11:092
2541599726,50cyclictest21758-21sh20:11:591
2541599707,45cyclictest17830-21ls20:46:211
25414996932,4cyclictest0-21swapper/019:26:060
2541499691,39cyclictest0-21swapper/021:41:050
25417996710,3cyclictest0-21swapper/318:54:083
2541499674,14cyclictest0-21swapper/016:51:050
2541499672,41cyclictest0-21swapper/018:21:070
2541799666,21cyclictest0-21swapper/318:06:073
2541599661,52cyclictest30913-1kworker/u9:019:06:121
2541499662,46cyclictest0-21swapper/019:06:060
2541799653,13cyclictest7950irq/56-dwc_otg_19:06:243
57372640,2sleep35740-21latency_hist17:05:553
2541599632,42cyclictest0-21swapper/121:29:171
25415996311,40cyclictest17805-21latency18:46:151
25415996311,40cyclictest17805-21latency18:46:151
2541499631,37cyclictest0-21swapper/017:36:000
2541599628,4cyclictest618-21apt-get21:06:001
2541599628,35cyclictest0-21swapper/121:32:521
2541499628,31cyclictest0-21swapper/019:16:070
2541499621,48cyclictest14790-21kworker/0:018:51:100
2541499621,34cyclictest0-21swapper/020:06:050
25414996213,26cyclictest29202-21aten_rbpower_cu16:41:080
25414996213,26cyclictest29202-21aten_rbpower_cu16:41:080
240532620,5sleep10-21swapper/116:26:131
2541599616,4cyclictest0-21swapper/120:10:411
2541599615,7cyclictest7950irq/56-dwc_otg_21:59:151
2541599612,11cyclictest13599-21ssh19:23:211
2541599611,46cyclictest0-21swapper/120:37:491
25415996111,35cyclictest30975-21cron18:15:541
2541599611,10cyclictest0-21swapper/119:57:281
25414996135,7cyclictest7850irq/56-dwc_otg_21:36:180
25414996135,7cyclictest7850irq/56-dwc_otg_21:36:180
2541499613,19cyclictest0-21swapper/018:11:050
2541499612,7cyclictest28932-21munin-run21:00:540
2541599607,37cyclictest0-21swapper/119:29:541
2541599606,4cyclictest7850irq/56-dwc_otg_21:42:131
2541599601,45cyclictest0-21swapper/117:11:051
2541499606,31cyclictest0-21swapper/017:46:080
2541499606,31cyclictest0-21swapper/017:46:080
2541499605,27cyclictest0-21swapper/018:41:060
2541799592,5cyclictest18016-21grep20:46:233
2541799591,4cyclictest22605-21iwlist17:51:133
2541799591,4cyclictest12745-21ssh21:20:103
2541599596,4cyclictest7750irq/56-dwc_otg20:23:201
2541599591,8cyclictest0-21swapper/120:17:441
2541599591,12cyclictest27580-21ssh21:36:591
2541599591,12cyclictest27580-21ssh21:36:591
2541599587,3cyclictest0-21swapper/119:39:541
2541599583,42cyclictest0-21swapper/119:46:341
2541599582,6cyclictest4410-21ssh18:31:051
2541599582,43cyclictest0-21swapper/120:26:201
2541599581,47cyclictest12756-21ssh21:20:111
2541499581,46cyclictest3177-21aten_rbpower_cu19:51:070
135052580,3sleep313495-21ssh18:41:153
2541799579,4cyclictest0-21swapper/319:14:113
2541799577,4cyclictest0-21swapper/319:05:163
2541799572,5cyclictest0-21swapper/321:56:073
25415995715,39cyclictest28938-21sh18:59:351
2541599571,48cyclictest0-21swapper/119:05:521
2541599571,44cyclictest0-21swapper/119:51:481
2541499575,32cyclictest0-21swapper/021:06:070
2541499571,29cyclictest25772-21/usr/sbin/munin18:56:230
2541499570,39cyclictest13841-21fschecks_time21:21:110
2541799565,4cyclictest7950irq/56-dwc_otg_18:27:193
2541799564,8cyclictest7950irq/56-dwc_otg_20:56:393
2541799561,4cyclictest7950irq/56-dwc_otg_17:56:203
2541599567,22cyclictest0-21swapper/118:06:071
2541599566,42cyclictest7950irq/56-dwc_otg_21:50:341
2541599565,8cyclictest7950irq/56-dwc_otg_18:43:371
2541599565,4cyclictest7750irq/56-dwc_otg18:21:071
2541599563,8cyclictest7950irq/56-dwc_otg_16:39:461
2541599563,43cyclictest5459-21ssh19:11:591
2541599562,43cyclictest0-21swapper/121:21:151
2541599562,11cyclictest7950irq/56-dwc_otg_20:34:591
2541599561,45cyclictest0-21swapper/116:47:281
2541599561,43cyclictest0-21swapper/121:53:421
2541599561,11cyclictest5203-21/usr/sbin/munin21:11:251
2541499566,21cyclictest0-21swapper/018:31:000
2541499562,9cyclictest0-21swapper/020:26:070
202782560,2sleep20-21swapper/217:45:552
2541799554,4cyclictest7950irq/56-dwc_otg_19:31:333
2541799553,4cyclictest7850irq/56-dwc_otg_17:50:543
2541799553,4cyclictest7850irq/56-dwc_otg_17:50:543
2541799551,8cyclictest7950irq/56-dwc_otg_20:06:183
2541799551,3cyclictest7950irq/56-dwc_otg_17:11:113
2541799551,14cyclictest0-21swapper/321:08:563
2541599552,45cyclictest14951-21ssh20:43:081
2541599552,3cyclictest7750irq/56-dwc_otg20:51:101
2541599551,9cyclictest0-21swapper/121:05:161
2541599551,9cyclictest0-21swapper/121:05:161
2541599551,9cyclictest0-21swapper/116:32:041
2541599551,9cyclictest0-21swapper/116:32:041
2541599551,44cyclictest0-21swapper/119:34:201
25414995524,16cyclictest19606-21cat17:41:220
2541499552,32cyclictest0-21swapper/020:21:050
2541499551,35cyclictest0-21swapper/021:51:070
2541499551,32cyclictest11365-21modprobe20:00:540
2541799549,3cyclictest0-21swapper/318:34:083
2541799544,6cyclictest8542-21wget20:36:063
2541799541,3cyclictest0-21swapper/320:41:433
25417995413,4cyclictest7512-21ssh20:34:563
2541799541,12cyclictest0-21swapper/320:28:543
2541799541,11cyclictest7750irq/56-dwc_otg19:25:543
2541599541,8cyclictest0-21swapper/117:56:151
2541599541,5cyclictest22427-21fschecks_count17:51:111
2541599541,4cyclictest9900-21ssh19:17:171
2541599541,45cyclictest0-21swapper/118:01:441
25415995414,30cyclictest908-21runrttasks18:35:561
2541599541,11cyclictest13930-21ssh20:02:581
2541499546,24cyclictest11721-21ssh20:01:080
25414995425,4cyclictest3976-21diskmemload20:15:540
25414995425,13cyclictest7850irq/56-dwc_otg_17:25:540
291672530,2sleep30-21swapper/318:59:593
2541799537,4cyclictest7950irq/56-dwc_otg_19:40:013
2541799537,4cyclictest7850irq/56-dwc_otg_19:26:123
2541799534,4cyclictest7750irq/56-dwc_otg17:16:073
2541799534,4cyclictest7750irq/56-dwc_otg17:16:073
2541799532,9cyclictest7950irq/56-dwc_otg_21:51:563
2541799531,9cyclictest7950irq/56-dwc_otg_18:40:553
2541799531,7cyclictest30925-21sh21:41:193
2541799531,6cyclictest606-21cut18:21:083
25417995316,4cyclictest29240-21/usr/sbin/munin21:01:113
25417995316,4cyclictest29240-21/usr/sbin/munin21:01:113
2541799531,2cyclictest0-21swapper/321:49:423
2541799531,15cyclictest0-21swapper/321:22:393
2541599533,8cyclictest7850irq/56-dwc_otg_18:54:251
2541599533,3cyclictest0-21swapper/117:16:041
2541599533,3cyclictest0-21swapper/117:16:041
25414995324,7cyclictest12709-21wget20:41:070
25414995320,14cyclictest7750irq/56-dwc_otg18:26:230
193932530,5sleep32541799cyclictest21:27:103
2541799528,4cyclictest0-21swapper/319:19:413
2541799521,9cyclictest7950irq/56-dwc_otg_20:00:103
2541799521,5cyclictest23139-21ssh20:14:303
2541799521,3cyclictest7850irq/56-dwc_otg_21:31:053
2541799521,3cyclictest0-21swapper/321:36:113
2541799521,3cyclictest0-21swapper/321:36:113
2541799521,11cyclictest0-21swapper/319:45:423
2541799521,11cyclictest0-21swapper/319:45:423
2541599523,37cyclictest0-21swapper/119:45:391
2541599523,37cyclictest0-21swapper/119:45:391
25414995230,7cyclictest7950irq/56-dwc_otg_19:45:550
25414995230,7cyclictest7950irq/56-dwc_otg_19:45:550
2541499522,35cyclictest2794-21modprobe19:50:530
2541499521,13cyclictest26051-21uniq18:01:130
2541799518,5cyclictest0-21swapper/318:05:543
2541799511,4cyclictest3122-21/usr/sbin/munin19:51:223
25417995111,7cyclictest26992-21cron16:35:533
25417995111,7cyclictest26992-21cron16:35:533
2541599512,9cyclictest0-21swapper/117:37:231
2541599511,42cyclictest0-21swapper/117:50:511
2541599511,42cyclictest0-21swapper/117:50:511
2541599511,38cyclictest0-21swapper/116:42:001
2541599511,38cyclictest0-21swapper/116:42:001
2541499515,25cyclictest0-21swapper/020:51:060
2541499514,29cyclictest0-21swapper/020:16:120
2541499513,23cyclictest0-21swapper/017:16:080
2541499513,23cyclictest0-21swapper/017:16:080
2541499513,21cyclictest0-21swapper/019:36:020
25414995128,5cyclictest7950irq/56-dwc_otg_21:26:140
2541799509,4cyclictest0-21swapper/318:49:523
2541799509,4cyclictest0-21swapper/318:49:523
2541799501,9cyclictest7950irq/56-dwc_otg_20:51:413
2541799501,8cyclictest8105-21ssh21:14:153
25415995018,28cyclictest14552-21seq17:30:151
25415995011,35cyclictest5136-21grep17:01:211
2541599500,12cyclictest7950irq/56-dwc_otg_20:57:411
2541499509,26cyclictest0-21swapper/016:56:060
250422503,11sleep00-21swapper/016:29:370
2541799491,9cyclictest7950irq/56-dwc_otg_19:45:553
2541599492,7cyclictest2672-21fschecks_count18:26:121
2541599492,11cyclictest0-21swapper/117:31:141
2541499492,30cyclictest27910-21fschecks_time18:06:110
2541499492,26cyclictest0-21swapper/020:46:110
25414994918,17cyclictest4634-21ssh21:49:090
2541499491,28cyclictest908-21runrttasks18:16:200
2541499491,28cyclictest4326-21aten_rbpower_en20:31:080
2541499491,25cyclictest22180-21/usr/sbin/munin17:51:050
2541499491,24cyclictest2248750ssh21:31:160
2541799484,9cyclictest7950irq/56-dwc_otg_16:56:203
2541799481,3cyclictest0-21swapper/318:16:113
2541699488,25cyclictest0-21swapper/216:46:092
2541599482,6cyclictest20216-21latency_hist17:45:551
2541599482,6cyclictest11504-21fschecks_count17:21:111
25414994827,6cyclictest7950irq/56-dwc_otg_18:36:200
25414994821,6cyclictest7850irq/56-dwc_otg_21:56:210
2541499481,7cyclictest17574-21cut18:46:130
2541499481,7cyclictest17574-21cut18:46:130
2541499481,13cyclictest0-21swapper/017:10:550
2541599473,8cyclictest7750irq/56-dwc_otg16:52:211
2541599471,39cyclictest0-21swapper/117:00:131
2541499472,33cyclictest0-21swapper/021:02:570
2541499472,33cyclictest0-21swapper/021:02:570
2541499471,31cyclictest61-21kswapd019:05:540
25414994711,19cyclictest15081-21/usr/sbin/munin17:31:110
2541799462,6cyclictest908-21runrttasks16:48:113
2541799461,4cyclictest6013-21cpuspeed17:06:093
2541799461,3cyclictest0-21swapper/320:18:303
2541799461,2cyclictest0-21swapper/317:31:053
2541499467,25cyclictest0-21swapper/016:36:080
2541499452,28cyclictest0-21swapper/019:34:200
25414994515,14cyclictest0-21swapper/019:21:140
2541499451,29cyclictest0-21swapper/016:31:140
2541499451,29cyclictest0-21swapper/016:31:140
2541599447,23cyclictest0-21swapper/117:06:081
25415994410,31cyclictest32371-21grep18:16:221
25414994424,4cyclictest7850irq/56-dwc_otg_16:46:210
74632430,4sleep2301ktimersoftd/221:51:452
2541799431,3cyclictest7950irq/56-dwc_otg_17:36:053
2541499437,19cyclictest0-21swapper/017:56:170
2541699424,11cyclictest0-21swapper/217:26:082
2541499423,24cyclictest0-21swapper/017:11:190
25414994211,22cyclictest0-21swapper/017:03:050
245432410,2sleep20-21swapper/219:37:102
2541699401,29cyclictest7750irq/56-dwc_otg21:45:282
293442390,19sleep27950irq/56-dwc_otg_18:10:552
25417993812,5cyclictest19182-21missed_timers17:41:163
2541799371,13cyclictest0-21swapper/316:38:293
2541699371,4cyclictest7850irq/56-dwc_otg_16:41:052
2541699371,4cyclictest7850irq/56-dwc_otg_16:41:052
25416993710,19cyclictest0-21swapper/218:52:572
238772377,13sleep30-21swapper/316:26:123
2541699367,4cyclictest0-21swapper/219:06:082
2541699366,16cyclictest17233-21unixbench_singl20:06:212
2541699361,5cyclictest0-21swapper/217:31:072
25416993611,14cyclictest7950irq/56-dwc_otg_21:21:072
2541699352,10cyclictest0-21swapper/219:03:462
2541799341,9cyclictest0-21swapper/317:29:463
2541699347,14cyclictest7950irq/56-dwc_otg_21:26:082
2541699344,15cyclictest7950irq/56-dwc_otg_21:31:112
2541699341,18cyclictest2033450ssh20:11:092
25416993410,5cyclictest0-21swapper/219:55:282
105932340,4sleep229-21rcuc/219:18:302
2541699335,5cyclictest7750irq/56-dwc_otg18:36:152
2541699334,14cyclictest8469-21/usr/sbin/munin20:36:062
2541699333,5cyclictest7950irq/56-dwc_otg_16:56:102
2541699333,3cyclictest7950irq/56-dwc_otg_20:56:172
2541699333,18cyclictest0-21swapper/220:52:252
2541699333,14cyclictest7850irq/56-dwc_otg_21:05:552
2541699333,14cyclictest7850irq/56-dwc_otg_21:05:552
25416993313,5cyclictest29-21rcuc/220:21:202
2541699331,20cyclictest31172-21ls18:15:552
156722330,4sleep231-21ksoftirqd/218:43:472
2541799327,21cyclictest0-21swapper/318:11:083
2541699328,5cyclictest29-21rcuc/218:50:502
2541699328,5cyclictest29-21rcuc/218:50:502
2541699322,6cyclictest20387-21/usr/sbin/munin17:46:052
2541699322,6cyclictest20387-21/usr/sbin/munin17:46:052
25416993218,4cyclictest13611-21sh22:00:252
25416993217,7cyclictest7850irq/56-dwc_otg_18:56:152
25416993214,4cyclictest7850irq/56-dwc_otg_21:38:032
25416993214,4cyclictest7850irq/56-dwc_otg_21:38:032
2541699318,12cyclictest0-21swapper/220:46:282
2541699317,5cyclictest7850irq/56-dwc_otg_18:01:202
2541699312,14cyclictest7850irq/56-dwc_otg_18:31:142
2541699312,11cyclictest0-21swapper/221:11:072
2541699311,7cyclictest7388-21/usr/sbin/munin19:56:112
25416993116,5cyclictest2733350wget16:36:092
2541699311,5cyclictest25092-21sed17:56:202
25416993115,14cyclictest14992-21ssh20:43:132
2541699311,3cyclictest0-21swapper/221:50:492
2541699311,22cyclictest1445-21ls18:21:202
25416993111,3cyclictest0-21swapper/219:28:302
25417993013,4cyclictest908-21runrttasks16:51:493
2541799301,12cyclictest7950irq/56-dwc_otg_17:21:173
2541699309,4cyclictest0-21swapper/221:16:092
2541699306,10cyclictest7950irq/56-dwc_otg_16:31:202
2541699306,10cyclictest7950irq/56-dwc_otg_16:31:202
2541699305,7cyclictest7850irq/56-dwc_otg_20:26:072
2541699304,11cyclictest7950irq/56-dwc_otg_17:10:542
2541699303,9cyclictest7850irq/56-dwc_otg_17:03:182
2541699302,6cyclictest4289-21aten_rbpower_cu20:31:072
2541699302,18cyclictest0-21swapper/220:16:212
25416993018,10cyclictest7950irq/56-dwc_otg_19:41:192
25416993018,10cyclictest7950irq/56-dwc_otg_19:41:192
25416993016,10cyclictest0-21swapper/217:21:222
25416993010,4cyclictest0-21swapper/219:33:262
2541699294,12cyclictest7950irq/56-dwc_otg_18:26:142
2541799281,10cyclictest29149-21runrttasks16:41:003
2541799281,10cyclictest29149-21runrttasks16:41:003
2541699287,12cyclictest0-21swapper/219:11:582
2541699286,6cyclictest7950irq/56-dwc_otg_17:16:132
2541699286,6cyclictest7950irq/56-dwc_otg_17:16:132
2541699283,11cyclictest7950irq/56-dwc_otg_19:21:592
2541699282,12cyclictest7950irq/56-dwc_otg_17:35:552
2541699274,4cyclictest31826-21wc18:16:152
250542273,15sleep20-21swapper/216:29:472
2541699262,5cyclictest7850irq/56-dwc_otg_17:52:032
2541699257,15cyclictest0-21swapper/216:50:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional