You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 00:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat May 09, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26468998913,9cyclictest0-21swapper/017:51:070
228292840,6sleep12647099cyclictest17:50:561
2646899835,47cyclictest0-21swapper/020:06:080
2647099823,12cyclictest23193-21ssh20:11:061
133612820,2sleep213337-21ssh21:56:142
26468997915,9cyclictest0-21swapper/019:21:070
26468997915,9cyclictest0-21swapper/019:21:070
26468997722,25cyclictest0-21swapper/018:31:100
2647099763,7cyclictest0-21swapper/116:30:591
2646899732,38cyclictest0-21swapper/016:46:090
26470997219,36cyclictest3053-21sh21:04:531
26468997261,4cyclictest9-21ksoftirqd/020:11:160
148922700,1sleep10-21swapper/121:57:261
26468996935,5cyclictest0-21swapper/020:56:090
2646899687,22cyclictest0-21swapper/017:46:020
2646899681,23cyclictest0-21swapper/020:36:080
2647099677,42cyclictest0-21swapper/121:06:191
26468996739,10cyclictest0-21swapper/016:30:590
26468996733,10cyclictest6603-21ssh19:51:080
26470996616,42cyclictest7371-21ssh19:51:191
2646899662,38cyclictest3542-21ssh20:26:130
2646899662,38cyclictest3542-21ssh20:26:130
26470996519,41cyclictest3203-21wget16:56:101
2646899656,5cyclictest0-21swapper/019:46:090
2646899656,5cyclictest0-21swapper/019:46:090
26468996511,39cyclictest0-21swapper/020:41:080
130292650,5sleep22647199cyclictest20:36:512
2647099647,40cyclictest0-21swapper/121:24:551
26468996448,5cyclictest101ktimersoftd/021:36:090
292362630,1sleep30-21swapper/320:17:273
2646899638,40cyclictest0-21swapper/017:21:090
26468996328,16cyclictest0-21swapper/019:06:080
26468996311,31cyclictest0-21swapper/021:01:250
2647099629,6cyclictest22404-21ssh19:31:111
2647099622,3cyclictest7750irq/56-dwc_otg17:16:171
2647099622,3cyclictest7750irq/56-dwc_otg17:16:171
26470996216,30cyclictest24611-21idleruntime-cro17:55:561
26468996216,27cyclictest0-21swapper/017:06:090
26468996216,26cyclictest27800-21unixbench_singl19:36:240
2646899614,8cyclictest25859-21ssh18:56:100
26468996129,5cyclictest23613-21grep19:31:260
26468996117,20cyclictest0-21swapper/021:11:110
26470996013,11cyclictest7950irq/56-dwc_otg_18:41:081
2646899604,24cyclictest16518-21ssh21:21:100
2647099594,6cyclictest24106-21ssh20:51:121
2647099592,6cyclictest12331-21/usr/sbin/munin21:16:231
2647099592,5cyclictest8150-21apt-get21:11:031
2647099583,9cyclictest7850irq/56-dwc_otg_20:31:211
2647099583,9cyclictest7850irq/56-dwc_otg_20:08:361
2647099582,7cyclictest29301-21ssh21:36:131
26468995815,23cyclictest10556-21fschecks_count17:16:140
26468995815,23cyclictest10556-21fschecks_count17:16:140
2647099574,4cyclictest7750irq/56-dwc_otg17:10:571
2647099571,3cyclictest0-21swapper/120:41:501
2646899572,29cyclictest0-21swapper/021:26:240
2646899572,29cyclictest0-21swapper/021:26:240
2646899571,4cyclictest8912-21ssh21:51:080
2646899571,4cyclictest8912-21ssh21:51:080
26468995713,26cyclictest13381-21fschecks_time21:56:140
2646899571,26cyclictest25049-21ls17:56:130
89072560,2sleep10-21swapper/119:14:411
2647099567,5cyclictest0-21swapper/119:25:561
2647099567,5cyclictest0-21swapper/119:25:561
2647099561,7cyclictest7750irq/56-dwc_otg19:26:171
2647099561,5cyclictest1941-21sort19:06:131
2647099561,4cyclictest0-21swapper/120:04:521
2647099561,3cyclictest0-21swapper/121:28:251
2647099561,3cyclictest0-21swapper/121:28:251
26468995638,6cyclictest29649-21/usr/sbin/munin18:06:240
26468995633,3cyclictest0-21swapper/020:46:080
26468995627,14cyclictest7850irq/56-dwc_otg_20:01:120
261012563,13sleep30-21swapper/316:29:513
2647099556,4cyclictest0-21swapper/120:26:101
2647099556,4cyclictest0-21swapper/120:26:101
2647099551,11cyclictest7850irq/56-dwc_otg_19:00:481
2646899553,29cyclictest0-21swapper/016:51:250
26468995517,4cyclictest0-21swapper/019:01:090
2646899551,33cyclictest0-21swapper/021:16:080
2647099548,4cyclictest7750irq/56-dwc_otg18:46:251
2647099546,10cyclictest7950irq/56-dwc_otg_18:01:151
2647099542,6cyclictest31357-21dhclient-script16:42:081
2647099542,11cyclictest0-21swapper/119:17:581
2647099541,4cyclictest7750irq/56-dwc_otg20:36:411
2646899548,25cyclictest14224-21seq17:26:150
26468995427,14cyclictest1945-21grep18:21:160
26468995423,5cyclictest24218-21fschecks_count20:51:140
2646899542,27cyclictest30761-21wget19:41:100
26468995422,20cyclictest18457-21fschecks_count19:26:130
2646899541,38cyclictest0-21swapper/018:46:080
2646899541,26cyclictest0-21swapper/018:51:070
2647099539,5cyclictest10160-21wc17:15:571
2647099531,7cyclictest27620-21taskset21:34:121
2647099531,6cyclictest8377-21ssh21:50:361
2647099531,10cyclictest7750irq/56-dwc_otg17:56:191
26468995330,6cyclictest7850irq/56-dwc_otg_16:56:130
26468995327,8cyclictest7850irq/56-dwc_otg_16:40:560
2646899531,4cyclictest30429-21grep16:41:130
2646899531,30cyclictest8180-21proc_pri20:31:190
2647099528,3cyclictest0-21swapper/119:44:311
2647099527,4cyclictest0-21swapper/119:01:191
2647099523,5cyclictest3924-1kworker/u9:119:48:591
2647099523,5cyclictest3924-1kworker/u9:119:48:591
2647099521,9cyclictest0-21swapper/117:37:561
2647099521,4cyclictest0-21swapper/120:50:451
2647099521,4cyclictest0-21swapper/118:36:121
2647099521,10cyclictest20325-21grep17:41:231
2646899529,38cyclictest101ktimersoftd/017:11:070
2646899522,33cyclictest14665-21ssh18:41:250
26468995220,17cyclictest0-21swapper/018:11:090
26468995218,19cyclictest4816-21diskmemload20:00:570
26468995218,19cyclictest4816-21diskmemload20:00:570
2647199512,5cyclictest7850irq/56-dwc_otg_19:36:082
2647099515,9cyclictest7950irq/56-dwc_otg_20:25:201
2647099512,3cyclictest0-21swapper/119:36:221
2647099511,8cyclictest1543-21aten_rbpower_vo18:21:101
2647099511,6cyclictest0-21swapper/121:41:211
2647099511,12cyclictest0-21swapper/119:56:101
2647099511,12cyclictest0-21swapper/119:56:101
2647099511,10cyclictest22009-21fschecks_time18:51:131
26468995129,10cyclictest7909-21latency_hist21:10:570
26468995114,24cyclictest0-21swapper/018:26:110
26468995114,24cyclictest0-21swapper/018:26:110
2646899511,30cyclictest0-21swapper/020:21:070
2647099504,8cyclictest7950irq/56-dwc_otg_20:56:261
2647099502,7cyclictest7850irq/56-dwc_otg_17:29:591
2647099502,7cyclictest7750irq/56-dwc_otg18:31:041
26470995012,5cyclictest32067-21/usr/sbin/munin18:16:121
2647099501,11cyclictest7950irq/56-dwc_otg_18:28:021
2647099501,11cyclictest7950irq/56-dwc_otg_18:28:021
2646899505,27cyclictest13-21rcuc/018:38:410
2647099493,4cyclictest0-21swapper/117:32:551
2647099491,7cyclictest0-21swapper/121:53:041
2647099491,7cyclictest0-21swapper/121:53:041
2647099491,5cyclictest7850irq/56-dwc_otg_18:09:121
26470994914,4cyclictest29055-21sh20:17:071
2646899499,37cyclictest101ktimersoftd/021:41:160
2646899492,20cyclictest0-21swapper/021:50:560
258442493,11sleep00-21swapper/016:26:280
2647099481,4cyclictest7850irq/56-dwc_otg_17:24:431
2646899484,32cyclictest9-21ksoftirqd/019:13:190
26468994830,8cyclictest0-21swapper/020:16:420
26468994828,13cyclictest101ktimersoftd/019:20:040
2647099472,9cyclictest0-21swapper/117:00:591
2646899473,26cyclictest0-21swapper/017:36:180
26468994718,19cyclictest0-21swapper/018:20:440
2646899471,27cyclictest9-21ksoftirqd/018:01:200
2646899471,21cyclictest24742-21/usr/sbin/munin21:31:140
2647199462,13cyclictest0-21swapper/216:56:092
2646899467,23cyclictest0-21swapper/017:31:100
26471994523,6cyclictest17630-21aten_rbpower_en18:46:102
2647099451,7cyclictest25577-21systemd-journal16:55:201
2647299443,27cyclictest0-21swapper/316:30:593
2647199448,26cyclictest0-21swapper/219:01:112
2647199445,4cyclictest0-21swapper/221:21:102
2647099441,4cyclictest31503-21taskset18:11:511
2646899446,22cyclictest5243-21ls17:01:150
178412440,2sleep30-21swapper/320:43:043
2647299437,22cyclictest0-21swapper/321:01:023
2647299434,15cyclictest7950irq/56-dwc_otg_18:16:123
2647199438,16cyclictest7950irq/56-dwc_otg_21:10:572
26471994318,5cyclictest10652-21latency_hist19:55:572
2647099432,8cyclictest505-21rs:main1
2647099431,6cyclictest0-21swapper/116:50:041
2646899436,23cyclictest9-21ksoftirqd/017:45:160
289802420,2sleep30-21swapper/318:06:153
26472994218,5cyclictest5207-21fschecks_time17:01:143
2647199425,9cyclictest0-21swapper/220:01:062
2647299415,26cyclictest10728-21ssh19:56:023
2647299415,26cyclictest10728-21ssh19:56:023
2647199417,24cyclictest0-21swapper/219:28:142
26471994118,5cyclictest21111-21latency_hist17:45:572
231552410,5sleep22647199cyclictest19:31:212
67922400,2sleep30-21swapper/318:31:593
62682400,1sleep30-21swapper/321:08:123
2647299404,31cyclictest7950irq/56-dwc_otg_21:51:073
2647299404,31cyclictest7950irq/56-dwc_otg_21:51:073
26472994017,9cyclictest14424-21fschecks_time19:21:123
26472994017,9cyclictest14424-21fschecks_time19:21:123
2647199407,25cyclictest0-21swapper/217:26:112
2647199403,24cyclictest0-21swapper/216:30:592
2647299391,27cyclictest21595-21ls17:46:163
26472993911,13cyclictest7850irq/56-dwc_otg_19:00:563
26471993917,5cyclictest11494-21sh21:53:462
26471993917,5cyclictest11494-21sh21:53:462
2647199391,20cyclictest0-21swapper/219:06:222
2647199385,6cyclictest7950irq/56-dwc_otg_18:41:092
2647199384,6cyclictest0-21swapper/217:21:242
2647199381,19cyclictest7850irq/56-dwc_otg_17:16:112
2647199381,19cyclictest7850irq/56-dwc_otg_17:16:112
2647199375,10cyclictest7950irq/56-dwc_otg_18:56:092
2647199374,4cyclictest7750irq/56-dwc_otg19:21:232
2647199374,4cyclictest7750irq/56-dwc_otg19:21:232
26471993714,6cyclictest29966-21sed21:36:222
2647299366,13cyclictest7950irq/56-dwc_otg_19:16:243
2647299365,19cyclictest4773-21ssh21:46:093
2647299362,3cyclictest0-21swapper/317:21:083
2647299361,21cyclictest30396-21uniq16:41:123
2647199363,18cyclictest8764-21sort17:11:142
26471993624,8cyclictest301ktimersoftd/221:31:072
26471993618,11cyclictest27631-21fschecks_count20:16:132
26471993615,5cyclictest13087-21nfs4_client21:16:192
48652350,6sleep22647199cyclictest19:48:362
48652350,6sleep22647199cyclictest19:48:362
26472993511,7cyclictest9142-21sensors21:11:233
2647199358,6cyclictest7750irq/56-dwc_otg21:00:572
2647199355,4cyclictest3202-21ssh20:26:082
2647199355,4cyclictest3202-21ssh20:26:082
2647199353,7cyclictest7950irq/56-dwc_otg_19:45:572
26471993514,9cyclictest908-21runrttasks19:12:162
26471993510,5cyclictest7850irq/56-dwc_otg_18:30:562
26471993510,5cyclictest7850irq/56-dwc_otg_18:30:562
2647299346,6cyclictest7950irq/56-dwc_otg_20:36:143
2647299346,15cyclictest7950irq/56-dwc_otg_19:46:233
2647299346,15cyclictest7950irq/56-dwc_otg_19:46:233
2647299346,14cyclictest7950irq/56-dwc_otg_19:55:573
26472993415,5cyclictest908-21runrttasks20:09:553
2647299341,26cyclictest0-21swapper/320:22:053
2647199347,16cyclictest9198-21switchtime21:11:232
2647199342,22cyclictest0-21swapper/218:06:322
2647199342,14cyclictest7850irq/56-dwc_otg_18:36:232
231782341,3sleep30-21swapper/319:31:233
2647299338,12cyclictest0-21swapper/321:41:143
2647299337,14cyclictest0-21swapper/317:41:103
2647299334,14cyclictest7850irq/56-dwc_otg_18:46:093
26472993324,6cyclictest1037-21latency_hist16:50:563
2647299331,3cyclictest7950irq/56-dwc_otg_19:26:073
2647299331,22cyclictest0-21swapper/316:36:093
2647199339,6cyclictest9450irq/48-DMA09:12:252
2647199338,5cyclictest7950irq/56-dwc_otg_18:05:562
2647199336,5cyclictest0-21swapper/218:16:062
26471993328,3cyclictest301ktimersoftd/220:46:072
2647199331,24cyclictest0-21swapper/220:45:252
2647199331,23cyclictest957-21uniq21:41:112
2647299328,9cyclictest0-21swapper/321:56:003
2647299328,15cyclictest0-21swapper/319:04:143
2647299327,11cyclictest7950irq/56-dwc_otg_18:41:143
2647299325,6cyclictest7850irq/56-dwc_otg_16:51:143
2647299325,22cyclictest7950irq/56-dwc_otg_17:56:093
2647299324,13cyclictest7950irq/56-dwc_otg_19:36:143
2647299323,16cyclictest7850irq/56-dwc_otg_19:41:183
2647299321,4cyclictest7950irq/56-dwc_otg_18:36:083
26472993210,4cyclictest0-21swapper/321:16:093
2647199329,12cyclictest0-21swapper/219:17:072
2647199327,12cyclictest7850irq/56-dwc_otg_20:31:122
2647199326,11cyclictest7950irq/56-dwc_otg_20:00:562
2647199326,11cyclictest7950irq/56-dwc_otg_20:00:562
2647199322,5cyclictest0-21swapper/218:15:572
2647199321,6cyclictest22916-21latency_hist20:10:562
26471993211,9cyclictest4816-21diskmemload21:01:132
2647299318,11cyclictest7950irq/56-dwc_otg_18:25:573
2647299315,14cyclictest7950irq/56-dwc_otg_21:36:213
2647299315,11cyclictest7950irq/56-dwc_otg_19:11:173
2647299312,3cyclictest0-21swapper/320:56:063
26472993121,7cyclictest1570750ssh20:01:193
2647299311,7cyclictest23800-21ps17:51:223
26472993116,4cyclictest908-21runrttasks20:48:273
2647299311,4cyclictest7950irq/56-dwc_otg_21:21:073
2647299311,14cyclictest25787-21ls21:31:223
2647199315,12cyclictest7850irq/56-dwc_otg_21:46:102
2647199312,7cyclictest0-21swapper/218:21:082
26471993125,4cyclictest31538-21aten_rbpower_cu20:21:092
2647199312,3cyclictest7850irq/56-dwc_otg_18:31:092
2647199312,13cyclictest7950irq/56-dwc_otg_17:01:252
2647199311,8cyclictest21087-21diskmemload21:26:152
2647199311,8cyclictest21087-21diskmemload21:26:152
2647199311,4cyclictest24424-21unixbench_multi20:11:232
2647199311,21cyclictest29869-21latency_hist16:40:562
2647199311,13cyclictest7950irq/56-dwc_otg_16:55:562
2647299306,12cyclictest20650-21taskset21:26:093
2647299306,12cyclictest20650-21taskset21:26:093
2647299305,14cyclictest7950irq/56-dwc_otg_20:29:543
2647299305,14cyclictest7950irq/56-dwc_otg_20:29:543
2647299303,19cyclictest0-21swapper/317:07:083
2647299302,3cyclictest7340-21ssh20:31:063
2647299301,6cyclictest25577-21systemd-journal18:30:563
2647299301,6cyclictest25577-21systemd-journal18:30:563
2647299301,15cyclictest7750irq/56-dwc_otg19:06:093
2647299301,11cyclictest0-21swapper/320:52:063
2647199308,5cyclictest7750irq/56-dwc_otg20:51:202
2647199306,15cyclictest0-21swapper/217:06:092
2647199302,19cyclictest0-21swapper/216:42:532
2647199301,24cyclictest0-21swapper/218:51:102
257002305,10sleep10-21swapper/116:26:251
249442300,2sleep2301ktimersoftd/217:56:122
2647299291,24cyclictest0-21swapper/317:36:103
2647299291,18cyclictest23473-21cut20:11:123
2647199297,12cyclictest0-21swapper/217:34:202
2647199292,13cyclictest7950irq/56-dwc_otg_17:36:122
2647299284,6cyclictest7850irq/56-dwc_otg_18:01:233
2647299282,19cyclictest0-21swapper/318:15:403
2647299282,19cyclictest0-21swapper/317:17:073
2647299282,19cyclictest0-21swapper/317:17:073
2647299282,12cyclictest7950irq/56-dwc_otg_18:51:103
26472992817,7cyclictest15946-21sort17:31:133
2647199282,8cyclictest7850irq/56-dwc_otg_17:51:072
2647299271,15cyclictest0-21swapper/317:11:143
2647199274,5cyclictest7950irq/56-dwc_otg_16:46:222
259962273,15sleep20-21swapper/216:28:292
2647299264,4cyclictest7850irq/56-dwc_otg_17:26:243
2647299261,22cyclictest0-21swapper/316:56:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional