You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 23:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 27, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1593126590,7sleep00-21swapper/016:30:500
1663221060,4sleep013-21rcuc/019:36:030
198182740,2sleep20-21swapper/218:10:512
198182740,2sleep20-21swapper/218:10:512
122922740,2sleep10-21swapper/117:49:121
16201997215,14cyclictest0-21swapper/020:26:010
271982700,2sleep00-21swapper/019:09:340
16201997037,3cyclictest0-21swapper/018:26:020
1620199651,6cyclictest6513-21ssh21:21:010
203502640,2sleep10-21swapper/119:40:431
16201996431,2cyclictest0-21swapper/020:51:030
1620199622,31cyclictest0-21swapper/017:20:510
1620199618,28cyclictest9-21ksoftirqd/020:16:190
153102610,5sleep10-21swapper/116:26:171
175862600,1sleep10-21swapper/121:33:481
16201996016,26cyclictest9-21ksoftirqd/019:50:500
108132600,4sleep11620299cyclictest21:26:031
16201995939,5cyclictest3914-21latency18:41:110
16201995836,9cyclictest7950irq/56-dwc_otg_21:11:040
16201995831,5cyclictest2029-21head19:56:190
16201995827,10cyclictest0-21swapper/020:10:500
1620199581,9cyclictest21247-21sed19:01:190
45252570,2sleep00-21swapper/021:17:490
1620199571,39cyclictest10019-21timerandwakeup17:41:170
1620199571,29cyclictest29713-21apt-get20:30:560
16201995637,4cyclictest101ktimersoftd/021:26:070
1620199562,26cyclictest18743-21ls21:35:500
1620199561,8cyclictest22896-21sed18:16:190
271702550,2sleep30-21swapper/317:01:043
16204995511,20cyclictest7950irq/56-dwc_otg_18:46:023
1620199556,4cyclictest9-21ksoftirqd/017:56:070
16201995527,11cyclictest7950irq/56-dwc_otg_18:15:490
1620199552,50cyclictest0-21swapper/019:21:010
1620199552,34cyclictest0-21swapper/017:31:180
1620199552,25cyclictest9-21ksoftirqd/018:31:030
1620199551,51cyclictest16912-21cat18:56:180
1620199551,41cyclictest32427-21ssh18:36:150
1620199551,35cyclictest27754-21unixbench-2d21:06:200
16201995430,6cyclictest7850irq/56-dwc_otg_17:50:500
1620199541,43cyclictest27237-21kworker/0:121:49:580
15472540,2sleep20-21swapper/221:53:342
1620499534,8cyclictest7950irq/56-dwc_otg_21:16:023
16201995332,4cyclictest7850irq/56-dwc_otg_21:36:030
16201995324,6cyclictest101ktimersoftd/019:11:010
16201995320,18cyclictest9-21ksoftirqd/019:51:100
1620199531,28cyclictest7295-21wget17:36:040
254942520,1sleep20-21swapper/221:43:072
254942520,1sleep20-21swapper/221:43:072
1620499524,7cyclictest7950irq/56-dwc_otg_18:56:033
1620299526,17cyclictest7750irq/56-dwc_otg19:45:501
16201995216,21cyclictest0-21swapper/018:21:120
1620199521,25cyclictest2195350ssh19:41:180
1620199521,14cyclictest10857-21switchtime20:46:170
1620199521,14cyclictest10857-21switchtime20:46:170
1620299512,10cyclictest7750irq/56-dwc_otg19:11:021
1620199512,26cyclictest29393-21grep17:06:090
1620199511,4cyclictest0-21swapper/018:46:040
1620199511,25cyclictest26309-21apt-config21:05:510
1620499502,7cyclictest0-21swapper/320:16:033
16201995023,5cyclictest9188-21grep19:26:160
16201995023,5cyclictest9188-21grep19:26:160
1620199501,27cyclictest5867-21aten_rbpower_vo20:41:050
1620199501,27cyclictest5199-21cron17:30:500
1620199501,25cyclictest13567-21ssh19:31:170
1620499493,4cyclictest7750irq/56-dwc_otg19:26:023
1620499493,4cyclictest7750irq/56-dwc_otg19:26:023
1620399491,15cyclictest28885-21ls19:50:512
1620299491,7cyclictest25539-21ssh20:26:011
1620199493,20cyclictest9-21ksoftirqd/019:16:070
16201994919,15cyclictest9-21ksoftirqd/021:51:170
1620199491,8cyclictest13-21rcuc/017:51:060
1620199491,26cyclictest23175-21munin-node21:41:000
1620199491,26cyclictest23175-21munin-node21:41:000
16204994824,8cyclictest7950irq/56-dwc_otg_21:21:043
1620199481,29cyclictest13-21rcuc/017:00:560
1620299471,27cyclictest0-21swapper/116:31:171
1620199476,29cyclictest9-21ksoftirqd/016:50:490
1620199471,26cyclictest8643-21cron20:05:490
1620199471,20cyclictest18359-21fschecks_time16:36:070
1620399467,28cyclictest7950irq/56-dwc_otg_19:21:032
1620299468,25cyclictest0-21swapper/118:06:041
1620299468,25cyclictest0-21swapper/118:06:041
1620199465,31cyclictest13-21rcuc/016:33:330
1620199464,34cyclictest7526-21kworker/0:218:54:080
1620499452,3cyclictest0-21swapper/317:16:033
1620299457,25cyclictest0-21swapper/116:56:041
16201994524,11cyclictest19951-21ssh20:57:100
16201994523,4cyclictest14089-21cat20:11:160
1620199451,20cyclictest4395-21sed21:56:170
53822440,5sleep11620299cyclictest21:58:001
1620499447,24cyclictest0-21swapper/321:25:573
1620499440,12cyclictest18176-21ssh20:56:023
1620399447,24cyclictest20579-21ls18:11:142
1620299447,23cyclictest0-21swapper/117:31:031
1620299447,22cyclictest0-21swapper/117:55:551
16201994422,12cyclictest7850irq/56-dwc_otg_20:20:530
16201994422,12cyclictest7850irq/56-dwc_otg_20:20:530
16201994414,5cyclictest1565-21grep20:36:030
1620499437,24cyclictest0-21swapper/319:15:563
1620499437,24cyclictest0-21swapper/316:36:033
16204994311,19cyclictest28427-21kworker/u8:320:26:163
16202994323,4cyclictest908-21runrttasks17:41:081
16202994311,23cyclictest0-21swapper/120:11:031
1620199431,28cyclictest25939-21ntpq16:56:120
1620499427,26cyclictest0-21swapper/318:20:553
1620499423,6cyclictest7850irq/56-dwc_otg_21:31:013
1620499421,3cyclictest0-21swapper/318:50:553
1620299421,21cyclictest0-21swapper/120:51:031
1620199421,24cyclictest2515-21ntpq17:21:110
1620199421,22cyclictest0-21swapper/018:06:130
1620199421,22cyclictest0-21swapper/018:06:130
23362410,2sleep10-21swapper/119:56:501
1620499418,18cyclictest29761-21ssh21:48:233
1620499417,24cyclictest0-21swapper/316:51:023
16204994120,9cyclictest7950irq/56-dwc_otg_21:51:093
16204994110,16cyclictest13868-21ssh19:31:283
16203994119,7cyclictest2736050wget21:46:022
16202994124,7cyclictest1256050wget19:31:031
1620199411,22cyclictest23461-21acpi16:50:590
16201994111,5cyclictest13-21rcuc/016:42:260
16201994111,5cyclictest13-21rcuc/016:42:260
1620499407,23cyclictest0-21swapper/316:56:043
16204994020,4cyclictest7950irq/56-dwc_otg_17:46:023
16204994019,9cyclictest7950irq/56-dwc_otg_21:55:593
16204994013,14cyclictest381ktimersoftd/320:53:143
1620499401,19cyclictest27432-21apt-get18:31:023
16203994011,5cyclictest26831-21latency_hist17:00:502
1620399401,11cyclictest0-21swapper/217:36:022
1620399400,35cyclictest7850irq/56-dwc_otg_21:26:232
16202994020,5cyclictest5622-21sh20:40:521
16201994017,5cyclictest31002-21cut17:11:070
16201994013,14cyclictest17545-21grep18:01:190
1620499398,19cyclictest24733-21ssh20:24:583
1620499398,19cyclictest24733-21ssh20:24:583
1620499396,17cyclictest7950irq/56-dwc_otg_21:11:043
1620499392,8cyclictest12685-21apt-get17:51:013
16204993919,8cyclictest7750irq/56-dwc_otg19:01:143
16204993913,14cyclictest7950irq/56-dwc_otg_17:56:203
16204993912,13cyclictest7950irq/56-dwc_otg_18:41:553
16204993910,10cyclictest7950irq/56-dwc_otg_21:01:153
1620399397,23cyclictest19981-21ssh19:01:042
1620399391,9cyclictest7950irq/56-dwc_otg_21:06:372
1620299391,28cyclictest22345-21taskset20:21:171
1620299391,28cyclictest22345-21taskset20:21:171
300402380,13sleep339-21ksoftirqd/320:31:063
248112380,1sleep10-21swapper/121:03:291
1620499388,9cyclictest7850irq/56-dwc_otg_21:43:133
1620499388,9cyclictest7850irq/56-dwc_otg_21:43:133
1620499387,18cyclictest6946-21ssh20:02:433
1620499386,18cyclictest7950irq/56-dwc_otg_19:51:043
1620499386,14cyclictest7950irq/56-dwc_otg_19:45:233
1620499383,19cyclictest19861-21taskset19:39:503
1620499383,14cyclictest7950irq/56-dwc_otg_18:11:043
16204993822,5cyclictest5337-21ssh20:40:503
16204993820,8cyclictest7850irq/56-dwc_otg_18:39:313
16204993820,7cyclictest7750irq/56-dwc_otg21:37:403
1620399389,15cyclictest18487-21/usr/sbin/munin16:36:092
1620399387,21cyclictest0-21swapper/218:16:052
1620399385,22cyclictest20172-21ls21:36:172
16203993821,14cyclictest11-21rcu_preempt17:41:022
1620299389,5cyclictest17327-21ls18:01:181
1620299388,6cyclictest7750irq/56-dwc_otg19:46:111
1620299382,23cyclictest26600-21kworker/1:119:51:001
1620299381,32cyclictest20288-21grep18:11:091
15402380,7sleep11620299cyclictest20:36:031
266342370,5sleep11620299cyclictest18:26:191
1620499374,30cyclictest5745-21ssh20:41:013
1620399372,31cyclictest7950irq/56-dwc_otg_21:56:012
1620399371,26cyclictest10527-21cat21:25:512
16202993716,7cyclictest11230-21cron18:50:491
16202993716,4cyclictest6873-21sh19:23:491
254062360,2sleep20-21swapper/221:04:342
1620499367,4cyclictest9875-21/usr/sbin/munin20:46:123
1620499367,4cyclictest9875-21/usr/sbin/munin20:46:123
1620499364,8cyclictest0-21swapper/319:11:013
16204993618,7cyclictest7950irq/56-dwc_otg_20:11:113
16204993611,12cyclictest0-21swapper/319:21:033
1620399369,12cyclictest7950irq/56-dwc_otg_18:46:032
1620399361,6cyclictest24026-21fschecks_time18:21:072
16203993615,4cyclictest7750irq/56-dwc_otg20:16:082
16202993612,4cyclictest0-21swapper/118:56:051
1620299361,22cyclictest0-21swapper/116:46:041
16204993520,5cyclictest381ktimersoftd/318:04:143
1620499351,4cyclictest7950irq/56-dwc_otg_17:31:013
1620499351,3cyclictest0-21swapper/317:21:013
16203993527,6cyclictest12800-21fschecks_time19:31:072
16203993522,5cyclictest3167950wget18:36:052
16203993512,7cyclictest12529-21processes18:51:142
1620399351,23cyclictest29970-21ls20:31:042
1620399351,16cyclictest26090-21latency20:26:102
1620299359,9cyclictest32479-21ssh19:16:011
1620299357,15cyclictest0-21swapper/117:36:041
1620299357,13cyclictest7850irq/56-dwc_otg_21:55:501
1620299355,20cyclictest7750irq/56-dwc_otg19:06:081
16202993521,4cyclictest7950irq/56-dwc_otg_21:05:561
1620299352,12cyclictest7950irq/56-dwc_otg_17:55:501
1620299351,5cyclictest28381-21ssh18:31:171
16202993514,5cyclictest662050wget21:21:041
1620299351,12cyclictest0-21swapper/116:36:011
1620499345,4cyclictest7850irq/56-dwc_otg_16:41:013
1620499345,4cyclictest7850irq/56-dwc_otg_16:41:013
1620499345,17cyclictest37-21rcuc/316:32:033
16204993415,4cyclictest7750irq/56-dwc_otg17:06:163
1620399344,13cyclictest7850irq/56-dwc_otg_17:16:062
1620399342,23cyclictest7750irq/56-dwc_otg19:15:572
16203993421,6cyclictest13134-21ssh20:11:012
1620399342,13cyclictest28251-21ssh19:11:022
1620399341,8cyclictest20767-21ssh19:41:032
16203993410,13cyclictest0-21swapper/221:18:032
16203993410,12cyclictest0-21swapper/219:56:062
1620299342,5cyclictest32476-21/usr/sbin/munin17:16:071
16202993418,9cyclictest28115-21seq17:01:171
16202993412,13cyclictest9055-21ssh20:06:041
1620499334,14cyclictest2246-21ssh19:56:403
16204993312,9cyclictest7950irq/56-dwc_otg_20:08:243
1620399337,19cyclictest7850irq/56-dwc_otg_21:31:022
1620399336,6cyclictest7950irq/56-dwc_otg_19:36:182
1620399335,19cyclictest7850irq/56-dwc_otg_17:26:112
1620399333,21cyclictest7750irq/56-dwc_otg16:54:052
1620399332,21cyclictest0-21swapper/221:00:502
1620399331,7cyclictest484750ssh18:41:412
16203993316,4cyclictest9127-21sh19:26:152
16203993316,4cyclictest9127-21sh19:26:152
16203993312,4cyclictest0-21swapper/220:36:062
1620399331,23cyclictest23227-21latency_hist16:50:502
16203993312,3cyclictest0-21swapper/219:06:192
1620399331,20cyclictest31326-21missed_timers21:11:112
1620399331,18cyclictest24992-21kworker/2:218:31:132
1620299337,15cyclictest0-21swapper/119:01:041
1620299336,17cyclictest0-21swapper/118:20:561
1620299334,15cyclictest0-21swapper/120:01:021
1620299332,7cyclictest31091-21cron21:50:491
16202993326,5cyclictest18194-21ssh20:56:031
16202993317,9cyclictest30957-21ssh20:31:171
1620299331,6cyclictest7850irq/56-dwc_otg_18:16:031
1620299331,22cyclictest6180-21latency_hist21:20:501
16202993310,4cyclictest0-21swapper/118:45:181
16204993217,4cyclictest24404-21fschecks_count19:06:083
1620499321,18cyclictest0-21swapper/321:05:593
1620399327,4cyclictest0-21swapper/220:46:112
1620399327,4cyclictest0-21swapper/220:46:112
1620399327,11cyclictest7850irq/56-dwc_otg_18:26:192
1620399326,18cyclictest7850irq/56-dwc_otg_17:08:452
16203993221,5cyclictest7750irq/56-dwc_otg17:56:072
1620399321,8cyclictest0-21swapper/218:57:292
16203993215,4cyclictest0-21swapper/220:06:072
1620399321,22cyclictest0-21swapper/220:55:292
1620399321,21cyclictest23518-21ssh20:22:442
1620399321,21cyclictest23518-21ssh20:22:442
1620299327,15cyclictest0-21swapper/121:36:031
1620299324,7cyclictest7750irq/56-dwc_otg20:16:141
1620299323,14cyclictest7850irq/56-dwc_otg_18:50:541
16202993217,5cyclictest922-21sh18:37:091
1620299321,3cyclictest0-21swapper/117:11:181
1620499315,14cyclictest39-21ksoftirqd/319:49:233
16204993110,5cyclictest0-21swapper/318:30:513
1620399313,19cyclictest7850irq/56-dwc_otg_20:41:192
16203993118,7cyclictest7750irq/56-dwc_otg17:31:092
16203993116,9cyclictest7750irq/56-dwc_otg17:55:352
1620299319,4cyclictest0-21swapper/119:26:221
1620299319,4cyclictest0-21swapper/119:26:221
1620299312,20cyclictest0-21swapper/116:45:021
1620299312,20cyclictest0-21swapper/116:45:021
16202993114,9cyclictest1836-21/usr/sbin/munin17:21:161
1620299311,20cyclictest26874-21rm21:45:391
1620299311,20cyclictest26874-21rm21:45:391
1620299311,20cyclictest0-21swapper/116:50:591
1620499307,14cyclictest0-21swapper/317:11:033
1620499306,13cyclictest7850irq/56-dwc_otg_18:17:553
16204993010,4cyclictest7850irq/56-dwc_otg_17:30:183
1620399306,13cyclictest7850irq/56-dwc_otg_19:51:112
1620299308,5cyclictest3399-1kworker/u9:220:46:041
1620299308,5cyclictest3399-1kworker/u9:220:46:041
1620299302,17cyclictest0-21swapper/117:08:081
150952304,10sleep20-21swapper/216:26:152
1620499295,13cyclictest7950irq/56-dwc_otg_17:43:223
1620499291,6cyclictest7950irq/56-dwc_otg_16:46:093
1620399298,14cyclictest121512chrt17:47:242
1620399293,4cyclictest0-21swapper/217:00:562
1620399293,11cyclictest7750irq/56-dwc_otg20:01:122
1620399291,21cyclictest0-21swapper/216:32:112
1620399291,18cyclictest17074-21ntpq18:01:132
16204992812,4cyclictest7950irq/56-dwc_otg_17:37:483
1620499281,15cyclictest19051-21ls18:06:163
1620499281,15cyclictest19051-21ls18:06:163
1620399283,11cyclictest7850irq/56-dwc_otg_17:21:112
1620399282,4cyclictest0-21swapper/216:42:462
1620399282,4cyclictest0-21swapper/216:42:462
1620299283,15cyclictest0-21swapper/117:26:021
1620299281,6cyclictest7750irq/56-dwc_otg21:11:081
159122284,15sleep30-21swapper/316:30:383
1620399271,17cyclictest0-21swapper/217:11:102
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional