You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 16:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 06, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271722660,2sleep337-21rcuc/316:25:573
283522430,3sleep10-21swapper/116:25:591
4188991056,67cyclictest0-21swapper/021:20:520
41889910457,18cyclictest7950irq/56-dwc_otg_20:10:530
229092870,3sleep3381ktimersoftd/321:50:553
136142800,4sleep0418899cyclictest18:25:560
418899786,25cyclictest0-21swapper/017:50:520
4188997618,11cyclictest0-21swapper/020:30:520
4188997618,11cyclictest0-21swapper/020:30:520
278552760,5sleep1418999cyclictest20:00:581
278552760,5sleep1418999cyclictest20:00:581
104582760,4sleep2419099cyclictest21:35:562
214212750,4sleep013-21rcuc/021:49:030
418899726,58cyclictest0-21swapper/020:05:460
418899701,33cyclictest0-21swapper/021:55:510
29332700,2sleep00-21swapper/017:55:570
211872700,1sleep20-21swapper/221:48:362
188392690,1sleep30-21swapper/320:28:063
418899685,46cyclictest9577-21cut20:55:530
419099651,11cyclictest0-21swapper/217:30:402
419099651,11cyclictest0-21swapper/217:30:402
418999652,10cyclictest7750irq/56-dwc_otg20:35:511
4188996533,4cyclictest16775-21latency_hist19:10:390
4188996512,13cyclictest0-21swapper/019:20:530
418899641,48cyclictest0-21swapper/021:06:050
419099622,6cyclictest7750irq/56-dwc_otg21:16:022
418899621,49cyclictest25370-21cat20:35:570
418899621,30cyclictest0-21swapper/018:00:510
419099616,10cyclictest7950irq/56-dwc_otg_18:46:012
419099613,9cyclictest61-21kswapd018:55:492
418899614,54cyclictest101ktimersoftd/016:31:060
4188996025,17cyclictest7850irq/56-dwc_otg_17:11:070
4188996016,5cyclictest0-21swapper/020:50:510
418899601,49cyclictest28151-21munin-node18:45:500
418899601,32cyclictest20449-21proc_pri18:36:030
4191995925,8cyclictest381ktimersoftd/317:45:503
419099596,4cyclictest0-21swapper/219:40:172
419099596,4cyclictest0-21swapper/219:40:172
419099592,13cyclictest0-21swapper/220:10:212
419099591,5cyclictest24209-21rm21:12:562
4190995911,5cyclictest26259-21taskset20:36:092
4189995921,10cyclictest7850irq/56-dwc_otg_21:10:511
4188995932,15cyclictest0-21swapper/020:41:070
418899591,7cyclictest11477-21grep21:36:080
4188995910,40cyclictest0-21swapper/020:15:530
419099581,15cyclictest10728-21ssh20:56:072
418899581,31cyclictest30018-21wget19:25:540
419099573,13cyclictest15867-21taskset21:41:282
419099571,17cyclictest0-21swapper/220:13:202
419099571,16cyclictest7950irq/56-dwc_otg_19:51:042
4188995728,5cyclictest101ktimersoftd/020:50:390
418899572,32cyclictest0-21swapper/019:00:500
4188995714,25cyclictest7298-21head21:31:070
4188995714,25cyclictest7298-21head21:31:070
418899571,30cyclictest19583-21ntp_kernel_pll_19:51:010
71042560,3sleep3381ktimersoftd/318:58:443
419099567,10cyclictest0-21swapper/219:22:582
419099563,4cyclictest7950irq/56-dwc_otg_17:35:562
419099561,16cyclictest0-21swapper/219:59:132
419099561,12cyclictest0-21swapper/219:02:482
419099561,10cyclictest0-21swapper/218:23:562
418899566,34cyclictest9-21ksoftirqd/018:20:570
4188995624,5cyclictest15406-21aten_rbpower_po18:30:510
418899561,27cyclictest21156-21latency_hist19:15:400
418899561,27cyclictest21156-21latency_hist19:15:400
419099551,23cyclictest22941-21cut21:50:552
4189995513,29cyclictest0-21swapper/118:10:511
4188995528,9cyclictest9875-21grep18:15:540
4188995528,15cyclictest101ktimersoftd/019:30:550
419099546,4cyclictest7750irq/56-dwc_otg17:10:542
4190995418,4cyclictest24522-21sh20:35:342
4190995418,4cyclictest24522-21sh20:35:342
419099541,6cyclictest7850irq/56-dwc_otg_18:54:152
419099541,14cyclictest0-21swapper/220:45:062
419099541,13cyclictest0-21swapper/220:30:392
4188995427,5cyclictest7850irq/56-dwc_otg_20:25:390
418899542,34cyclictest29477-21/usr/sbin/munin17:41:030
418899542,32cyclictest0-21swapper/021:26:080
4188995423,14cyclictest7850irq/56-dwc_otg_19:18:320
418899541,24cyclictest26040-21ls17:30:530
4190995317,5cyclictest17851-21aten_rbpower_po21:05:542
419099531,17cyclictest0-21swapper/218:16:022
419099531,14cyclictest0-21swapper/219:20:202
419099531,10cyclictest308472sleep220:04:542
419099531,10cyclictest308472sleep220:04:542
4190995310,3cyclictest0-21swapper/219:44:532
418899533,12cyclictest7850irq/56-dwc_otg_18:55:490
4188995329,10cyclictest17575-21grep20:26:070
418899532,39cyclictest14665-21aten_rbpower_vo19:45:530
418899532,33cyclictest0-21swapper/018:55:390
4188995313,22cyclictest16918-21aten_rbpower_po17:05:530
4188995235,8cyclictest101ktimersoftd/020:05:010
4188995235,8cyclictest101ktimersoftd/020:05:010
4188995233,9cyclictest27269-21grep21:16:080
4188995228,11cyclictest13728-21rm19:44:500
418899522,30cyclictest0-21swapper/021:42:550
418899522,29cyclictest0-21swapper/019:36:120
418899522,29cyclictest0-21swapper/019:36:120
419099512,3cyclictest0-21swapper/220:18:492
4189995120,10cyclictest7850irq/56-dwc_otg_19:10:511
4189995120,10cyclictest7850irq/56-dwc_otg_19:10:511
418899515,5cyclictest0-21swapper/018:40:520
418899515,5cyclictest0-21swapper/018:40:520
418899512,26cyclictest0-21swapper/017:20:590
4188995120,20cyclictest0-21swapper/016:56:040
185152510,35sleep30-21swapper/321:06:023
419099506,40cyclictest13840-21missed_timers16:56:002
419099504,6cyclictest7950irq/56-dwc_otg_19:31:072
419099501,11cyclictest0-21swapper/219:28:122
418899504,31cyclictest9-21ksoftirqd/019:56:080
4188995030,10cyclictest25396-21chrt17:27:560
4188995030,10cyclictest25396-21chrt17:27:560
4188995028,11cyclictest0-21swapper/021:04:480
4188995028,11cyclictest0-21swapper/021:04:480
4188995027,12cyclictest24761-21ssh21:52:330
419099491,6cyclictest0-21swapper/221:33:012
419099491,6cyclictest0-21swapper/221:33:012
418899494,28cyclictest28832-21head17:36:070
4188994922,5cyclictest6242-21wget18:05:530
418999484,8cyclictest0-21swapper/119:55:521
418899481,14cyclictest0-21swapper/016:49:470
38502483,10sleep00-21swapper/016:29:480
305892480,4sleep2301ktimersoftd/217:41:062
419199472,22cyclictest7750irq/56-dwc_otg19:00:493
419099471,14cyclictest0-21swapper/221:22:482
4189994718,12cyclictest14964-21diskmemload20:50:511
4188994719,6cyclictest23068-21unixbench-2d21:11:060
418899471,30cyclictest0-21swapper/016:50:540
4188994710,20cyclictest0-21swapper/016:36:070
418999467,25cyclictest0-21swapper/117:20:521
418899464,29cyclictest9-21ksoftirqd/017:04:190
4188994630,5cyclictest7850irq/56-dwc_otg_17:18:550
4188994627,9cyclictest32501-21chrt17:46:190
418899452,25cyclictest0-21swapper/018:11:110
419199447,25cyclictest0-21swapper/319:50:523
419099447,24cyclictest0-21swapper/216:30:522
419099441,15cyclictest0-21swapper/218:36:442
4190994410,4cyclictest14151-21ls21:00:582
4190994410,4cyclictest14151-21ls21:00:582
4189994427,14cyclictest23-21ksoftirqd/118:45:531
419199434,5cyclictest7950irq/56-dwc_otg_19:08:333
419199431,18cyclictest7950irq/56-dwc_otg_17:01:003
418999439,25cyclictest0-21swapper/116:40:511
418999427,25cyclictest0-21swapper/117:30:511
418999427,23cyclictest0-21swapper/118:00:441
212112420,5sleep2419099cyclictest17:16:022
83152410,32sleep10-21swapper/119:00:391
4190994119,6cyclictest301ktimersoftd/218:31:012
418999419,17cyclictest21-21rcuc/119:45:501
418999418,24cyclictest0-21swapper/118:40:521
418999418,24cyclictest0-21swapper/118:40:521
418999416,23cyclictest0-21swapper/117:05:521
4189994115,6cyclictest908-21runrttasks21:25:401
419099401,11cyclictest0-21swapper/219:08:172
418999406,27cyclictest0-21swapper/120:30:521
418999406,27cyclictest0-21swapper/120:30:521
301832400,2sleep10-21swapper/117:41:011
418999396,10cyclictest16387-21ssh20:25:501
4189993921,4cyclictest908-21runrttasks19:52:331
4188993916,15cyclictest0-21swapper/016:44:350
4191993818,5cyclictest15404-21aten_rbpower_po18:30:513
419099381,26cyclictest27531-21ls17:35:392
419099381,18cyclictest6077-21ssh20:51:012
418999387,25cyclictest0-21swapper/116:50:511
4189993817,5cyclictest12688-21sh19:05:401
4189993813,13cyclictest908-21runrttasks20:12:101
4189993812,7cyclictest14858-21unixbench_singl21:01:061
4189993812,7cyclictest14858-21unixbench_singl21:01:061
4191993721,6cyclictest10681-21unixbench_multi20:56:063
419099379,9cyclictest0-21swapper/218:10:422
419099374,21cyclictest11676-21grep16:50:552
4190993715,5cyclictest610750wget16:35:532
4190993714,5cyclictest7132-21timerandwakeup18:06:062
418999377,24cyclictest0-21swapper/118:05:531
418999377,24cyclictest0-21swapper/117:25:531
418999377,24cyclictest0-21swapper/117:25:531
418999377,24cyclictest0-21swapper/117:00:521
418999376,23cyclictest0-21swapper/120:40:521
418999371,18cyclictest0-21swapper/120:16:381
224692370,2sleep10-21swapper/118:38:491
13002370,4sleep2301ktimersoftd/220:45:562
419199366,15cyclictest7950irq/56-dwc_otg_21:40:593
4191993621,4cyclictest908-21runrttasks16:40:533
419199361,22cyclictest5758-21ssh19:35:403
4191993610,13cyclictest7950irq/56-dwc_otg_21:46:023
419099361,23cyclictest24641-21grep18:41:012
419099361,23cyclictest24641-21grep18:41:012
418999367,22cyclictest0-21swapper/118:25:531
418999367,22cyclictest0-21swapper/117:35:521
418999364,8cyclictest7750irq/56-dwc_otg21:55:521
418999364,11cyclictest0-21swapper/119:20:521
4189993618,6cyclictest5394-21ssh19:35:131
418999361,3cyclictest0-21swapper/121:40:391
419199358,14cyclictest0-21swapper/320:55:103
419199357,6cyclictest7950irq/56-dwc_otg_19:45:393
419199357,13cyclictest0-21swapper/320:19:503
419199351,24cyclictest12365-21ls16:51:043
419099351,14cyclictest7850irq/56-dwc_otg_16:50:382
418999356,6cyclictest32404-21grep17:46:071
4189993515,5cyclictest6389-21tune2fs21:30:561
4189993515,5cyclictest6389-21tune2fs21:30:561
418999351,30cyclictest15884-21latency18:30:571
418999351,25cyclictest14772-21ls21:40:581
251702350,1sleep30-21swapper/319:20:303
419199345,15cyclictest7950irq/56-dwc_otg_21:10:563
419199345,14cyclictest7950irq/56-dwc_otg_19:25:393
419199344,21cyclictest0-21swapper/321:35:513
419199341,5cyclictest0-21swapper/319:48:423
4191993411,7cyclictest0-21swapper/318:50:533
419199340,19cyclictest0-21swapper/321:30:593
419199340,19cyclictest0-21swapper/321:30:593
418999349,6cyclictest0-21swapper/119:36:111
418999349,6cyclictest0-21swapper/119:36:111
418999343,5cyclictest7750irq/56-dwc_otg21:25:531
4189993414,5cyclictest972250wget16:45:531
418999341,13cyclictest7950irq/56-dwc_otg_20:20:571
419199336,5cyclictest7950irq/56-dwc_otg_18:45:523
419199333,4cyclictest7950irq/56-dwc_otg_20:00:503
419199333,4cyclictest7950irq/56-dwc_otg_20:00:503
419199331,27cyclictest30804-21ls22:00:403
419099331,6cyclictest7850irq/56-dwc_otg_19:10:512
419099331,6cyclictest7850irq/56-dwc_otg_19:10:512
419099331,23cyclictest0-21swapper/217:55:272
4190993310,5cyclictest17982-21unixbench_singl17:06:072
418999338,5cyclictest26160-21grep21:15:551
418999337,16cyclictest0-21swapper/120:05:561
418999336,4cyclictest7750irq/56-dwc_otg19:25:521
418999334,15cyclictest7950irq/56-dwc_otg_20:46:081
418999334,13cyclictest0-21swapper/121:55:401
418999332,7cyclictest653-21ssh18:51:001
418999331,9cyclictest731-21aten_rbpower_po17:50:531
4189993315,9cyclictest24584-21sh19:19:291
419199327,15cyclictest0-21swapper/320:20:523
419199327,14cyclictest0-21swapper/318:35:523
419199327,12cyclictest0-21swapper/319:14:093
419199327,12cyclictest0-21swapper/319:14:093
419199325,14cyclictest7950irq/56-dwc_otg_16:46:033
419199323,14cyclictest7950irq/56-dwc_otg_20:31:083
419199323,14cyclictest7950irq/56-dwc_otg_20:31:083
419199322,22cyclictest0-21swapper/317:57:563
4191993210,4cyclictest0-21swapper/320:41:033
419099329,10cyclictest0-21swapper/220:24:002
4190993217,4cyclictest15452-21sh19:46:042
419099321,16cyclictest0-21swapper/216:40:552
418999323,13cyclictest7850irq/56-dwc_otg_19:41:241
418999322,15cyclictest7850irq/56-dwc_otg_18:20:521
418999321,22cyclictest8742-21cpuspeed19:00:511
268732320,2sleep30-21swapper/317:31:053
419199319,4cyclictest0-21swapper/321:24:483
419199318,12cyclictest0-21swapper/318:44:253
419199318,12cyclictest0-21swapper/318:44:253
419199318,10cyclictest0-21swapper/319:30:263
419199317,15cyclictest0-21swapper/321:15:543
419199317,10cyclictest7750irq/56-dwc_otg20:35:533
419199315,12cyclictest0-21swapper/321:27:013
419199312,16cyclictest0-21swapper/321:01:063
419199312,16cyclictest0-21swapper/321:01:063
4191993110,5cyclictest18855-21df17:10:543
4190993123,5cyclictest5717-21idleruntime-cro21:30:392
419099311,22cyclictest27003-21cpuspeed21:55:542
418999317,21cyclictest0-21swapper/117:55:511
418999315,13cyclictest7950irq/56-dwc_otg_20:56:061
418999312,9cyclictest0-21swapper/116:30:591
4189993113,5cyclictest14964-21diskmemload21:50:021
419199306,12cyclictest7950irq/56-dwc_otg_20:45:563
419199305,3cyclictest0-21swapper/320:05:433
419199305,15cyclictest0-21swapper/317:40:523
419199304,6cyclictest0-21swapper/320:11:173
4191993015,5cyclictest25525-21sh19:58:053
419199301,15cyclictest7950irq/56-dwc_otg_18:21:053
419099303,13cyclictest7950irq/56-dwc_otg_17:56:002
419099302,12cyclictest0-21swapper/218:26:082
419099301,25cyclictest4727-21ls18:00:572
419099301,22cyclictest7950irq/56-dwc_otg_17:00:522
418999306,6cyclictest1092-21ntpd21:06:031
418999304,7cyclictest19174-21grep17:10:591
419199293,16cyclictest23135-21switchtime17:21:043
4191992921,5cyclictest5311-21grep16:31:063
419199291,14cyclictest7950irq/56-dwc_otg_17:36:013
419099291,13cyclictest7850irq/56-dwc_otg_17:22:482
418999291,23cyclictest0-21swapper/118:15:581
73412281,4sleep30-21swapper/316:37:403
419199285,5cyclictest7950irq/56-dwc_otg_17:16:063
419199284,11cyclictest7950irq/56-dwc_otg_17:55:393
419199282,13cyclictest7950irq/56-dwc_otg_18:05:393
4191992820,5cyclictest8221-21fschecks_count18:10:553
419199281,4cyclictest7750irq/56-dwc_otg18:16:053
419199281,16cyclictest7095-21ssh19:36:193
419199281,16cyclictest7095-21ssh19:36:193
419099281,5cyclictest31618-21fschecks_time17:45:572
418999286,10cyclictest7850irq/56-dwc_otg_16:36:011
4189992813,5cyclictest13413-21/usr/sbin/munin16:55:531
419199273,6cyclictest7850irq/56-dwc_otg_17:26:003
419199273,6cyclictest7850irq/56-dwc_otg_17:26:003
419199271,6cyclictest0-21swapper/316:55:503
419199263,6cyclictest7750irq/56-dwc_otg17:08:223
418999266,14cyclictest0-21swapper/117:15:511
38592263,14sleep20-21swapper/216:29:542
419199253,6cyclictest39-21ksoftirqd/318:06:013
419199251,6cyclictest13698-21tune2fs18:25:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional