You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 16:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Fri Apr 10, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
258062940,1sleep00-21swapper/021:37:290
50872900,3sleep25081-21apt-get17:30:532
232562870,2sleep023258-21aten_rbpower_cu18:20:540
232562870,2sleep023258-21aten_rbpower_cu18:20:540
71042730,3sleep07099-21ssh21:15:590
71042730,3sleep07099-21ssh21:15:590
1546599721,41cyclictest0-21swapper/018:05:520
149582720,5sleep21547899cyclictest18:54:142
15481996911,18cyclictest7750irq/56-dwc_otg20:40:533
15481996734,6cyclictest7950irq/56-dwc_otg_19:25:513
179362640,1sleep20-21swapper/220:12:322
15465996424,7cyclictest0-21swapper/019:50:490
149012643,11sleep10-21swapper/116:27:361
15481996027,18cyclictest7850irq/56-dwc_otg_21:20:473
1547399604,6cyclictest7750irq/56-dwc_otg19:25:531
15465996025,5cyclictest7850irq/56-dwc_otg_18:56:070
1546599592,32cyclictest31144-21latency_hist18:35:420
1546599581,40cyclictest15900-21ls16:30:580
1546599581,30cyclictest30223-21grep17:10:550
1546599581,30cyclictest30223-21grep17:10:550
15465995811,24cyclictest6425-21grep19:21:090
141662580,3sleep01546599cyclictest21:24:320
1548199572,7cyclictest0-21swapper/321:00:523
1546599577,41cyclictest0-21swapper/019:40:420
15465995738,10cyclictest12905-21kworker/0:020:18:050
1546599565,36cyclictest0-21swapper/018:45:540
1546599565,36cyclictest0-21swapper/018:45:540
15465995628,5cyclictest0-21swapper/021:51:080
15465995626,5cyclictest0-21swapper/019:55:520
1546599557,25cyclictest10252-21ssh20:41:080
1546599552,50cyclictest11611-21wget20:05:550
1546599551,7cyclictest3035-21munin-plugin-st17:25:410
1546599551,39cyclictest13-21rcuc/018:51:080
1546599551,30cyclictest32450-21ssh21:45:490
1546599551,30cyclictest32450-21ssh21:45:490
1546599551,28cyclictest29477-21seq20:26:070
1547399547,28cyclictest0-21swapper/116:35:461
15465995430,6cyclictest13-21rcuc/020:00:550
15465995417,34cyclictest101ktimersoftd/019:45:440
1546599541,18cyclictest605-21ls17:16:080
1546599539,26cyclictest13-21rcuc/021:05:410
1546599535,25cyclictest21446-21/usr/sbin/munin19:01:050
1546599535,25cyclictest21446-21/usr/sbin/munin19:01:050
15465995331,7cyclictest7850irq/56-dwc_otg_20:36:090
15465995329,6cyclictest31682-21grep21:06:090
1546599532,28cyclictest0-21swapper/022:00:410
15465995320,9cyclictest24534-21apt-config16:55:410
1546599531,28cyclictest15271-21unixbench-2d17:56:080
1546599531,23cyclictest0-21swapper/016:45:420
1546599531,23cyclictest0-21swapper/016:45:420
1546599521,35cyclictest26621-21tr17:00:550
1548199517,28cyclictest0-21swapper/318:55:463
15481995120,13cyclictest7850irq/56-dwc_otg_18:45:553
15481995120,13cyclictest7850irq/56-dwc_otg_18:45:553
15481995112,13cyclictest808-21ntpq21:46:053
15481995112,13cyclictest808-21ntpq21:46:053
15473995116,24cyclictest7750irq/56-dwc_otg18:35:411
1546599514,28cyclictest16348-21ssh20:11:000
15465995133,5cyclictest7750irq/56-dwc_otg19:26:090
1546599512,28cyclictest22550-21missed_timers20:56:020
15465995117,21cyclictest0-21swapper/021:32:020
1546599511,29cyclictest26629-21gpgv19:45:420
1546599511,29cyclictest26629-21gpgv19:45:420
180032501,4sleep018012-21ssh21:28:480
15481995019,19cyclictest26605-21diskmemload20:17:553
1546599502,7cyclictest17649-21ssh20:50:540
15465995022,7cyclictest1664-21grep20:31:090
1546599502,23cyclictest0-21swapper/020:21:050
1546599501,22cyclictest0-21swapper/019:16:060
15481994914,21cyclictest10342-21ssh20:04:133
15481994914,16cyclictest18507-21fschecks_time19:35:573
15481994910,20cyclictest7950irq/56-dwc_otg_20:46:033
1547899493,18cyclictest7950irq/56-dwc_otg_21:45:482
1547899493,18cyclictest7950irq/56-dwc_otg_21:45:482
15465994932,5cyclictest17829-21idleruntime-cro19:35:410
15465994925,11cyclictest7950irq/56-dwc_otg_17:31:030
15465994923,10cyclictest0-21swapper/018:35:460
1546599491,28cyclictest17109-21apt-get16:35:470
1548199489,24cyclictest7950irq/56-dwc_otg_21:26:013
1548199488,17cyclictest0-21swapper/320:37:533
1548199487,24cyclictest24835-21ssh19:05:533
15478994813,9cyclictest0-21swapper/218:05:522
1547399481,26cyclictest0-21swapper/119:50:491
15465994828,6cyclictest7950irq/56-dwc_otg_18:16:050
15465994822,5cyclictest29955-21proc_pri19:11:050
1546599482,23cyclictest0-21swapper/018:30:410
1546599481,43cyclictest0-21swapper/018:00:590
1546599481,31cyclictest25670-21ps19:06:050
129822480,2sleep20-21swapper/219:29:442
1548199477,27cyclictest0-21swapper/318:00:533
1548199477,25cyclictest0-21swapper/318:20:543
1548199477,25cyclictest0-21swapper/318:20:543
1548199477,24cyclictest0-21swapper/317:15:473
15481994713,16cyclictest6874-21grep21:15:563
15481994713,16cyclictest6874-21grep21:15:563
15481994713,15cyclictest31789-21ssh19:13:323
1546599471,9cyclictest25885-21unixbench_multi16:56:080
1546599471,34cyclictest3173-21/usr/sbin/munin18:41:080
149062473,38sleep00-21swapper/016:27:390
1548199467,25cyclictest0-21swapper/316:30:473
1548199467,24cyclictest16333-21ssh19:33:063
15481994616,15cyclictest23366-21/usr/sbin/munin16:50:593
15481994611,18cyclictest0-21swapper/320:21:093
15481994610,12cyclictest22627-21cron16:50:403
1547899467,24cyclictest0-21swapper/217:15:552
1547899467,24cyclictest0-21swapper/216:35:462
15478994621,15cyclictest301ktimersoftd/221:35:562
15465994629,9cyclictest101ktimersoftd/021:11:180
15465994625,6cyclictest7850irq/56-dwc_otg_21:41:150
15465994623,5cyclictest908-21runrttasks20:46:540
15465994619,19cyclictest0-21swapper/017:48:360
1548199457,24cyclictest0-21swapper/317:45:473
15481994511,14cyclictest30514-21ssh21:05:553
15478994519,11cyclictest2494650wget19:05:542
15473994511,8cyclictest0-21swapper/120:25:521
1546599458,10cyclictest7750irq/56-dwc_otg17:35:530
15465994527,5cyclictest7850irq/56-dwc_otg_17:41:060
324312440,6sleep121-21rcuc/117:15:591
185022440,5sleep31548199cyclictest16:36:083
1548199447,23cyclictest0-21swapper/318:05:543
15481994421,9cyclictest22345-21ls18:16:073
15481994417,11cyclictest27567-21/usr/sbin/munin17:01:083
15481994417,11cyclictest11342-21ssh21:58:263
15481994416,13cyclictest21069-21ssh20:54:513
1548199435,22cyclictest8620-21sh19:24:463
1548199433,8cyclictest7950irq/56-dwc_otg_18:30:533
15481994312,18cyclictest5464-21ssh21:14:253
15481994310,22cyclictest1141-21sh19:53:073
15473994317,5cyclictest14132-21sh20:08:231
1548199429,23cyclictest14879-21ssh18:54:063
1548199428,13cyclictest32724-21latency_hist20:30:413
1548199427,26cyclictest26775-21ssh19:45:473
1548199425,9cyclictest0-21swapper/321:00:173
15481994215,14cyclictest19429-21/usr/sbin/munin21:30:553
15481994213,13cyclictest18009-21ssh20:12:403
15481994210,11cyclictest20668-21/usr/sbin/munin19:00:543
15481994210,11cyclictest20668-21/usr/sbin/munin19:00:543
1547899427,25cyclictest7850irq/56-dwc_otg_21:25:402
1547899426,9cyclictest0-21swapper/218:00:542
15478994221,13cyclictest301ktimersoftd/218:35:572
15478994220,15cyclictest8833-21sh21:55:552
1547399426,6cyclictest7850irq/56-dwc_otg_20:15:541
1547399423,4cyclictest7950irq/56-dwc_otg_18:56:021
1546599422,9cyclictest29520-21grep17:06:080
1546599422,19cyclictest0-21swapper/017:28:530
1548199415,12cyclictest2595-21sh19:16:433
15481994117,11cyclictest27652-21ssh21:40:393
15481994114,7cyclictest9689-21ls17:41:083
15481994114,11cyclictest2955-21apt-get19:55:533
15481994111,12cyclictest3026-21ssh18:40:423
1547899419,15cyclictest7850irq/56-dwc_otg_17:25:422
1547899416,19cyclictest29232-21aten_rbpower_vo19:10:552
15478994116,5cyclictest23991-21sh19:04:512
15478994116,5cyclictest23991-21sh19:04:512
1547399419,12cyclictest7750irq/56-dwc_otg21:15:501
1547399419,12cyclictest7750irq/56-dwc_otg21:15:501
1547399415,25cyclictest20528-21ssh19:00:511
1547399415,25cyclictest20528-21ssh19:00:511
15473994111,5cyclictest215852sh20:55:421
15473994110,5cyclictest0-21swapper/121:06:091
1546599412,29cyclictest0-21swapper/016:48:180
1546599412,21cyclictest0-21swapper/017:51:480
1548199408,12cyclictest28032-21sh21:40:453
15481994015,16cyclictest23978-21ssh19:41:303
15481994015,16cyclictest23978-21ssh19:41:303
15481994011,12cyclictest24790-21sshd16:55:533
1547399404,19cyclictest0-21swapper/118:45:541
1547399404,19cyclictest0-21swapper/118:45:541
15473994017,6cyclictest28147-21/usr/sbin/munin21:40:531
15465994013,21cyclictest0-21swapper/018:13:450
151222405,21sleep20-21swapper/216:30:282
1548199396,18cyclictest7950irq/56-dwc_otg_18:40:453
1547899391,20cyclictest0-21swapper/221:29:012
1547399397,17cyclictest19524-21aten_rbpower_en21:30:541
15473993919,4cyclictest1093650wget21:20:521
15473993911,16cyclictest5450-1kworker/u9:019:24:581
1548199389,19cyclictest13414-21ssh20:07:083
1548199388,12cyclictest20968-21taskset18:13:053
1548199388,11cyclictest0-21swapper/317:50:583
1548199385,22cyclictest0-21swapper/317:05:533
1548199383,9cyclictest7950irq/56-dwc_otg_17:14:433
1548199383,9cyclictest7950irq/56-dwc_otg_17:14:433
15481993817,5cyclictest437-21/usr/sbin/munin20:31:063
1548199381,25cyclictest7950irq/56-dwc_otg_21:55:223
1548199381,25cyclictest7464-21ls17:36:033
1547899385,27cyclictest7950irq/56-dwc_otg_20:55:522
1547899383,29cyclictest7850irq/56-dwc_otg_21:51:072
1547899383,25cyclictest7850irq/56-dwc_otg_18:35:422
1547899383,21cyclictest7850irq/56-dwc_otg_18:50:422
1547899383,21cyclictest7850irq/56-dwc_otg_18:50:422
1547899381,24cyclictest28676-21cut17:05:572
1547399383,23cyclictest7750irq/56-dwc_otg16:40:581
1547399383,23cyclictest7750irq/56-dwc_otg16:40:581
15473993812,15cyclictest0-21swapper/119:16:001
1548199376,27cyclictest0-21swapper/317:25:533
1548199372,3cyclictest0-21swapper/317:55:523
1547899374,5cyclictest11-21rcu_preempt16:57:132
1547899374,24cyclictest7850irq/56-dwc_otg_20:25:412
1547899373,9cyclictest0-21swapper/219:50:492
1547899373,27cyclictest7850irq/56-dwc_otg_16:45:412
1547899373,27cyclictest7850irq/56-dwc_otg_16:45:412
1547899373,25cyclictest7850irq/56-dwc_otg_19:21:032
1547899372,12cyclictest0-21swapper/221:15:502
1547899372,12cyclictest0-21swapper/221:15:502
15478993718,8cyclictest30177-21/usr/sbin/munin17:11:092
15478993718,8cyclictest30177-21/usr/sbin/munin17:11:092
15478993713,5cyclictest908-21runrttasks18:20:522
15478993713,5cyclictest908-21runrttasks18:20:522
15478993712,6cyclictest29232-21cat21:41:072
1547899371,13cyclictest19444-21unixbench_singl19:36:082
1547399378,15cyclictest19383-21sh18:10:411
1547399374,23cyclictest7850irq/56-dwc_otg_16:45:591
15473993720,4cyclictest14378-21sh18:53:131
1547399371,5cyclictest0-21swapper/120:30:521
15473993714,16cyclictest0-21swapper/117:59:241
15473993713,6cyclictest882350wget17:40:561
1547399371,22cyclictest7850irq/56-dwc_otg_19:06:021
15473993710,17cyclictest0-21swapper/118:40:541
1548199367,19cyclictest0-21swapper/317:20:553
15481993617,14cyclictest26202-21grep18:26:083
1547899365,24cyclictest7850irq/56-dwc_otg_19:45:422
1547899365,24cyclictest7850irq/56-dwc_otg_19:45:422
1547899364,11cyclictest7850irq/56-dwc_otg_19:56:092
15478993615,5cyclictest20392-21smartpower21:31:062
15478993615,10cyclictest27740-21sh21:01:352
15478993612,5cyclictest0-21swapper/221:06:082
1547399369,13cyclictest0-21swapper/121:11:431
1547399366,20cyclictest31356-21ssh18:35:541
1547399364,15cyclictest7950irq/56-dwc_otg_19:45:061
1547399364,15cyclictest7950irq/56-dwc_otg_19:45:061
15473993626,7cyclictest520-21cron18:30:401
15473993622,4cyclictest8308-21timerandwakeup20:01:071
15473993615,6cyclictest16115-21aten_rbpower_po18:00:541
15473993610,10cyclictest7950irq/56-dwc_otg_17:50:411
1547899353,8cyclictest0-21swapper/221:10:512
1547899351,4cyclictest7850irq/56-dwc_otg_20:41:082
15478993513,5cyclictest29-21rcuc/220:51:082
1547399357,15cyclictest4300-21/usr/sbin/munin21:51:051
1547399354,25cyclictest7750irq/56-dwc_otg18:16:021
1547399352,8cyclictest7750irq/56-dwc_otg16:50:531
1547399352,6cyclictest5957-21ssh19:59:041
15473993512,16cyclictest7750irq/56-dwc_otg17:25:411
15473993510,13cyclictest0-21swapper/121:00:531
1548199347,9cyclictest7950irq/56-dwc_otg_17:33:593
15481993410,11cyclictest20621-21chrt16:43:083
15481993410,11cyclictest20621-21chrt16:43:083
1547899348,5cyclictest7850irq/56-dwc_otg_18:41:092
1547899345,7cyclictest7950irq/56-dwc_otg_20:36:062
1547899344,7cyclictest28710-21wget20:25:552
1547399348,5cyclictest7950irq/56-dwc_otg_18:20:541
1547399348,5cyclictest7950irq/56-dwc_otg_18:20:541
1547399346,4cyclictest7750irq/56-dwc_otg17:06:071
1547399343,9cyclictest9949-21grep21:56:081
1547399342,6cyclictest9450irq/48-DMA09:12:251
15473993424,6cyclictest221ktimersoftd/117:50:511
1547399341,23cyclictest19216-21ssh19:36:061
1547899338,16cyclictest0-21swapper/219:47:172
1547899337,6cyclictest7950irq/56-dwc_otg_19:16:032
1547899337,4cyclictest7950irq/56-dwc_otg_16:50:532
1547899336,5cyclictest20352-21sh20:15:582
1547899333,11cyclictest7950irq/56-dwc_otg_18:20:422
15478993316,10cyclictest301ktimersoftd/217:29:202
15478993313,13cyclictest10572-21ssh20:04:372
1547899331,26cyclictest26754-21ls17:00:562
15478993311,4cyclictest7850irq/56-dwc_otg_18:25:512
1547399339,10cyclictest0-21swapper/121:26:341
1547399336,15cyclictest0-21swapper/119:45:551
1547399333,7cyclictest7850irq/56-dwc_otg_21:46:011
1547399333,7cyclictest7850irq/56-dwc_otg_21:46:011
15473993318,4cyclictest20678-21unixbench-2d18:11:081
15473993316,4cyclictest17181-21ssh20:11:131
15473993315,8cyclictest7850irq/56-dwc_otg_17:10:521
15473993315,8cyclictest7850irq/56-dwc_otg_17:10:521
1547399331,15cyclictest7950irq/56-dwc_otg_20:46:041
1547899329,7cyclictest0-21swapper/219:30:592
1547899328,5cyclictest7850irq/56-dwc_otg_20:05:522
1547899328,4cyclictest0-21swapper/220:33:142
1547899326,6cyclictest7950irq/56-dwc_otg_18:59:232
15478993210,3cyclictest0-21swapper/220:46:112
1547399327,15cyclictest0-21swapper/119:30:431
1547399322,4cyclictest25883-21grep16:56:081
1547399322,21cyclictest7850irq/56-dwc_otg_17:26:011
1547399322,10cyclictest0-21swapper/117:39:361
1547399321,25cyclictest0-21swapper/120:40:521
15473993210,13cyclictest25477-21ssh21:36:541
15473993210,11cyclictest5554-21ntpq20:36:041
1547899317,15cyclictest0-21swapper/216:45:552
1547899317,15cyclictest0-21swapper/216:30:482
1547899312,20cyclictest7850irq/56-dwc_otg_18:13:412
15478993110,11cyclictest15984-21sed18:00:422
1547399317,6cyclictest0-21swapper/117:31:461
1547399311,8cyclictest25577-21systemd-journal20:55:551
1547899302,11cyclictest7950irq/56-dwc_otg_17:41:032
15478993010,5cyclictest13862-21seq17:54:262
1547399306,4cyclictest0-21swapper/120:21:001
1547399306,14cyclictest0-21swapper/119:10:541
1547399303,17cyclictest7950irq/56-dwc_otg_17:05:411
150882294,16sleep3917-21snmpd16:30:003
1547899274,10cyclictest7950irq/56-dwc_otg_17:45:582
1547899251,4cyclictest6875-21/usr/sbin/munin17:35:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional