You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 19:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Apr 09, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
371924970,4sleep20-21swapper/216:30:432
371924970,4sleep20-21swapper/216:30:432
282121680,4sleep1377199cyclictest19:33:211
57242980,2sleep30-21swapper/318:57:323
70172950,35sleep10-21swapper/120:16:411
377099915,65cyclictest0-21swapper/019:40:500
194992850,2sleep10-21swapper/121:50:421
194992850,2sleep10-21swapper/121:50:421
27792830,2sleep10-21swapper/121:30:411
55992820,6sleep0377099cyclictest20:55:090
377099826,64cyclictest0-21swapper/016:30:540
377099775,21cyclictest22894-21ssh21:15:540
269042760,5sleep3377399cyclictest18:44:523
269042760,5sleep3377399cyclictest18:44:523
116632740,5sleep3377399cyclictest20:22:353
226432730,2sleep10-21swapper/121:54:121
3770997118,25cyclictest0-21swapper/021:45:550
3770997118,25cyclictest0-21swapper/021:45:550
86722700,1sleep30-21swapper/320:19:413
63142700,4sleep09-21ksoftirqd/021:34:460
377299706,9cyclictest7750irq/56-dwc_otg21:55:532
377099701,47cyclictest0-21swapper/018:30:530
133862700,2sleep30-21swapper/319:46:003
377099672,38cyclictest0-21swapper/016:50:520
80682650,4sleep18046-21ssh19:00:571
377399655,15cyclictest0-21swapper/317:05:523
377399655,15cyclictest0-21swapper/317:05:523
3773996522,22cyclictest22243-21ssh21:15:103
7442640,1sleep30-21swapper/318:51:123
3770996436,18cyclictest0-21swapper/021:10:580
3770996428,22cyclictest18092-21ls19:51:060
275342640,1sleep30-21swapper/320:02:423
3773996313,10cyclictest7950irq/56-dwc_otg_21:45:533
3773996313,10cyclictest7950irq/56-dwc_otg_21:45:533
377099633,52cyclictest6794-21kworker/0:116:40:560
3773996119,29cyclictest26888-21latency_hist21:20:413
377099611,52cyclictest8289-21latency_hist19:40:410
377099601,42cyclictest22794-21ntpq20:36:030
377299593,12cyclictest0-21swapper/217:10:512
377099595,51cyclictest0-21swapper/021:25:460
15762590,1sleep20-21swapper/221:28:452
3773995820,18cyclictest8935-21ssh20:58:393
377099583,46cyclictest0-21swapper/021:38:520
377099581,43cyclictest0-21swapper/021:25:400
3773995728,23cyclictest3774-21taskset16:30:443
3773995724,14cyclictest9951-21/usr/sbin/munin16:46:023
377099571,39cyclictest31031-21wget17:45:550
3770995713,25cyclictest0-21swapper/018:45:570
377399563,8cyclictest0-21swapper/317:40:513
3773995622,11cyclictest14950-21rm19:09:133
3773995615,21cyclictest2070550ssh19:55:243
3770995632,8cyclictest29829-21sed19:26:090
3770995629,13cyclictest7950irq/56-dwc_otg_19:21:030
3773995524,12cyclictest10173-21ntp_states18:16:053
3773995523,18cyclictest11964-21/usr/sbin/munin18:21:093
3773995519,15cyclictest29716-21wc19:26:083
3773995518,16cyclictest25074-21unixbench_singl21:56:083
33432550,2sleep10-21swapper/120:12:351
183242550,2sleep00-21swapper/017:10:540
377099548,14cyclictest30609-21ssh20:45:570
3770995437,5cyclictest32481-21wc18:51:000
3770995430,9cyclictest0-21swapper/020:07:580
377099542,49cyclictest0-21swapper/018:55:580
377099542,44cyclictest5831-21fschecks_count16:35:570
377099542,40cyclictest0-21swapper/021:00:410
3770995416,12cyclictest0-21swapper/021:55:530
377099541,35cyclictest0-21swapper/019:05:550
3773995321,20cyclictest8998-21ssh21:37:073
3773995319,21cyclictest20442-21open_files21:51:053
3771995315,24cyclictest0-21swapper/118:10:471
3770995336,3cyclictest0-21swapper/020:16:040
3773995221,11cyclictest14879-21rm21:45:133
3773995219,12cyclictest17871-21apt-get20:30:543
3773995218,22cyclictest6884-21ssh19:38:083
377099526,28cyclictest19526-21rm18:35:570
3770995232,4cyclictest0-21swapper/018:41:020
3770995232,4cyclictest0-21swapper/018:41:020
377099521,48cyclictest0-21swapper/020:41:030
377099521,38cyclictest0-21swapper/018:06:070
3773995120,13cyclictest23092-21ssh19:57:113
3773995120,13cyclictest23092-21ssh19:57:113
3773995118,19cyclictest7950irq/56-dwc_otg_17:36:083
3773995117,19cyclictest7950irq/56-dwc_otg_21:07:003
3773995113,21cyclictest7950irq/56-dwc_otg_17:30:533
377099512,31cyclictest0-21swapper/017:26:090
3770995121,26cyclictest27357-21aten_rbpower_vo17:35:550
377099511,37cyclictest25577-21systemd-journal18:17:240
377099511,37cyclictest0-21swapper/016:55:520
377099511,26cyclictest8986-21unixbench_multi19:01:070
3773995019,18cyclictest29593-21ssh21:23:263
3773995017,18cyclictest29490-21ssh20:44:443
3773995017,18cyclictest29411-21ssh20:05:413
3773995016,21cyclictest22278-21ssh19:17:243
3770995033,3cyclictest0-21swapper/018:00:570
3770995028,14cyclictest20357-21fschecks_time17:15:580
377099502,38cyclictest9-21ksoftirqd/019:15:530
377099502,37cyclictest0-21swapper/018:21:090
377099501,45cyclictest0-21swapper/017:05:580
377099501,45cyclictest0-21swapper/017:05:580
377099501,36cyclictest25476-21aten_rbpower_vo20:00:550
377099501,34cyclictest13617-21ntpq19:46:020
3773994931,8cyclictest5600-21apt-get16:35:533
3773994925,11cyclictest7325-21apt-get18:10:473
3773994918,11cyclictest22810-1kworker/u9:219:30:543
3773994914,20cyclictest2617-21ssh20:11:153
3770994927,5cyclictest3298-21grep17:56:080
377099492,35cyclictest0-21swapper/019:56:010
377099492,35cyclictest0-21swapper/019:56:010
3770994921,19cyclictest0-21swapper/020:27:130
3770994916,5cyclictest101ktimersoftd/017:41:070
377099491,40cyclictest0-21swapper/020:12:120
377099491,34cyclictest10812-21sed20:21:100
3773994820,13cyclictest10927-21sh19:42:573
3773994818,12cyclictest19354-21ssh19:14:393
3773994818,12cyclictest19354-21ssh19:14:393
3773994817,11cyclictest12242-21taskset16:51:363
3773994815,18cyclictest426-21ssh20:48:243
3773994815,12cyclictest5567-21taskset21:33:253
377199487,26cyclictest0-21swapper/118:50:521
3770994829,6cyclictest7850irq/56-dwc_otg_21:06:010
3770994824,12cyclictest0-21swapper/019:34:050
377099482,34cyclictest11861-21chrt21:01:250
377099482,34cyclictest11861-21chrt21:01:250
377099481,32cyclictest0-21swapper/017:53:370
3770994813,22cyclictest0-21swapper/017:30:530
3773994718,14cyclictest12019-21rm21:01:423
3773994718,14cyclictest12019-21rm21:01:423
3773994717,12cyclictest5655-21ssh20:55:153
3773994717,11cyclictest26797-21ssh19:23:033
3773994714,18cyclictest7950irq/56-dwc_otg_20:28:073
3773994713,11cyclictest7950irq/56-dwc_otg_20:36:513
3770994731,4cyclictest7850irq/56-dwc_otg_19:10:510
3770994731,4cyclictest7850irq/56-dwc_otg_19:10:510
377099472,34cyclictest14662-21basename18:30:400
377099471,34cyclictest0-21swapper/017:21:090
377099471,33cyclictest0-21swapper/018:10:570
377099471,29cyclictest0-21swapper/021:51:020
377399466,26cyclictest0-21swapper/317:45:543
3773994617,10cyclictest7950irq/56-dwc_otg_16:41:043
3773994615,12cyclictest7291-21taskset18:10:283
3770994624,7cyclictest18615-21ntpq20:31:030
377099461,36cyclictest0-21swapper/016:50:410
34702462,13sleep00-21swapper/016:30:210
34702462,13sleep00-21swapper/016:30:210
377399459,18cyclictest7950irq/56-dwc_otg_18:38:243
3773994514,19cyclictest28542-21ps18:46:053
3773994513,11cyclictest16058-21chrt17:04:173
3773994512,11cyclictest9877-21ssh19:02:333
377099452,30cyclictest0-21swapper/021:40:540
26682455,27sleep12748-21/usr/sbin/munin16:26:051
26682455,27sleep12748-21/usr/sbin/munin16:26:051
377399449,20cyclictest7950irq/56-dwc_otg_18:02:473
3773994420,11cyclictest17191-21ssh18:32:403
3773994411,13cyclictest21599-21latency_hist17:20:413
377199447,25cyclictest0-21swapper/120:40:521
3771994411,7cyclictest9450irq/48-DMA09:12:251
3771994411,7cyclictest9450irq/48-DMA09:12:251
3770994419,13cyclictest908-21runrttasks17:04:260
377399437,22cyclictest0-21swapper/317:25:543
3773994313,14cyclictest12808-21/usr/sbin/munin16:55:503
3773994312,15cyclictest32428-21sed21:26:063
377199437,22cyclictest0-21swapper/116:50:531
377199431,4cyclictest7750irq/56-dwc_otg19:50:571
3772994219,5cyclictest951250wget18:15:552
377199429,23cyclictest0-21swapper/117:30:531
377199427,25cyclictest0-21swapper/116:30:541
3771994220,10cyclictest28835-21sh19:25:571
3773994118,12cyclictest3628-21taskset17:59:433
377299411,6cyclictest0-21swapper/217:00:532
3771994119,5cyclictest28518-21sh20:04:291
213022410,2sleep20-21swapper/218:37:222
3773994010,14cyclictest23162-21seq17:22:533
377299407,23cyclictest0-21swapper/217:30:532
3771994013,4cyclictest0-21swapper/118:35:571
3773993910,11cyclictest0-21swapper/318:25:553
3772993913,8cyclictest24058-21unixbench-2d21:16:082
3772993912,15cyclictest0-21swapper/220:37:262
3771993917,12cyclictest908-21runrttasks20:29:121
3771993822,5cyclictest1290550wget16:55:531
3771993818,8cyclictest16482-21wget17:05:551
3771993818,8cyclictest16482-21wget17:05:551
3771993817,5cyclictest874250wget19:40:541
3773993722,12cyclictest1596-21taskset17:52:043
3773993710,17cyclictest7950irq/56-dwc_otg_17:12:293
377299371,7cyclictest0-21swapper/218:45:532
3772993716,4cyclictest6353-21sh18:58:412
377199377,15cyclictest0-21swapper/121:35:541
377199373,13cyclictest7750irq/56-dwc_otg17:10:511
3771993718,4cyclictest15049-21aten_rbpower_en18:30:541
3771993717,13cyclictest14794-21diskstats21:05:561
114542370,2sleep20-21swapper/219:05:242
54262360,1sleep20-21swapper/220:54:512
377299369,11cyclictest0-21swapper/221:33:542
377299366,6cyclictest0-21swapper/220:26:092
377299362,7cyclictest9592-21ssh20:20:552
3772993616,6cyclictest939-21lldpd19:15:262
3772993616,6cyclictest939-21lldpd19:15:262
3772993611,6cyclictest8821-21ssh20:58:272
377199367,5cyclictest7750irq/56-dwc_otg20:46:001
377199362,5cyclictest0-21swapper/118:20:531
3771993617,12cyclictest30629-21sh21:25:191
3771993616,5cyclictest12329-21sh20:23:471
28822360,6sleep2377299cyclictest17:56:042
145202361,4sleep10-21swapper/119:08:261
377299358,7cyclictest7950irq/56-dwc_otg_21:46:082
377299358,7cyclictest7950irq/56-dwc_otg_21:46:082
377299356,6cyclictest7850irq/56-dwc_otg_20:00:552
377299356,13cyclictest7850irq/56-dwc_otg_19:16:042
377299355,4cyclictest7950irq/56-dwc_otg_19:05:502
377299354,6cyclictest21225-21ssh19:55:542
377299354,6cyclictest21225-21ssh19:55:542
3772993516,5cyclictest13111-21aten_rbpower_cu18:25:552
3772993516,4cyclictest7750irq/56-dwc_otg21:11:062
3772993515,11cyclictest5648-21sed18:05:412
377299351,20cyclictest14888-21fschecks_time21:05:572
377299350,17cyclictest0-21swapper/220:30:542
377199359,7cyclictest0-21swapper/117:00:531
377199358,11cyclictest7850irq/56-dwc_otg_18:10:411
3771993516,4cyclictest4177-21uname18:56:011
377199351,5cyclictest7750irq/56-dwc_otg19:35:521
233702351,28sleep10-21swapper/121:16:001
39142340,10sleep2301ktimersoftd/220:13:362
377299345,6cyclictest9306-21aten_rbpower_po16:45:532
377299342,18cyclictest7950irq/56-dwc_otg_18:31:032
377299342,18cyclictest11139-21latency_hist21:40:412
377299341,9cyclictest20221-21latency21:51:012
377299341,7cyclictest613-21grep18:51:072
377299341,7cyclictest12644-21latency_hist16:55:412
3772993414,4cyclictest8239-21/usr/sbin/munin16:41:042
377299341,23cyclictest14228-21latency_hist21:05:412
377299341,23cyclictest14228-21latency_hist21:05:412
377199343,6cyclictest0-21swapper/119:45:521
3771993415,7cyclictest285352chrt17:37:201
3771993414,5cyclictest31133-21/usr/sbin/munin17:45:561
3771993413,4cyclictest0-21swapper/116:35:511
377299338,7cyclictest7950irq/56-dwc_otg_16:31:042
377299337,8cyclictest7850irq/56-dwc_otg_19:55:412
377299337,6cyclictest7950irq/56-dwc_otg_18:20:572
377299337,16cyclictest0-21swapper/220:05:542
377299335,14cyclictest7950irq/56-dwc_otg_20:45:402
377299333,5cyclictest7750irq/56-dwc_otg18:41:062
377299333,5cyclictest7750irq/56-dwc_otg18:41:062
377299332,5cyclictest29482-21ssh19:26:062
3772993325,6cyclictest8602-21ssh19:40:502
3772993324,6cyclictest908-21runrttasks19:50:412
3772993314,5cyclictest908-21runrttasks16:36:082
377199337,5cyclictest7850irq/56-dwc_otg_18:45:581
377199335,5cyclictest7950irq/56-dwc_otg_22:00:411
3771993318,5cyclictest9861-21if_err_eth018:16:001
3771993317,4cyclictest2960-21/usr/sbin/munin21:31:041
3771993312,4cyclictest21319-21timerandwakeup19:16:061
377299328,6cyclictest4750irq/17-3f00b88019:20:572
377299327,6cyclictest0-21swapper/220:16:052
377299322,17cyclictest7850irq/56-dwc_otg_19:30:592
377299321,7cyclictest7850irq/56-dwc_otg_17:06:082
377299321,7cyclictest7850irq/56-dwc_otg_17:06:082
377299321,4cyclictest7850irq/56-dwc_otg_20:46:002
377199328,12cyclictest18784-21zcat20:31:061
3771993224,5cyclictest21531-21fschecks_time19:55:581
3771993224,5cyclictest21531-21fschecks_time19:55:581
377199321,7cyclictest2385350ssh18:40:591
377199321,7cyclictest2385350ssh18:40:591
3771993217,4cyclictest18202-21sh19:12:331
3771993217,4cyclictest18202-21sh19:12:331
3771993214,9cyclictest31603-21sh20:07:331
3771993214,4cyclictest908-21runrttasks18:03:081
377199321,3cyclictest7750irq/56-dwc_otg21:43:521
3771993212,16cyclictest7950irq/56-dwc_otg_19:20:521
377199321,11cyclictest0-21swapper/121:13:271
377299316,4cyclictest0-21swapper/219:36:042
377299315,6cyclictest7850irq/56-dwc_otg_21:20:592
377299315,5cyclictest7850irq/56-dwc_otg_17:46:032
377299312,14cyclictest7950irq/56-dwc_otg_17:50:532
377299311,5cyclictest27921-21ntpq17:36:032
377299311,13cyclictest7850irq/56-dwc_otg_17:41:072
377199312,13cyclictest7750irq/56-dwc_otg20:36:081
377199311,5cyclictest0-21swapper/118:25:531
121592310,2sleep212160-21/usr/sbin/munin21:41:052
377299303,6cyclictest24351-21/usr/sbin/munin17:26:052
377299301,19cyclictest0-21swapper/216:59:512
377199305,10cyclictest9279-21aten_rbpower_en16:45:521
377199304,14cyclictest7950irq/56-dwc_otg_17:26:011
377199301,13cyclictest0-21swapper/120:51:221
31412305,15sleep30-21swapper/316:26:113
31412305,15sleep30-21swapper/316:26:113
377299296,10cyclictest7850irq/56-dwc_otg_18:10:442
377299291,5cyclictest5883-21aten_rbpower_vo18:05:542
3771992923,4cyclictest30832-21latency_hist17:45:411
3771992918,4cyclictest7730-21fschecks_time16:40:561
377199291,10cyclictest0-21swapper/120:55:461
377299285,4cyclictest7950irq/56-dwc_otg_17:21:082
377299272,14cyclictest0-21swapper/217:18:312
377199272,8cyclictest7950irq/56-dwc_otg_17:20:581
377199271,5cyclictest0-21swapper/117:55:531
377199271,4cyclictest917-21snmpd17:53:371
377199271,3cyclictest7950irq/56-dwc_otg_17:15:511
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional