You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 13:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Dec 13, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1060125250,5sleep010604-21unixbench_singl19:05:320
1133622710,2sleep20-21swapper/219:08:212
844821620,5sleep11198399cyclictest20:28:141
119829910414,63cyclictest0-21swapper/000:23:010
1198299881,75cyclictest0-21swapper/000:28:010
16932860,7sleep11198399cyclictest21:40:021
11982998336,34cyclictest4671-21grep20:18:100
1198299819,68cyclictest0-21swapper/021:27:590
1198299791,61cyclictest0-21swapper/000:12:590
1198299771,5cyclictest13185-21cron23:45:010
1198299771,56cyclictest0-21swapper/020:57:550
1198299769,53cyclictest14034-21unixbench_singl22:13:110
1198299762,53cyclictest28360-21grep21:23:110
11982997618,2cyclictest0-21swapper/020:02:580
1198299752,8cyclictest0-21swapper/019:43:120
1198299751,61cyclictest14130-21cron20:45:010
1198299751,56cyclictest0-21swapper/019:17:590
1198499742,47cyclictest4885-21grep20:18:122
1198499725,58cyclictest29347-21/usr/sbin/munin21:28:032
1198299728,4cyclictest5049-21unixbench-2d23:18:130
1198299721,60cyclictest22909-21awk22:38:130
1198299712,7cyclictest0-21swapper/019:48:310
1198299711,60cyclictest0-21swapper/022:51:180
1198299701,55cyclictest17064-21ntpq20:53:050
11982997015,39cyclictest9333-21wget22:02:590
1198299696,3cyclictest0-21swapper/023:53:000
1198299691,59cyclictest0-21swapper/019:35:100
11982996915,43cyclictest13992-21/usr/sbin/munin23:48:030
1198299681,54cyclictest15752-21grep20:48:140
1198299681,52cyclictest23689-21wget21:12:580
11982996813,37cyclictest7950irq/56-dwc_otg_20:40:010
11982996645,7cyclictest7750irq/56-dwc_otg21:35:010
1198299662,7cyclictest0-21swapper/019:24:190
1198299661,8cyclictest0-21swapper/021:46:450
1198299652,53cyclictest0-21swapper/022:18:010
11982996513,34cyclictest7950irq/56-dwc_otg_00:20:010
1198499648,45cyclictest5778-21/usr/sbin/munin23:23:072
1198499646,38cyclictest7750irq/56-dwc_otg22:02:592
1198299641,8cyclictest24716-21cat22:43:090
11982996413,36cyclictest7850irq/56-dwc_otg_23:40:010
11982996410,40cyclictest3155-21ls23:13:090
1198599636,24cyclictest0-21swapper/321:32:593
1198499635,36cyclictest7750irq/56-dwc_otg23:18:122
1198299638,39cyclictest6776-21chrt21:55:150
1198299632,8cyclictest0-21swapper/022:32:240
1198299631,48cyclictest0-21swapper/023:07:580
1198299631,48cyclictest0-21swapper/023:07:580
1198299631,47cyclictest0-21swapper/020:28:540
1198299631,46cyclictest0-21swapper/020:07:250
1198299624,43cyclictest2560-21ntp_kernel_pll_20:13:080
1198299622,48cyclictest0-21swapper/019:28:100
1198299622,48cyclictest0-21swapper/019:28:100
1198299622,47cyclictest379-21grep00:38:140
1198299621,47cyclictest6112-21fschecks_time23:23:000
116912623,14sleep10-21swapper/119:09:461
1198499612,5cyclictest0-21swapper/223:52:582
1198499611,5cyclictest28402-21diskstats00:28:002
1198299611,9cyclictest0-21swapper/000:09:280
1198299611,48cyclictest0-21swapper/021:09:120
1198299611,48cyclictest0-21swapper/021:09:120
11982996112,37cyclictest0-21swapper/020:34:180
1198299611,12cyclictest0-21swapper/022:23:080
1198299611,12cyclictest0-21swapper/022:23:080
1198499603,37cyclictest336-21/usr/sbin/munin23:08:072
1198499603,37cyclictest336-21/usr/sbin/munin23:08:072
1198299609,39cyclictest0-21swapper/021:17:370
1198299607,41cyclictest0-21swapper/023:34:190
1198299601,47cyclictest0-21swapper/023:55:470
1198299601,47cyclictest0-21swapper/023:55:470
1198299601,46cyclictest19515-21unixbench_multi22:28:110
1198299601,45cyclictest0-21swapper/000:01:090
11982996014,33cyclictest7750irq/56-dwc_otg19:54:000
1198499591,4cyclictest0-21swapper/223:48:042
1198299599,39cyclictest19890-21chrt19:32:510
1198299599,39cyclictest19890-21chrt19:32:510
1198299591,45cyclictest5974-21fschecks_count21:53:150
11982995914,35cyclictest0-21swapper/021:38:460
1198299591,12cyclictest0-21swapper/021:41:470
11982995817,29cyclictest8841-21taskset23:28:560
1198299581,44cyclictest29106-21taskset22:56:400
1198299581,44cyclictest19849-21taskset21:01:090
1198299581,11cyclictest0-21swapper/020:23:320
1198499574,49cyclictest0-21swapper/222:27:552
1198499572,44cyclictest0-21swapper/200:04:132
11984995716,38cyclictest10350-21unixbench_singl20:33:142
1198499571,4cyclictest16390-21/usr/sbin/munin20:53:042
1198499571,4cyclictest0-21swapper/223:13:072
1198299578,3cyclictest0-21swapper/022:05:360
1198299572,45cyclictest0-21swapper/000:30:410
11982995710,36cyclictest28840-21chrt19:57:020
11984995613,28cyclictest23499-21/usr/sbin/munin19:43:112
1198399562,11cyclictest7750irq/56-dwc_otg22:47:541
1198299562,41cyclictest0-21swapper/022:45:550
1198499553,4cyclictest7750irq/56-dwc_otg22:11:012
1198499553,4cyclictest0-21swapper/223:02:552
1198499551,5cyclictest14147-21latency_hist20:45:022
1198499551,5cyclictest13311-21sed23:45:022
1198499541,5cyclictest7950irq/56-dwc_otg_00:17:572
1198499541,3cyclictest0-21swapper/220:48:032
11984995410,41cyclictest31346-21latency_hist00:35:022
1198499532,5cyclictest21803-21apt-get21:07:542
1198499532,5cyclictest21803-21apt-get21:07:542
1198499532,4cyclictest0-21swapper/221:22:572
1198499531,4cyclictest28835-21sshd19:56:592
1198499531,4cyclictest0-21swapper/200:36:262
11984995310,40cyclictest6184-21missed_timers20:23:092
1198299531,2cyclictest0-21swapper/019:11:190
1198499529,39cyclictest21333-21unixbench_singl22:33:142
1198499524,37cyclictest7850irq/56-dwc_otg_20:40:012
1198499522,3cyclictest0-21swapper/222:22:572
1198499522,3cyclictest0-21swapper/222:22:572
1198499521,6cyclictest1869-21/usr/sbin/munin20:13:082
11984995215,33cyclictest10293-21grep23:38:022
1198499521,4cyclictest7950irq/56-dwc_otg_21:41:122
1198499521,4cyclictest26388-21apt-get00:22:592
1198499521,43cyclictest0-21swapper/223:29:162
1198499516,41cyclictest27057-21latency_hist22:50:022
1198499512,46cyclictest358-21cut20:08:042
1198499511,7cyclictest24129-21fschecks_time22:43:002
1198499511,6cyclictest13436-21chrt19:13:412
1198499511,4cyclictest0-21swapper/222:57:022
1198499511,3cyclictest7950irq/56-dwc_otg_22:37:572
1198499508,39cyclictest8828-21latency_hist22:00:022
1198499502,6cyclictest22115-21grep00:08:102
1198499502,37cyclictest0-21swapper/221:17:352
1198499501,6cyclictest32612-21/usr/sbin/munin21:38:102
1198499501,6cyclictest23725-21cpu21:12:582
1198499501,5cyclictest0-21swapper/200:11:552
1198499501,4cyclictest27005-21chrt19:51:362
1198499501,4cyclictest0-21swapper/221:32:572
1198499501,46cyclictest12432-21munin-plugin-st22:10:012
11984995014,33cyclictest199292chrt21:02:122
1198299502,35cyclictest0-21swapper/023:03:080
1198499492,43cyclictest18956-21proc_pri20:58:082
1198499491,6cyclictest0-21swapper/220:27:592
1198499491,4cyclictest18065-21chrt19:27:252
1198499491,4cyclictest18065-21chrt19:27:252
1198499491,42cyclictest128-21systemd-journal22:55:022
1198499491,38cyclictest7850irq/56-dwc_otg_22:19:252
1198399493,8cyclictest0-21swapper/121:32:591
1198599482,11cyclictest0-21swapper/323:47:583
1198499485,40cyclictest0-21swapper/219:32:592
1198499485,40cyclictest0-21swapper/219:32:592
1198499482,4cyclictest0-21swapper/219:23:002
1198499482,42cyclictest128-21systemd-journal23:35:012
1198499482,3cyclictest0-21swapper/223:57:582
1198499482,3cyclictest0-21swapper/223:57:582
1198499481,5cyclictest0-21swapper/219:38:112
1198499481,4cyclictest6551-21munin-run21:55:012
1198499481,4cyclictest3777-21chrt21:46:472
1198499471,4cyclictest25326-21apt-get19:47:562
87242461,37sleep38905-21cut20:30:023
1198499462,5cyclictest7950irq/56-dwc_otg_19:16:242
1198399467,24cyclictest0-21swapper/122:25:071
10732460,10sleep123-21ksoftirqd/120:08:131
1198599457,24cyclictest0-21swapper/322:25:073
1198599457,24cyclictest0-21swapper/321:40:083
1198499456,4cyclictest7750irq/56-dwc_otg20:02:582
1198399457,26cyclictest0-21swapper/121:52:571
1198399451,11cyclictest0-21swapper/120:32:591
1198599447,24cyclictest0-21swapper/322:47:573
1198399441,7cyclictest7750irq/56-dwc_otg23:17:561
1198599437,24cyclictest0-21swapper/320:52:573
1198399427,26cyclictest0-21swapper/121:27:591
1198399427,25cyclictest0-21swapper/121:17:591
1198599417,23cyclictest0-21swapper/319:58:013
1198399416,11cyclictest7850irq/56-dwc_otg_20:43:001
1198399413,5cyclictest0-21swapper/121:22:561
128802400,5sleep11198399cyclictest19:13:161
1198599407,24cyclictest0-21swapper/320:08:003
1198399407,25cyclictest0-21swapper/100:38:001
1198599397,23cyclictest0-21swapper/319:23:003
1198599387,22cyclictest0-21swapper/322:38:013
1198599387,21cyclictest0-21swapper/322:12:563
1198599387,21cyclictest0-21swapper/321:37:563
1198599387,21cyclictest0-21swapper/320:17:583
1198599386,22cyclictest0-21swapper/322:57:573
11985993816,5cyclictest1125-21runrttasks20:58:063
11985993715,6cyclictest25558-21forks19:48:013
11985993710,10cyclictest7950irq/56-dwc_otg_20:23:163
1198399377,24cyclictest0-21swapper/120:15:071
11983993717,12cyclictest23-21ksoftirqd/100:20:011
11983993626,6cyclictest20644-21latency_hist00:05:021
1198399361,25cyclictest0-21swapper/122:12:561
1198599357,22cyclictest0-21swapper/321:52:573
1198599351,24cyclictest4234-21cut23:18:023
1198399352,7cyclictest7750irq/56-dwc_otg23:12:551
1198599349,12cyclictest7950irq/56-dwc_otg_00:33:083
1198599347,13cyclictest7850irq/56-dwc_otg_21:48:103
1198599344,7cyclictest28554-21head22:53:103
1198599342,2cyclictest0-21swapper/300:37:573
11985993417,6cyclictest9868-21missed_timers22:03:053
1198599341,23cyclictest2531-21idleruntime23:13:013
11985993411,10cyclictest7950irq/56-dwc_otg_22:23:043
11985993411,10cyclictest7950irq/56-dwc_otg_22:23:043
1198399344,6cyclictest7850irq/56-dwc_otg_23:58:151
1198399344,6cyclictest7850irq/56-dwc_otg_23:58:151
1198399342,20cyclictest0-21swapper/123:48:021
1198599337,24cyclictest0-21swapper/300:12:593
1198599337,16cyclictest0-21swapper/321:27:593
1198599336,6cyclictest7950irq/56-dwc_otg_22:18:143
1198599334,5cyclictest7750irq/56-dwc_otg23:28:043
1198599331,22cyclictest19099-21timerwakeupswit19:28:133
1198599331,22cyclictest19099-21timerwakeupswit19:28:133
1198599331,21cyclictest13755-21ls20:43:143
1198599331,21cyclictest12329-21latency23:43:033
1198399339,6cyclictest6709-21grep20:23:161
1198399335,6cyclictest7750irq/56-dwc_otg22:43:001
1198399332,21cyclictest0-21swapper/123:31:541
1198399331,28cyclictest20172-21cut19:33:031
1198399331,28cyclictest20172-21cut19:33:031
11983993311,5cyclictest7750irq/56-dwc_otg23:05:021
1198599327,15cyclictest0-21swapper/319:38:013
11985993213,8cyclictest23815-21sh21:12:593
1198399327,15cyclictest0-21swapper/119:23:001
1198399327,14cyclictest7950irq/56-dwc_otg_20:15:021
1198399327,10cyclictest0-21swapper/119:42:561
1198399321,8cyclictest32065-21latency_hist20:05:011
1198399321,24cyclictest27442-21cut19:53:041
1198399321,21cyclictest15770-21uniq23:53:011
1198399321,20cyclictest5848-21wget23:22:571
114212320,4sleep339-21ksoftirqd/322:08:013
1198599319,6cyclictest381ktimersoftd/321:08:103
1198599319,6cyclictest381ktimersoftd/321:08:103
1198599318,7cyclictest24115-21fschecks_time22:43:003
1198599318,5cyclictest7950irq/56-dwc_otg_20:47:583
1198599317,14cyclictest0-21swapper/320:38:003
1198599316,2cyclictest0-21swapper/319:42:563
1198599314,13cyclictest7950irq/56-dwc_otg_21:18:033
1198599313,6cyclictest7950irq/56-dwc_otg_23:05:023
11985993114,9cyclictest1125-21runrttasks00:21:143
1198599311,22cyclictest21477-21latency00:08:033
11985993111,3cyclictest7750irq/56-dwc_otg19:18:033
1198399316,5cyclictest7750irq/56-dwc_otg21:40:081
1198399316,17cyclictest0-21swapper/122:10:021
1198399314,5cyclictest7750irq/56-dwc_otg00:07:551
11983993115,5cyclictest19251-21/usr/sbin/munin20:58:111
1198399311,27cyclictest24991-21ls21:15:011
1198399311,24cyclictest22791-21ls21:08:091
1198399311,24cyclictest22791-21ls21:08:091
1198399311,21cyclictest20321-21uniq21:03:021
1198399311,20cyclictest23709-21timerandwakeup00:13:121
1198399311,20cyclictest22259-21uniq22:38:041
1198399311,15cyclictest7850irq/56-dwc_otg_20:55:021
1198599304,6cyclictest7950irq/56-dwc_otg_19:53:093
11985993010,5cyclictest28230-21timerwakeupswit21:23:103
1198399308,5cyclictest7750irq/56-dwc_otg23:38:011
1198399305,6cyclictest7750irq/56-dwc_otg22:00:011
11983993023,5cyclictest7678-21wget23:27:571
1198399301,4cyclictest7750irq/56-dwc_otg19:27:571
1198399301,4cyclictest7750irq/56-dwc_otg19:27:571
1198599291,23cyclictest21288-21ls19:35:013
1198599291,23cyclictest21288-21ls19:35:013
11985992912,11cyclictest10137-21timerandwakeup20:33:133
1198399295,5cyclictest7850irq/56-dwc_otg_00:33:101
1198399292,5cyclictest14039-21timerandwakeup19:15:301
11983992910,4cyclictest221ktimersoftd/119:59:051
1198599287,12cyclictest0-21swapper/319:13:003
1198599284,4cyclictest7950irq/56-dwc_otg_00:03:013
1198599281,8cyclictest2759-21switchtime20:13:123
1198599281,6cyclictest17303-21cut23:58:013
1198599281,6cyclictest17303-21cut23:58:013
11985992814,5cyclictest366-21aten_rbpower_cu23:07:573
11985992814,5cyclictest366-21aten_rbpower_cu23:07:573
11985992812,9cyclictest31286-21memory20:03:083
1198599281,16cyclictest25076-21ls00:18:033
11985992810,6cyclictest0-21swapper/300:28:003
1198399288,7cyclictest7850irq/56-dwc_otg_20:48:001
1198399288,5cyclictest7750irq/56-dwc_otg22:58:021
1198399283,4cyclictest0-21swapper/119:37:581
1198399283,19cyclictest23-21ksoftirqd/119:47:561
1198399283,12cyclictest7750irq/56-dwc_otg23:43:001
1198399282,13cyclictest0-21swapper/122:53:001
1198399281,6cyclictest26808-21/usr/sbin/munin00:23:061
11983992812,5cyclictest3985-21aten_rbpower_vo21:47:571
11983992812,5cyclictest28803-21cat00:28:051
1198399281,16cyclictest7750irq/56-dwc_otg23:06:271
1198399281,16cyclictest7750irq/56-dwc_otg23:06:271
1198599272,11cyclictest7850irq/56-dwc_otg_23:23:033
1198399277,10cyclictest7950irq/56-dwc_otg_20:38:091
1198399276,11cyclictest7750irq/56-dwc_otg22:03:011
1198399274,5cyclictest7950irq/56-dwc_otg_22:33:081
1198399273,4cyclictest7750irq/56-dwc_otg22:23:081
1198399273,4cyclictest7750irq/56-dwc_otg22:23:081
1198399271,7cyclictest14940-21uniq22:18:041
1198599265,4cyclictest7850irq/56-dwc_otg_21:55:263
1198599263,6cyclictest7950irq/56-dwc_otg_21:03:003
1198599261,11cyclictest7750irq/56-dwc_otg23:37:123
110022263,14sleep30-21swapper/319:08:153
1198599252,13cyclictest7950irq/56-dwc_otg_23:52:483
11985992516,6cyclictest21304-21sed22:33:143
11985992418,4cyclictest9635-21latency_hist23:35:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional