You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 11:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Mar 30, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
536399645,38cyclictest14133-21ssh00:05:151
536399615,49cyclictest7733-21ssh21:55:121
536399613,10cyclictest131rcu_preempt22:45:141
535599614,49cyclictest25651-21ssh22:05:120
537199602,46cyclictest3598-21ssh23:05:122
536399583,40cyclictest5147-21ssh00:00:121
537199543,45cyclictest26136-21apt-get20:05:132
536399524,39cyclictest17691-21ssh00:25:131
536399503,6cyclictest131rcu_preempt23:45:121
537199492,42cyclictest9027-21apt-get22:50:102
537199490,44cyclictest8089-21apt-get20:20:132
536399495,37cyclictest31142-21ssh21:50:151
538399473,28cyclictest28366irq/58-eth021:45:133
537199471,42cyclictest0-21swapper/219:15:122
536399471,37cyclictest30642-21ssh23:20:141
537199461,39cyclictest22346irq/40-dwc2_hso19:40:142
537199460,43cyclictest0-21swapper/220:40:122
535599454,32cyclictest12524-21ssh23:10:120
537199432,30cyclictest0-21swapper/222:05:122
537199430,33cyclictest1743-21apt-get23:40:132
537199430,28cyclictest23483-21diskmemload23:18:552
537199421,36cyclictest0-21swapper/221:15:142
537199421,33cyclictest9038-21sh22:12:552
537199421,29cyclictest28582-21ssh23:55:132
536399421,38cyclictest27501-21ssh21:30:121
538399413,28cyclictest28366irq/58-eth022:35:143
537199413,31cyclictest5601-21apt-get22:30:152
537199412,34cyclictest0-21swapper/200:15:142
537199411,34cyclictest0-21swapper/221:40:132
537199411,21cyclictest7150-21sh23:25:132
537199410,36cyclictest924-21snmpd23:20:122
537199410,29cyclictest23483-21diskmemload22:44:552
537199410,29cyclictest2137-21ssh22:26:172
536399414,6cyclictest131rcu_preempt21:15:141
537199403,27cyclictest16144-21ssh23:30:112
537199402,34cyclictest0-21swapper/221:55:132
537199401,36cyclictest0-21swapper/223:45:132
537199401,35cyclictest0-21swapper/220:25:112
537199400,37cyclictest20093-21apt-get22:20:132
537199400,37cyclictest20093-21apt-get22:20:122
537199400,26cyclictest23483-21diskmemload00:32:102
538399392,22cyclictest28366irq/58-eth022:10:133
537199393,30cyclictest0-21swapper/219:55:102
537199391,32cyclictest4071-21apt-get21:35:112
537199390,26cyclictest20512-21ssh00:07:242
537199390,25cyclictest0-21swapper/223:13:032
536399393,29cyclictest26724-21ssh00:30:131
535599391,28cyclictest21599-21ssh23:15:130
5355993911,3cyclictest131rcu_preempt23:00:110
538399382,23cyclictest28366irq/58-eth021:05:133
537199381,33cyclictest0-21swapper/221:05:122
537199381,24cyclictest0-21swapper/221:26:482
537199380,27cyclictest20856-21ssh22:37:162
537199380,25cyclictest0-21swapper/200:01:392
536399381,30cyclictest16695-21ssh22:00:141
538399372,30cyclictest28366irq/58-eth023:30:133
537199372,32cyclictest22346irq/40-dwc2_hso00:35:142
537199372,32cyclictest22346irq/40-dwc2_hso00:35:132
537199372,31cyclictest11591-21apt-get19:50:132
537199371,32cyclictest0-21swapper/221:45:132
537199371,31cyclictest30916-21apt-get20:10:122
537199371,20cyclictest14963-21ssh21:21:152
537199370,34cyclictest0-21swapper/222:45:132
537199370,32cyclictest23047-21apt-get00:10:132
537199370,32cyclictest11162-21ssh22:15:152
537199370,24cyclictest25096-21ssh22:04:482
5363993724,10cyclictest14522-21ssh22:35:131
5363993714,12cyclictest5346-21cyclictest20:30:141
535599371,32cyclictest18028-21rm22:55:110
537199361,30cyclictest31156-21ssh21:30:252
537199361,29cyclictest6792-21apt-get19:45:122
537199360,22cyclictest23483-21diskmemload23:35:452
537199351,29cyclictest2388-21ntp_states21:50:262
537199351,29cyclictest0-21swapper/220:15:132
537199350,23cyclictest25767-21wc20:35:222
537199350,22cyclictest24388-21sh22:57:172
538399344,20cyclictest28366irq/58-eth022:00:143
537199341,28cyclictest0-21swapper/220:30:142
537199341,25cyclictest26988-21ssh23:53:312
537199340,32cyclictest15532-21ssh00:22:562
537199340,25cyclictest24239-21ssh00:27:322
536399343,26cyclictest20115-21ssh22:20:131
536399343,26cyclictest20115-21ssh22:20:121
536399343,24cyclictest9495-21ssh21:20:111
538399333,19cyclictest28366irq/58-eth000:05:133
537199331,28cyclictest28406-21cat19:30:252
537199331,24cyclictest27043-21apt-get23:00:092
5371993311,19cyclictest23829-21ssh21:10:042
535599332,21cyclictest29109-21ssh22:25:140
5355993321,8cyclictest0-21swapper/019:40:140
538399321,6cyclictest0-21swapper/323:35:123
537199321,27cyclictest23724-21grep19:25:292
537199321,27cyclictest15374-21hwlatdetect19:20:162
537199321,23cyclictest18640-21vmstat21:00:332
537199321,23cyclictest18640-21vmstat21:00:322
537199320,28cyclictest573-21tr19:35:242
537199320,25cyclictest13847-21sshd20:55:382
5363993220,6cyclictest0-21swapper/100:20:121
5355993215,5cyclictest0-21swapper/020:40:110
538399313,19cyclictest28366irq/58-eth020:45:123
537199311,28cyclictest3090-21ntp_192.168.11420:45:242
537199311,26cyclictest4527-21cpu20:50:132
536399312,22cyclictest11544-21sh21:38:371
535599313,16cyclictest2199-21ssh22:10:130
537199301,26cyclictest7373-21kthreadcore19:10:222
537199301,26cyclictest21337-21df20:00:152
535599304,18cyclictest12-21ksoftirqd/021:30:120
535599301,22cyclictest23551-21ssh22:40:130
538399293,13cyclictest28366irq/58-eth020:55:123
538399292,4cyclictest0-21swapper/321:25:173
538399291,9cyclictest0-21swapper/323:20:143
538399291,8cyclictest0-21swapper/319:40:133
538399291,4cyclictest0-21swapper/322:05:113
538399291,12cyclictest28366irq/58-eth021:20:113
535599294,16cyclictest0-21swapper/000:30:140
535599292,22cyclictest22099-21ssh21:45:130
536399286,10cyclictest0-21swapper/122:40:131
536399282,19cyclictest27073-21ssh23:00:121
535599283,8cyclictest12-21ksoftirqd/023:35:140
5355992817,6cyclictest0-21swapper/019:30:110
538399274,4cyclictest0-21swapper/319:20:123
538399272,4cyclictest0-21swapper/320:25:123
538399271,5cyclictest0-21swapper/323:55:193
538399271,5cyclictest0-21swapper/322:55:183
538399271,4cyclictest0-21swapper/321:30:153
536399272,18cyclictest73-21kswapd023:05:121
536399271,19cyclictest18528-21ssh21:25:181
5363992711,13cyclictest7142-21ssh23:25:121
535599277,6cyclictest12-21ksoftirqd/021:25:270
535599272,20cyclictest9847-21ssh21:36:350
535599272,18cyclictest28576-21ssh23:55:110
535599272,14cyclictest5621-21sh22:30:140
5355992714,8cyclictest531-21usb-storage19:55:110
535599271,18cyclictest23483-21diskmemload23:05:130
538399267,4cyclictest0-21swapper/322:32:353
538399261,6cyclictest0-21swapper/322:25:263
538399261,6cyclictest0-21swapper/319:45:233
538399261,5cyclictest0-21swapper/300:22:043
536399267,10cyclictest21-21ksoftirqd/123:35:121
536399262,18cyclictest16169-21ssh23:30:141
536399262,15cyclictest29090-21ssh22:25:131
535599269,10cyclictest12-21ksoftirqd/023:26:130
535599265,12cyclictest0-21swapper/020:20:120
535599262,18cyclictest32084-21ssh00:15:160
5355992614,8cyclictest12-21ksoftirqd/020:10:140
538399254,6cyclictest0-21swapper/300:25:083
538399252,4cyclictest0-21swapper/321:35:223
538399251,6cyclictest0-21swapper/323:15:033
538399251,6cyclictest0-21swapper/300:30:313
538399251,5cyclictest0-21swapper/323:50:293
538399251,5cyclictest0-21swapper/323:25:043
538399251,5cyclictest0-21swapper/321:40:183
538399251,5cyclictest0-21swapper/300:00:213
536399257,4cyclictest5760-21sed22:10:231
536399252,18cyclictest18804-21sshd21:41:351
5363992515,7cyclictest14848-21sshd22:51:381
536399251,18cyclictest12544-21ssh23:10:131
5363992511,10cyclictest30725-21cron21:50:001
535599251,5cyclictest12-21ksoftirqd/020:00:160
538399242,4cyclictest0-21swapper/321:55:203
538399242,3cyclictest0-21swapper/319:25:143
538399241,9cyclictest0-21swapper/322:40:313
538399241,6cyclictest0-21swapper/323:15:153
538399241,6cyclictest0-21swapper/321:15:183
538399241,6cyclictest0-21swapper/319:30:203
538399241,6cyclictest0-21swapper/300:15:203
538399241,5cyclictest0-21swapper/322:45:303
538399241,5cyclictest0-21swapper/322:16:203
538399241,5cyclictest0-21swapper/321:13:443
538399241,5cyclictest0-21swapper/319:15:193
538399241,5cyclictest0-21swapper/300:35:263
538399241,5cyclictest0-21swapper/300:35:263
537199241,20cyclictest5375-21latency_hist19:10:022
536399241,19cyclictest5631-21sh22:30:141
536399241,17cyclictest73-21kswapd022:55:111
535599247,12cyclictest12-21ksoftirqd/000:38:460
535599247,12cyclictest12-21ksoftirqd/000:38:450
535599243,15cyclictest0-21swapper/020:15:130
535599241,17cyclictest23077-21ssh00:10:150
538399231,7cyclictest0-21swapper/300:10:333
538399231,6cyclictest0-21swapper/321:50:173
538399231,6cyclictest0-21swapper/320:35:183
538399231,5cyclictest0-21swapper/323:05:143
538399231,5cyclictest0-21swapper/322:20:273
538399231,5cyclictest0-21swapper/322:20:263
538399231,5cyclictest0-21swapper/320:30:213
538399231,5cyclictest0-21swapper/319:55:193
538399231,5cyclictest0-21swapper/319:55:013
538399231,5cyclictest0-21swapper/319:40:023
538399231,5cyclictest0-21swapper/319:10:213
538399231,4cyclictest0-21swapper/323:00:133
538399231,4cyclictest0-21swapper/320:15:293
538399231,19cyclictest0-21swapper/323:40:223
536399233,12cyclictest0-21swapper/120:45:111
536399232,14cyclictest21586-21ssh23:15:121
536399230,18cyclictest14360-21ls21:00:181
536399230,18cyclictest14360-21ls21:00:171
535599239,11cyclictest12-21ksoftirqd/021:10:130
535599236,6cyclictest12-21ksoftirqd/021:55:250
535599233,13cyclictest0-21swapper/020:50:130
535599232,4cyclictest12-21ksoftirqd/021:18:430
538399222,5cyclictest0-21swapper/323:45:193
538399221,6cyclictest0-21swapper/322:50:473
538399221,6cyclictest0-21swapper/320:40:203
538399221,6cyclictest0-21swapper/320:20:153
538399221,5cyclictest0-21swapper/321:00:203
538399221,5cyclictest0-21swapper/321:00:193
536399222,16cyclictest3242-21diskmemload00:35:151
536399222,16cyclictest3242-21diskmemload00:35:141
536399221,16cyclictest30960-21ls20:40:271
536399221,16cyclictest1774-21ssh23:40:141
536399221,15cyclictest3189-21ls00:15:241
536399220,19cyclictest13030-21cut20:25:141
535599229,3cyclictest131rcu_preempt00:29:400
535599229,3cyclictest131rcu_preempt00:09:030
535599222,14cyclictest15579-21ssh00:22:590
535599221,18cyclictest20084-21cut19:25:160
5355992210,6cyclictest131rcu_preempt22:35:230
535599220,18cyclictest22601-21cut20:35:160
538399211,6cyclictest0-21swapper/320:10:203
538399211,6cyclictest0-21swapper/320:00:143
538399211,5cyclictest0-21swapper/320:50:163
538399211,5cyclictest0-21swapper/320:05:203
536399217,11cyclictest30640-21cut23:55:211
536399211,15cyclictest24787-21apt-get19:30:091
535599219,4cyclictest12-21ksoftirqd/022:48:510
535599217,6cyclictest131rcu_preempt22:21:330
535599217,6cyclictest131rcu_preempt22:21:320
535599217,5cyclictest12-21ksoftirqd/021:53:380
535599215,8cyclictest12-21ksoftirqd/023:24:360
535599214,3cyclictest131rcu_preempt21:43:420
5355992112,6cyclictest6851-21cut00:00:190
5355992112,3cyclictest0-21swapper/022:51:150
5355992111,7cyclictest21343-21ssh22:00:300
536399201,9cyclictest24033-21tr00:10:211
5363992012,5cyclictest20774-21kthreadcore21:05:211
536399201,16cyclictest6777-21find19:45:131
5363992011,5cyclictest28300-21usage_Anybus21:10:301
5363992010,5cyclictest27644-21kthreadcore22:05:191
535599208,3cyclictest131rcu_preempt23:53:030
535599207,3cyclictest131rcu_preempt23:45:430
535599204,11cyclictest131rcu_preempt23:30:030
5355992013,5cyclictest0-21swapper/020:05:130
535599201,11cyclictest0-21swapper/023:43:350
5355992010,4cyclictest0-21swapper/022:15:360
538399191,6cyclictest0-21swapper/319:10:023
536399192,8cyclictest5346-21cyclictest19:10:121
536399192,4cyclictest53146usb-storage19:55:051
536399191,14cyclictest23363-21ssh23:50:261
536399191,14cyclictest12148-21cut22:15:211
535599197,4cyclictest12-21ksoftirqd/019:50:280
535599196,6cyclictest12-21ksoftirqd/021:00:300
535599196,6cyclictest12-21ksoftirqd/021:00:290
535599191,9cyclictest12-21ksoftirqd/021:20:540
5355991914,3cyclictest0-21swapper/021:05:320
5355991910,4cyclictest12-21ksoftirqd/020:55:150
535599190,16cyclictest10178-21ls19:15:020
535599190,13cyclictest10061-21latency19:45:230
536399186,9cyclictest25886-21ntp_kernel_pll_20:35:251
536399185,6cyclictest21-21ksoftirqd/119:15:141
536399181,4cyclictest0-21swapper/120:05:131
536399181,14cyclictest11918-21sed20:20:291
535599186,4cyclictest12-21ksoftirqd/020:25:260
535599185,10cyclictest21012-21wc20:30:230
535599183,4cyclictest12-21ksoftirqd/020:45:160
5355991812,4cyclictest0-21swapper/019:35:110
536399179,5cyclictest687-21in:imuxsock19:45:001
536399174,10cyclictest633-21Async12:11:531
536399171,7cyclictest16500-21kthreadcore19:20:181
5363991711,4cyclictest0-21swapper/120:15:171
5363991711,4cyclictest0-21swapper/119:50:141
536399171,12cyclictest9232-21ls20:55:021
536399171,12cyclictest29821-21grep19:35:171
536399170,14cyclictest20129-21fschecks_time19:25:171
535599174,5cyclictest12-21ksoftirqd/019:15:220
5355991710,5cyclictest18583-21ntp_kernel_err19:20:240
536399162,7cyclictest9773-21irqrtprio20:55:161
536399162,10cyclictest31026-21uniq20:10:171
536399141,10cyclictest5412-21latency_hist19:10:021
535599141,10cyclictest5356-21latency_hist19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional