You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 06:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Tue May 26, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2291996716,9cyclictest131rcu_preempt20:50:140
229199636,10cyclictest131rcu_preempt23:25:130
233299583,36cyclictest20717-21ssh21:10:122
229199552,41cyclictest20552-21ssh23:15:130
231499544,24cyclictest2423-21ssh23:05:131
229199543,8cyclictest131rcu_preempt21:10:130
233299518,19cyclictest27-21ksoftirqd/219:20:122
229199503,39cyclictest17593-21ssh00:25:150
229199503,39cyclictest17593-21ssh00:25:150
231499491,39cyclictest27276-21ssh22:25:131
233299472,24cyclictest27-21ksoftirqd/219:50:132
233299472,24cyclictest27-21ksoftirqd/219:50:132
229199475,8cyclictest131rcu_preempt20:15:140
229199475,8cyclictest131rcu_preempt20:15:140
233299461,30cyclictest0-21swapper/222:45:142
229199461,30cyclictest23419-21ssh22:05:120
2332994525,10cyclictest0-21swapper/223:10:122
231499452,35cyclictest1091-21ssh23:40:121
231499451,29cyclictest8564-21ssh00:20:141
229199444,26cyclictest29836-21ssh21:15:130
229199401,32cyclictest13860-21ssh00:05:140
233299385,25cyclictest9224-21ssh22:15:132
233299380,30cyclictest28582-21ssh21:50:132
229199383,32cyclictest19139-21ssh23:50:120
235199371,20cyclictest35966irq/58-eth022:30:143
233299372,18cyclictest5807-21usage_EthcTask23:40:292
231499373,29cyclictest10409-21ssh21:40:121
2314993716,15cyclictest21-21ksoftirqd/121:05:131
2291993710,20cyclictest12-21ksoftirqd/020:20:140
235199361,22cyclictest35966irq/58-eth000:15:133
233299352,27cyclictest10789-21cat21:20:302
2332993521,11cyclictest0-21swapper/222:10:132
2351993417,13cyclictest0-21swapper/300:25:153
2351993417,13cyclictest0-21swapper/300:25:153
235199341,21cyclictest35966irq/58-eth022:15:123
229199344,24cyclictest29635-21rm23:20:120
231499333,25cyclictest26557-21ssh00:30:121
231499331,22cyclictest3172-21ssh00:35:131
2291993313,16cyclictest12-21ksoftirqd/022:20:140
233299325,16cyclictest27346irq/40-dwc2_hso20:15:142
233299325,16cyclictest27346irq/40-dwc2_hso20:15:132
233299322,25cyclictest19940-21rm23:30:292
233299321,26cyclictest16831-21ssh22:55:122
231499324,17cyclictest21-21ksoftirqd/119:30:111
235199314,17cyclictest35966irq/58-eth023:15:133
235199312,19cyclictest35966irq/58-eth019:55:123
233299314,16cyclictest27346irq/40-dwc2_hso00:10:112
233299313,21cyclictest0-21swapper/222:00:292
233299313,16cyclictest27346irq/40-dwc2_hso00:05:132
233299312,13cyclictest13125-21kworker/u8:021:30:302
229199313,22cyclictest31081-21ssh22:45:150
233299302,23cyclictest8563-21diskmemload00:20:142
233299302,18cyclictest12960-21ssh22:35:132
233299301,16cyclictest31934-21ssh00:15:142
231499301,22cyclictest7743-21ssh22:50:121
235199293,19cyclictest33-21ksoftirqd/323:35:123
2351992916,9cyclictest35966irq/58-eth022:55:243
2351992912,11cyclictest35966irq/58-eth023:05:133
233299296,18cyclictest29130-21ssh22:06:292
233299293,20cyclictest0-21swapper/222:22:452
233299293,15cyclictest29852-21ssh21:15:152
233299292,14cyclictest19518-21ssh21:45:142
231499292,22cyclictest30832-21ssh22:08:301
229199294,20cyclictest17016-21sh21:42:330
229199293,18cyclictest12-21ksoftirqd/022:40:130
229199291,7cyclictest131rcu_preempt23:35:120
235199284,11cyclictest35966irq/58-eth023:50:123
235199283,15cyclictest35966irq/58-eth021:45:123
235199283,14cyclictest35966irq/58-eth000:10:103
233299286,6cyclictest0-21swapper/221:42:332
233299286,12cyclictest27346irq/40-dwc2_hso19:45:122
233299284,14cyclictest27346irq/40-dwc2_hso20:40:122
233299283,20cyclictest0-21swapper/222:32:302
233299283,19cyclictest27346irq/40-dwc2_hso19:25:142
233299282,21cyclictest6272-21ssh23:25:152
2332992820,6cyclictest27-21ksoftirqd/221:35:122
231499283,17cyclictest20324-21diskmemload21:25:161
229199283,19cyclictest12950-21ssh22:35:130
229199280,19cyclictest0-21swapper/019:20:120
235199272,16cyclictest0-21swapper/321:20:313
233299274,16cyclictest27346irq/40-dwc2_hso21:05:142
233299273,18cyclictest27346irq/40-dwc2_hso20:30:122
233299272,17cyclictest11765-21users19:15:312
2332992713,11cyclictest0-21swapper/220:55:162
2332992712,13cyclictest27-21ksoftirqd/220:05:122
231499275,16cyclictest24742-21ssh21:30:131
231499272,19cyclictest967-21ssh21:50:291
231499272,19cyclictest24992-21sh22:59:291
231499271,3cyclictest0-21swapper/123:35:131
229199272,7cyclictest131rcu_preempt20:40:120
229199272,19cyclictest18563-21cat22:00:300
229199272,19cyclictest13853-21cat22:15:290
229199272,18cyclictest4322-21uptime22:10:290
229199271,3cyclictest131rcu_preempt20:30:120
2291992710,14cyclictest12-21ksoftirqd/000:10:120
235199265,11cyclictest35966irq/58-eth023:00:123
235199263,15cyclictest0-21swapper/323:40:293
235199263,14cyclictest0-21swapper/322:08:293
233299263,17cyclictest27346irq/40-dwc2_hso19:35:112
233299263,16cyclictest27346irq/40-dwc2_hso19:55:132
233299263,15cyclictest27346irq/40-dwc2_hso23:20:132
233299263,14cyclictest27346irq/40-dwc2_hso19:40:112
2332992612,10cyclictest27-21ksoftirqd/221:30:012
2332992611,7cyclictest1-21systemd00:27:522
2332992611,7cyclictest1-21systemd00:27:512
231499262,18cyclictest22499-21cat22:20:291
2351992514,9cyclictest0-21swapper/322:30:023
2351992511,9cyclictest35966irq/58-eth000:20:143
2351992511,8cyclictest41-21kworker/3:121:35:133
2351992511,8cyclictest35966irq/58-eth022:20:133
235199251,15cyclictest35966irq/58-eth019:45:133
233299252,18cyclictest13472-21ssh21:59:292
233299252,11cyclictest1524-21ssh22:27:382
233299251,13cyclictest22124-21ssh22:40:152
233299251,11cyclictest3008566ssh00:30:232
2332992511,11cyclictest28464-21timerwakeupswit23:35:292
229199254,15cyclictest0-21swapper/019:35:110
229199251,18cyclictest515-21ssh22:26:290
235199243,16cyclictest0-21swapper/321:25:153
235199242,16cyclictest35966irq/58-eth021:55:153
2332992413,4cyclictest27-21ksoftirqd/223:46:582
233299241,16cyclictest20561-21ssh23:15:152
233299241,15cyclictest27346irq/40-dwc2_hso20:00:132
233299240,11cyclictest9624-21find20:25:142
229199243,16cyclictest28321-21ssh23:55:150
229199242,16cyclictest3934-21ssh22:30:140
2291992413,6cyclictest7761-21sh22:50:130
235199232,13cyclictest35966irq/58-eth023:30:133
235199231,13cyclictest0-21swapper/321:15:143
233299232,16cyclictest5571-21tr00:35:232
233299231,7cyclictest29765-21timerandwakeup23:00:292
231499234,9cyclictest0-21swapper/120:45:131
231499232,9cyclictest4495-21munin-run00:00:011
231499231,4cyclictest131rcu_preempt22:35:191
231499231,17cyclictest31306-21apt-get19:40:101
235199223,13cyclictest0-21swapper/322:01:293
235199223,13cyclictest0-21swapper/321:42:333
233299228,5cyclictest4681-21latency_hist00:00:022
233299228,4cyclictest13589-21latency_hist00:05:022
2332992211,5cyclictest8018-21ssh23:06:202
233299221,11cyclictest19283-21cut23:50:152
231499226,11cyclictest21-21ksoftirqd/100:05:291
231499222,14cyclictest19150-21sh23:50:141
2314992214,5cyclictest29901-21unixbench_multi23:00:301
231499221,15cyclictest22168-21cut22:40:161
2314992211,3cyclictest131rcu_preempt23:30:241
229199224,13cyclictest20324-21diskmemload23:40:130
229199223,13cyclictest131rcu_preempt21:50:130
229199222,14cyclictest24368-21users21:45:300
2291992212,5cyclictest0-21swapper/000:00:140
229199221,18cyclictest3882-21cut19:45:150
229199221,15cyclictest5424-21cut21:55:170
235199213,12cyclictest0-21swapper/322:50:133
235199211,13cyclictest0-21swapper/300:35:133
233299211,17cyclictest27813-21fschecks_time20:10:162
233299211,16cyclictest28994-21fschecks_count20:45:172
2332992110,8cyclictest0-21swapper/220:35:182
231499212,12cyclictest3936-21sh22:30:141
231499211,18cyclictest10481-21ls19:15:221
231499211,15cyclictest6468-21ssh21:20:131
231499211,15cyclictest11640-21cut23:10:161
229199217,4cyclictest131rcu_preempt00:30:310
2291992116,3cyclictest20324-21diskmemload23:10:110
2291992112,7cyclictest20317-21cron21:10:010
229199211,16cyclictest6272-21cut20:55:160
229199211,14cyclictest13408-21sshd20:25:310
235199208,9cyclictest0-21swapper/323:10:203
2351992011,7cyclictest0-21swapper/321:30:153
233299202,9cyclictest131rcu_preempt22:51:352
2332992012,5cyclictest0-21swapper/220:50:252
233299201,11cyclictest2621-21sort19:10:152
233299201,10cyclictest8646-21tail20:20:282
233299201,10cyclictest24098-21kthreadcore19:30:202
231499206,9cyclictest2269-21cyclictest20:32:351
231499203,14cyclictest0-21swapper/120:00:151
231499202,12cyclictest16464-21ssh22:18:301
231499201,16cyclictest18253-21ls22:00:271
231499201,15cyclictest21148-21ntp_192.168.11400:25:261
231499201,15cyclictest21148-21ntp_192.168.11400:25:251
231499200,10cyclictest0-21swapper/121:37:401
229199209,8cyclictest21668-21apt-get19:30:130
229199207,5cyclictest131rcu_preempt21:25:010
229199203,10cyclictest0-21swapper/020:45:140
2291992012,6cyclictest11761-21cron00:40:010
235199199,8cyclictest0-21swapper/322:45:453
235199199,7cyclictest0-21swapper/323:55:173
235199194,9cyclictest0-21swapper/300:00:263
235199193,11cyclictest0-21swapper/319:43:303
235199192,9cyclictest35966irq/58-eth020:15:143
235199192,9cyclictest35966irq/58-eth020:15:133
235199192,13cyclictest0-21swapper/323:29:293
235199192,11cyclictest0-21swapper/323:20:133
235199192,11cyclictest0-21swapper/319:30:133
235199192,10cyclictest0-21swapper/320:25:163
2351991913,4cyclictest0-21swapper/321:50:293
235199191,12cyclictest0-21swapper/319:25:283
235199191,11cyclictest0-21swapper/319:20:173
2351991910,6cyclictest0-21swapper/300:32:533
235199190,13cyclictest0-21swapper/320:05:133
233299191,9cyclictest11141-21fschecks_time21:00:162
231499199,8cyclictest2525-21df_inode19:10:141
231499194,11cyclictest5269-21ssh21:55:141
231499193,9cyclictest0-21swapper/121:15:291
2314991911,5cyclictest3598-21latency22:10:221
2314991911,4cyclictest6273-21cpu23:25:141
229199199,8cyclictest13605-21tr21:00:210
229199195,9cyclictest0-21swapper/023:00:120
229199191,16cyclictest12055-21ls00:20:240
229199190,15cyclictest12070-21switchtime19:50:270
229199190,15cyclictest12070-21switchtime19:50:270
229199190,12cyclictest32303-21ls20:15:010
235199189,4cyclictest0-21swapper/323:49:193
235199188,7cyclictest0-21swapper/322:10:263
235199182,11cyclictest0-21swapper/320:32:353
235199182,10cyclictest0-21swapper/321:00:043
235199182,10cyclictest0-21swapper/320:45:143
235199182,10cyclictest0-21swapper/319:15:143
2351991812,4cyclictest0-21swapper/322:40:213
235199181,13cyclictest0-21swapper/300:05:213
2351991810,6cyclictest0-21swapper/321:10:563
231499187,6cyclictest0-21swapper/121:49:311
231499187,3cyclictest131rcu_preempt23:15:311
231499185,10cyclictest20548-21timerandwakeup19:25:291
231499183,12cyclictest7701-21tr20:20:211
231499182,13cyclictest2175-21kthreadcore20:50:211
231499181,9cyclictest7079-21ntp_kernel_pll_19:45:251
2314991813,3cyclictest0-21swapper/120:55:281
2314991813,3cyclictest0-21swapper/100:10:171
231499181,14cyclictest31067-21cut23:20:211
231499181,13cyclictest2347-21awk22:45:261
231499181,12cyclictest0-21swapper/120:05:171
2314991810,6cyclictest31045-21sshd19:38:301
229199185,10cyclictest2748-21grep19:40:280
229199182,6cyclictest10984-21kthreadcore23:45:200
229199182,4cyclictest12-21ksoftirqd/023:05:120
229199182,12cyclictest1370-21diskmemload21:35:130
229199181,5cyclictest0-21swapper/022:55:180
2291991812,3cyclictest12-21ksoftirqd/020:35:320
229199181,15cyclictest15793-21kthreadcore19:55:200
2291991811,4cyclictest15664-21irqcore23:30:160
229199181,13cyclictest24264-21apt-get21:30:030
235199179,6cyclictest0-21swapper/322:35:393
235199173,8cyclictest0-21swapper/320:40:113
235199172,10cyclictest0-21swapper/319:10:143
235199171,10cyclictest35966irq/58-eth021:05:133
233299171,8cyclictest2384-21latency_hist19:10:022
231499178,7cyclictest28183-21irqrtprio20:10:191
231499178,6cyclictest24295-21netstat21:10:231
231499172,11cyclictest22742-21ntp_states20:35:261
231499171,7cyclictest16646-21cron19:25:011
231499171,14cyclictest11166-21kthreadcore19:50:221
231499171,14cyclictest11166-21kthreadcore19:50:211
231499171,13cyclictest4919-21ssh00:00:141
231499171,13cyclictest24257-21cut20:40:151
229199177,8cyclictest8127-21cron00:20:010
229199172,10cyclictest0-21swapper/019:15:140
229199171,6cyclictest17079-21fschecks_time19:25:170
229199171,14cyclictest3519-21kthreadcore19:10:180
229199171,13cyclictest27533-21latency_hist20:10:020
229199171,13cyclictest18028-21find20:00:120
2291991710,4cyclictest16165-21irqrtprio21:25:190
235199161,8cyclictest0-21swapper/319:35:103
235199161,12cyclictest0-21swapper/320:10:163
231499168,6cyclictest17784-21cron20:00:001
231499165,9cyclictest13409-21cat20:25:301
231499163,10cyclictest11276-21if_eth121:00:181
231499162,11cyclictest10124-21ssh23:45:131
231499161,12cyclictest31929-21ssh00:15:131
235199152,9cyclictest0-21swapper/319:50:133
235199152,9cyclictest0-21swapper/319:50:133
235199151,12cyclictest0-21swapper/320:20:163
231499151,11cyclictest32430-21apt-get20:15:131
231499151,11cyclictest32430-21apt-get20:15:131
229199151,10cyclictest2305-21grep19:10:010
235199132,9cyclictest0-21swapper/320:55:153
235199131,10cyclictest0-21swapper/320:50:293
231499131,9cyclictest2327-21latency_hist19:10:011
235199121,9cyclictest0-21swapper/320:35:233
235199111,8cyclictest0-21swapper/320:00:183
235199101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional