You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-21 - 02:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Sep 21, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2512299652,46cyclictest9712-21ssh21:45:122
2510399612,45cyclictest29168-21ssh21:20:110
2510399556,37cyclictest27979-21ssh23:25:140
2511499511,7cyclictest131rcu_preempt23:10:141
25130994725,12cyclictest35966irq/58-eth022:35:153
2513099453,26cyclictest35966irq/58-eth023:40:133
2511499451,35cyclictest17075-21sh23:55:121
2512299444,27cyclictest261rcuc/221:50:172
25114994418,8cyclictest131rcu_preempt19:15:131
2511499432,27cyclictest2623-21ssh00:05:141
25130994218,17cyclictest35966irq/58-eth022:05:133
25122994017,19cyclictest27-21ksoftirqd/222:55:142
25114994025,7cyclictest0-21swapper/121:25:121
2510399401,30cyclictest14911-21ssh00:30:130
25103994010,18cyclictest11090-21ssh21:10:140
25103994010,18cyclictest11090-21ssh21:10:130
25130993921,13cyclictest35966irq/58-eth000:35:143
2512299386,20cyclictest27-21ksoftirqd/220:55:132
2511499382,5cyclictest131rcu_preempt22:15:131
2510399382,30cyclictest9693-21ssh21:45:120
25130993721,8cyclictest0-21swapper/300:10:153
2511499372,26cyclictest11830-21ssh22:40:121
25122993623,8cyclictest27-21ksoftirqd/223:35:162
25122993623,8cyclictest27-21ksoftirqd/223:35:162
2512299362,28cyclictest3464-21sh21:59:242
2511499363,21cyclictest21850-21kworker/u8:023:05:131
25103993629,4cyclictest15481-21apt-get23:00:150
25103993629,4cyclictest15481-21apt-get23:00:140
25122993524,8cyclictest0-21swapper/223:05:132
25114993523,7cyclictest211ksoftirqd/123:35:161
25114993523,7cyclictest211ksoftirqd/123:35:151
25103993530,3cyclictest18798-21apt-get21:50:170
25103993527,5cyclictest4912-21apt-get20:30:160
25103993521,10cyclictest8119-21apt-get22:20:130
2513099345,20cyclictest35966irq/58-eth000:15:143
2513099341,22cyclictest35966irq/58-eth020:45:123
25122993422,5cyclictest27-21ksoftirqd/219:30:152
2512299342,24cyclictest26226-21ntp_192.168.11423:40:262
2511499343,7cyclictest131rcu_preempt19:10:161
2510399348,16cyclictest18258-21kthreadcore21:30:240
2510399348,15cyclictest19836-21cut20:10:210
25103993412,16cyclictest73-21kswapd023:20:120
25103993411,9cyclictest11795-21grep20:00:230
2513099334,18cyclictest35966irq/58-eth022:10:153
2513099333,22cyclictest35966irq/58-eth022:45:143
2513099331,6cyclictest0-21swapper/323:45:153
2512299333,9cyclictest4556-21ssh23:30:142
2511499332,28cyclictest11605-21ssh00:10:141
2511499332,25cyclictest0-21swapper/122:35:141
2510399335,9cyclictest17202-21diskstats23:55:150
25103993328,2cyclictest24452-21apt-get23:05:140
2512299323,13cyclictest11828-21sh22:40:112
2510399328,8cyclictest8856-21grep21:05:230
2510399327,11cyclictest8165-21ntp_192.168.11423:30:230
2510399326,11cyclictest7381-21timerwakeupswit19:55:280
2510399324,10cyclictest30471-21usage_Anybus00:00:310
2510399323,22cyclictest13438-21ssh22:05:130
25103993226,3cyclictest986-21apt-get23:10:130
25130993125,3cyclictest35966irq/58-eth023:55:123
25130993123,5cyclictest35966irq/58-eth022:00:133
25130993121,6cyclictest35966irq/58-eth021:35:143
2512299312,23cyclictest20522-21ssh00:15:172
2510399318,8cyclictest20155-21ssh23:37:350
2510399318,8cyclictest20155-21ssh23:37:340
2510399317,9cyclictest2947-21ps19:20:180
2510399316,8cyclictest10691-21diskmemload00:11:510
25103993124,3cyclictest6386-21ssh22:55:130
25103993116,9cyclictest0-21swapper/019:50:130
25103993114,14cyclictest0-21swapper/022:45:150
25103993113,14cyclictest0-21swapper/023:45:140
2513099304,9cyclictest0-21swapper/321:45:013
2513099303,14cyclictest35966irq/58-eth021:25:133
2512299301,23cyclictest29180-21ssh21:20:132
2511499301,23cyclictest17150-21ssh22:25:141
2510399308,10cyclictest10339-21irqrtprio20:35:160
2510399306,9cyclictest5959-21sh22:17:570
2510399305,10cyclictest22930-21/usr/sbin/munin23:40:180
2512299293,21cyclictest0-21swapper/223:13:252
2512299293,20cyclictest17078-21ssh23:55:112
2512299292,21cyclictest4351-21ssh21:40:252
2511499297,19cyclictest26161-21ssh22:30:151
2510399296,7cyclictest31245-21ssh00:20:190
2510399296,7cyclictest26549-21/usr/sbin/munin19:45:160
2510399295,7cyclictest0-21swapper/023:51:210
2510399295,11cyclictest25356-21cut21:35:190
2510399294,9cyclictest10889-21ssh00:09:400
2510399294,11cyclictest4294-21ntp_192.168.11421:40:240
2510399292,22cyclictest5932-21ssh21:25:130
2513099283,14cyclictest35966irq/58-eth021:20:123
2513099281,10cyclictest0-21swapper/300:10:023
2512299283,10cyclictest0-21swapper/200:34:252
2512299282,19cyclictest0-21swapper/219:50:142
2512299282,14cyclictest5780-21ssh21:25:042
25122992818,5cyclictest27-21ksoftirqd/223:20:122
2512299281,18cyclictest32624-21fschecks_time20:25:182
25122992811,11cyclictest0-21swapper/223:25:122
2511499282,20cyclictest6286-21ssh21:41:241
25114992815,10cyclictest0-21swapper/120:05:131
2511499281,23cyclictest14993-21ssh21:30:151
2510399285,7cyclictest0-21swapper/022:14:260
2510399285,11cyclictest16547-21kthreadcore20:40:210
2510399285,11cyclictest12093-21grep23:15:270
2510399282,21cyclictest2802-21ssh22:35:150
2510399282,19cyclictest20104-21rm21:15:130
25103992812,13cyclictest0-21swapper/022:30:150
25103992811,13cyclictest0-21swapper/000:15:160
2513099271,7cyclictest35966irq/58-eth000:00:133
2513099271,6cyclictest0-21swapper/323:17:013
25130992711,10cyclictest35966irq/58-eth022:57:153
2512299273,19cyclictest0-21swapper/219:40:132
2512299272,20cyclictest628-21tcf-agent00:03:362
2512299271,21cyclictest4419-21ssh22:00:152
2512299270,23cyclictest8118-21ssh22:20:122
2511499273,19cyclictest29927-21ssh22:50:161
2511499272,22cyclictest18530-21ntp_192.168.11400:30:251
2510399274,10cyclictest24845-21ssh22:28:570
2510399273,8cyclictest4393-21apt-get22:00:130
2510399273,8cyclictest15420-21ntp_192.168.11422:40:240
2513099265,6cyclictest0-21swapper/322:54:113
2513099262,14cyclictest35966irq/58-eth023:30:113
2513099261,9cyclictest0-21swapper/321:30:223
2513099261,6cyclictest0-21swapper/320:20:213
2512299264,12cyclictest27346irq/40-dwc2_hso20:10:122
2512299261,19cyclictest20905-21ssh22:45:162
2512299261,18cyclictest3627-21find19:55:132
2510399264,10cyclictest24040-21fschecks_count00:35:160
2510399263,10cyclictest3060-21sh21:58:570
25103992616,5cyclictest30042-21apt-get19:15:130
2513099255,4cyclictest35966irq/58-eth022:25:163
2513099252,4cyclictest0-21swapper/300:33:423
2513099251,7cyclictest0-21swapper/321:45:193
2513099251,10cyclictest0-21swapper/323:10:173
2513099251,10cyclictest0-21swapper/323:00:293
2513099251,10cyclictest0-21swapper/323:00:283
2513099251,10cyclictest0-21swapper/321:10:193
2513099251,10cyclictest0-21swapper/321:10:183
2512299254,12cyclictest27346irq/40-dwc2_hso19:15:132
2512299253,14cyclictest27346irq/40-dwc2_hso00:05:112
2511499252,19cyclictest27954-21ssh00:18:411
2511499251,20cyclictest27839-21ssh21:55:151
2510399259,13cyclictest10048-21kthreadcore19:25:220
2510399255,8cyclictest21156-21grep20:45:210
2510399255,7cyclictest0-21swapper/022:50:310
25103992515,6cyclictest5931-21ssh00:25:130
25103992515,6cyclictest5931-21ssh00:25:120
25103992513,9cyclictest29041-21fschecks_count20:55:170
2513099247,3cyclictest0-21swapper/322:44:233
2513099243,14cyclictest0-21swapper/321:59:253
2513099241,7cyclictest0-21swapper/323:20:153
2513099241,7cyclictest0-21swapper/320:00:293
2513099241,6cyclictest0-21swapper/323:25:303
2513099241,6cyclictest0-21swapper/322:30:253
2513099241,6cyclictest0-21swapper/322:15:143
2513099241,6cyclictest0-21swapper/321:55:003
2513099241,6cyclictest0-21swapper/320:10:003
2513099241,6cyclictest0-21swapper/319:25:273
2513099241,5cyclictest35966irq/58-eth019:15:133
2513099241,20cyclictest0-21swapper/322:20:313
2512299242,13cyclictest15510-21ssh23:00:162
2512299242,13cyclictest15510-21ssh23:00:152
25122992413,8cyclictest27-21ksoftirqd/221:35:132
2512299241,18cyclictest28279-21ssh21:19:252
2512299241,17cyclictest27346irq/40-dwc2_hso00:25:132
2512299241,17cyclictest27346irq/40-dwc2_hso00:25:122
25122992410,12cyclictest29392-21kthreadcore22:30:232
2511499244,10cyclictest0-21swapper/120:45:131
2511499243,13cyclictest24033-21ssh21:35:141
25114992412,6cyclictest21-21ksoftirqd/119:55:131
2511499240,17cyclictest31637-21ssh23:45:151
2510399243,9cyclictest6074-21latency_hist21:05:010
2513099232,6cyclictest0-21swapper/321:15:163
2513099231,7cyclictest0-21swapper/323:50:153
2513099231,7cyclictest0-21swapper/321:05:243
2513099231,7cyclictest0-21swapper/319:35:013
2513099231,7cyclictest0-21swapper/300:25:153
2513099231,7cyclictest0-21swapper/300:25:143
2513099231,6cyclictest0-21swapper/323:35:193
2513099231,6cyclictest0-21swapper/323:35:183
2513099231,6cyclictest0-21swapper/323:05:313
2513099231,6cyclictest0-21swapper/321:00:173
2513099231,6cyclictest0-21swapper/320:55:223
2513099231,6cyclictest0-21swapper/320:40:163
2513099231,6cyclictest0-21swapper/320:30:273
2513099231,6cyclictest0-21swapper/320:10:163
2513099231,6cyclictest0-21swapper/319:50:253
2513099231,6cyclictest0-21swapper/319:20:293
2513099231,5cyclictest0-21swapper/319:15:013
2513099231,5cyclictest0-21swapper/300:20:143
2512299239,12cyclictest21021-1kworker/u9:122:50:252
2512299232,16cyclictest0-21swapper/221:30:262
25122992312,9cyclictest5070-21ssh00:24:272
2512299231,17cyclictest17418-21ls22:25:182
2512299231,15cyclictest27346irq/40-dwc2_hso20:35:122
2512299230,20cyclictest2445-21find19:20:122
2511499235,11cyclictest0-21swapper/121:05:131
2511499232,16cyclictest73-21kswapd021:50:171
25114992316,4cyclictest0-21swapper/122:00:151
25114992314,5cyclictest26029-21/usr/sbin/munin00:00:271
25114992312,8cyclictest22580-21cron20:15:021
25103992312,7cyclictest32465-21apt-get20:25:130
25103992310,9cyclictest13424-21fschecks_time20:05:170
25103992310,10cyclictest24611-21/usr/sbin/munin20:50:190
2513099221,7cyclictest0-21swapper/320:35:283
2513099221,7cyclictest0-21swapper/319:45:163
2513099221,7cyclictest0-21swapper/319:35:163
2513099221,6cyclictest0-21swapper/320:50:183
2513099221,6cyclictest0-21swapper/319:55:253
2513099221,6cyclictest0-21swapper/319:40:193
2512299229,11cyclictest2993-21fschecks_time22:35:182
2512299222,18cyclictest0-21swapper/220:50:152
25122992215,5cyclictest15779-21ssh00:10:332
2511499229,9cyclictest10691-21diskmemload22:10:151
2511499223,10cyclictest15501-21ssh23:00:151
2511499223,10cyclictest15501-21ssh23:00:141
2511499221,18cyclictest27407-21cron20:20:021
2511499221,12cyclictest0-21swapper/121:21:571
2511499220,18cyclictest31362-21ls23:45:011
2510399224,15cyclictest29780-21cut20:20:220
25103992213,6cyclictest25064-21ntp_192.168.11419:40:240
2513099211,6cyclictest0-21swapper/320:15:153
2513099211,5cyclictest0-21swapper/320:25:203
2512299219,3cyclictest15038-21irqrtprio20:40:192
2512299214,14cyclictest11968-21kthreadcore21:10:202
2512299214,14cyclictest11968-21kthreadcore21:10:192
2512299212,16cyclictest13755-21irqrtprio20:05:192
25122992111,8cyclictest0-21swapper/219:45:172
2512299211,16cyclictest16909-21latency22:05:242
2512299211,14cyclictest27346irq/40-dwc2_hso23:15:182
2512299211,10cyclictest8135-21sh23:50:132
2512299211,10cyclictest31795-21fschecks_count23:45:162
2512299211,10cyclictest10108-21kthreadcore20:00:192
2511499216,5cyclictest131rcu_preempt20:40:131
2511499212,14cyclictest19890-21ssh22:07:241
25114992114,3cyclictest21-21ksoftirqd/122:45:141
25114992112,5cyclictest5898-21/usr/sbin/munin00:25:221
25114992112,5cyclictest5898-21/usr/sbin/munin00:25:221
2511499211,17cyclictest30018-21kthreadcore23:25:211
2510399219,9cyclictest20485-21sed19:35:290
2510399219,9cyclictest12277-21sed19:30:180
2510399211,13cyclictest25473-21ls19:10:190
2512299202,15cyclictest6334-21fschecks_time21:05:182
2512299202,10cyclictest19916-21kthreadcore19:35:232
2512299201,16cyclictest24033-21latency_hist20:50:022
2512299201,14cyclictest261rcuc/222:15:142
2512299201,10cyclictest24644-21kthreadcore00:35:212
2512299201,10cyclictest23641-21kthreadcore22:10:212
2512299200,10cyclictest28337-21kthreadcore19:10:242
2511499208,8cyclictest21-21ksoftirqd/119:50:151
2511499202,12cyclictest21-21ksoftirqd/119:45:121
2511499201,9cyclictest0-21swapper/119:25:161
2511499201,16cyclictest6619-21fschecks_time22:55:161
2511499201,16cyclictest30326-21kthreadcore00:20:171
2511499201,15cyclictest15361-21missed_timers19:30:261
2511499201,13cyclictest3202-21kthreadcore21:00:191
2510399204,13cyclictest22799-21/usr/sbin/munin20:15:180
2512299198,9cyclictest1515-21cut21:00:142
2512299192,9cyclictest9159-21kthreadcore19:25:212
2512299192,14cyclictest27814-21fschecks_time20:20:172
2512299191,9cyclictest5753-21kthreadcore20:30:222
2512299191,14cyclictest24515-21kthreadcore20:15:202
2511499197,9cyclictest13199-21sshd20:35:251
2511499193,11cyclictest10102-21ssh23:15:201
25114991911,5cyclictest0-21swapper/119:40:251
2511499190,8cyclictest13886-21unixbench_singl21:45:301
2511499190,16cyclictest11392-21idleruntime21:10:181
2511499190,16cyclictest11392-21idleruntime21:10:171
2511499189,7cyclictest20218-21diskstats21:15:151
2511499187,3cyclictest131rcu_preempt19:20:301
2511499186,7cyclictest131rcu_preempt22:23:251
2511499184,3cyclictest131rcu_preempt00:35:221
2511499183,9cyclictest0-21swapper/119:35:121
2511499183,11cyclictest27711-21cpuspeed20:20:151
2511499182,11cyclictest22695-21ssh23:20:261
2511499181,12cyclictest9065-21usage_EthcTask23:30:311
2511499173,9cyclictest21-21ksoftirqd/120:00:141
2511499172,12cyclictest25544-21kthreadcore20:50:191
2511499172,11cyclictest11796-21ntp_states23:50:251
2511499172,10cyclictest0-21swapper/120:30:031
2511499171,12cyclictest4596-21idleruntime-cro20:30:011
2511499170,14cyclictest30271-21kthreadcore20:55:201
2510399165,9cyclictest25210-21latency_hist19:10:020
2511499141,9cyclictest25143-21latency_hist19:10:011
2512299121,8cyclictest25132-21latency_hist19:10:022
2513099111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional