You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 10:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 18, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22161995646,7cyclictest35966irq/58-eth023:10:113
2214099522,36cyclictest12024-21ssh23:00:131
2213199512,44cyclictest8391-21ssh22:40:110
2213199509,39cyclictest2964-21ssh22:55:140
2213199495,7cyclictest131rcu_preempt20:00:130
2213199494,35cyclictest4807-21ssh22:20:120
2213199493,41cyclictest30042-21ssh23:10:120
22131994932,7cyclictest12-21ksoftirqd/023:05:150
2213199483,7cyclictest131rcu_preempt21:30:120
2216199471,2cyclictest0-21swapper/322:30:123
2214099461,37cyclictest2877-21ssh21:25:131
2213199462,13cyclictest17467-21sh00:15:150
2213199462,13cyclictest17467-21sh00:15:150
2213199461,38cyclictest1293-21ssh23:30:120
2216199431,4cyclictest0-21swapper/321:45:133
2214099434,32cyclictest12061-21ssh00:30:131
2213199432,12cyclictest25738-21apt-get20:55:040
2213199425,9cyclictest0-21swapper/000:00:140
2213199423,28cyclictest12-21ksoftirqd/021:35:130
2213199423,28cyclictest12-21ksoftirqd/021:35:120
2213199422,30cyclictest26442-21ssh00:20:130
2213199415,33cyclictest10259-21ssh23:35:140
2213199401,30cyclictest8292-21rm21:10:120
2216199392,3cyclictest0-21swapper/321:10:113
2215099392,30cyclictest17396-21ssh22:45:122
2214099394,21cyclictest26347-21ssh22:50:131
2213199390,35cyclictest13847-21ssh22:25:130
22161993826,9cyclictest35966irq/58-eth023:35:123
2214099385,22cyclictest21-21ksoftirqd/123:55:141
2214099381,29cyclictest8404-21ssh22:40:131
2213199381,9cyclictest6664-21latency_hist20:35:020
2213199381,11cyclictest3775-21tr21:25:190
2213199371,12cyclictest20783-21latency_hist23:05:010
2213199371,10cyclictest3462-21ssh22:00:180
2215099369,12cyclictest27-21ksoftirqd/222:19:212
22150993611,9cyclictest24622-21apt-get23:25:132
2213199367,4cyclictest31095-21kthreadcore20:25:170
2213199361,11cyclictest25638-21needreboot19:10:220
2216199355,26cyclictest35966irq/58-eth000:15:133
2216199355,26cyclictest35966irq/58-eth000:15:133
22161993524,6cyclictest35966irq/58-eth022:40:113
2213199352,24cyclictest18405-21cut21:50:210
2213199352,22cyclictest19232-21ssh22:10:130
2213199352,22cyclictest19232-21ssh22:10:130
2213199352,19cyclictest20049-21kworker/u8:022:30:140
2213199351,12cyclictest25212-21unixbench_singl00:35:290
22131993511,20cyclictest15257-21sh23:20:010
2213199351,11cyclictest6881-21if_eth021:45:170
2213199351,11cyclictest5604-21kthreadcore23:50:190
2213199351,10cyclictest28498-21cat20:20:280
2216199341,3cyclictest0-21swapper/300:25:143
2214099341,26cyclictest17395-21ssh22:45:121
2213199345,5cyclictest15334-21kthreadcore00:30:230
2213199342,26cyclictest31914-21sh00:05:120
2213199341,9cyclictest24600-21/usr/sbin/munin23:25:280
2213199341,11cyclictest18986-21grep20:10:280
2213199341,10cyclictest24427-21latency_hist23:25:010
2213199341,10cyclictest13786-21cat22:05:240
2216199332,2cyclictest0-21swapper/319:15:133
2216199332,24cyclictest35966irq/58-eth023:00:133
2216199331,5cyclictest0-21swapper/322:55:133
2213199334,18cyclictest13897-21ssh23:55:110
2213199334,15cyclictest29966-21ssh21:40:120
2213199332,11cyclictest26525-21perl21:20:160
2213199331,8cyclictest26031-21cat21:20:020
2213199331,13cyclictest804-21ssh22:35:200
2213199331,12cyclictest22484-21grep19:40:270
2213199331,11cyclictest30258-21switchtime22:50:270
2213199331,10cyclictest7913-21timerandwakeup19:25:280
2213199331,10cyclictest4037-21latency_hist19:25:010
2213199331,10cyclictest27735-21kthreadcore19:15:210
2213199331,10cyclictest11753-21tr00:10:210
2215099323,21cyclictest0-21swapper/200:30:142
2215099322,19cyclictest31768-21ssh22:35:132
2214099321,20cyclictest26288-21ssh21:20:131
2213199321,11cyclictest24094-21irqrtprio19:45:170
2213199321,11cyclictest20208-21unixbench_singl20:45:270
2213199321,11cyclictest12237-21kthreadcore20:05:190
2213199321,10cyclictest2374-21grep21:00:290
2216199312,4cyclictest0-21swapper/321:05:133
2216199312,3cyclictest0-21swapper/321:40:133
2216199311,16cyclictest35966irq/58-eth000:30:123
2215099318,8cyclictest0-21swapper/222:40:142
2214099313,15cyclictest21-21ksoftirqd/123:25:131
2213199313,9cyclictest29319-21kthreadcore19:50:180
2213199312,7cyclictest638-21users22:15:310
2213199310,10cyclictest6780-21ntpq21:05:230
2216199302,5cyclictest0-21swapper/323:15:123
22161993017,8cyclictest35966irq/58-eth021:50:133
2216199301,4cyclictest0-21swapper/323:30:133
2216199301,4cyclictest0-21swapper/321:30:123
2215099304,15cyclictest27346irq/40-dwc2_hso22:05:132
2215099302,21cyclictest0-21swapper/220:35:142
2215099302,12cyclictest17280-21ssh21:15:132
2215099300,16cyclictest0-21swapper/222:30:132
2214099302,20cyclictest28286-21sh23:45:121
2213199301,10cyclictest6688-21grep00:25:250
2213199301,10cyclictest3724-21kthreadcore19:55:210
2213199301,10cyclictest30291-21cut23:45:200
2213199301,10cyclictest25212-21irqcore21:55:170
2213199301,10cyclictest21383-21kthreadcore23:40:220
2213199300,10cyclictest14165-21kthreadcore20:40:220
2216199293,16cyclictest0-21swapper/322:25:143
2216199292,17cyclictest35966irq/58-eth023:50:123
2216199291,3cyclictest0-21swapper/319:30:123
2215099294,22cyclictest0-21swapper/221:40:132
2215099291,18cyclictest27-21ksoftirqd/221:55:122
2214099291,21cyclictest24708-21ssh21:55:111
2213199297,3cyclictest0-21swapper/019:10:010
2213199291,9cyclictest11335-21ls20:40:010
2213199291,10cyclictest19961-21sort20:15:100
2216199286,19cyclictest35966irq/58-eth023:05:143
2216199284,16cyclictest35966irq/58-eth000:35:143
2216199282,19cyclictest0-21swapper/321:28:233
2216199281,5cyclictest0-21swapper/321:35:133
2216199281,5cyclictest0-21swapper/321:35:123
2216199281,4cyclictest0-21swapper/322:00:113
2216199281,3cyclictest0-21swapper/320:55:113
2216199281,3cyclictest0-21swapper/300:00:143
2215099285,13cyclictest27346irq/40-dwc2_hso22:50:132
2215099283,10cyclictest0-21swapper/223:33:212
2215099282,10cyclictest0-21swapper/219:40:132
2215099281,15cyclictest27346irq/40-dwc2_hso22:20:132
2214099282,20cyclictest10273-21ssh22:05:131
2214099282,19cyclictest30956-21kthreadcore22:15:211
22140992811,8cyclictest21-21ksoftirqd/100:00:401
2213199281,10cyclictest21268-21switchtime22:45:270
2216199272,21cyclictest35966irq/58-eth000:10:113
2216199271,23cyclictest0-21swapper/300:10:013
2215099273,7cyclictest0-21swapper/221:38:212
2215099273,7cyclictest0-21swapper/221:38:202
2215099273,14cyclictest27346irq/40-dwc2_hso19:35:132
2215099272,6cyclictest27-21ksoftirqd/223:41:302
22140992711,4cyclictest21-21ksoftirqd/100:08:521
2216199261,5cyclictest0-21swapper/322:20:123
2216199261,4cyclictest0-21swapper/320:10:133
2216199261,3cyclictest0-21swapper/323:56:403
2216199261,3cyclictest0-21swapper/322:47:293
2216199261,3cyclictest0-21swapper/322:40:023
2216199261,3cyclictest0-21swapper/322:08:343
2215099262,7cyclictest271ksoftirqd/200:17:472
2215099262,7cyclictest271ksoftirqd/200:17:472
2215099262,18cyclictest21069-21ssh00:35:122
2215099261,14cyclictest6613-21ssh23:15:132
2214099269,8cyclictest21-21ksoftirqd/122:39:481
22140992610,8cyclictest21-21ksoftirqd/122:10:331
22140992610,8cyclictest21-21ksoftirqd/122:10:321
2213199261,19cyclictest844-21munin-node19:30:120
2216199251,3cyclictest0-21swapper/323:49:173
2216199251,3cyclictest0-21swapper/323:20:363
2216199251,3cyclictest0-21swapper/322:50:163
2216199251,3cyclictest0-21swapper/321:15:583
2216199251,3cyclictest0-21swapper/320:25:113
2216199251,3cyclictest0-21swapper/319:45:213
2216199251,3cyclictest0-21swapper/300:20:203
2216199251,21cyclictest0-21swapper/322:10:193
2216199251,21cyclictest0-21swapper/322:10:183
2216199251,21cyclictest0-21swapper/321:20:583
2215099256,12cyclictest27-21ksoftirqd/223:51:282
2215099253,6cyclictest27-21ksoftirqd/222:58:202
2215099253,14cyclictest27-21ksoftirqd/223:03:172
2215099253,14cyclictest27-21ksoftirqd/221:51:022
22150992513,8cyclictest13817-21apt-get22:25:132
2215099250,9cyclictest2096-21grep21:00:282
2214099259,8cyclictest21-21ksoftirqd/123:34:411
2214099259,8cyclictest21-21ksoftirqd/121:39:021
2214099259,8cyclictest21-21ksoftirqd/121:39:011
2214099259,8cyclictest21-21ksoftirqd/121:30:281
2214099259,7cyclictest21-21ksoftirqd/120:25:121
2214099259,3cyclictest21-21ksoftirqd/122:29:481
2214099258,9cyclictest21-21ksoftirqd/122:20:381
2214099258,10cyclictest21-21ksoftirqd/121:10:051
2214099256,10cyclictest21-21ksoftirqd/100:22:341
2214099255,10cyclictest21-21ksoftirqd/119:35:121
2214099253,19cyclictest6603-21ssh23:15:111
2214099251,20cyclictest15629-21ssh21:50:131
22140992511,7cyclictest21-21ksoftirqd/122:30:251
22140992511,4cyclictest21-21ksoftirqd/123:10:011
22140992510,5cyclictest21-21ksoftirqd/121:41:551
2216199243,17cyclictest35966irq/58-eth023:25:123
2216199242,20cyclictest0-21swapper/319:10:153
2216199241,3cyclictest0-21swapper/321:55:203
2216199241,3cyclictest0-21swapper/321:00:233
2216199241,3cyclictest0-21swapper/319:50:233
2216199241,2cyclictest0-21swapper/322:15:183
2216199241,2cyclictest0-21swapper/320:20:163
2215099248,10cyclictest131rcu_preempt21:46:272
2215099247,3cyclictest131rcu_preempt21:21:262
2215099245,13cyclictest27-21ksoftirqd/221:14:502
2215099242,8cyclictest27-21ksoftirqd/223:10:492
2215099242,5cyclictest27-21ksoftirqd/200:24:182
2215099242,4cyclictest0-21swapper/220:00:122
2215099242,15cyclictest0-21swapper/223:55:052
22150992415,7cyclictest261rcuc/223:22:232
2215099241,10cyclictest2138-21kthreadcore22:00:162
2214099249,4cyclictest21-21ksoftirqd/100:37:331
2214099248,7cyclictest21-21ksoftirqd/123:12:061
2214099247,7cyclictest21-21ksoftirqd/100:18:291
2214099247,7cyclictest21-21ksoftirqd/100:18:291
2214099245,3cyclictest131rcu_preempt20:20:301
2216199231,6cyclictest0-21swapper/320:40:133
2216199231,4cyclictest0-21swapper/320:50:203
2216199231,3cyclictest0-21swapper/320:35:113
2216199231,3cyclictest0-21swapper/319:55:153
2216199231,3cyclictest0-21swapper/319:35:273
2216199231,2cyclictest0-21swapper/323:40:203
2216199231,2cyclictest0-21swapper/320:00:153
2215099236,7cyclictest27-21ksoftirqd/200:29:132
2215099235,3cyclictest131rcu_preempt00:05:242
2215099234,13cyclictest27346irq/40-dwc2_hso19:55:132
2215099233,2cyclictest131rcu_preempt23:05:322
2215099232,4cyclictest27-21ksoftirqd/222:11:262
2215099232,4cyclictest27-21ksoftirqd/222:11:252
2215099231,6cyclictest27-21ksoftirqd/219:10:222
22150992314,7cyclictest261rcuc/220:30:242
2215099231,2cyclictest0-21swapper/220:40:252
22150992312,8cyclictest261rcuc/200:14:182
2215099231,18cyclictest28306-21sh23:45:132
2214099237,8cyclictest21-21ksoftirqd/121:45:591
2214099237,4cyclictest21-21ksoftirqd/123:40:101
2214099236,9cyclictest21-21ksoftirqd/122:00:091
2214099236,4cyclictest21-21ksoftirqd/122:55:451
2214099234,4cyclictest21-21ksoftirqd/121:19:501
2214099234,3cyclictest131rcu_preempt19:15:281
22140992312,6cyclictest0-21swapper/120:45:121
22140992310,3cyclictest131rcu_preempt21:05:231
2216199221,4cyclictest0-21swapper/319:20:143
2216199221,3cyclictest0-21swapper/320:45:163
2216199221,3cyclictest0-21swapper/320:30:233
2216199221,3cyclictest0-21swapper/320:15:103
2216199221,3cyclictest0-21swapper/320:05:263
2216199221,3cyclictest0-21swapper/319:40:123
2216199221,3cyclictest0-21swapper/319:25:233
2215099228,8cyclictest261rcuc/223:35:112
22150992216,4cyclictest0-21swapper/220:25:262
2215099221,4cyclictest0-21swapper/221:31:302
22150992212,4cyclictest261rcuc/200:02:282
2214099228,4cyclictest21-21ksoftirqd/123:51:001
2214099227,3cyclictest131rcu_preempt20:10:011
2214099226,4cyclictest21-21ksoftirqd/123:37:061
22140992210,4cyclictest21-21ksoftirqd/100:25:201
22140992210,4cyclictest21-21ksoftirqd/100:10:241
22140992210,3cyclictest21-21ksoftirqd/120:00:161
2215099214,14cyclictest25865-21cut20:55:022
2215099212,10cyclictest7909-21latency_hist21:10:022
2215099212,10cyclictest16471-21cut20:45:162
22150992112,8cyclictest261rcuc/219:45:262
2215099211,15cyclictest27066-21apt-get19:15:122
2215099211,12cyclictest31001-21kthreadcore19:50:202
2215099211,11cyclictest3214-21sh21:25:162
2215099210,4cyclictest27-21ksoftirqd/220:20:202
2214099218,7cyclictest21-21ksoftirqd/123:20:351
2214099218,4cyclictest21-21ksoftirqd/120:30:261
2214099217,3cyclictest131rcu_preempt20:35:251
2214099217,3cyclictest0-21swapper/120:40:261
2214099214,3cyclictest131rcu_preempt19:10:171
2214099213,10cyclictest0-21swapper/119:40:141
22140992114,5cyclictest536-21usb-storage20:55:051
22140992110,5cyclictest21-21ksoftirqd/119:20:151
2215099202,4cyclictest27-21ksoftirqd/220:05:172
2215099202,15cyclictest10889-21kthreadcore19:30:192
2215099201,9cyclictest28382-21kthreadcore20:55:192
22150992015,3cyclictest261rcuc/220:10:132
2214099206,4cyclictest21-21ksoftirqd/120:15:131
2214099202,3cyclictest131rcu_preempt19:35:011
2214099202,14cyclictest30613-21apt-get21:00:041
22140992012,5cyclictest0-21swapper/119:55:251
2214099201,10cyclictest21-21ksoftirqd/119:55:011
2216199191,3cyclictest0-21swapper/319:10:023
2215099193,9cyclictest22115-21cyclictest19:10:002
2215099191,9cyclictest6106-21tail19:25:212
2215099191,15cyclictest20111-21fschecks_count20:15:152
22150991911,3cyclictest0-21swapper/219:20:122
2214099194,2cyclictest131rcu_preempt20:10:231
2213199196,10cyclictest25764-21apt-config20:55:020
2214099189,6cyclictest7905-21cat19:25:271
2214099183,3cyclictest131rcu_preempt19:45:151
22140991813,3cyclictest0-21swapper/120:50:121
2213199185,10cyclictest17336-21wc19:35:260
22140991610,4cyclictest21-21ksoftirqd/119:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional