You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 04:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu Aug 27, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3082199702,61cyclictest35966irq/58-eth021:30:133
3080999571,4cyclictest0-21swapper/220:55:132
3080999561,5cyclictest0-21swapper/219:55:142
3080999551,5cyclictest0-21swapper/219:25:142
3080999551,5cyclictest0-21swapper/219:25:132
3080999551,45cyclictest0-21swapper/223:15:132
3080999551,3cyclictest0-21swapper/200:30:092
3080999551,3cyclictest0-21swapper/200:30:092
3080999541,47cyclictest0-21swapper/221:10:122
3080999522,2cyclictest27977-21apt-get22:10:142
3080999522,2cyclictest27977-21apt-get22:10:142
3080999521,37cyclictest0-21swapper/200:25:132
3080999521,2cyclictest10996-21ssh23:30:142
3080999511,2cyclictest0-21swapper/220:05:142
3080199512,38cyclictest22867-21ssh22:25:111
3080999503,44cyclictest27346irq/40-dwc2_hso20:50:132
3080999491,2cyclictest0-21swapper/221:50:142
3080999491,2cyclictest0-21swapper/220:25:122
3080999481,2cyclictest0-21swapper/219:50:142
3080199481,7cyclictest21-21ksoftirqd/100:25:131
3080199481,39cyclictest26613-21ssh22:45:101
3080199481,39cyclictest26613-21ssh22:45:091
3080999471,2cyclictest0-21swapper/223:50:132
3080199473,7cyclictest131rcu_preempt22:15:121
3080999461,7cyclictest0-21swapper/219:30:142
30801994610,4cyclictest131rcu_preempt21:50:121
30821994533,9cyclictest35966irq/58-eth023:25:123
3082199452,29cyclictest35966irq/58-eth023:45:133
3080999452,2cyclictest20218-21apt-get20:40:122
3080999451,38cyclictest0-21swapper/200:35:122
3080999451,2cyclictest0-21swapper/221:30:132
3080999450,3cyclictest25294-21apt-get23:20:122
3080199451,32cyclictest29450-21ssh21:35:111
3079399441,36cyclictest9887-21ssh22:00:120
3079399441,36cyclictest9887-21ssh22:00:110
3080999433,34cyclictest0-21swapper/219:35:122
3080999433,2cyclictest0-21swapper/223:40:132
3080999431,3cyclictest17587-21ntp_states20:00:242
3080999431,3cyclictest0-21swapper/219:15:122
3080199436,8cyclictest21-21ksoftirqd/100:35:121
3080999422,2cyclictest3367-21apt-get22:50:142
3080999421,36cyclictest0-21swapper/222:55:122
3079399422,32cyclictest7219-21sh23:10:120
3082199411,23cyclictest35966irq/58-eth000:10:123
3080999412,6cyclictest0-21swapper/220:45:132
3080999411,36cyclictest0-21swapper/221:35:112
3080999411,2cyclictest0-21swapper/221:25:142
3080999410,5cyclictest15948-21cat21:05:302
3080199415,10cyclictest21-21ksoftirqd/122:40:111
3080199414,33cyclictest16724-21ssh21:10:101
3080199411,35cyclictest11396-21ssh21:25:141
3080199411,31cyclictest15192-21ssh21:45:121
3080199401,29cyclictest20050-21ssh23:35:131
3082199396,20cyclictest35966irq/58-eth021:25:133
3082199393,24cyclictest35966irq/58-eth022:25:143
3082199393,22cyclictest35966irq/58-eth023:10:133
3082199393,22cyclictest35966irq/58-eth022:45:103
3082199393,22cyclictest35966irq/58-eth022:45:103
30821993920,13cyclictest35966irq/58-eth022:05:133
3080999394,32cyclictest28677-21kthreadcore21:15:222
3080999392,32cyclictest0-21swapper/222:35:132
3080999391,5cyclictest2134-21ssh23:25:162
3080999391,3cyclictest7198-21apt-get23:10:112
3080999382,27cyclictest22880-21ssh22:25:122
3080999381,7cyclictest17904-21timerwakeupswit22:20:312
3080999381,3cyclictest13271-21ntp_states00:05:302
3080999372,26cyclictest26633-21ssh22:45:112
3080999372,26cyclictest26633-21ssh22:45:112
3080999371,9cyclictest930-21snmpd00:13:312
3080999371,32cyclictest0-21swapper/200:00:132
3080999371,26cyclictest23717-21ssh23:35:252
3080999370,3cyclictest32131-21apt-get19:45:132
3080199373,27cyclictest9552-21ssh00:05:161
3080999361,6cyclictest14810-21sh21:45:012
3080999361,4cyclictest0-21swapper/221:20:142
3080999361,3cyclictest0-21swapper/220:10:132
3080199363,27cyclictest18533-21ssh00:10:121
3080199361,3cyclictest131rcu_preempt19:35:121
3082199354,19cyclictest35966irq/58-eth021:40:133
3082199352,22cyclictest35966irq/58-eth022:35:133
3080999351,7cyclictest4647-21grep21:55:282
3080999351,7cyclictest1920-21ntp_192.168.11419:10:222
3080999351,30cyclictest22802-21mailstats22:05:272
3080999343,9cyclictest0-21swapper/222:01:012
3080999343,9cyclictest0-21swapper/222:01:012
3080999343,28cyclictest1166-21cut20:20:172
3080999342,27cyclictest6612-21ssh00:20:232
3080999341,6cyclictest14964-21usage_EthcTask20:30:302
3080999341,3cyclictest0-21swapper/219:20:202
3080999341,30cyclictest11353-21/usr/sbin/munin21:00:302
3080999341,2cyclictest0-21swapper/223:45:132
3080999341,29cyclictest0-21swapper/220:15:112
3080999332,28cyclictest27533-21fschecks_time19:40:202
3080999331,6cyclictest16360-21diskmemload23:56:512
3080999331,4cyclictest131rcu_preempt23:00:122
3080999321,5cyclictest8847-21timerwakeupswit22:15:312
3080999321,3cyclictest0-21swapper/222:30:152
3080999321,28cyclictest0-21swapper/222:43:402
30809993212,14cyclictest664-21ntpd23:05:122
3080999320,4cyclictest31208-21netstat00:15:222
3079399327,3cyclictest131rcu_preempt00:25:120
3082199311,20cyclictest35966irq/58-eth022:40:113
3080199311,22cyclictest29069-21ssh23:40:131
30793993114,5cyclictest12-21ksoftirqd/023:35:330
3080999307,21cyclictest15157-21/usr/sbin/munin21:45:172
3080999300,3cyclictest19945-21sshd20:35:412
3080199306,15cyclictest21-21ksoftirqd/120:35:141
3080199303,18cyclictest7210-21ssh23:10:121
3079399303,7cyclictest12-21ksoftirqd/022:55:270
3082199293,19cyclictest35966irq/58-eth023:40:123
30793992910,5cyclictest12-21ksoftirqd/022:30:310
3082199283,16cyclictest0-21swapper/322:01:013
3082199283,16cyclictest0-21swapper/322:01:013
3080199289,3cyclictest131rcu_preempt23:25:131
3080199283,18cyclictest13870-21ssh22:20:151
3080199282,20cyclictest8014-21ssh21:58:161
3079399289,7cyclictest12-21ksoftirqd/021:54:260
3079399284,5cyclictest12-21ksoftirqd/020:20:130
30793992824,2cyclictest16211-21ssh23:15:110
3079399281,8cyclictest12-21ksoftirqd/022:20:150
3079399281,7cyclictest3314-21apt-get19:15:120
3079399281,22cyclictest27999-21ssh22:10:130
3079399281,22cyclictest27999-21ssh22:10:120
3079399281,19cyclictest25324-21sh23:20:140
3082199273,17cyclictest35966irq/58-eth020:00:103
30821992714,9cyclictest35966irq/58-eth021:10:123
3080199272,18cyclictest9897-21ssh22:00:131
3080199272,18cyclictest9897-21ssh22:00:131
3080199271,18cyclictest18957-21ssh22:05:131
3080199271,17cyclictest6372-21uniq21:40:151
3079399278,3cyclictest0-21swapper/000:30:100
3079399278,3cyclictest0-21swapper/000:30:090
3079399272,4cyclictest12-21ksoftirqd/000:05:170
3079399272,4cyclictest12-21ksoftirqd/000:00:310
30793992718,7cyclictest30558-21ssh23:05:110
30793992712,4cyclictest12-21ksoftirqd/022:09:380
3079399271,21cyclictest31306-21ssh00:35:120
3079399269,8cyclictest12-21ksoftirqd/022:18:120
3079399264,9cyclictest131rcu_preempt21:10:090
30793992618,2cyclictest12-21ksoftirqd/023:47:370
3079399261,5cyclictest12-21ksoftirqd/021:10:010
30793992615,7cyclictest20025-21cron21:30:010
30793992613,7cyclictest0-21swapper/000:10:370
30793992613,4cyclictest12-21ksoftirqd/000:20:280
30793992612,4cyclictest12-21ksoftirqd/023:25:160
3082199252,17cyclictest0-21swapper/321:20:143
3080199259,3cyclictest131rcu_preempt19:40:231
3080199254,13cyclictest3395-21ssh22:50:161
30801992512,8cyclictest201rcuc/120:40:131
30801992511,7cyclictest0-21swapper/123:17:161
3079399258,7cyclictest12-21ksoftirqd/022:28:360
3079399258,3cyclictest131rcu_preempt21:30:240
3079399253,3cyclictest131rcu_preempt21:15:280
3079399253,19cyclictest11002-21ssh23:30:140
3079399253,17cyclictest3400-21sshd22:50:160
3079399252,6cyclictest12-21ksoftirqd/019:55:240
30793992511,3cyclictest0-21swapper/021:35:550
3082199242,14cyclictest35966irq/58-eth000:35:123
3080999241,5cyclictest30815-21latency_hist19:10:012
3080199245,9cyclictest14475-21kworker/u8:122:35:121
3080199245,13cyclictest25325-21ssh23:20:141
3080199242,20cyclictest25765-21ssh21:15:121
3080199242,12cyclictest21-21ksoftirqd/119:15:121
30801992415,4cyclictest21-21ksoftirqd/122:30:211
30801992410,7cyclictest0-21swapper/123:00:121
3080199240,4cyclictest131rcu_preempt23:30:121
3079399246,3cyclictest131rcu_preempt19:10:260
3079399245,3cyclictest131rcu_preempt21:20:270
3079399245,3cyclictest131rcu_preempt20:30:120
3079399244,5cyclictest12-21ksoftirqd/020:40:190
3079399243,11cyclictest12-21ksoftirqd/021:57:470
30793992410,5cyclictest12-21ksoftirqd/023:41:050
30821992311,9cyclictest35966irq/58-eth000:00:133
3082199231,16cyclictest0-21swapper/321:36:163
3082199231,11cyclictest35966irq/58-eth021:55:133
3080199238,7cyclictest21-21ksoftirqd/121:05:281
3080199235,7cyclictest21-21ksoftirqd/100:04:501
3080199231,18cyclictest17718-21apt-get19:30:131
3079399235,3cyclictest131rcu_preempt22:45:290
3079399235,3cyclictest131rcu_preempt22:45:290
3079399235,2cyclictest131rcu_preempt19:25:300
3079399235,2cyclictest131rcu_preempt19:25:300
3079399234,4cyclictest131rcu_preempt21:45:290
3079399234,3cyclictest131rcu_preempt19:35:290
3079399232,6cyclictest12-21ksoftirqd/023:00:280
3079399232,3cyclictest131rcu_preempt22:41:080
3079399231,5cyclictest12-21ksoftirqd/022:35:100
30793992310,3cyclictest131rcu_preempt19:30:160
3082199222,14cyclictest0-21swapper/323:19:163
3082199221,13cyclictest35966irq/58-eth023:50:143
3082199220,14cyclictest35966irq/58-eth019:55:133
3080199229,4cyclictest21-21ksoftirqd/122:15:021
3080199229,4cyclictest21-21ksoftirqd/122:15:021
3080199224,5cyclictest21-21ksoftirqd/100:15:281
3080199223,11cyclictest131rcu_preempt23:57:291
30801992210,3cyclictest21-21ksoftirqd/121:33:441
3079399229,3cyclictest131rcu_preempt19:40:290
3079399228,3cyclictest131rcu_preempt19:45:260
3079399227,6cyclictest131rcu_preempt19:20:100
3079399227,5cyclictest131rcu_preempt20:25:270
3079399224,4cyclictest12-21ksoftirqd/021:40:590
3079399224,3cyclictest131rcu_preempt20:15:260
3079399223,3cyclictest131rcu_preempt00:15:220
3079399222,4cyclictest12-21ksoftirqd/023:50:030
30821992115,4cyclictest321rcuc/321:50:143
30821992111,7cyclictest0-21swapper/322:55:123
3082199211,16cyclictest35966irq/58-eth022:30:143
3082199211,12cyclictest35966irq/58-eth023:00:123
3080199218,9cyclictest12031-21ssh23:47:161
3080199217,8cyclictest0-21swapper/122:56:101
3080199217,4cyclictest131rcu_preempt19:10:101
3080199211,15cyclictest6953-21find21:00:121
30801992111,4cyclictest131rcu_preempt20:30:171
30801992110,2cyclictest21-21ksoftirqd/121:20:231
30801992110,2cyclictest21-21ksoftirqd/120:45:231
30801992110,2cyclictest21-21ksoftirqd/100:22:391
3079399219,3cyclictest0-21swapper/023:56:090
3079399218,3cyclictest131rcu_preempt20:00:140
3079399214,3cyclictest131rcu_preempt21:00:290
3082199209,8cyclictest0-21swapper/323:38:213
3082199208,7cyclictest35966irq/58-eth023:05:143
30821992010,8cyclictest0-21swapper/323:22:143
30821992010,8cyclictest0-21swapper/300:07:233
30821992010,7cyclictest35966irq/58-eth000:25:123
3080199208,9cyclictest21-21ksoftirqd/120:05:141
3080199208,2cyclictest21-21ksoftirqd/120:10:211
3080199207,8cyclictest131rcu_preempt20:50:261
3080199207,6cyclictest21-21ksoftirqd/123:51:211
3080199203,11cyclictest131rcu_preempt19:25:201
3080199203,11cyclictest131rcu_preempt19:25:201
30801992013,5cyclictest0-21swapper/120:15:151
30801992012,5cyclictest0-21swapper/120:55:191
30801992012,5cyclictest0-21swapper/120:20:191
3080199201,16cyclictest13847-21kworker/1:220:00:171
30801992010,3cyclictest131rcu_preempt23:05:361
3079399205,2cyclictest131rcu_preempt20:05:260
3079399204,3cyclictest131rcu_preempt20:15:020
3079399202,3cyclictest131rcu_preempt20:55:200
30793992010,4cyclictest25024-21apt-get20:45:130
3079399200,2cyclictest131rcu_preempt20:35:270
3082199199,7cyclictest35966irq/58-eth021:15:133
3082199199,6cyclictest0-21swapper/323:32:363
3082199199,6cyclictest0-21swapper/322:22:413
3082199199,3cyclictest0-21swapper/322:16:173
3082199198,8cyclictest0-21swapper/322:51:203
3082199198,6cyclictest0-21swapper/323:59:403
3082199192,10cyclictest35966irq/58-eth020:30:113
3082199191,13cyclictest0-21swapper/319:10:273
30821991910,6cyclictest0-21swapper/300:30:193
30821991910,6cyclictest0-21swapper/300:30:193
30821991910,6cyclictest0-21swapper/300:18:133
3080199196,7cyclictest22138-21apt-get00:30:101
3080199196,7cyclictest22138-21apt-get00:30:091
3080199194,4cyclictest131rcu_preempt20:25:161
30801991911,6cyclictest0-21swapper/119:50:281
3080199190,11cyclictest8206-21df19:20:111
3079399192,13cyclictest29806-21apt-get20:50:130
3079399191,8cyclictest4710-21grep19:50:160
3082199182,10cyclictest0-21swapper/320:55:153
3082199182,10cyclictest0-21swapper/320:05:153
30821991812,4cyclictest0-21swapper/322:11:173
30821991812,4cyclictest0-21swapper/322:11:173
30821991811,4cyclictest0-21swapper/300:20:203
30821991810,6cyclictest0-21swapper/321:45:053
3080199189,6cyclictest0-21swapper/119:55:191
3080199181,8cyclictest131rcu_preempt19:45:291
3082199172,9cyclictest0-21swapper/319:40:143
3082199172,10cyclictest0-21swapper/321:05:153
3082199172,10cyclictest0-21swapper/320:45:133
3082199172,10cyclictest0-21swapper/320:40:123
3082199172,10cyclictest0-21swapper/319:35:123
3082199172,10cyclictest0-21swapper/319:30:153
30821991712,3cyclictest33-21ksoftirqd/320:10:253
30821991711,4cyclictest0-21swapper/319:45:183
3082199171,13cyclictest0-21swapper/319:20:143
3079399175,2cyclictest131rcu_preempt19:10:010
3082199162,10cyclictest0-21swapper/320:35:143
3082199162,10cyclictest0-21swapper/320:15:133
3082199152,9cyclictest0-21swapper/321:00:143
3082199151,8cyclictest0-21swapper/319:25:283
3082199151,8cyclictest0-21swapper/319:25:283
3082199142,10cyclictest0-21swapper/319:50:143
3082199141,10cyclictest0-21swapper/319:15:133
3080199142,10cyclictest30681-21latency_hist19:10:011
3082199132,9cyclictest0-21swapper/320:25:143
3082199132,7cyclictest0-21swapper/320:50:123
3082199121,8cyclictest0-21swapper/320:20:133
308219991,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional