You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 18:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Tue Dec 09, 2025 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
233299766,59cyclictest9724-21ssh11:30:141
233299726,50cyclictest8620-21ssh09:45:181
233299692,48cyclictest11607-21ssh10:30:131
2349995940,13cyclictest29566irq/58-eth009:30:163
233299574,7cyclictest131rcu_preempt10:05:141
233299534,34cyclictest16027-21ssh09:50:111
234999504,26cyclictest29566irq/58-eth011:05:133
234199507,28cyclictest0-21swapper/211:30:142
234199492,43cyclictest26346irq/40-dwc2_hso08:55:152
232199493,6cyclictest131rcu_preempt11:10:120
234199481,41cyclictest28831-21apt-get10:20:132
232199473,8cyclictest131rcu_preempt10:40:130
234199454,30cyclictest0-21swapper/209:45:172
234199451,32cyclictest0-21swapper/210:05:152
232199455,25cyclictest2383-21ssh11:05:130
234199442,38cyclictest0-21swapper/207:15:132
234199441,33cyclictest0-21swapper/209:00:122
234199441,29cyclictest0-21swapper/208:10:132
232199442,38cyclictest18541-21ssh11:15:150
234199433,30cyclictest0-21swapper/211:20:152
234199431,37cyclictest0-21swapper/211:50:152
234199431,35cyclictest0-21swapper/211:25:162
234199421,36cyclictest0-21swapper/208:45:132
233299421,33cyclictest2378-21ssh11:05:121
234199412,33cyclictest0-21swapper/212:00:122
234199411,37cyclictest0-21swapper/209:40:122
232199412,4cyclictest131rcu_preempt11:30:130
2341994026,9cyclictest1305-21ssh10:43:312
234199401,36cyclictest16979-21apt-get07:25:162
234199401,35cyclictest0-21swapper/210:35:152
234199401,35cyclictest0-21swapper/209:25:142
234199401,33cyclictest0-21swapper/212:25:152
234199401,29cyclictest14459-21apt-get12:35:132
234999391,25cyclictest29566irq/58-eth010:10:153
234999383,23cyclictest29566irq/58-eth008:15:133
234199382,33cyclictest0-21swapper/211:00:132
234199381,33cyclictest8692-21apt-get07:50:112
234199381,32cyclictest26048-21apt-get09:35:132
234199380,29cyclictest10125-21sshd09:23:302
2332993820,12cyclictest31154-21ssh12:25:141
2332993814,14cyclictest20914-21diskmemload09:27:051
232199382,28cyclictest15195-21ssh12:15:130
234199372,18cyclictest29692-21ssh11:40:312
234199372,18cyclictest22472-21/usr/sbin/munin10:55:312
234199372,18cyclictest22392-21ssh12:19:322
234199372,18cyclictest18755-21ssh10:34:302
234199371,33cyclictest31474-21apt-get07:40:142
234199371,33cyclictest21487-21rm09:10:172
232199374,24cyclictest32121-21ssh12:05:120
234999363,21cyclictest29566irq/58-eth009:25:133
234999362,20cyclictest29566irq/58-eth011:10:133
234199361,32cyclictest16148-21apt-get11:55:142
234199361,31cyclictest18358-21apt-get08:00:152
234199360,32cyclictest26346irq/40-dwc2_hso12:05:112
232199365,28cyclictest23399-21ssh09:55:140
232199361,23cyclictest10462-21ssh10:50:140
234999352,20cyclictest29566irq/58-eth009:40:133
234199352,29cyclictest0-21swapper/211:35:132
234199352,29cyclictest0-21swapper/207:30:112
234199351,31cyclictest21920-21kthreadcore09:30:242
234199351,18cyclictest0-21swapper/211:17:402
234199350,21cyclictest2214-21sh09:17:062
234999344,16cyclictest29566irq/58-eth008:25:113
234199341,24cyclictest989466ssh10:49:452
2332993416,14cyclictest0-21swapper/112:05:111
2332993412,13cyclictest24039-21ssh09:31:361
234199332,28cyclictest16049-21kthreadcore08:30:192
234199331,29cyclictest27301-21timerwakeupswit09:55:292
234199331,29cyclictest22549-21kthreadcore08:35:212
234199321,29cyclictest0-21swapper/210:28:332
234199321,28cyclictest7817-21latency_hist11:50:022
234199321,27cyclictest7764-21apt-get12:10:182
234199321,27cyclictest4511-21kthreadcore08:50:222
234199321,27cyclictest30215-21cut07:35:282
234199321,24cyclictest5948-21ntpq07:10:252
234199320,29cyclictest6444-21apt-get12:30:142
233299322,21cyclictest18551-21ssh10:55:081
2332993213,6cyclictest20914-21diskmemload11:00:051
233299320,15cyclictest0-21swapper/110:43:321
234999315,18cyclictest0-21swapper/309:55:143
234199312,26cyclictest0-21swapper/208:25:112
234199311,21cyclictest4368-21tr11:05:222
234199311,20cyclictest24351-21kthreadcore10:15:242
234199310,18cyclictest10445-21apt-get10:50:142
2332993112,7cyclictest15417-21ssh11:10:471
2332993112,6cyclictest3514-21ssh10:24:291
2332993111,7cyclictest3058-21ssh09:18:511
2332993110,7cyclictest32199-21ssh11:44:071
234999305,13cyclictest29566irq/58-eth010:30:143
234199302,25cyclictest672-21irqcore08:15:182
234199302,25cyclictest16193-21fschecks_count09:50:152
234199302,25cyclictest0-21swapper/211:10:112
234199301,27cyclictest0-21swapper/212:24:282
234199301,26cyclictest8246-21kthreadcore08:20:202
234199301,26cyclictest16401-21cpuspeed09:05:142
234199301,25cyclictest24012-21kthreadcore08:05:202
234199300,27cyclictest13485-21/usr/sbin/munin07:55:132
234199300,26cyclictest26346irq/40-dwc2_hso10:10:162
233299308,7cyclictest26061-21ssh10:38:441
2332993011,7cyclictest9949-21ssh09:23:121
232199306,3cyclictest131rcu_preempt12:35:130
234999295,13cyclictest29566irq/58-eth010:50:143
234199291,25cyclictest20914-21diskmemload10:00:372
234199291,25cyclictest20914-21diskmemload10:00:372
234199291,25cyclictest13364-21kthreadcore07:20:192
234199290,25cyclictest2415-21latency_hist07:10:022
2332992911,10cyclictest15491-21ssh10:50:551
232199292,21cyclictest1805-21ssh11:25:160
232199292,19cyclictest27749-21latency12:00:220
234999288,12cyclictest29566irq/58-eth012:20:143
234999283,18cyclictest0-21swapper/311:55:313
234999283,16cyclictest29566irq/58-eth011:20:153
2332992816,10cyclictest0-21swapper/111:20:141
2332992812,6cyclictest12626-21ssh12:32:411
2332992811,6cyclictest18565-21ssh10:11:021
2332992810,6cyclictest26684-21ssh09:11:311
233299280,8cyclictest0-21swapper/110:00:301
233299280,8cyclictest0-21swapper/110:00:291
2321992812,10cyclictest7769-21ssh12:10:200
232199281,19cyclictest18665-21ssh10:55:160
233299277,7cyclictest7448-21ssh11:49:441
233299272,19cyclictest8371-21ssh11:50:191
2332992715,9cyclictest25212-21timerwakeupswit10:15:301
233299270,11cyclictest8308-21ssh10:47:031
232199272,19cyclictest31141-21usage_EthcTask11:00:310
232199271,21cyclictest8049-21ssh11:50:160
234999269,11cyclictest29566irq/58-eth010:25:023
234999263,16cyclictest0-21swapper/311:50:203
234999263,15cyclictest0-21swapper/310:55:083
234199260,23cyclictest8613-21latency_hist07:50:022
234199260,19cyclictest29080-21sshd08:44:432
233299262,20cyclictest15176-21ssh12:15:121
2332992616,8cyclictest1715-21kthreadcore09:40:181
233299260,10cyclictest17439-21cat07:55:321
234999253,14cyclictest0-21swapper/312:19:313
234999252,17cyclictest0-21swapper/311:00:103
2332992519,2cyclictest0-21swapper/112:00:111
2332992517,6cyclictest20914-21diskmemload11:57:331
2332992515,8cyclictest22976-21usage_EthcTask11:15:351
233299250,8cyclictest5565-21ps08:20:161
234999243,15cyclictest0-21swapper/312:00:223
234999243,12cyclictest29566irq/58-eth010:35:133
234999243,11cyclictest0-21swapper/311:25:143
2332992416,6cyclictest4047-21fschecks_time07:45:161
2332992416,6cyclictest21190-21ntpq11:35:261
2332992416,6cyclictest20914-21diskmemload11:26:531
2332992416,6cyclictest20914-21diskmemload09:56:421
2332992416,6cyclictest13387-21ssh12:12:311
2332992416,6cyclictest12080-21kthreadcore09:00:191
2332992415,7cyclictest4173-21timerwakeupswit08:15:281
2332992415,7cyclictest27295-21kthreadcore07:35:201
2332992414,8cyclictest4941-21kthreadcore10:25:211
2332992414,7cyclictest8961-21/usr/sbin/munin07:50:151
2332992413,9cyclictest26926-21sort12:20:271
2332992413,5cyclictest0-21swapper/109:35:141
2332992412,5cyclictest0-21swapper/107:25:151
233299240,9cyclictest14660-21grep12:35:161
233299240,8cyclictest0-21swapper/108:45:251
232199241,17cyclictest23086-21rm12:20:140
233299237,8cyclictest0-21swapper/108:30:231
2332992313,8cyclictest22522-21usage_Anybus08:00:321
233299231,19cyclictest28137-21ls08:40:271
232199234,12cyclictest0-21swapper/010:20:140
232199233,15cyclictest31015-21ssh12:25:040
232199232,16cyclictest25279-21ssh11:40:150
2321992313,5cyclictest0-21swapper/009:25:120
232199231,18cyclictest2570-21ls07:10:150
232199231,17cyclictest4234-21ssh10:25:150
232199231,17cyclictest16174-21ssh11:55:140
233299228,11cyclictest0-21swapper/107:40:171
2332992212,8cyclictest25897-21kthreadcore08:05:211
2332992212,7cyclictest3047-21irqrtprio07:10:181
2332992211,9cyclictest19186-21kthreadcore09:05:221
232199221,16cyclictest402-21ls08:15:140
233299217,11cyclictest0-21swapper/108:35:271
2332992113,6cyclictest29234-21tr08:10:211
2332992111,8cyclictest14652-21latency_hist08:30:011
2332992110,8cyclictest3322-21kthreadcore08:50:201
233299210,7cyclictest25282-21ntp_states07:30:251
232199215,7cyclictest131rcu_preempt10:15:130
232199214,11cyclictest0-21swapper/009:15:110
232199213,12cyclictest0-21swapper/007:50:120
2321992111,7cyclictest31220-21cron07:40:010
2321992111,6cyclictest20914-21diskmemload11:20:160
232199211,16cyclictest15258-21sed10:30:240
232199211,13cyclictest1975-21ls08:50:180
232199210,18cyclictest17308-21ls10:10:320
234999209,9cyclictest0-21swapper/312:29:503
234999203,11cyclictest0-21swapper/311:40:323
2349992012,6cyclictest0-21swapper/312:10:233
234999201,13cyclictest29566irq/58-eth009:10:153
2332992012,6cyclictest14695-21kthreadcore07:20:211
2332992012,6cyclictest11158-21latency_hist09:00:021
232199202,15cyclictest18890-21irqrtprio08:00:210
232199202,10cyclictest3934-21kthreadcore09:40:220
232199201,9cyclictest15820-21latency_hist09:50:010
232199201,12cyclictest505-21ls08:45:280
234999199,7cyclictest29566irq/58-eth011:30:143
234999193,10cyclictest0-21swapper/310:40:073
234999191,11cyclictest0-21swapper/309:20:133
2349991910,7cyclictest0-21swapper/312:05:193
2349991910,6cyclictest0-21swapper/312:31:523
2349991910,6cyclictest0-21swapper/311:17:173
2349991910,5cyclictest0-21swapper/309:15:133
232199198,8cyclictest13133-21awk10:10:010
232199196,4cyclictest131rcu_preempt09:31:320
232199193,11cyclictest0-21swapper/008:20:140
232199193,10cyclictest662-21ntpd08:25:110
232199192,10cyclictest0-21swapper/010:00:150
232199192,10cyclictest0-21swapper/010:00:140
2321991911,6cyclictest27322-21sshd09:12:320
232199191,10cyclictest8390-21kthreadcore12:30:220
234999189,6cyclictest0-21swapper/311:36:253
234999189,6cyclictest0-21swapper/309:52:073
234999187,8cyclictest0-21swapper/310:01:533
234999187,8cyclictest0-21swapper/310:01:523
234999183,10cyclictest0-21swapper/311:47:313
234999183,10cyclictest0-21swapper/310:45:063
234999182,10cyclictest0-21swapper/310:25:023
234999182,10cyclictest0-21swapper/309:00:143
234999182,10cyclictest0-21swapper/308:20:143
234999182,10cyclictest0-21swapper/307:20:153
234999181,12cyclictest0-21swapper/309:00:013
234999181,12cyclictest0-21swapper/307:30:013
234999181,11cyclictest0-21swapper/308:00:013
234999181,10cyclictest0-21swapper/307:10:173
2349991810,6cyclictest0-21swapper/312:35:233
233299181,14cyclictest9353-21kthreadcore07:15:211
232199188,8cyclictest29521-21load09:35:230
232199182,9cyclictest0-21swapper/010:45:120
232199182,11cyclictest0-21swapper/008:30:140
232199182,11cyclictest0-21swapper/008:05:140
232199181,14cyclictest31725-21timerwakeupswit08:10:280
232199181,14cyclictest19398-21kthreadcore07:25:230
2321991811,4cyclictest11202-21grep07:15:290
2321991810,6cyclictest17522-21cat09:50:190
234999179,6cyclictest0-21swapper/309:35:583
234999172,10cyclictest0-21swapper/308:40:143
234999172,10cyclictest0-21swapper/308:05:143
234999172,10cyclictest0-21swapper/308:00:173
234999172,10cyclictest0-21swapper/307:15:143
2349991712,3cyclictest0-21swapper/310:15:273
2349991710,4cyclictest0-21swapper/310:05:203
232199179,6cyclictest167-21systemd-network11:36:510
232199179,6cyclictest167-21systemd-network07:34:160
232199173,3cyclictest131rcu_preempt09:10:010
232199172,10cyclictest0-21swapper/008:55:170
232199172,10cyclictest0-21swapper/007:20:150
232199171,14cyclictest25016-21irqcore08:40:180
232199171,13cyclictest4109-21ntp_192.168.11411:45:220
232199171,13cyclictest21658-21kthreadcore10:35:220
232199171,12cyclictest3961-21diskstats07:45:150
234999162,11cyclictest0-21swapper/309:45:253
234999162,10cyclictest0-21swapper/308:50:153
234999162,10cyclictest0-21swapper/308:10:153
234999161,12cyclictest0-21swapper/308:45:243
232199165,9cyclictest31824-21irqcore07:40:210
232199164,9cyclictest4168-21fschecks_count09:20:170
232199162,10cyclictest0-21swapper/009:00:140
232199161,12cyclictest20292-21sed08:35:170
234999151,8cyclictest0-21swapper/308:30:143
234999151,11cyclictest0-21swapper/307:40:193
232199152,10cyclictest2443-21grep07:10:020
232199151,11cyclictest13689-21fschecks_time07:55:160
234999142,6cyclictest0-21swapper/307:30:123
234999141,8cyclictest0-21swapper/307:35:163
234999141,6cyclictest0-21swapper/309:10:013
234999141,6cyclictest0-21swapper/307:45:143
234999131,9cyclictest0-21swapper/307:55:023
234999131,7cyclictest0-21swapper/308:35:273
233299131,10cyclictest2308-21cyclictest07:10:021
234999111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional