You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 02:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Dec 22, 2025 12:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3146299662,45cyclictest12739-21ssh11:35:150
3146299609,39cyclictest26090-21ssh10:10:130
3147899551,33cyclictest8741-21ssh10:00:132
3147899541,29cyclictest29777-21ssh09:35:112
3147099502,37cyclictest32374-21ssh09:55:141
3146299502,41cyclictest29792-21ssh09:35:110
3148699496,24cyclictest28366irq/58-eth012:00:153
31462994932,7cyclictest12-21ksoftirqd/010:55:140
3147099483,35cyclictest12218-21ssh09:25:141
3147099476,7cyclictest131rcu_preempt07:35:101
3146299465,11cyclictest131rcu_preempt08:00:150
3147899441,33cyclictest0-21swapper/210:30:132
3146299432,36cyclictest27563-21ssh11:25:120
3148699422,22cyclictest28366irq/58-eth010:35:143
3148699414,22cyclictest28366irq/58-eth009:55:133
3148699411,26cyclictest28366irq/58-eth009:10:143
3146299411,32cyclictest3926-21ssh11:30:130
3147899403,23cyclictest27-21ksoftirqd/209:30:132
3147899402,21cyclictest12636-21ssh10:39:172
3146299402,7cyclictest131rcu_preempt08:15:140
3148699395,22cyclictest28366irq/58-eth010:50:133
3147099397,19cyclictest21-21ksoftirqd/111:05:141
3146299396,29cyclictest14898-21ssh09:45:130
3146299396,29cyclictest14898-21ssh09:45:120
3146299394,31cyclictest10098-21ssh11:15:140
3148699382,19cyclictest28366irq/58-eth011:10:153
3147899381,16cyclictest0-21swapper/208:30:132
3148699371,22cyclictest28366irq/58-eth008:15:143
3147099372,25cyclictest13333-21ssh12:15:151
3146299372,27cyclictest27070-21ssh09:15:130
3146299362,26cyclictest18886-21ssh11:20:130
31486993513,13cyclictest28366irq/58-eth010:40:143
3147899351,18cyclictest15321-21ssh11:55:142
3148699342,22cyclictest28366irq/58-eth012:20:133
31486993415,13cyclictest28366irq/58-eth009:40:133
3147899343,16cyclictest11139-21ssh10:20:142
3146299343,23cyclictest6172-21ssh09:40:130
3147099331,26cyclictest4563-21ssh12:10:131
3146299333,20cyclictest151rcuc/011:50:130
31462993318,7cyclictest0-21swapper/012:20:120
3148699322,14cyclictest28366irq/58-eth011:30:123
3147899322,23cyclictest31080-21ssh10:50:122
3146299322,4cyclictest32356-21apt-get09:55:130
31462993219,8cyclictest121ksoftirqd/010:20:160
3147899312,22cyclictest12440-21grep09:25:182
3146299313,19cyclictest25051-21ssh11:05:130
3146299312,23cyclictest23606-21ssh10:07:170
3146299311,19cyclictest31085-21ssh10:50:130
3148699305,17cyclictest28366irq/58-eth012:35:163
3148699303,15cyclictest28366irq/58-eth011:40:143
3146299302,19cyclictest7329-21df_inode12:30:180
3148699293,14cyclictest28366irq/58-eth011:25:133
3148699292,17cyclictest28366irq/58-eth009:30:143
3148699291,19cyclictest28366irq/58-eth009:15:133
3147899292,20cyclictest0-21swapper/208:55:152
3147899292,20cyclictest0-21swapper/207:50:142
3147899291,14cyclictest16025-21ssh12:35:152
3147099292,19cyclictest1454-21ssh11:10:161
3146299292,19cyclictest9965-21ssh10:35:510
31462992917,6cyclictest121ksoftirqd/008:30:140
31462992915,10cyclictest21047-21ssh09:30:140
3147899283,16cyclictest22346irq/40-dwc2_hso09:00:132
3147899283,16cyclictest22346irq/40-dwc2_hso08:45:132
3147899283,16cyclictest22346irq/40-dwc2_hso07:10:152
3147899282,20cyclictest0-21swapper/208:10:152
3147899281,15cyclictest6185-21sh09:40:152
3147099283,18cyclictest21464-21ssh11:40:141
3147099282,20cyclictest11821-21ssh10:38:161
3147099281,5cyclictest21-21ksoftirqd/109:40:131
3146299283,18cyclictest19866-21ssh10:25:160
3148699273,15cyclictest0-21swapper/310:07:163
3147899279,7cyclictest88166lldpd09:10:182
3147899273,9cyclictest0-21swapper/209:49:172
3147899273,9cyclictest0-21swapper/209:49:172
3147899273,17cyclictest22346irq/40-dwc2_hso11:35:162
3147899273,16cyclictest22346irq/40-dwc2_hso08:40:142
3147899273,15cyclictest22346irq/40-dwc2_hso11:20:142
3147899272,10cyclictest0-21swapper/208:20:152
31462992713,7cyclictest31450-21cyclictest07:10:150
31462992712,6cyclictest12-21ksoftirqd/007:15:150
3147899267,10cyclictest411-21cron07:45:002
3147899264,5cyclictest17429-21ssh10:05:142
3147899261,13cyclictest23654-21ssh09:50:132
3147099262,18cyclictest16267-21ssh11:00:151
3147099261,19cyclictest20224-21ssh11:55:341
3147099261,16cyclictest10091-21sh11:15:141
3146299262,18cyclictest28531-21ssh10:30:120
3148699259,8cyclictest28366irq/58-eth009:00:133
3147899253,17cyclictest27-21ksoftirqd/212:29:172
3147899252,18cyclictest8592-21ssh10:17:172
3147899252,17cyclictest22346irq/40-dwc2_hso09:55:142
3147899251,17cyclictest22346irq/40-dwc2_hso10:10:122
3147099258,9cyclictest21-21ksoftirqd/110:55:141
3147099252,18cyclictest3573-21ssh09:20:161
3147099251,19cyclictest17448-21ssh10:05:151
3146299257,4cyclictest131rcu_preempt09:00:130
3146299255,15cyclictest12-21ksoftirqd/012:00:140
3146299255,13cyclictest0-21swapper/010:00:140
31462992521,2cyclictest0-21swapper/012:10:150
3146299251,7cyclictest131rcu_preempt08:25:130
3146299251,7cyclictest131rcu_preempt08:25:120
3146299251,19cyclictest22420-21ssh10:45:150
3148699243,17cyclictest28366irq/58-eth008:20:143
3147899242,14cyclictest22346irq/40-dwc2_hso07:25:142
3147899241,9cyclictest24093-21ssh12:00:142
3147899241,5cyclictest0-21swapper/207:55:142
3147899241,18cyclictest22346irq/40-dwc2_hso10:55:152
3147099247,8cyclictest21-21ksoftirqd/110:49:031
3147099242,16cyclictest19959-21df10:25:171
3146299241,18cyclictest9850-21ls11:15:020
3148699233,13cyclictest0-21swapper/309:25:173
3148699232,13cyclictest28366irq/58-eth010:55:133
3147899232,6cyclictest0-21swapper/212:14:172
3147899232,4cyclictest0-21swapper/207:30:152
3147899232,16cyclictest0-21swapper/210:28:172
3147899232,14cyclictest22346irq/40-dwc2_hso12:20:122
31478992316,4cyclictest16449-21ssh12:15:242
31478992316,4cyclictest0-21swapper/209:25:012
3147099237,3cyclictest131rcu_preempt09:15:201
3147099236,7cyclictest21-21ksoftirqd/107:10:241
3147099232,3cyclictest131rcu_preempt09:30:201
31470992314,5cyclictest16000-21sh12:35:141
31470992312,7cyclictest16391-21cron08:35:011
31470992311,6cyclictest131rcu_preempt11:48:581
31470992311,4cyclictest21-21ksoftirqd/107:55:171
3147099230,14cyclictest21-21ksoftirqd/108:15:231
3146299234,3cyclictest131rcu_preempt11:45:140
3146299234,12cyclictest0-21swapper/011:00:130
3146299233,17cyclictest18314-21ssh09:10:130
3146299232,2cyclictest131rcu_preempt09:25:140
31462992311,3cyclictest131rcu_preempt10:40:220
3146299231,10cyclictest29575-21dpkg08:15:020
3148699223,13cyclictest0-21swapper/309:45:173
3148699223,13cyclictest0-21swapper/309:45:173
3147899228,11cyclictest0-21swapper/208:50:172
3147899228,11cyclictest0-21swapper/208:50:162
3147899226,7cyclictest27-21ksoftirqd/210:50:022
3147899224,9cyclictest30589-21sh09:15:242
3147899223,15cyclictest22346irq/40-dwc2_hso11:40:132
3147899222,6cyclictest0-21swapper/212:30:182
3147899222,6cyclictest0-21swapper/211:10:162
3147899222,15cyclictest22346irq/40-dwc2_hso07:35:122
3147899221,15cyclictest22346irq/40-dwc2_hso09:05:142
3147899221,15cyclictest22346irq/40-dwc2_hso09:05:142
3147899221,11cyclictest1754-21grep11:45:292
3147899221,11cyclictest14698-21kthreadcore10:40:212
31478992210,8cyclictest6540-21/usr/sbin/munin11:50:202
3147099228,4cyclictest21-21ksoftirqd/109:36:071
3147099226,8cyclictest21-21ksoftirqd/111:32:361
3147099221,7cyclictest131rcu_preempt12:23:161
31470992213,6cyclictest31955-21cron09:55:011
31470992212,5cyclictest0-21swapper/108:50:151
31470992212,5cyclictest0-21swapper/108:50:151
3147099221,18cyclictest32570-21ls12:05:021
3147099221,16cyclictest7219-21ssh12:30:161
31470992210,3cyclictest21-21ksoftirqd/110:51:371
3146299222,13cyclictest12-21ksoftirqd/009:20:140
3146299221,15cyclictest23252-21ls07:35:010
3148699218,10cyclictest0-21swapper/311:20:213
3148699213,12cyclictest0-21swapper/311:15:153
3148699213,11cyclictest28366irq/58-eth010:20:143
3147899213,9cyclictest8192-21kthreadcore08:25:192
3147899213,9cyclictest8192-21kthreadcore08:25:192
3147899212,5cyclictest0-21swapper/208:15:042
3147899212,16cyclictest22346irq/40-dwc2_hso08:05:132
3147899212,15cyclictest0-21swapper/212:07:172
3147899212,11cyclictest7783-21tail07:15:302
3147899211,11cyclictest12748-21kthreadcore11:15:232
3147899211,10cyclictest27962-21kthreadcore11:05:242
3147899211,10cyclictest19726-21latency11:00:232
31478992110,8cyclictest8968-21tune2fs07:20:172
3147099219,6cyclictest131rcu_preempt10:20:281
3147099219,10cyclictest0-21swapper/112:28:031
3147099218,4cyclictest21-21ksoftirqd/107:25:171
3147099218,3cyclictest21-21ksoftirqd/108:20:261
3147099216,8cyclictest21-21ksoftirqd/109:46:371
3147099216,8cyclictest21-21ksoftirqd/109:46:361
3147099212,14cyclictest21938-21ssh09:10:251
31470992115,4cyclictest0-21swapper/107:40:131
3147099211,3cyclictest131rcu_preempt12:05:071
31470992112,6cyclictest21-21ksoftirqd/111:35:161
3147099211,16cyclictest17440-21kthreadcore08:35:231
3146299212,6cyclictest17025-21ntp_states12:15:270
3146299211,17cyclictest23375-21kthreadcore08:40:210
3148699202,10cyclictest0-21swapper/310:10:133
31486992013,4cyclictest0-21swapper/311:55:253
3147899206,7cyclictest16707-21df08:35:172
3147899203,14cyclictest4282-21if_eth111:30:172
3147899201,9cyclictest3031-21kthreadcore07:45:232
3147899201,9cyclictest29396-21kthreadcore11:25:212
3147899201,7cyclictest15152-21apt-get08:00:152
3147099209,9cyclictest0-21swapper/111:21:581
3147099206,6cyclictest21-21ksoftirqd/108:00:261
3147099205,9cyclictest21-21ksoftirqd/110:01:331
3147099203,6cyclictest131rcu_preempt10:30:131
3147099201,9cyclictest21-21ksoftirqd/111:25:191
31470992010,4cyclictest21-21ksoftirqd/110:10:391
3147099200,13cyclictest25156-21ls08:40:271
31462992012,4cyclictest121ksoftirqd/012:35:150
3146299201,10cyclictest19454-21ssh11:55:310
3148699199,7cyclictest28366irq/58-eth011:35:153
3148699199,7cyclictest0-21swapper/311:05:393
3148699199,6cyclictest0-21swapper/311:47:073
3148699199,6cyclictest0-21swapper/310:01:533
3148699193,11cyclictest0-21swapper/312:27:173
3148699193,11cyclictest0-21swapper/309:38:163
3148699192,11cyclictest0-21swapper/310:45:153
3148699192,10cyclictest0-21swapper/308:45:143
3148699191,13cyclictest0-21swapper/307:35:203
3148699191,11cyclictest0-21swapper/308:00:153
31486991910,6cyclictest0-21swapper/312:11:253
31486991910,4cyclictest0-21swapper/311:00:583
3147099198,5cyclictest21-21ksoftirqd/110:16:141
3147099197,4cyclictest21-21ksoftirqd/107:15:191
3147099194,3cyclictest131rcu_preempt08:25:231
3147099194,3cyclictest131rcu_preempt08:25:231
31470991914,3cyclictest0-21swapper/109:00:181
31470991913,4cyclictest0-21swapper/107:45:201
3147099191,14cyclictest3575-21apt-get08:55:131
3147099191,14cyclictest13656-21ssh10:40:141
31470991910,3cyclictest131rcu_preempt11:53:151
3146299198,6cyclictest0-21swapper/012:27:170
3146299197,9cyclictest394-21awk07:40:330
3146299191,8cyclictest10705-21cron10:20:000
3146299191,7cyclictest25090-21ntp_192.168.11411:40:250
31462991912,5cyclictest0-21swapper/008:35:250
3146299191,11cyclictest31287-21cut08:50:180
3146299191,11cyclictest31287-21cut08:50:180
3146299190,15cyclictest10437-21cut07:55:160
3148699189,6cyclictest0-21swapper/312:18:133
3148699189,6cyclictest0-21swapper/310:25:093
3148699183,11cyclictest0-21swapper/312:06:063
3148699182,11cyclictest0-21swapper/310:17:183
3148699182,11cyclictest0-21swapper/309:50:143
3148699182,10cyclictest0-21swapper/309:05:143
3148699182,10cyclictest0-21swapper/309:05:143
3148699182,10cyclictest0-21swapper/307:45:153
3148699181,13cyclictest0-21swapper/308:35:313
3148699181,11cyclictest0-21swapper/309:20:173
3147099188,8cyclictest13267-21apt-get09:05:131
3147099188,8cyclictest13267-21apt-get09:05:131
3147099180,12cyclictest24971-21cut08:10:171
3146299182,13cyclictest24021-21tail09:50:180
3146299182,11cyclictest0-21swapper/008:20:150
3146299181,15cyclictest8185-21kthreadcore07:50:220
31462991810,4cyclictest0-21swapper/008:55:140
3148699179,6cyclictest0-21swapper/311:50:083
3148699179,6cyclictest0-21swapper/310:30:023
3148699172,9cyclictest0-21swapper/307:10:163
3148699172,10cyclictest0-21swapper/312:30:163
3148699172,10cyclictest0-21swapper/308:40:153
3148699172,10cyclictest0-21swapper/307:30:153
3148699172,10cyclictest0-21swapper/307:25:153
3148699172,10cyclictest0-21swapper/307:20:143
3148699171,12cyclictest0-21swapper/308:10:223
3148699171,10cyclictest0-21swapper/308:50:163
3148699171,10cyclictest0-21swapper/308:50:153
3148699171,10cyclictest0-21swapper/307:40:243
3147099178,7cyclictest19983-21apt-get08:05:131
3147099175,4cyclictest21-21ksoftirqd/108:45:291
3147099173,8cyclictest21-21ksoftirqd/107:50:181
3147099172,13cyclictest8914-21fschecks_count07:20:171
31470991712,3cyclictest0-21swapper/107:30:201
3146299175,9cyclictest31450-21cyclictest07:10:020
3146299171,2cyclictest131rcu_preempt07:45:140
3146299171,11cyclictest1121-21ssh12:06:070
3146299168,6cyclictest8887-21diskstats07:20:160
3146299162,9cyclictest0-21swapper/008:05:140
3146299162,11cyclictest25199-21kthreadcore07:35:180
3146299162,10cyclictest0-21swapper/009:05:140
3146299162,10cyclictest0-21swapper/009:05:130
3146299161,5cyclictest28580-21kthreadcore08:45:210
3146299160,12cyclictest0-21swapper/007:25:140
3148699152,8cyclictest0-21swapper/307:50:133
3148699142,8cyclictest0-21swapper/308:55:133
3147899143,8cyclictest0-21swapper/207:10:022
3147099141,10cyclictest31489-21grep07:10:021
3148699132,7cyclictest0-21swapper/308:05:133
3148699131,9cyclictest0-21swapper/308:00:023
3148699131,7cyclictest0-21swapper/308:30:153
3148699121,9cyclictest0-21swapper/308:25:163
3148699121,9cyclictest0-21swapper/308:25:163
3148699121,5cyclictest0-21swapper/307:10:013
3148699111,7cyclictest0-21swapper/307:15:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional