You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 01:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Fri Aug 07, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9882997017,9cyclictest131rcu_preempt11:15:131
9882997017,9cyclictest131rcu_preempt11:15:121
9882996711,12cyclictest131rcu_preempt11:10:121
989399581,35cyclictest0-21swapper/208:30:142
987399573,11cyclictest131rcu_preempt12:00:120
987399565,47cyclictest31712-21ssh12:30:130
987399552,39cyclictest4747-21ssh09:15:110
988299491,36cyclictest9913-21ssh10:30:121
987399493,36cyclictest20616-21ssh10:00:130
990299464,25cyclictest35966irq/58-eth011:15:133
990299464,25cyclictest35966irq/58-eth011:15:123
987399451,33cyclictest8202-21ssh09:35:150
987399441,37cyclictest6305-21ssh10:10:120
9873994413,17cyclictest12-21ksoftirqd/007:10:120
988299431,35cyclictest13704-21ssh09:20:131
987399434,29cyclictest11922-21ssh11:25:140
989399409,18cyclictest27-21ksoftirqd/211:20:142
989399409,18cyclictest27-21ksoftirqd/211:20:132
9893994029,2cyclictest22673-21sh09:25:122
9882994010,22cyclictest21-21ksoftirqd/110:45:111
987399404,30cyclictest0-21swapper/011:45:150
9873993920,14cyclictest628-21tcf-agent11:00:120
990299382,25cyclictest35966irq/58-eth007:50:123
987399381,28cyclictest28256-21ssh09:10:150
987399381,28cyclictest28256-21ssh09:10:140
9902993621,9cyclictest0-21swapper/312:15:133
989399362,18cyclictest15891-21ssh10:31:482
988299366,19cyclictest26097-21ssh09:45:131
988299362,7cyclictest131rcu_preempt09:00:131
987399361,9cyclictest26253-21apt-get11:15:140
987399361,9cyclictest26253-21apt-get11:15:130
9873993611,22cyclictest26300-21kthreadcore10:20:210
988299352,25cyclictest912-21ssh10:25:151
987399351,9cyclictest29255-21tr08:35:220
987399351,9cyclictest22104-21fschecks_count08:30:160
987399351,8cyclictest4464-21ssh09:50:230
987399351,8cyclictest4464-21ssh09:50:220
987399351,10cyclictest9518-21tr08:50:190
987399351,10cyclictest28655-21kthreadcore09:45:220
990299340,17cyclictest35966irq/58-eth010:05:143
987399343,23cyclictest28088-21ssh10:40:150
987399342,25cyclictest19115-21ssh12:05:120
987399342,10cyclictest5205-21irqcore12:15:200
987399341,9cyclictest19562-21irqrtprio10:35:190
987399341,9cyclictest19562-21irqrtprio10:35:180
987399341,9cyclictest15058-21grep07:45:270
987399341,9cyclictest13013-21kthreadcore10:30:220
987399341,9cyclictest10272-21kthreadcore08:15:230
987399341,8cyclictest10406-21kthreadcore12:35:200
9873993417,10cyclictest29532-21sh12:27:520
987399341,11cyclictest3565-21cut11:55:230
987399341,10cyclictest20451-21cut07:20:200
987399340,10cyclictest10110-21kthreadcore11:05:220
9902993325,6cyclictest0-21swapper/312:20:133
9902993318,11cyclictest35966irq/58-eth012:35:143
9893993322,9cyclictest20596-21ssh10:00:122
988299335,10cyclictest21-21ksoftirqd/110:20:121
9882993317,10cyclictest9865-21cyclictest09:55:121
9882993314,15cyclictest131rcu_preempt07:55:151
987399332,8cyclictest612-21unixbench_singl07:30:300
987399331,9cyclictest14064-21irqcore08:55:160
987399331,8cyclictest8645-21tr07:40:200
987399331,8cyclictest25310-21kthreadcore11:50:190
987399331,8cyclictest19487-21tr08:25:220
987399331,8cyclictest12671-21ssh09:55:200
987399331,10cyclictest24619-21fschecks_time07:25:160
987399331,10cyclictest22856-21tr11:30:210
987399330,9cyclictest25711-21kthreadcore10:55:230
988299322,6cyclictest21-21ksoftirqd/109:50:151
988299322,6cyclictest21-21ksoftirqd/109:50:141
987399325,8cyclictest25588-21apt-get08:00:110
987399325,24cyclictest29893-21ssh11:35:130
987399325,20cyclictest6456-21ssh11:40:130
987399323,6cyclictest31337-21kthreadcore08:05:240
987399323,6cyclictest16982-21latency_hist08:25:010
987399322,10cyclictest17534-21ssh09:20:270
987399321,9cyclictest4795-21ssh10:45:150
987399321,8cyclictest26516-21find09:25:280
987399321,11cyclictest18579-21kthreadcore09:40:190
987399320,10cyclictest20756-21sort11:10:210
990299312,18cyclictest35966irq/58-eth010:45:123
987399315,5cyclictest18201-21needreboot07:15:230
987399312,9cyclictest3729-21latency_hist08:45:020
987399312,9cyclictest29884-21diskstats10:05:170
987399312,8cyclictest16303-21irqcore07:50:160
987399311,9cyclictest31596-21ssh09:30:100
987399311,7cyclictest15686-21if_eth110:15:190
9873993112,16cyclictest2808-21/usr/sbin/munin11:20:150
9873993112,16cyclictest2808-21/usr/sbin/munin11:20:140
989399303,20cyclictest0-21swapper/210:45:142
9882993012,8cyclictest0-21swapper/108:40:261
987399301,9cyclictest4591-21ntp_states10:25:250
987399301,9cyclictest3947-21acpi08:45:110
987399301,9cyclictest1721-21fschecks_time07:35:160
987399301,8cyclictest9865-21cyclictest07:10:010
987399301,7cyclictest17482-21ntp_states12:20:270
989399294,15cyclictest27346irq/40-dwc2_hso09:05:102
9893992918,3cyclictest27835-21diskmemload09:40:262
9893992916,10cyclictest29448-21sshd10:20:472
989399291,21cyclictest27-21ksoftirqd/210:19:482
988299293,15cyclictest21-21ksoftirqd/108:30:141
988299292,19cyclictest8212-21ssh12:35:141
987399291,9cyclictest29162-21ssh12:10:180
987399291,9cyclictest20102-21ssh10:52:230
987399291,8cyclictest24616-21kthreadcore09:05:170
987399291,8cyclictest23047-21cron09:05:020
987399291,8cyclictest20867-21fschecks_count07:55:160
987399281,9cyclictest3449-21kthreadcore08:10:190
989399279,12cyclictest261rcuc/208:20:112
989399273,16cyclictest27346irq/40-dwc2_hso08:50:122
989399272,9cyclictest24930-21ssh10:36:472
989399272,9cyclictest24930-21ssh10:36:472
989399272,15cyclictest8359-21ssh11:05:152
989399271,11cyclictest24433-21apt-get11:50:122
989399264,16cyclictest27346irq/40-dwc2_hso07:55:132
989399263,15cyclictest27346irq/40-dwc2_hso10:25:122
9893992621,3cyclictest27-21ksoftirqd/211:40:122
9893992616,7cyclictest0-21swapper/212:15:132
988299269,12cyclictest21-21ksoftirqd/112:09:481
988299261,17cyclictest21-21ksoftirqd/107:30:141
9882992611,3cyclictest131rcu_preempt11:05:021
9882992610,12cyclictest21-21ksoftirqd/110:19:531
989399256,11cyclictest4058-21sshd09:14:382
989399256,11cyclictest4058-21sshd09:14:372
989399254,14cyclictest27346irq/40-dwc2_hso08:15:142
989399253,16cyclictest27-21ksoftirqd/210:06:162
989399253,15cyclictest27346irq/40-dwc2_hso07:40:132
989399253,15cyclictest27346irq/40-dwc2_hso07:35:132
9893992512,10cyclictest31233-21ssh09:45:532
9893992512,10cyclictest20229-21ssh12:22:302
989399251,16cyclictest0-21swapper/207:30:142
988299251,4cyclictest21-21ksoftirqd/111:55:151
9882992514,3cyclictest21-21ksoftirqd/109:15:281
989399249,9cyclictest131rcu_preempt12:07:322
989399248,2cyclictest131rcu_preempt12:02:092
989399244,13cyclictest27346irq/40-dwc2_hso07:25:132
9893992414,8cyclictest0-21swapper/209:35:142
9893992412,9cyclictest3197-21ssh12:30:302
9893992412,9cyclictest0-21swapper/212:11:412
9893992411,10cyclictest27750-21ssh12:25:442
9893992411,10cyclictest13149-21ssh12:35:382
9893992411,10cyclictest0-21swapper/211:58:092
9893992410,3cyclictest131rcu_preempt11:16:332
9893992410,3cyclictest131rcu_preempt11:16:332
9882992412,4cyclictest21-21ksoftirqd/108:20:291
990299238,10cyclictest0-21swapper/309:59:483
990299233,13cyclictest0-21swapper/310:31:493
990299231,16cyclictest0-21swapper/310:29:563
989399239,3cyclictest131rcu_preempt10:57:042
989399239,11cyclictest0-21swapper/207:20:252
989399239,10cyclictest1657-21uptime11:35:312
989399237,13cyclictest19433-21cat11:45:302
989399237,12cyclictest0-21swapper/209:20:262
989399235,3cyclictest27-21ksoftirqd/210:55:012
989399235,12cyclictest27-21ksoftirqd/207:45:252
989399233,10cyclictest21035-21kthreadcore09:00:212
9893992313,8cyclictest0-21swapper/211:14:262
9893992313,8cyclictest0-21swapper/209:33:452
9893992313,8cyclictest0-21swapper/208:55:242
9893992312,9cyclictest10874-21ssh09:17:042
9893992312,9cyclictest0-21swapper/211:00:372
9893992311,9cyclictest0-21swapper/210:44:272
989399231,14cyclictest27346irq/40-dwc2_hso09:55:132
989399231,13cyclictest20900-21ssh11:30:132
988299237,8cyclictest0-21swapper/112:27:571
988299233,15cyclictest17128-21ssh09:40:141
9882992311,3cyclictest21-21ksoftirqd/112:00:301
9882992310,7cyclictest0-21swapper/109:05:151
989399228,8cyclictest27-21ksoftirqd/211:28:472
989399226,13cyclictest28628-21kthreadcore08:35:222
989399221,17cyclictest17953-21kthreadcore07:15:212
9893992210,9cyclictest261rcuc/208:25:132
988299229,4cyclictest21-21ksoftirqd/112:20:311
988299229,2cyclictest21-21ksoftirqd/112:10:271
988299228,4cyclictest21-21ksoftirqd/107:10:151
988299227,5cyclictest21-21ksoftirqd/108:05:181
988299227,12cyclictest131rcu_preempt10:40:151
988299222,16cyclictest8360-21ssh11:05:141
9882992211,3cyclictest21-21ksoftirqd/110:01:021
9882992210,3cyclictest21-21ksoftirqd/111:30:331
9902992112,4cyclictest0-21swapper/310:40:213
9902992111,7cyclictest0-21swapper/311:45:163
990299211,14cyclictest35966irq/58-eth008:15:133
9902992110,7cyclictest35966irq/58-eth009:50:163
9902992110,7cyclictest35966irq/58-eth009:50:153
989399219,4cyclictest27-21ksoftirqd/210:10:062
989399216,12cyclictest13274-21latency07:10:212
989399214,13cyclictest0-21swapper/207:50:132
989399212,11cyclictest5177-21kthreadcore08:10:212
989399211,9cyclictest4154-21fschecks_time08:45:152
9893992113,5cyclictest6453-21ssh09:50:292
9893992113,5cyclictest6453-21ssh09:50:292
9893992112,3cyclictest0-21swapper/208:40:252
988299219,3cyclictest21-21ksoftirqd/112:20:021
988299218,4cyclictest21-21ksoftirqd/108:10:301
9882992115,4cyclictest0-21swapper/110:55:131
9882992115,4cyclictest0-21swapper/107:15:191
9882992115,3cyclictest21-21ksoftirqd/111:20:221
9882992115,3cyclictest21-21ksoftirqd/111:20:211
9882992113,4cyclictest630-21Async09:36:311
988299211,17cyclictest10114-21ls07:40:271
988299211,13cyclictest29773-21ssh10:05:151
988299211,10cyclictest6315-21ssh10:10:131
9882992110,9cyclictest0-21swapper/110:53:121
9882992110,3cyclictest21-21ksoftirqd/112:33:421
9882992110,3cyclictest21-21ksoftirqd/111:50:211
9882992110,3cyclictest21-21ksoftirqd/111:27:411
9882992110,3cyclictest21-21ksoftirqd/107:35:141
9882992110,2cyclictest21-21ksoftirqd/111:48:421
990299209,8cyclictest0-21swapper/310:13:253
990299203,11cyclictest0-21swapper/311:43:493
990299202,11cyclictest0-21swapper/311:20:073
990299202,11cyclictest0-21swapper/311:20:063
990299201,14cyclictest0-21swapper/311:05:133
989399201,15cyclictest25697-21fschecks_count08:00:152
989399201,10cyclictest31063-21kthreadcore08:05:222
988299207,2cyclictest21-21ksoftirqd/110:37:431
988299207,2cyclictest21-21ksoftirqd/110:37:421
988299201,15cyclictest28255-21ssh09:10:151
988299201,15cyclictest28255-21ssh09:10:141
9882992010,2cyclictest21-21ksoftirqd/111:41:261
9882992010,2cyclictest21-21ksoftirqd/109:35:121
990299199,8cyclictest0-21swapper/312:01:273
990299193,11cyclictest0-21swapper/311:55:133
990299193,11cyclictest0-21swapper/310:35:483
990299193,11cyclictest0-21swapper/310:35:473
9902991913,4cyclictest0-21swapper/309:40:293
9902991912,5cyclictest0-21swapper/309:00:113
9902991911,5cyclictest35966irq/58-eth009:35:143
988299198,2cyclictest21-21ksoftirqd/109:33:291
988299195,8cyclictest21-21ksoftirqd/108:25:161
988299191,4cyclictest0-21swapper/108:50:161
9882991913,4cyclictest0-21swapper/108:55:171
9882991913,4cyclictest0-21swapper/108:45:191
988299191,15cyclictest26397-21kthreadcore08:00:181
990299189,6cyclictest0-21swapper/310:51:453
990299188,7cyclictest0-21swapper/309:10:143
990299188,7cyclictest0-21swapper/309:10:133
990299187,9cyclictest33-21ksoftirqd/311:51:423
990299187,9cyclictest33-21ksoftirqd/310:21:483
990299182,10cyclictest0-21swapper/310:19:473
9902991812,4cyclictest0-21swapper/312:05:133
9902991811,5cyclictest35966irq/58-eth012:25:293
990299181,12cyclictest35966irq/58-eth010:55:143
9902991810,6cyclictest0-21swapper/312:30:513
9902991810,6cyclictest0-21swapper/311:32:033
9902991810,6cyclictest0-21swapper/311:26:533
9902991810,6cyclictest0-21swapper/309:34:593
9902991810,6cyclictest0-21swapper/309:23:423
9882991813,3cyclictest0-21swapper/107:45:251
9882991812,4cyclictest0-21swapper/107:20:151
9882991811,5cyclictest0-21swapper/107:25:271
988299181,13cyclictest22741-21sh09:25:121
990299179,6cyclictest0-21swapper/311:36:193
990299179,6cyclictest0-21swapper/311:13:393
990299179,5cyclictest35966irq/58-eth010:00:133
990299179,3cyclictest0-21swapper/311:01:533
990299177,8cyclictest33-21ksoftirqd/309:17:113
990299177,8cyclictest0-21swapper/309:47:453
990299172,9cyclictest0-21swapper/307:30:143
990299171,10cyclictest0-21swapper/308:40:133
988299173,11cyclictest26851-21fschecks_count08:35:171
988299171,3cyclictest0-21swapper/107:50:121
990299168,6cyclictest33-21ksoftirqd/312:11:093
990299168,6cyclictest0-21swapper/309:27:043
990299161,9cyclictest35966irq/58-eth008:10:113
9902991611,3cyclictest0-21swapper/307:43:123
988299161,12cyclictest9935-21ls07:10:021
988299161,11cyclictest8737-21kthreadcore08:15:201
990299159,4cyclictest35966irq/58-eth009:10:013
990299151,9cyclictest0-21swapper/308:10:023
990299151,7cyclictest0-21swapper/308:40:013
9902991510,3cyclictest0-21swapper/308:45:273
9902991510,3cyclictest0-21swapper/307:35:253
990299149,3cyclictest0-21swapper/308:25:183
990299141,8cyclictest0-21swapper/308:35:023
990299141,8cyclictest0-21swapper/307:15:023
990299133,7cyclictest0-21swapper/307:55:173
990299132,7cyclictest0-21swapper/308:00:123
990299131,9cyclictest0-21swapper/308:50:143
989399131,8cyclictest9934-21latency_hist07:10:012
990299127,3cyclictest35966irq/58-eth007:45:123
990299122,6cyclictest0-21swapper/308:20:133
990299122,6cyclictest0-21swapper/307:25:133
990299120,9cyclictest33-21ksoftirqd/307:20:133
990299115,4cyclictest0-21swapper/308:59:353
990299111,7cyclictest0-21swapper/307:15:213
99029991,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional