You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 17:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Thu Jun 25, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8755998816,17cyclictest131rcu_preempt11:55:132
8766996756,5cyclictest35966irq/58-eth012:15:133
8755996045,10cyclictest26827-21diskmemload12:15:132
874699607,37cyclictest22143-21ssh09:25:141
8766995637,12cyclictest35966irq/58-eth012:10:123
8766995637,12cyclictest35966irq/58-eth012:10:113
8755995614,33cyclictest27-21ksoftirqd/210:15:132
8766995327,16cyclictest35966irq/58-eth011:05:113
8766995033,11cyclictest35966irq/58-eth009:50:133
874699501,13cyclictest24675-21apt-get08:00:121
875599461,35cyclictest489-21ssh11:00:132
874699469,29cyclictest11167-21ssh12:35:131
8755994515,13cyclictest27-21ksoftirqd/208:30:122
8746994415,22cyclictest21-21ksoftirqd/111:20:121
874699434,32cyclictest5637-21sh10:45:141
874699433,32cyclictest13533-21ssh11:25:131
874699431,14cyclictest12426-21ssh12:00:121
874699430,7cyclictest11951-21ssh09:55:131
873899434,6cyclictest131rcu_preempt08:40:120
874699411,6cyclictest13000-21ssh09:20:121
873899412,36cyclictest9646-21ssh11:05:110
873899403,32cyclictest1808-21sh10:25:120
873899401,31cyclictest2114-21ssh12:30:100
873899392,32cyclictest30550-21sshd12:10:110
873899392,32cyclictest30550-21sshd12:10:100
873899391,26cyclictest18747-21ssh11:10:120
8755993811,9cyclictest27-21ksoftirqd/210:40:122
874699389,25cyclictest0-21swapper/109:45:131
876699371,24cyclictest35966irq/58-eth011:10:133
876699371,19cyclictest35966irq/58-eth009:45:123
875599374,23cyclictest0-21swapper/211:25:132
875599372,17cyclictest131rcu_preempt12:33:112
874699376,26cyclictest0-21swapper/110:40:121
874699373,9cyclictest17513-21/usr/sbin/munin11:45:231
874699371,12cyclictest19660-21ntp_states10:15:261
8766993624,5cyclictest35966irq/58-eth012:30:123
875599362,18cyclictest26676-21ssh11:50:152
874699364,18cyclictest21-21ksoftirqd/112:05:141
874699363,9cyclictest31347-21kthreadcore08:05:201
874699363,26cyclictest27216-21ssh09:10:111
874699361,9cyclictest978-21sed07:35:161
874699352,9cyclictest22399-21grep09:05:151
874699351,9cyclictest8108-21cat11:40:021
874699351,9cyclictest4364-21cut12:30:201
874699351,9cyclictest16337-21kthreadcore10:50:201
874699351,9cyclictest16337-21kthreadcore10:50:191
874699351,8cyclictest3114-21ssh10:25:191
874699351,8cyclictest1639-21ls10:05:261
874699351,25cyclictest8403-21ssh11:40:121
874699351,10cyclictest18069-21sed09:00:171
874699351,10cyclictest1645-21kthreadcore11:00:191
874699350,9cyclictest8748-21cut08:50:191
875599344,22cyclictest0-21swapper/209:55:142
875599343,19cyclictest30549-21ssh12:10:112
875599343,19cyclictest30549-21ssh12:10:102
8755993411,13cyclictest0-21swapper/207:25:142
8746993420,9cyclictest8730-21cyclictest11:15:121
874699341,9cyclictest6978-21if_eth108:15:181
874699341,9cyclictest31340-21kthreadcore08:40:201
874699341,9cyclictest29527-21grep12:25:291
874699341,9cyclictest29527-21grep12:25:291
874699341,9cyclictest29332-21kthreadcore11:50:221
874699341,9cyclictest21551-21kthreadcore10:35:191
874699341,9cyclictest19932-21df_inode07:55:151
874699341,9cyclictest19719-21kthreadcore07:20:191
874699341,9cyclictest18863-21df11:10:141
874699341,9cyclictest18300-21apt-config07:20:021
874699341,9cyclictest17925-21tr08:25:211
874699341,9cyclictest16210-21latency_hist12:20:011
874699341,9cyclictest10334-21kthreadcore07:10:211
874699341,8cyclictest28330-21tr08:35:201
875599332,23cyclictest10919-21ssh10:30:142
875599331,17cyclictest9662-21sshd11:05:122
874699332,22cyclictest22736-21ssh11:30:131
874699332,10cyclictest22609-21tr10:00:201
874699331,9cyclictest5825-21cut09:15:211
874699331,9cyclictest30653-21ssh12:10:131
874699331,9cyclictest30653-21ssh12:10:131
874699331,8cyclictest7212-21wc11:55:271
874699331,8cyclictest31570-21irqcore09:30:171
874699331,8cyclictest25406-21kthreadcore07:25:201
874699331,8cyclictest17548-21kthreadcore12:20:191
874699331,8cyclictest12926-21kthreadcore08:20:181
874699331,7cyclictest7789-21/usr/sbin/munin09:35:121
874699331,7cyclictest27406-21wc10:55:241
874699331,6cyclictest16151-21proc_pri08:55:271
874699330,9cyclictest2533-21kernelversion08:10:191
874699330,10cyclictest6877-21kthreadcore07:40:201
873899332,25cyclictest28184-21sd-accept07:30:120
875599326,16cyclictest27346irq/40-dwc2_hso11:35:132
875599325,16cyclictest27346irq/40-dwc2_hso08:20:122
875599321,24cyclictest1826-21ssh10:25:132
875599321,20cyclictest27346irq/40-dwc2_hso12:25:122
875599321,20cyclictest27346irq/40-dwc2_hso12:25:122
874699321,9cyclictest6359-21ntp_192.168.11409:50:251
874699321,9cyclictest31632-21ntp_states07:30:251
874699321,8cyclictest23692-21kthreadcore08:30:201
874699321,8cyclictest14759-21dpkg07:50:021
874699321,7cyclictest5662-21kthreadcore08:45:201
874699320,9cyclictest15284-21idleruntime07:50:151
874699320,8cyclictest9863-21fschecks_time11:05:131
874699320,8cyclictest19248-21kthreadcore09:40:211
874699320,10cyclictest19659-21cmp10:35:021
873899321,27cyclictest15956-21ssh10:15:130
874699312,21cyclictest25147-21ssh10:20:141
874699310,9cyclictest14341-21ssh10:13:351
873899313,21cyclictest2764-21ssh09:50:120
876699303,15cyclictest35966irq/58-eth011:45:123
875599304,17cyclictest27346irq/40-dwc2_hso07:55:122
875599302,16cyclictest5638-21sh10:45:142
873899302,22cyclictest1320-21ssh11:53:110
873899302,20cyclictest6901-21ssh10:10:130
875599293,18cyclictest27346irq/40-dwc2_hso08:05:132
875599292,21cyclictest0-21swapper/208:00:142
8755992910,10cyclictest30444-21basename08:40:012
874699291,9cyclictest8801-21latency_hist07:10:021
873899292,22cyclictest30183-21ssh10:05:130
876699283,15cyclictest35966irq/58-eth010:00:133
8766992813,10cyclictest35966irq/58-eth012:25:133
8766992813,10cyclictest35966irq/58-eth012:25:133
876699281,19cyclictest35966irq/58-eth009:05:133
875599286,10cyclictest27346irq/40-dwc2_hso11:30:132
875599285,14cyclictest27346irq/40-dwc2_hso08:45:122
875599283,19cyclictest0-21swapper/212:38:122
875599282,20cyclictest0-21swapper/211:15:132
875599282,19cyclictest0-21swapper/207:35:122
875599282,11cyclictest2732-21apt-get09:50:112
8755992812,12cyclictest26083-21ssh09:45:142
873899288,16cyclictest151rcuc/011:20:130
876699271,16cyclictest0-21swapper/311:25:133
875599272,19cyclictest0-21swapper/210:05:132
875599271,20cyclictest23874-21ssh10:55:152
875599271,13cyclictest17466-21sshd09:00:122
873899277,18cyclictest12-21ksoftirqd/007:45:110
873899272,19cyclictest18050-21ssh12:38:120
8738992715,7cyclictest0-21swapper/011:45:120
876699264,16cyclictest0-21swapper/311:00:143
876699262,18cyclictest0-21swapper/311:53:103
875599264,15cyclictest0-21swapper/211:45:222
875599262,9cyclictest17925-21ssh12:01:162
875599262,18cyclictest0-21swapper/209:05:142
875599261,15cyclictest16974-21sh09:40:142
873899263,13cyclictest5611-21ssh10:45:110
873899262,18cyclictest18931-21ssh11:26:090
8738992614,8cyclictest0-21swapper/009:40:130
876699252,17cyclictest0-21swapper/312:20:123
876699251,17cyclictest0-21swapper/310:10:143
875599252,16cyclictest0-21swapper/207:50:052
875599251,16cyclictest27346irq/40-dwc2_hso10:20:132
875599250,17cyclictest21752-21ls09:25:022
873899252,18cyclictest25374-21ssh12:25:130
873899252,18cyclictest25374-21ssh12:25:130
873899252,18cyclictest11493-21apt-get08:20:110
876699244,16cyclictest35966irq/58-eth011:35:123
876699242,15cyclictest35966irq/58-eth012:35:123
8766992411,7cyclictest0-21swapper/308:10:123
876699241,16cyclictest35966irq/58-eth010:15:123
875599248,6cyclictest10761-21tr11:40:212
875599245,13cyclictest27346irq/40-dwc2_hso10:10:132
875599243,8cyclictest0-21swapper/208:50:132
875599241,8cyclictest22528-21grep09:25:192
873899245,11cyclictest7791-21ssh09:35:120
873899242,18cyclictest7619-21ssh09:15:260
873899242,16cyclictest31200-21ssh09:30:140
873899241,16cyclictest22150-21sh09:25:140
876699234,11cyclictest0-21swapper/307:10:143
876699233,14cyclictest35966irq/58-eth009:55:133
8766992316,5cyclictest0-21swapper/310:30:233
875599232,6cyclictest0-21swapper/212:22:112
875599232,16cyclictest0-21swapper/212:05:112
8755992310,5cyclictest0-21swapper/211:20:262
873899232,15cyclictest27229-21sh09:10:130
8738992314,7cyclictest15569-21ssh09:55:250
873899231,17cyclictest14747-21ssh10:50:130
873899231,17cyclictest14747-21ssh10:50:120
876699223,12cyclictest0-21swapper/310:25:133
8766992211,8cyclictest0-21swapper/311:40:183
8766992210,9cyclictest35966irq/58-eth009:35:133
875599227,8cyclictest19937-21kworker/u8:310:35:222
875599221,8cyclictest1838166ssh10:50:242
875599221,8cyclictest1838166ssh10:50:242
8755992215,5cyclictest24087-21rm11:11:062
875599221,18cyclictest10755-21kthreadcore07:10:222
8755992211,6cyclictest0-21swapper/208:40:402
875599221,13cyclictest7418-21grep09:15:232
875599221,13cyclictest6679-21apt-get08:15:132
875599221,11cyclictest7874-21cpuspeed09:35:142
873899222,16cyclictest10918-21sh10:30:130
8738992213,7cyclictest167-21systemd-network10:00:430
8738992212,6cyclictest24961-21cron12:25:010
873899221,15cyclictest23875-21sh10:55:150
876699218,10cyclictest35966irq/58-eth010:20:143
876699212,13cyclictest35966irq/58-eth007:55:133
876699212,10cyclictest0-21swapper/309:30:133
8766992113,4cyclictest0-21swapper/309:00:123
8766992111,7cyclictest0-21swapper/309:10:133
876699211,14cyclictest0-21swapper/308:30:113
875599219,5cyclictest28319-21kthreadcore09:10:182
875599218,10cyclictest10109-21cat07:45:022
875599212,10cyclictest18392-21latency_hist07:20:022
875599211,11cyclictest2269-21missed_timers09:30:242
875599211,11cyclictest10382-21hwlatdetect07:45:142
873899212,15cyclictest0-21swapper/007:20:130
8738992114,4cyclictest0-21swapper/007:20:020
8738992110,8cyclictest31233-21ssh09:45:520
876699209,8cyclictest35966irq/58-eth011:15:123
876699206,10cyclictest35966irq/58-eth010:35:123
8766992011,7cyclictest0-21swapper/309:29:143
876699201,14cyclictest0-21swapper/309:40:163
876699201,12cyclictest0-21swapper/307:15:133
875599201,9cyclictest6448-21latency_hist08:15:012
875599201,16cyclictest28812-21irqrtprio07:30:182
875599201,10cyclictest21147-21cpuspeed10:00:142
875599201,10cyclictest18899-21irqcore07:20:162
875599201,10cyclictest17530-21kthreadcore08:25:192
875599200,15cyclictest12621-21find08:55:122
873899208,8cyclictest1434-21kworker/u8:112:05:110
873899205,10cyclictest8730-21cyclictest08:00:140
8738992011,6cyclictest17173-21apt11:45:010
873899200,17cyclictest16484-21ls08:25:170
873899200,15cyclictest25826-21find08:35:110
876699193,12cyclictest0-21swapper/309:20:143
876699193,11cyclictest0-21swapper/312:01:143
876699193,11cyclictest0-21swapper/311:57:123
8766991914,3cyclictest35966irq/58-eth011:20:113
876699191,12cyclictest0-21swapper/307:45:273
876699191,11cyclictest0-21swapper/307:40:153
876699191,11cyclictest0-21swapper/307:20:223
873899194,12cyclictest10323-21ssh12:15:230
873899192,13cyclictest0-21swapper/012:01:150
8738991913,4cyclictest4738-21sshd11:36:120
8738991911,6cyclictest28847-21ssh10:20:260
873899191,15cyclictest18807-21kthreadcore09:00:180
873899191,13cyclictest756-21ssh11:00:160
876699189,6cyclictest35966irq/58-eth009:15:113
876699188,6cyclictest0-21swapper/311:33:503
876699187,7cyclictest35966irq/58-eth010:05:133
876699183,10cyclictest0-21swapper/310:40:123
876699182,10cyclictest0-21swapper/308:45:133
876699182,10cyclictest0-21swapper/308:40:133
876699182,10cyclictest0-21swapper/308:35:133
8766991812,4cyclictest0-21swapper/310:55:313
876699181,13cyclictest35966irq/58-eth010:45:123
8766991810,6cyclictest0-21swapper/312:05:153
873899189,7cyclictest20476-21sed10:35:180
873899187,8cyclictest8730-21cyclictest08:15:140
873899183,12cyclictest6788-21cat08:45:270
8738991811,5cyclictest13343-21ssh09:20:170
8738991811,4cyclictest0-21swapper/008:30:220
873899181,11cyclictest4009-21latency_hist11:20:020
873899180,15cyclictest23396-21find07:25:120
876699173,10cyclictest0-21swapper/310:50:123
876699173,10cyclictest0-21swapper/310:50:123
876699172,10cyclictest0-21swapper/308:20:123
876699172,10cyclictest0-21swapper/308:00:143
876699172,10cyclictest0-21swapper/307:35:123
873899178,7cyclictest504-21apt-get07:35:100
873899176,8cyclictest614-21dbus-daemon08:10:330
873899175,8cyclictest12-21ksoftirqd/010:40:130
873899174,10cyclictest8868-21latency_hist07:10:020
8738991712,3cyclictest20172-21irqcore07:55:170
8738991711,4cyclictest0-21swapper/007:50:050
873899171,13cyclictest3303-21ssh11:55:130
873899171,13cyclictest25558-21kthreadcore11:30:220
873899171,13cyclictest11708-21grep08:50:280
876699162,10cyclictest0-21swapper/308:05:123
876699161,12cyclictest0-21swapper/307:25:263
876699161,10cyclictest0-21swapper/308:55:143
876699160,10cyclictest0-21swapper/308:25:153
873899165,8cyclictest899-21unixbench_multi08:05:270
873899163,10cyclictest13163-21cut08:55:180
873899161,12cyclictest22360-21uniq09:05:160
873899151,12cyclictest5557-21fschecks_time07:40:170
873899151,11cyclictest9050-21grep07:10:170
876699141,7cyclictest0-21swapper/308:15:183
875599147,5cyclictest8756-21latency_hist07:10:022
876699122,6cyclictest0-21swapper/307:30:143
876699121,9cyclictest0-21swapper/307:50:143
876699121,8cyclictest0-21swapper/308:50:123
876699111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional