You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 09:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Sep 04, 2026 00:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23041997952,17cyclictest35966irq/58-eth023:10:163
2303399633,24cyclictest1836466ssh21:15:122
23013995826,24cyclictest12-21ksoftirqd/000:20:130
23041995742,8cyclictest35966irq/58-eth021:15:123
2302099553,43cyclictest13653-21ssh21:30:171
2303399540,44cyclictest0-21swapper/219:40:132
23013995311,7cyclictest131rcu_preempt19:25:130
23013995311,7cyclictest131rcu_preempt19:25:130
2301399525,7cyclictest131rcu_preempt00:25:120
23020994926,12cyclictest131rcu_preempt23:25:121
2304199482,32cyclictest0-21swapper/323:30:173
2302099484,34cyclictest3357-21ssh22:35:121
2302099472,35cyclictest29169-21ssh00:00:131
23033994620,19cyclictest0-21swapper/222:45:132
2303399460,38cyclictest12713-21apt-get20:40:122
2303399460,34cyclictest0-21swapper/222:50:182
2301399461,37cyclictest6839-21ssh23:30:180
2303399442,25cyclictest16754-21ssh23:00:142
2303399441,35cyclictest6635-21apt-get20:00:132
2302099442,27cyclictest30918-21ssh22:50:171
2303399434,23cyclictest1770-21ssh23:45:122
2303399431,28cyclictest25878-21apt-get20:20:122
2303399430,39cyclictest30001-21diskmemload23:25:122
2303399411,33cyclictest21063-21apt-get20:15:132
2301399413,30cyclictest16768-21ssh23:00:130
2304199403,21cyclictest35966irq/58-eth023:35:153
2303399401,33cyclictest5803-21ssh00:05:122
2303399400,33cyclictest0-21swapper/221:30:192
2303399400,25cyclictest17621-21ssh00:29:072
2302099403,27cyclictest24049-21ssh00:15:131
2302099403,27cyclictest24049-21ssh00:15:131
2302099401,31cyclictest22342-21ssh22:10:171
2301399401,9cyclictest9243-21apt-get21:10:140
2301399401,8cyclictest0-21swapper/021:35:120
2304199393,24cyclictest35966irq/58-eth021:10:153
2303399391,32cyclictest31991-21apt-get21:40:162
2303399390,35cyclictest27606-21apt-get21:20:162
2303399390,35cyclictest0-21swapper/220:35:132
2303399390,34cyclictest0-21swapper/222:10:152
2303399390,34cyclictest0-21swapper/200:30:152
2303399380,34cyclictest0-21swapper/200:15:132
2303399380,34cyclictest0-21swapper/200:15:132
2302099380,29cyclictest11715-21ssh23:15:121
2303399371,32cyclictest10599-21wc23:30:322
2303399371,27cyclictest8950-21ssh22:00:372
2303399370,31cyclictest0-21swapper/221:50:152
2301399372,27cyclictest2681-21ssh23:10:160
2301399371,4cyclictest27240-21ssh21:55:130
2304199369,23cyclictest35966irq/58-eth023:40:133
2304199364,20cyclictest35966irq/58-eth019:50:143
23041993625,6cyclictest36066irq/59-eth023:00:133
2303399361,31cyclictest0-21swapper/221:25:152
2303399360,33cyclictest16001-21sshd20:06:342
2303399360,33cyclictest16001-21sshd20:06:342
2301399361,9cyclictest8588-21grep21:25:310
2301399361,8cyclictest5431-21ssh00:05:010
2304199353,15cyclictest35966irq/58-eth000:05:133
2304199352,20cyclictest35966irq/58-eth000:20:133
2303399353,28cyclictest7493-21apt-get22:55:112
2303399352,29cyclictest10940-21apt-get23:50:152
2303399351,28cyclictest23159-21apt-get19:10:132
2303399351,28cyclictest23159-21apt-get19:10:132
2303399351,26cyclictest30550-21ntp_states20:55:272
2303399350,32cyclictest8820-21sshd21:05:342
2303399350,31cyclictest27346irq/40-dwc2_hso22:40:152
2303399350,31cyclictest22326-21apt-get20:50:112
2303399350,31cyclictest0-21swapper/223:55:132
2303399350,30cyclictest0-21swapper/221:10:132
2301399352,9cyclictest31880-21kthreadcore20:25:200
2301399352,9cyclictest26002-21fschecks_count20:20:170
2301399352,8cyclictest30266-21kthreadcore19:50:200
2301399351,9cyclictest3061-21memory21:40:260
2301399351,9cyclictest26166-21grep23:05:150
2301399351,9cyclictest19006-21kthreadcore20:10:210
2301399351,8cyclictest27922-21netstat19:45:250
2301399351,10cyclictest29660-21tr22:30:250
2301399351,10cyclictest15007-21fschecks_count19:35:140
2303399341,31cyclictest17467-21kthreadcore23:35:212
2303399340,31cyclictest0-21swapper/222:35:132
2303399340,30cyclictest0-21swapper/200:20:122
2303399340,27cyclictest14558-21sshd21:46:332
2302099342,24cyclictest26703-21ssh22:30:161
2301399344,8cyclictest10324-21kthreadcore22:20:220
2301399342,4cyclictest0-21swapper/020:00:120
2301399341,9cyclictest21193-21threads20:45:280
2301399341,9cyclictest12571-21/usr/sbin/munin22:40:280
2301399341,9cyclictest1019-21sed19:20:190
2301399341,8cyclictest3328-21/usr/sbin/munin22:35:160
2301399341,8cyclictest21228-21sed23:20:180
2301399341,7cyclictest3963-21kthreadcore23:45:220
2301399341,10cyclictest8969-21fschecks_time21:45:150
2301399341,10cyclictest2958-21kthreadcore19:55:220
2301399341,10cyclictest14996-21find20:05:290
2301399341,10cyclictest14996-21find20:05:280
2301399340,10cyclictest16109-21tr22:05:240
2304199333,20cyclictest35966irq/58-eth000:25:123
2303399331,29cyclictest27229-21apt-get21:55:132
2303399331,29cyclictest2677-21ntp_192.168.11422:15:242
2303399331,29cyclictest0-21swapper/221:35:122
2303399331,23cyclictest19224-21rm23:18:382
2303399330,31cyclictest930-21snmpd23:12:112
2303399330,29cyclictest18294-21ntpq19:35:222
2303399330,28cyclictest0-21swapper/219:25:122
2303399330,28cyclictest0-21swapper/219:25:122
2302099334,19cyclictest844-21munin-node19:45:151
2302099333,25cyclictest21-21ksoftirqd/119:40:141
2302099333,18cyclictest21-21ksoftirqd/122:40:151
2301399337,5cyclictest26625-21cat22:10:340
2301399337,3cyclictest28321-21cut00:35:200
2301399331,9cyclictest20110-21tr00:30:230
2301399331,9cyclictest15749-21kthreadcore20:40:210
2301399331,8cyclictest27977-21/usr/sbin/munin19:15:280
2301399331,8cyclictest27977-21/usr/sbin/munin19:15:270
2301399331,8cyclictest27130-21kthreadcore23:40:220
2301399331,8cyclictest22308-21/usr/sbin/munin20:50:160
2301399331,8cyclictest15908-21/usr/sbin/munin23:35:280
2301399331,7cyclictest6487-21ntp_192.168.11420:30:220
2301399331,7cyclictest6487-21ntp_192.168.11420:30:220
2301399331,7cyclictest22094-21grep21:50:330
2301399331,7cyclictest18779-21irqcore21:15:180
2301399331,10cyclictest4118-21ssh22:00:150
2301399330,9cyclictest8809-21kthreadcore00:05:220
2301399330,8cyclictest10922-21/usr/sbin/munin23:50:170
2304199324,23cyclictest35966irq/58-eth023:50:163
2303399321,28cyclictest2065-21ssh23:09:492
2303399320,29cyclictest1453-21ssh22:33:162
2303399320,28cyclictest8847-21diskmemload22:09:112
2303399320,28cyclictest0-21swapper/219:45:132
2303399320,26cyclictest27995-21apt-get19:15:122
2303399320,26cyclictest27995-21apt-get19:15:122
23020993226,3cyclictest21-21ksoftirqd/123:40:131
2301399326,3cyclictest13133-21ssh22:56:210
2301399326,3cyclictest0-21swapper/021:32:130
2301399325,3cyclictest0-21swapper/019:30:180
2301399323,8cyclictest23565-21cron00:15:020
2301399321,9cyclictest21670-21ssh22:45:120
2301399321,9cyclictest21260-21wc22:25:280
2301399321,9cyclictest21260-21wc22:25:280
2301399321,8cyclictest26591-21ntp_192.168.11419:10:230
2301399321,8cyclictest26591-21ntp_192.168.11419:10:230
2301399321,8cyclictest25655-21awk20:20:010
2301399321,7cyclictest31179-21rm21:20:260
2301399321,7cyclictest24192-21fschecks_count00:15:170
2301399321,7cyclictest24192-21fschecks_count00:15:160
2301399320,9cyclictest605-21cut22:50:270
23041993121,6cyclictest35966irq/58-eth023:20:133
2303399314,24cyclictest29888-21cut19:50:192
2303399311,26cyclictest22078-21sshd20:47:442
2303399311,26cyclictest17498-21acpi22:25:132
2303399311,26cyclictest17498-21acpi22:25:122
2303399311,26cyclictest0-21swapper/220:30:122
2303399311,26cyclictest0-21swapper/220:30:112
2303399311,22cyclictest30822-21cat00:00:212
2303399311,19cyclictest0-21swapper/223:43:232
2303399310,28cyclictest24571-21ssh23:20:282
2303399310,28cyclictest0-21swapper/222:20:042
2303399310,27cyclictest5049-21sshd19:22:282
2303399310,26cyclictest0-21swapper/220:10:112
2303399310,23cyclictest2930-21ntp_kernel_pll_21:00:262
2303399310,18cyclictest0-21swapper/200:35:472
2302099313,19cyclictest21-21ksoftirqd/120:55:121
2302099312,23cyclictest15958-21ssh23:35:151
2301399316,3cyclictest0-21swapper/022:16:450
2301399311,9cyclictest20089-21irqcore19:40:180
2301399311,8cyclictest20173-21df_inode23:55:150
2301399311,6cyclictest8410-21irqrtprio20:35:190
2301399311,6cyclictest27545-21irqcore20:55:180
2301399311,5cyclictest30170-21cut23:25:150
23041993016,10cyclictest35966irq/58-eth000:00:143
2304199301,25cyclictest35966irq/58-eth021:30:183
2303399309,18cyclictest21658-21sh00:12:492
2303399300,28cyclictest2835-21sshd20:29:342
2301399301,9cyclictest12414-21cut23:15:190
2304199294,11cyclictest35966irq/58-eth021:40:163
2303399291,24cyclictest0-21swapper/219:55:132
2301399291,8cyclictest7857-21awk21:05:270
2301399291,8cyclictest31884-21/usr/sbin/munin21:00:260
2304199282,18cyclictest35966irq/58-eth000:10:133
23041992815,8cyclictest35966irq/58-eth023:05:123
23041992812,13cyclictest35966irq/58-eth022:40:163
23033992812,13cyclictest10078-21/usr/sbin/munin19:30:252
2302099289,4cyclictest131rcu_preempt21:55:131
2304199278,15cyclictest35966irq/58-eth023:25:153
23041992616,5cyclictest0-21swapper/322:55:123
23041992611,10cyclictest35966irq/58-eth021:25:143
2302099265,11cyclictest14796-21ssh00:10:111
2304199256,14cyclictest35966irq/58-eth023:15:143
2304199251,19cyclictest35966irq/58-eth022:30:173
2302099252,16cyclictest20063-21ssh23:55:141
2302099252,16cyclictest17539-21ssh22:25:161
2302099252,16cyclictest17539-21ssh22:25:151
2304199243,12cyclictest35966irq/58-eth022:45:133
2304199243,12cyclictest0-21swapper/322:00:133
2304199242,14cyclictest35966irq/58-eth021:20:143
2304199237,9cyclictest35966irq/58-eth021:45:133
2302099232,18cyclictest7525-21ssh22:55:141
2304199222,13cyclictest35966irq/58-eth019:20:143
2304199221,16cyclictest0-21swapper/322:15:133
23020992211,7cyclictest21-21ksoftirqd/123:00:141
2302099221,16cyclictest27887-21ssh00:35:151
23020992211,3cyclictest21-21ksoftirqd/121:45:131
2304199219,9cyclictest0-21swapper/322:50:193
2304199219,9cyclictest0-21swapper/300:18:493
2304199219,9cyclictest0-21swapper/300:18:493
23041992112,6cyclictest35966irq/58-eth022:25:143
23041992112,6cyclictest35966irq/58-eth022:25:143
2303399210,17cyclictest23038-21latency_hist19:10:022
2302099213,11cyclictest0-21swapper/119:20:131
23020992114,4cyclictest21-21ksoftirqd/122:20:021
2302099211,15cyclictest13378-21grep22:05:191
2302099211,15cyclictest10969-21ssh23:50:181
2302099211,12cyclictest2843-21cron20:30:021
2304199203,10cyclictest35966irq/58-eth000:30:153
2304199201,12cyclictest0-21swapper/321:55:173
2302099209,8cyclictest8326-21apt-get22:20:131
23020992014,4cyclictest0-21swapper/100:25:221
23020992011,7cyclictest3395-21irqstats23:10:221
2302099201,17cyclictest21563-21kthreadcore23:20:191
2302099200,18cyclictest4525-21ls21:05:161
2304199199,7cyclictest35966irq/58-eth023:55:143
2304199191,12cyclictest0-21swapper/320:30:143
2304199191,12cyclictest0-21swapper/320:30:143
2302099198,3cyclictest0-21swapper/119:50:191
2302099193,12cyclictest0-21swapper/119:10:111
2302099193,12cyclictest0-21swapper/119:10:111
2302099192,14cyclictest18016-21ssh21:50:161
2302099191,8cyclictest24235-21ssh21:35:201
2302099191,13cyclictest21682-21sh22:45:121
2302099190,7cyclictest17301-21kthreadcore20:10:181
2302099190,17cyclictest10524-21latency_hist23:50:021
2302099190,16cyclictest15527-21latency_hist23:35:011
2302099190,13cyclictest22248-21switchtime21:15:301
2304199189,6cyclictest35966irq/58-eth022:35:143
2304199189,6cyclictest0-21swapper/322:20:043
2304199182,10cyclictest0-21swapper/320:40:133
2304199182,10cyclictest0-21swapper/320:00:043
2304199182,10cyclictest0-21swapper/319:35:123
2304199182,10cyclictest0-21swapper/319:30:143
2304199181,12cyclictest0-21swapper/319:10:013
2302099186,3cyclictest131rcu_preempt00:05:241
2302099185,3cyclictest131rcu_preempt21:05:011
2302099185,10cyclictest8288-21apt21:45:001
2302099182,12cyclictest4378-21ntp_states00:20:271
2302099182,12cyclictest13539-21ntp_states19:30:271
2302099181,9cyclictest0-21swapper/121:10:521
2302099181,8cyclictest28432-21tr23:05:211
2302099181,6cyclictest19563-21irqstats00:30:221
2302099181,3cyclictest131rcu_preempt21:20:151
2304199179,6cyclictest0-21swapper/323:49:093
2304199179,6cyclictest0-21swapper/321:36:253
2304199179,6cyclictest0-21swapper/300:37:363
2304199179,3cyclictest0-21swapper/322:13:063
2304199178,6cyclictest0-21swapper/321:52:513
2304199172,8cyclictest35966irq/58-eth022:05:163
2304199172,10cyclictest0-21swapper/320:45:143
2304199172,10cyclictest0-21swapper/319:55:163
2304199172,10cyclictest0-21swapper/319:45:153
2304199171,10cyclictest0-21swapper/320:50:283
2302099173,11cyclictest6742-21cut19:25:211
2302099173,11cyclictest6742-21cut19:25:201
2302099173,11cyclictest30261-21usage_EthcTask20:20:321
2302099173,11cyclictest22493-21fschecks_time20:50:151
2302099173,10cyclictest0-21swapper/120:15:141
2302099172,11cyclictest32581-21apt-get19:20:011
2302099172,11cyclictest32581-21apt-get19:20:001
2302099171,8cyclictest3200-21fschecks_count20:30:151
2302099171,8cyclictest3200-21fschecks_count20:30:141
2302099171,8cyclictest14324-21kthreadcore20:05:231
2302099171,8cyclictest14324-21kthreadcore20:05:231
2302099171,13cyclictest7481-21ssh22:00:221
2304199162,9cyclictest0-21swapper/321:05:133
2304199161,10cyclictest0-21swapper/320:10:193
2302099167,7cyclictest17521-21apt-get20:45:121
2302099163,10cyclictest18124-21kthreadcore19:35:201
2302099162,10cyclictest2053-21ps19:55:191
2302099162,10cyclictest0-21swapper/120:35:141
2302099161,7cyclictest12822-21fschecks_count20:40:161
2302099161,12cyclictest5002-21sed21:25:211
2302099161,11cyclictest6822-21fschecks_time20:00:151
2304199152,9cyclictest0-21swapper/321:00:163
2304199151,9cyclictest0-21swapper/319:40:133
2304199151,12cyclictest0-21swapper/320:25:233
2304199151,12cyclictest0-21swapper/319:15:163
2304199151,12cyclictest0-21swapper/319:15:153
2302099154,9cyclictest23044-21latency_hist19:10:011
2304199149,3cyclictest0-21swapper/320:35:153
2304199148,4cyclictest35966irq/58-eth019:10:263
2304199148,4cyclictest35966irq/58-eth019:10:253
2304199141,9cyclictest35966irq/58-eth019:25:123
2304199141,9cyclictest35966irq/58-eth019:25:123
2304199141,8cyclictest0-21swapper/320:15:233
2304199141,7cyclictest0-21swapper/320:55:143
2304199141,11cyclictest0-21swapper/320:25:013
2301399142,9cyclictest23030-21ls19:10:010
2304199133,7cyclictest0-21swapper/320:05:163
2304199133,7cyclictest0-21swapper/320:05:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional