You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-13 - 06:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Jul 13, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
616399574,41cyclictest7778-21ssh00:00:142
615599551,43cyclictest26885-21ssh22:05:121
616399544,9cyclictest0-21swapper/220:15:132
616399534,46cyclictest14793-21ssh23:10:162
615599532,39cyclictest1071-21ssh21:15:131
617199478,26cyclictest35966irq/58-eth021:45:123
616399425,23cyclictest1064-21diskmemload21:15:122
6163994212,25cyclictest27346irq/40-dwc2_hso21:10:142
6163994212,21cyclictest30194-21readlink19:35:012
616399411,8cyclictest4305-21fschecks_time23:40:172
616399407,5cyclictest23090-21kthreadcore22:20:212
616399401,9cyclictest6435-21grep19:10:162
614399401,30cyclictest28220-21ssh21:30:120
6163993916,16cyclictest27-21ksoftirqd/220:00:132
616399391,11cyclictest14172-21kthreadcore19:50:202
615599393,28cyclictest11384-21ssh00:20:151
615599393,28cyclictest11384-21ssh00:20:151
615599391,31cyclictest7430-21ssh22:30:141
615599391,26cyclictest20267-21ssh22:55:141
614399391,28cyclictest22990-21ssh21:45:120
617199386,21cyclictest35966irq/58-eth021:35:133
616399381,9cyclictest29460-21apt-get00:30:152
616399381,13cyclictest20464-21wc22:35:292
616399372,19cyclictest15815-21ssh21:58:102
616399371,9cyclictest7732-21kthreadcore21:35:222
616399371,9cyclictest3192-21latency_hist22:10:012
616399371,9cyclictest23035-21ps21:05:282
616399371,9cyclictest15986-21kthreadcore21:40:212
616399371,13cyclictest25697-21fschecks_time22:40:162
616399362,18cyclictest5883-21ssh23:21:102
616399362,18cyclictest20346-21ssh00:25:122
616399362,10cyclictest20106-21kthreadcore20:30:212
616399362,10cyclictest0-21swapper/223:45:202
616399361,8cyclictest24103-21diskmemload22:47:102
616399361,12cyclictest31164-21ssh23:55:142
616399360,9cyclictest14277-21kthreadcore20:25:182
616399360,8cyclictest10480-21kthreadcore19:45:212
615599363,29cyclictest5824-21ssh23:05:131
617199353,21cyclictest35966irq/58-eth021:10:133
616399356,5cyclictest21349-21sed22:00:222
616399351,8cyclictest2862-21kthreadcore21:50:232
616399351,11cyclictest13628-21latency21:20:232
616399351,10cyclictest9062-21if_eth120:20:182
616399351,10cyclictest22361-21kthreadcore21:25:242
616399350,7cyclictest4916-21latency_hist20:50:012
614399352,23cyclictest151rcuc/022:35:140
617199346,19cyclictest35966irq/58-eth023:50:133
616399345,6cyclictest28079-21kthreadcore00:10:232
616399342,12cyclictest19341-21latency_hist21:05:012
616399341,9cyclictest9589-21netstat00:35:252
616399341,9cyclictest29981-21kthreadcore21:30:202
616399341,7cyclictest18526-21/usr/sbin/munin23:30:132
616399341,15cyclictest27728-21ssh23:35:172
616399341,13cyclictest31227-21irqcore22:25:162
616399341,13cyclictest23635-21kthreadcore22:55:222
616399340,10cyclictest25879-21kthreadcore20:35:222
617199332,21cyclictest35966irq/58-eth019:35:123
617199331,14cyclictest35966irq/58-eth022:45:133
616399332,19cyclictest27346irq/40-dwc2_hso23:15:142
616399331,9cyclictest30734-21kthreadcore20:40:232
616399331,9cyclictest21762-21awk20:00:012
616399331,9cyclictest12700-21cut19:15:202
616399331,9cyclictest0-21swapper/219:40:182
616399331,8cyclictest6188-21ssh00:15:292
616399330,9cyclictest23519-21irqcore21:45:192
6155993326,4cyclictest0-21swapper/120:25:121
6171993220,6cyclictest0-21swapper/320:10:143
616399326,4cyclictest10238-21irqrtprio23:25:182
616399322,4cyclictest22298-21diskstats23:50:162
616399321,8cyclictest9776-21latency_hist20:55:012
616399321,8cyclictest19623-21timerandwakeup19:20:292
616399321,10cyclictest16257-21users00:20:352
616399321,10cyclictest16257-21users00:20:342
616399317,21cyclictest0-21swapper/222:11:522
616399311,3cyclictest0-21swapper/222:50:292
616399311,10cyclictest23296-21ssh00:07:372
616399310,12cyclictest20083-21ssh22:18:362
614399313,15cyclictest22170-21ssh23:50:120
614399312,22cyclictest6018-21ssh00:35:140
614399311,3cyclictest131rcu_preempt22:25:110
616399309,3cyclictest456-21ssh23:00:272
616399301,9cyclictest11731-21tr20:55:222
615599304,18cyclictest22981-21ssh21:45:121
616399292,6cyclictest0-21swapper/222:34:102
616399291,9cyclictest8762-21kthreadcore23:05:222
616399291,9cyclictest2574-21latency_hist19:40:012
616399291,7cyclictest25418-21latency_hist19:30:012
616399291,7cyclictest25418-21latency_hist19:30:012
614399291,22cyclictest29282-21ssh23:00:130
616399285,5cyclictest1023-21kthreadcore20:10:222
615599283,20cyclictest13158-21ssh23:45:141
615599282,20cyclictest30933-21ssh22:41:101
615599282,19cyclictest15464-21ssh23:26:461
614399284,12cyclictest6135-21cyclictest19:40:140
6143992813,13cyclictest24103-21diskmemload23:30:130
615599274,18cyclictest131rcu_preempt23:52:341
615599272,19cyclictest7995-21ssh22:46:421
615599272,19cyclictest29241-21ssh23:16:101
615599272,19cyclictest16652-21sh21:59:101
614399272,18cyclictest8527-21ssh21:35:260
614399272,17cyclictest8701-21ssh21:55:120
617199269,11cyclictest35966irq/58-eth023:30:143
615599263,18cyclictest19268-21ssh21:25:141
615599262,8cyclictest0-21swapper/120:00:121
615599262,16cyclictest12516-21sh22:15:101
614399263,20cyclictest21679-21sh22:20:140
615599252,15cyclictest27554-21ssh23:35:141
6155992511,8cyclictest21-21ksoftirqd/121:50:131
615599251,16cyclictest30751-21ssh22:25:101
6155992510,5cyclictest0-21swapper/121:24:391
614399252,19cyclictest7428-21sh22:30:150
617199242,14cyclictest35966irq/58-eth021:55:123
6155992412,7cyclictest0-21swapper/122:01:481
6155992412,3cyclictest0-21swapper/123:34:271
6155992411,7cyclictest201rcuc/119:20:131
6143992411,7cyclictest12-21ksoftirqd/021:40:120
614399241,17cyclictest11157-21ssh22:50:140
6171992313,7cyclictest35966irq/58-eth022:05:143
615599232,6cyclictest21-21ksoftirqd/120:45:141
6155992311,10cyclictest0-21swapper/121:32:181
614399236,11cyclictest1546rcuc/000:00:150
614399235,8cyclictest0-21swapper/022:15:100
614399232,17cyclictest20394-21ssh00:25:140
6143992314,7cyclictest12-21ksoftirqd/022:00:130
614399231,17cyclictest638-21ssh00:30:260
6143992311,3cyclictest131rcu_preempt22:40:250
614399231,12cyclictest11291-21cat19:45:270
6143992310,5cyclictest6135-21cyclictest20:45:130
617199223,13cyclictest0-21swapper/323:36:473
6171992213,6cyclictest35966irq/58-eth023:45:133
617199221,15cyclictest0-21swapper/322:35:143
615599229,3cyclictest0-21swapper/122:51:311
6155992213,6cyclictest0-21swapper/123:10:231
6155992211,8cyclictest201rcuc/100:10:131
615599221,15cyclictest6002-21ssh00:35:131
6155992210,3cyclictest21-21ksoftirqd/100:33:081
6155992210,3cyclictest21-21ksoftirqd/100:04:151
614399228,4cyclictest131rcu_preempt22:55:260
614399226,13cyclictest13129-21ssh23:45:120
614399223,11cyclictest0-21swapper/023:40:140
614399222,18cyclictest15463-21sh23:26:460
614399222,13cyclictest23799-21ssh23:15:140
614399222,13cyclictest15555-21kworker/u8:221:10:140
614399221,8cyclictest9067-21munin-run23:25:010
6143992212,6cyclictest15051-21grep23:10:180
614399221,17cyclictest11081-21find19:15:130
614399220,9cyclictest2843-21tr21:50:230
617199219,9cyclictest0-21swapper/300:33:193
617199212,16cyclictest35966irq/58-eth023:00:123
6171992111,7cyclictest35966irq/58-eth021:25:143
617199211,12cyclictest35966irq/58-eth000:00:133
615599219,3cyclictest0-21swapper/123:43:131
615599219,3cyclictest0-21swapper/100:27:441
615599217,4cyclictest21-21ksoftirqd/121:39:431
615599213,14cyclictest2372-21df00:15:151
615599210,6cyclictest26677-21kthreadcore21:10:221
614399214,9cyclictest0-21swapper/020:20:140
614399212,17cyclictest18728-21ssh00:23:320
614399212,17cyclictest18728-21ssh00:23:310
614399212,15cyclictest17425-21idleruntime19:55:180
6143992113,5cyclictest7012-21usage_EthcTask00:15:320
614399211,16cyclictest22141-21uniq20:00:140
614399211,16cyclictest16732-21ssh00:05:130
614399211,15cyclictest26867-21find22:05:110
6143992110,8cyclictest12-21ksoftirqd/021:25:140
617199209,7cyclictest35966irq/58-eth023:15:153
617199202,13cyclictest0-21swapper/322:15:113
617199202,11cyclictest0-21swapper/322:50:103
6171992015,3cyclictest36066irq/59-eth023:27:213
6171992012,6cyclictest35966irq/58-eth022:30:133
6171992011,6cyclictest0-21swapper/300:10:293
617199201,15cyclictest0-21swapper/300:25:163
615599209,3cyclictest21-21ksoftirqd/119:50:141
615599208,10cyclictest0-21swapper/122:22:491
615599207,4cyclictest21-21ksoftirqd/123:00:081
615599201,16cyclictest17512-21irqcore19:55:201
615599201,15cyclictest8855-21df20:20:151
6155992011,5cyclictest12304-21latency_hist22:15:021
6155992010,4cyclictest131rcu_preempt20:50:171
6155992010,3cyclictest21-21ksoftirqd/121:40:131
615599200,14cyclictest16441-21find22:35:121
6143992014,3cyclictest0-21swapper/020:50:140
6143992012,5cyclictest0-21swapper/019:20:270
6143992011,7cyclictest24101-21cron21:10:010
614399201,15cyclictest3531-21ssh22:10:150
614399201,15cyclictest25508-21find19:30:120
614399201,14cyclictest9431-21ntp_192.168.11423:05:250
614399201,14cyclictest2250-21sh22:45:150
614399201,13cyclictest0-21swapper/019:50:150
614399201,10cyclictest12313-21kthreadcore21:20:210
6143992010,6cyclictest29600-21kthreadcore23:35:220
617199195,11cyclictest35966irq/58-eth000:05:143
617199193,11cyclictest0-21swapper/321:53:103
6171991914,3cyclictest35966irq/58-eth023:05:283
6171991911,5cyclictest0-21swapper/321:40:183
6171991910,6cyclictest0-21swapper/321:30:223
6171991910,6cyclictest0-21swapper/300:35:583
616399193,11cyclictest0-21swapper/220:05:152
615599199,2cyclictest131rcu_preempt23:20:091
615599197,3cyclictest21-21ksoftirqd/121:05:201
615599196,4cyclictest21-21ksoftirqd/120:05:181
615599193,13cyclictest0-21swapper/119:10:141
615599192,15cyclictest31894-21kthreadcore23:55:211
6155991913,4cyclictest0-21swapper/100:05:171
6155991912,5cyclictest0-21swapper/119:45:111
615599191,15cyclictest11944-21kthreadcore20:55:231
614399196,10cyclictest1506-21grep19:35:270
614399192,14cyclictest0-21swapper/020:40:140
614399192,12cyclictest29313-21ntp_192.168.11400:10:260
6143991912,4cyclictest31113-21/usr/sbin/munin23:55:190
614399191,15cyclictest6957-21kthreadcore19:10:190
6143991911,5cyclictest1817-21kthreadcore21:15:200
6143991911,5cyclictest0-21swapper/020:05:240
614399191,10cyclictest24393-21threads19:25:280
614399191,10cyclictest24393-21threads19:25:270
617199189,6cyclictest35966irq/58-eth022:55:133
617199189,6cyclictest35966irq/58-eth022:00:133
617199183,12cyclictest35966irq/58-eth022:20:143
617199183,10cyclictest0-21swapper/322:11:113
617199182,11cyclictest0-21swapper/323:10:153
617199182,10cyclictest0-21swapper/320:15:143
6171991813,3cyclictest35966irq/58-eth022:25:193
617199181,13cyclictest35966irq/58-eth021:20:133
617199181,11cyclictest0-21swapper/319:55:153
6171991810,5cyclictest35966irq/58-eth023:20:133
615599181,6cyclictest30119-21latency_hist19:35:011
6155991813,3cyclictest0-21swapper/119:10:011
615599181,15cyclictest30518-21kthreadcore20:40:241
614399185,10cyclictest2705-21grep20:10:280
614399183,11cyclictest14488-21idleruntime-cro21:00:000
614399181,13cyclictest8527-21munin-run20:20:010
617199179,6cyclictest0-21swapper/321:17:303
617199179,6cyclictest0-21swapper/300:23:093
617199179,6cyclictest0-21swapper/300:23:093
617199179,5cyclictest35966irq/58-eth023:55:133
617199171,8cyclictest0-21swapper/320:55:013
6171991711,4cyclictest0-21swapper/300:15:213
617199171,11cyclictest0-21swapper/323:40:143
617199171,11cyclictest0-21swapper/322:40:153
617199171,10cyclictest0-21swapper/320:30:203
615599172,10cyclictest0-21swapper/120:30:151
615599171,14cyclictest30065-21apt-get19:35:041
615599171,12cyclictest23244-21diskstats20:35:151
6143991712,3cyclictest14740-21apt21:00:090
614399171,13cyclictest18345-21apt-get20:30:120
617199162,10cyclictest0-21swapper/320:45:143
617199161,12cyclictest0-21swapper/319:55:023
617199161,11cyclictest0-21swapper/319:15:133
617199161,10cyclictest0-21swapper/321:00:103
616399161,7cyclictest6195-21latency_hist19:10:012
615599169,5cyclictest0-21swapper/120:15:181
615599164,9cyclictest7419-21latency_hist19:45:011
615599162,11cyclictest22762-21tr19:25:201
615599162,11cyclictest22762-21tr19:25:201
615599162,10cyclictest0-21swapper/119:15:141
6155991610,4cyclictest0-21swapper/121:00:131
6155991610,4cyclictest0-21swapper/120:10:191
614399168,6cyclictest13552-21sshd20:25:100
614399167,5cyclictest0-21swapper/020:35:160
617199152,10cyclictest0-21swapper/320:20:143
614399151,10cyclictest6187-21cat19:10:020
617199141,9cyclictest35966irq/58-eth020:55:153
617199141,8cyclictest35966irq/58-eth019:20:133
617199141,10cyclictest0-21swapper/320:25:133
617199131,9cyclictest35966irq/58-eth019:40:143
617199131,4cyclictest0-21swapper/319:25:133
617199131,4cyclictest0-21swapper/319:25:133
617199131,10cyclictest0-21swapper/320:05:333
617199127,3cyclictest0-21swapper/319:49:353
617199121,9cyclictest0-21swapper/319:10:223
617199121,8cyclictest0-21swapper/319:30:163
617199121,7cyclictest0-21swapper/321:10:023
617199121,5cyclictest0-21swapper/320:05:013
617199111,7cyclictest0-21swapper/320:40:193
617199111,6cyclictest0-21swapper/320:35:223
61719991,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional