You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 05:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Aug 08, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1335399675,45cyclictest16253-21ssh23:25:120
13376996215,37cyclictest0-21swapper/222:30:132
1335399551,29cyclictest17773-21ssh00:20:130
1335399522,39cyclictest8602-21ssh21:15:130
1335399514,39cyclictest26714-21ssh21:25:130
1336499501,44cyclictest23508-21ssh22:35:121
1335399503,34cyclictest27206-21ssh22:55:110
1338899495,32cyclictest35966irq/58-eth022:55:123
13388994921,18cyclictest35966irq/58-eth000:25:123
1336499498,6cyclictest131rcu_preempt21:35:141
13376994828,13cyclictest27200-21sh22:55:122
1335399473,32cyclictest28996-21ssh22:20:150
1336499463,37cyclictest3325-21ssh21:30:131
1335399463,33cyclictest23231-21ssh00:05:130
1336499453,5cyclictest131rcu_preempt22:10:121
1336499445,34cyclictest1788-21ssh23:35:141
1338899436,25cyclictest35966irq/58-eth020:35:133
13376994226,12cyclictest0-21swapper/221:50:112
13376994125,13cyclictest32565-21ssh22:40:132
13376994117,12cyclictest27283-21ssh22:18:352
13376993929,7cyclictest31973-21apt-get21:10:132
13376993919,17cyclictest0-21swapper/221:00:132
13376993910,13cyclictest31980-21ssh22:39:502
13376993830,5cyclictest14251-21apt-get00:00:132
13376993823,12cyclictest27346irq/40-dwc2_hso19:20:112
13376993810,12cyclictest31579-21diskmemload21:35:002
13364993813,15cyclictest13337-21cyclictest22:45:121
1335399380,28cyclictest5222-21ssh23:55:110
1335399380,28cyclictest5222-21ssh23:55:110
13376993726,8cyclictest19777-21apt-get23:45:162
13376993715,17cyclictest0-21swapper/219:45:142
13376993714,11cyclictest0-21swapper/200:30:142
1336499373,27cyclictest30376-21ssh21:45:141
1336499372,5cyclictest131rcu_preempt00:15:131
1336499371,33cyclictest3720-21ssh23:00:121
13376993628,5cyclictest10773-21apt-get23:40:152
13376993624,9cyclictest1725-21apt-get22:05:132
1337699362,28cyclictest29261-21ssh21:44:112
13376993612,8cyclictest3371-21ssh21:46:102
13376993612,7cyclictest0-21swapper/222:02:502
13376993529,3cyclictest30597-21apt-get23:15:132
1337699352,15cyclictest5283-21kworker/u8:022:53:122
13376993520,11cyclictest0-21swapper/221:35:142
13376993520,10cyclictest25242-21ssh23:30:122
13376993517,14cyclictest5531-21apt-get22:25:112
13376993510,7cyclictest0-21swapper/222:14:132
1336499354,25cyclictest5244-21ssh23:55:141
1336499354,25cyclictest5244-21ssh23:55:131
1336499353,25cyclictest27230-21rm22:55:131
1336499352,27cyclictest0-21swapper/120:50:121
1336499352,23cyclictest17655-21diskmemload21:20:131
1336499351,29cyclictest1748-21ssh22:05:131
1337699349,12cyclictest0-21swapper/200:21:242
1337699347,11cyclictest0-21swapper/223:53:092
13376993418,12cyclictest0-21swapper/219:25:132
13376993416,14cyclictest27346irq/40-dwc2_hso21:15:132
13376993415,9cyclictest15480-21cut20:50:212
13376993414,16cyclictest7340-21awk23:20:142
13376993413,18cyclictest3731-21xargs00:10:282
13376993411,7cyclictest31579-21diskmemload22:46:182
1336499342,29cyclictest5563-21ssh22:25:121
1338899332,20cyclictest35966irq/58-eth023:30:123
1337699336,10cyclictest0-21swapper/200:35:042
13376993326,3cyclictest12653-21apt-get23:05:122
13376993316,14cyclictest0-21swapper/220:20:122
13376993315,9cyclictest21598-21/usr/sbin/munin23:10:152
13376993314,16cyclictest361-21ntpq20:00:242
13376993314,15cyclictest7231-21cut19:35:212
1336499332,26cyclictest10778-21ssh23:40:121
1336499332,22cyclictest18193-21ssh22:50:131
1336499331,24cyclictest7229-21ssh23:20:111
13388993214,9cyclictest35966irq/58-eth021:25:143
1337699328,11cyclictest0-21swapper/223:56:532
1337699328,11cyclictest0-21swapper/223:56:522
13376993215,14cyclictest26353-21latency_hist21:25:022
13376993215,13cyclictest5956-21kthreadcore23:00:212
13376993215,13cyclictest11519-21entropy20:15:162
1336499322,21cyclictest28664-21ssh23:50:111
1337699319,9cyclictest22650-21sh23:27:202
1337699319,8cyclictest0-21swapper/200:17:232
1337699313,21cyclictest0-21swapper/220:30:132
13376993115,9cyclictest8499-21ssh23:37:452
13376993115,8cyclictest30829-21rm21:25:292
13376993115,12cyclictest27002-21/usr/sbin/munin21:05:122
13376993114,9cyclictest3420-21grep19:30:212
13376993114,14cyclictest3073-21dpkg20:40:132
13376993114,14cyclictest3073-21dpkg20:40:122
13376993114,14cyclictest28703-21ssh00:06:152
1336499315,17cyclictest32556-21ssh22:40:131
1336499311,20cyclictest3442-21ssh00:30:151
13353993118,10cyclictest12633-21apt-get20:50:130
13376993022,5cyclictest27346irq/40-dwc2_hso19:55:122
13376993015,12cyclictest20260-21irqrtprio19:50:192
13376993015,12cyclictest20260-21irqrtprio19:50:182
13376993014,13cyclictest13965-21irqcore19:10:172
13376993013,14cyclictest11598-21sed20:45:282
1336499301,23cyclictest17798-21ssh00:20:131
1335399303,16cyclictest25241-21ssh23:30:120
1335399302,20cyclictest1773-21ssh23:35:120
13353993017,8cyclictest12285-21apt-get21:35:150
1338899291,23cyclictest36066irq/59-eth023:25:113
13376992914,13cyclictest1983-21ssh22:21:122
13376992914,13cyclictest18128-21kthreadcore21:55:192
13376992913,12cyclictest30824-21fschecks_time20:35:162
1335399291,20cyclictest19807-21ssh23:45:160
1338899283,18cyclictest0-21swapper/321:51:123
13376992814,6cyclictest25240-21cat20:25:292
13376992814,11cyclictest31795-21ssh00:25:402
13376992814,11cyclictest10037-21munin-node19:40:092
1336499289,16cyclictest21-21ksoftirqd/123:15:121
1336499288,8cyclictest131rcu_preempt21:40:131
13364992818,8cyclictest630-21Async09:36:311
1335399282,20cyclictest12427-21ssh21:51:120
13388992718,5cyclictest35966irq/58-eth023:35:133
13388992713,11cyclictest35966irq/58-eth023:15:133
1337699279,14cyclictest1730-21/usr/sbin/munin20:05:122
13376992714,9cyclictest11379-21cat20:15:022
13376992714,10cyclictest0-21swapper/219:15:142
13376992712,7cyclictest17394-21/usr/sbin/munin20:55:172
1336499272,19cyclictest12651-21ssh23:05:111
1335399274,5cyclictest12-21ksoftirqd/021:58:120
1338899262,17cyclictest35966irq/58-eth022:07:013
1338899261,18cyclictest0-21swapper/322:20:163
1336499262,16cyclictest14271-21ssh00:00:161
1335399264,6cyclictest12-21ksoftirqd/022:17:110
1335399262,18cyclictest7853-21ssh22:07:010
1335399262,18cyclictest5563-21ssh22:41:120
1338899252,16cyclictest35966irq/58-eth022:53:123
13388992516,4cyclictest35966irq/58-eth022:10:133
1336499252,18cyclictest19682-21apt-get19:50:121
1336499252,18cyclictest19682-21apt-get19:50:121
1336499252,16cyclictest6983-21ssh21:50:141
1336499252,16cyclictest16058-21ssh21:55:121
1335399254,5cyclictest12-21ksoftirqd/022:52:120
1335399254,4cyclictest151rcuc/023:01:330
13364992414,6cyclictest0-21swapper/122:20:031
1335399244,5cyclictest12-21ksoftirqd/022:46:260
1335399244,5cyclictest12-21ksoftirqd/022:37:440
1335399244,4cyclictest12-21ksoftirqd/022:14:080
1335399244,4cyclictest12-21ksoftirqd/000:15:010
1335399243,6cyclictest12-21ksoftirqd/023:20:270
1335399243,4cyclictest151rcuc/023:40:130
1335399243,4cyclictest151rcuc/000:34:470
1335399241,11cyclictest0-21swapper/021:31:010
1338899233,13cyclictest0-21swapper/321:44:113
1336499236,13cyclictest32697-21sed00:10:181
13364992310,3cyclictest131rcu_preempt23:30:291
1335399237,10cyclictest0-21swapper/019:45:130
1335399234,5cyclictest12-21ksoftirqd/023:07:030
1335399234,5cyclictest12-21ksoftirqd/020:35:200
1335399234,5cyclictest12-21ksoftirqd/000:29:250
1335399234,4cyclictest12-21ksoftirqd/023:15:300
1335399234,4cyclictest12-21ksoftirqd/021:13:220
1335399234,4cyclictest12-21ksoftirqd/019:25:180
1335399233,6cyclictest12-21ksoftirqd/022:25:130
1335399233,5cyclictest12-21ksoftirqd/000:35:330
1335399233,4cyclictest12-21ksoftirqd/020:05:250
1335399232,5cyclictest151rcuc/022:33:340
1335399232,16cyclictest0-21swapper/021:00:120
13353992317,2cyclictest17628-21apt-get21:20:120
1335399231,6cyclictest12-21ksoftirqd/021:42:120
1335399231,19cyclictest10110-21ls19:40:110
1335399231,16cyclictest0-21swapper/020:25:150
1335399231,15cyclictest8746-21ssh00:15:130
1338899224,10cyclictest0-21swapper/300:05:123
13364992213,6cyclictest0-21swapper/120:10:241
13364992210,3cyclictest131rcu_preempt00:07:131
1335399229,10cyclictest25170-21apt-get22:00:140
1335399227,5cyclictest0-21swapper/020:40:120
1335399227,5cyclictest0-21swapper/020:40:120
1335399225,5cyclictest12-21ksoftirqd/019:30:190
1335399224,4cyclictest12-21ksoftirqd/023:10:380
1335399224,4cyclictest12-21ksoftirqd/000:00:160
1335399223,4cyclictest12-21ksoftirqd/020:15:260
1335399222,4cyclictest12-21ksoftirqd/023:53:480
1338899219,9cyclictest0-21swapper/300:21:253
1338899215,11cyclictest0-21swapper/323:55:113
1338899215,11cyclictest0-21swapper/323:55:113
1336499218,7cyclictest23711-1kworker/1:2H19:40:101
1336499218,3cyclictest131rcu_preempt23:10:251
1336499211,7cyclictest20542-21ps23:45:221
1336499211,15cyclictest13289-21kthreadcore20:15:211
1335399215,3cyclictest0-21swapper/019:55:030
13353992113,6cyclictest1-21systemd21:46:580
1338899209,8cyclictest0-21swapper/322:27:313
1338899209,7cyclictest0-21swapper/300:30:163
1338899208,7cyclictest35966irq/58-eth021:45:263
1338899205,10cyclictest0-21swapper/321:15:163
1338899203,11cyclictest0-21swapper/322:01:113
13388992011,6cyclictest0-21swapper/321:39:443
1338899201,12cyclictest35966irq/58-eth023:20:133
13388992010,8cyclictest0-21swapper/322:43:273
13388992010,8cyclictest0-21swapper/300:11:213
1336499209,8cyclictest16225-21apt-get23:25:121
1336499209,6cyclictest0-21swapper/122:15:121
1336499202,13cyclictest664-21ntpd22:02:011
1336499201,8cyclictest15261-21irqstats22:30:181
1336499201,15cyclictest19547-21cut19:15:201
1336499201,12cyclictest16387-21ls00:35:301
1336499201,10cyclictest25713-21wc21:00:271
1335399203,4cyclictest12-21ksoftirqd/019:20:170
1338899199,3cyclictest0-21swapper/322:47:543
1338899198,9cyclictest0-21swapper/300:04:063
1338899198,6cyclictest35966irq/58-eth023:00:133
1338899193,11cyclictest0-21swapper/323:43:123
1338899192,11cyclictest0-21swapper/319:55:053
1338899192,10cyclictest0-21swapper/319:50:143
1338899192,10cyclictest0-21swapper/319:50:133
1338899192,10cyclictest0-21swapper/319:20:133
13388991913,3cyclictest0-21swapper/320:20:123
1338899191,13cyclictest0-21swapper/321:00:163
1338899191,11cyclictest0-21swapper/321:05:283
13388991910,7cyclictest0-21swapper/321:21:013
1336499198,8cyclictest29449-21df_inode20:00:131
1336499193,12cyclictest0-21swapper/100:25:131
1336499192,14cyclictest26904-21grep19:20:281
1336499192,12cyclictest13802-21grep19:10:171
1336499191,13cyclictest26809-21df21:25:141
1336499191,12cyclictest20731-21ls20:55:231
1336499191,10cyclictest9013-21irqcore21:15:171
1335399195,7cyclictest12-21ksoftirqd/019:50:160
1335399195,7cyclictest12-21ksoftirqd/019:50:160
1335399195,4cyclictest0-21swapper/019:35:120
1335399194,12cyclictest29959-21irqrtprio20:00:160
1335399191,15cyclictest17003-21kthreadcore20:20:190
1338899189,6cyclictest35966irq/58-eth020:10:123
1338899189,6cyclictest0-21swapper/323:53:053
1338899189,5cyclictest0-21swapper/322:30:323
1338899183,10cyclictest0-21swapper/323:13:123
1338899183,10cyclictest0-21swapper/321:59:113
1338899182,9cyclictest0-21swapper/319:30:143
1338899182,11cyclictest0-21swapper/300:19:133
1338899182,10cyclictest0-21swapper/320:40:133
1338899182,10cyclictest0-21swapper/320:40:133
1338899182,10cyclictest0-21swapper/320:05:153
1338899182,10cyclictest0-21swapper/319:40:053
1338899182,10cyclictest0-21swapper/319:15:143
1338899182,10cyclictest0-21swapper/300:35:153
13388991811,4cyclictest0-21swapper/322:35:123
13388991811,4cyclictest0-21swapper/322:15:133
13388991810,5cyclictest35966irq/58-eth021:30:123
1336499188,7cyclictest19661-21ntpq20:20:251
1336499183,11cyclictest3080-21kthreadcore20:05:201
1336499182,6cyclictest764-21grep19:30:171
1336499181,13cyclictest30644-21apt-get20:35:131
13364991810,6cyclictest9702-21sshd19:36:371
1335399183,7cyclictest12-21ksoftirqd/020:56:470
1335399182,11cyclictest0-21swapper/021:05:130
1335399182,11cyclictest0-21swapper/020:10:130
1335399182,10cyclictest0-21swapper/020:30:130
1338899178,6cyclictest0-21swapper/321:10:143
1338899174,4cyclictest0-21swapper/320:25:133
1338899172,10cyclictest0-21swapper/320:15:153
13388991712,3cyclictest321rcuc/323:06:403
13388991710,5cyclictest35966irq/58-eth023:45:123
1337699178,7cyclictest13337-21cyclictest19:10:022
1336499172,5cyclictest135-21systemd-journal20:50:011
1336499171,9cyclictest4836-21cut20:40:201
1336499171,9cyclictest4836-21cut20:40:191
1336499171,9cyclictest16186-21kthreadcore19:45:201
1336499171,13cyclictest27891-21apt-get19:25:121
1336499171,12cyclictest21182-21fschecks_time20:25:161
1335399175,7cyclictest12-21ksoftirqd/020:45:140
13353991711,4cyclictest12-21ksoftirqd/019:20:010
1338899161,8cyclictest0-21swapper/320:30:283
13388991611,2cyclictest0-21swapper/320:50:113
1336499162,11cyclictest24220-21apt-get19:55:021
1336499161,12cyclictest26195-21sed20:30:181
1335399161,12cyclictest15533-21kthreadcore19:10:190
1336499151,6cyclictest844-21munin-node21:05:121
1335399151,11cyclictest13415-21latency_hist19:10:010
1338899149,3cyclictest0-21swapper/320:01:523
1338899149,3cyclictest0-21swapper/319:45:243
1338899149,3cyclictest0-21swapper/319:37:213
1338899149,3cyclictest0-21swapper/319:28:333
1338899144,6cyclictest0-21swapper/320:55:133
13388991410,3cyclictest36066irq/59-eth020:45:163
1338899131,5cyclictest0-21swapper/319:10:263
1336499132,8cyclictest13381-21latency_hist19:10:011
1338899101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional