You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 03:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue May 05, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1162999552,10cyclictest12-21ksoftirqd/020:30:110
1164199533,8cyclictest131rcu_preempt19:15:131
1164199522,9cyclictest131rcu_preempt22:20:131
1165299492,24cyclictest22433-21ssh22:00:142
11652994918,25cyclictest27-21ksoftirqd/222:55:132
1164199461,32cyclictest21709-21ssh23:30:141
1164199442,34cyclictest10139-21ssh21:35:121
1166099433,19cyclictest35966irq/58-eth000:35:123
1164199433,20cyclictest11665-21ssh22:10:251
1162999433,34cyclictest28467-21ssh00:10:130
11629994330,6cyclictest0-21swapper/023:00:140
1162999432,32cyclictest167-21systemd-network00:35:120
1166099424,24cyclictest35966irq/58-eth023:00:133
1164199413,31cyclictest3804-21ssh23:20:131
1164199413,31cyclictest3804-21ssh23:20:131
1164199401,29cyclictest12777-21rm23:25:151
1166099393,21cyclictest35966irq/58-eth022:40:103
1166099392,22cyclictest35966irq/58-eth022:45:123
1165299391,31cyclictest4544-21ssh21:50:132
1162999381,31cyclictest16875-21ssh22:15:110
1165299372,18cyclictest5546-21kthreadcore22:25:232
1165299361,26cyclictest20353-21sh22:35:112
1164199364,23cyclictest5775-21ssh22:45:121
1162999362,26cyclictest13680-21missed_timers21:35:230
1162999361,29cyclictest11390-21ssh22:30:120
1165299351,16cyclictest22854-21ssh00:25:132
1165299351,16cyclictest22854-21ssh00:25:132
1165299341,4cyclictest131rcu_preempt23:00:132
1166099338,16cyclictest35966irq/58-eth022:05:123
1165299325,18cyclictest27346irq/40-dwc2_hso23:40:132
1164199322,25cyclictest25078-21ssh23:50:131
1164199322,19cyclictest31827-21ssh00:30:121
11629993225,3cyclictest15688-21ssh21:20:130
1166099311,20cyclictest35966irq/58-eth000:05:173
1165299312,24cyclictest930-21snmpd19:20:142
1165299312,19cyclictest27346irq/40-dwc2_hso22:30:122
1164199315,16cyclictest1546-21ssh23:55:121
1164199312,23cyclictest16145-21ssh23:45:131
1162999312,22cyclictest9305-21ssh23:05:150
1166099306,15cyclictest35966irq/58-eth020:55:123
1166099303,13cyclictest35966irq/58-eth022:50:143
1166099302,16cyclictest35966irq/58-eth023:10:113
1166099302,16cyclictest0-21swapper/321:45:113
1165299304,16cyclictest27727-21kworker/u8:022:50:152
1165299302,23cyclictest14706-21ssh22:12:512
1165299302,21cyclictest0-21swapper/220:35:132
1165299300,22cyclictest31362-21ssh22:05:142
1164199306,19cyclictest21-21ksoftirqd/122:15:131
11641993011,13cyclictest30210-21ssh21:10:111
1166099292,17cyclictest35966irq/58-eth019:35:123
1165299293,18cyclictest27346irq/40-dwc2_hso23:30:132
1165299293,11cyclictest0-21swapper/219:40:132
1165299292,19cyclictest27346irq/40-dwc2_hso21:15:142
1165299292,17cyclictest27346irq/40-dwc2_hso00:20:112
11652992912,12cyclictest12776-21sh23:25:152
11641992911,10cyclictest14111-21kworker/1:020:20:121
11641992911,10cyclictest14111-21kworker/1:020:20:121
11629992915,10cyclictest151rcuc/021:30:130
1166099283,18cyclictest0-21swapper/322:32:543
1166099281,17cyclictest35966irq/58-eth000:15:133
1165299284,17cyclictest27346irq/40-dwc2_hso21:05:152
1165299284,14cyclictest27346irq/40-dwc2_hso19:25:142
1165299282,21cyclictest32688-21ssh22:22:522
1165299282,19cyclictest27346irq/40-dwc2_hso21:35:122
1165299282,11cyclictest27-21ksoftirqd/221:33:232
11641992818,6cyclictest19083-21sh21:40:141
11641992818,5cyclictest0-21swapper/120:15:141
1162999289,13cyclictest14668-1kworker/u9:021:25:130
1162999281,23cyclictest6755-21ssh21:15:140
1162999281,21cyclictest7226-21ssh23:40:130
1166099273,13cyclictest35966irq/58-eth021:20:133
1165299273,16cyclictest27346irq/40-dwc2_hso00:10:122
1165299272,15cyclictest1562-21ssh23:55:132
1165299272,11cyclictest0-21swapper/219:55:132
1165299272,10cyclictest0-21swapper/221:00:142
1165299272,10cyclictest0-21swapper/221:00:142
1165299271,21cyclictest3631-21ssh21:49:232
11652992712,12cyclictest0-21swapper/223:15:022
1162999274,6cyclictest0-21swapper/023:35:120
1162999274,17cyclictest19595-21ssh00:05:170
1162999274,16cyclictest14797-21ssh22:50:140
1162999272,19cyclictest30903-21ssh00:29:230
1162999272,19cyclictest30903-21ssh00:29:230
1162999272,18cyclictest15148-21sh22:13:230
1162999271,20cyclictest18182-21ssh23:10:130
11629992711,8cyclictest12-21ksoftirqd/022:01:490
1165299269,14cyclictest14113-21cron20:20:012
1165299264,5cyclictest0-21swapper/220:20:122
1165299264,5cyclictest0-21swapper/220:20:122
1165299263,19cyclictest27346irq/40-dwc2_hso00:35:112
1165299262,17cyclictest29818-21diskmemload21:40:142
11652992611,9cyclictest20506-21ssh22:15:232
1165299261,10cyclictest30598-21ssh23:35:122
1162999263,19cyclictest20355-21ssh22:35:110
1166099254,10cyclictest35966irq/58-eth021:25:133
1166099252,17cyclictest0-21swapper/323:59:223
1166099252,16cyclictest35966irq/58-eth021:30:123
1165299254,15cyclictest27346irq/40-dwc2_hso00:00:122
1165299254,13cyclictest27346irq/40-dwc2_hso19:10:132
1165299254,10cyclictest27346irq/40-dwc2_hso20:55:122
1165299252,16cyclictest29237-21ssh22:40:102
1165299250,21cyclictest17747-21ls19:50:022
1164199252,18cyclictest9228-21ssh22:27:531
11641992514,7cyclictest0-21swapper/100:35:131
11629992515,7cyclictest16093-21/usr/sbin/munin23:45:180
1162999251,16cyclictest31347-21ssh22:05:120
1166099245,11cyclictest35966irq/58-eth022:35:123
1165299243,12cyclictest27346irq/40-dwc2_hso00:05:162
1165299242,17cyclictest27346irq/40-dwc2_hso19:50:142
1165299242,17cyclictest27346irq/40-dwc2_hso00:30:132
1165299242,17cyclictest17027-21ssh21:55:232
1165299240,15cyclictest4954-21ls20:10:162
1164199242,17cyclictest28019-21rm21:45:131
1164199242,16cyclictest844-21munin-node21:50:131
1162999248,3cyclictest131rcu_preempt00:00:140
1162999246,9cyclictest12-21ksoftirqd/022:43:380
11629992411,4cyclictest12-21ksoftirqd/019:10:140
1166099236,12cyclictest33-21ksoftirqd/300:13:233
1166099233,13cyclictest0-21swapper/323:15:053
1166099233,11cyclictest0-21swapper/323:35:123
1165299232,18cyclictest4993-21kthreadcore23:20:202
1165299232,18cyclictest4993-21kthreadcore23:20:202
1165299232,17cyclictest0-21swapper/222:45:132
1165299232,14cyclictest27346irq/40-dwc2_hso19:15:122
1165299230,15cyclictest5800-21latency_hist20:45:012
1164199232,4cyclictest131rcu_preempt20:40:121
1164199232,14cyclictest844-21munin-node22:30:131
1162999239,8cyclictest12-21ksoftirqd/000:19:560
1162999237,4cyclictest12-21ksoftirqd/023:30:450
1162999231,17cyclictest29818-21diskmemload23:15:140
11629992310,4cyclictest0-21swapper/022:24:190
1166099223,13cyclictest0-21swapper/300:29:233
1166099223,13cyclictest0-21swapper/300:29:233
11660992213,5cyclictest0-21swapper/300:35:013
11660992212,6cyclictest0-21swapper/321:35:213
11660992211,7cyclictest35966irq/58-eth000:20:103
1166099221,15cyclictest35966irq/58-eth021:10:113
11660992210,7cyclictest35966irq/58-eth023:25:153
1165299223,10cyclictest16790-21kthreadcore21:20:202
1165299222,17cyclictest5276-21ssh00:15:172
11652992212,8cyclictest31090-21ssh21:27:282
1165299221,17cyclictest28775-21cut23:15:202
1165299221,17cyclictest1427-21sed21:10:252
1164199221,18cyclictest399-21cut23:00:171
1164199221,17cyclictest27226-21ssh23:15:141
1162999229,7cyclictest12-21ksoftirqd/021:52:440
1162999229,6cyclictest0-21swapper/021:41:270
1162999227,3cyclictest131rcu_preempt23:50:130
1162999225,9cyclictest12-21ksoftirqd/023:20:130
1162999225,9cyclictest12-21ksoftirqd/023:20:130
1162999222,13cyclictest0-21swapper/019:45:130
1162999221,14cyclictest32058-21cut00:30:160
1166099217,9cyclictest35966irq/58-eth023:05:153
1166099214,11cyclictest35966irq/58-eth021:55:143
1166099212,12cyclictest0-21swapper/322:00:133
1166099212,10cyclictest35966irq/58-eth023:20:133
1166099212,10cyclictest35966irq/58-eth023:20:133
11660992111,7cyclictest0-21swapper/322:30:013
1165299212,12cyclictest27346irq/40-dwc2_hso23:05:142
11652992115,4cyclictest0-21swapper/219:35:302
1165299211,16cyclictest19387-21fschecks_time20:25:152
1165299211,15cyclictest27346irq/40-dwc2_hso20:50:142
1165299211,10cyclictest27776-21sed20:30:282
1164199214,8cyclictest11497-21cat21:15:311
1164199213,8cyclictest0-21swapper/121:25:131
1164199212,14cyclictest25919-21ssh22:00:231
1164199211,7cyclictest19285-21ntp_192.168.11421:20:251
1164199211,11cyclictest287766ssh23:35:301
11641992110,5cyclictest0-21swapper/100:25:131
11641992110,5cyclictest0-21swapper/100:25:131
1164199210,17cyclictest28744-21ls00:10:161
1162999219,4cyclictest12-21ksoftirqd/022:26:590
1162999218,7cyclictest12-21ksoftirqd/000:22:030
1162999217,7cyclictest12-21ksoftirqd/021:05:270
1162999217,4cyclictest12-21ksoftirqd/022:47:250
1162999216,4cyclictest12-21ksoftirqd/021:45:470
1162999216,2cyclictest131rcu_preempt22:56:100
1162999216,12cyclictest3722-21unixbench_multi20:05:290
1162999215,2cyclictest131rcu_preempt21:56:430
1162999212,5cyclictest12-21ksoftirqd/023:28:570
1162999212,16cyclictest16714-21fschecks_count19:15:150
11629992113,6cyclictest0-21swapper/020:45:280
1162999211,17cyclictest14428-21ls20:50:290
1162999211,14cyclictest31118-21timerandwakeup20:00:290
1166099204,9cyclictest35966irq/58-eth022:15:133
11660992013,5cyclictest0-21swapper/323:50:133
11660992011,6cyclictest0-21swapper/323:40:163
1166099201,13cyclictest0-21swapper/320:50:293
1165299206,11cyclictest31272-21unixbench_singl20:00:302
1165299201,9cyclictest26047-21kthreadcore23:50:192
11652992012,5cyclictest16245-21ssh23:45:142
1165299201,16cyclictest2069-21wc19:30:272
1165299200,10cyclictest3787-21unixbench_singl20:05:292
1164199207,10cyclictest14453-21grep00:00:311
11641992011,6cyclictest25890-21sshd22:36:231
1164199201,14cyclictest28921-21timerandwakeup21:05:301
1164199201,14cyclictest23224-21ssh00:05:301
1162999207,7cyclictest12-21ksoftirqd/019:50:190
1162999207,3cyclictest131rcu_preempt20:55:180
1162999202,7cyclictest12-21ksoftirqd/020:20:200
1162999202,7cyclictest12-21ksoftirqd/020:20:200
1162999202,15cyclictest24220-21kthreadcore19:55:200
11629992012,5cyclictest6060-21usage_EthcTask23:55:300
1162999201,15cyclictest30222-21sh21:10:130
1162999200,10cyclictest23755-21tr19:20:210
1166099193,11cyclictest0-21swapper/321:52:233
11660991913,4cyclictest0-21swapper/323:45:253
11660991913,4cyclictest0-21swapper/323:30:253
11660991913,4cyclictest0-21swapper/322:20:233
11660991913,4cyclictest0-21swapper/321:40:133
11660991913,4cyclictest0-21swapper/321:16:473
11660991910,6cyclictest35966irq/58-eth022:10:133
1165299191,3cyclictest0-21swapper/220:45:132
1164199199,7cyclictest11325-21kthreadcore19:40:211
1164199194,5cyclictest0-21swapper/122:55:191
1164199192,15cyclictest5455-21irqcore00:15:181
1164199192,13cyclictest17183-21ssh21:55:261
1164199191,7cyclictest1669-21users22:40:291
11641991913,4cyclictest21481-21sshd23:10:221
1164199191,14cyclictest1126-21apt-get21:30:121
1164199191,10cyclictest21028-21wc19:50:231
1162999199,2cyclictest12-21ksoftirqd/020:10:140
1162999196,3cyclictest12-21ksoftirqd/021:00:020
1162999196,3cyclictest12-21ksoftirqd/021:00:020
11629991910,6cyclictest902-21kthreadcore19:30:210
1166099182,11cyclictest0-21swapper/300:00:143
11660991812,4cyclictest0-21swapper/319:30:283
1166099181,12cyclictest0-21swapper/319:25:233
11660991810,6cyclictest0-21swapper/322:55:023
1164199186,9cyclictest24155-21tune2fs20:30:151
1164199181,9cyclictest4623-21kthreadcore19:35:191
1164199181,6cyclictest0-21swapper/123:05:221
1164199181,14cyclictest8666-21tr23:40:191
1164199181,13cyclictest31921-21ps22:05:181
1164199181,12cyclictest14932-21ssh22:50:171
1162999184,11cyclictest11514-21kthreadcore20:15:200
1162999182,13cyclictest0-21swapper/020:25:130
11629991812,4cyclictest12-21ksoftirqd/020:40:180
1166099177,7cyclictest0-21swapper/321:00:183
1166099177,7cyclictest0-21swapper/321:00:183
1166099172,9cyclictest0-21swapper/320:30:133
1166099172,10cyclictest0-21swapper/319:45:133
11660991711,4cyclictest0-21swapper/319:50:233
1164199179,5cyclictest10856-21fschecks_count20:50:171
1164199173,11cyclictest23620-21kthreadcore19:20:211
1164199173,11cyclictest2225-21kthreadcore20:05:211
1164199172,11cyclictest22731-21grep19:55:161
1164199172,10cyclictest4845-21fschecks_count20:10:151
1164199172,10cyclictest0-21swapper/119:10:141
11641991711,3cyclictest20205-21awk21:00:011
1164199171,13cyclictest13860-21ssh00:20:111
11641991710,5cyclictest16529-21sendmail_mailqu19:45:261
1162999172,12cyclictest32385-21cat20:35:260
1162999172,11cyclictest26548-21tail19:25:180
1162999171,13cyclictest6778-21ntp_192.168.11419:35:240
1162999171,12cyclictest9251-21kthreadcore19:40:180
1166099162,10cyclictest0-21swapper/320:25:133
1166099161,7cyclictest0-21swapper/320:10:273
1166099161,12cyclictest0-21swapper/321:05:253
1165299160,13cyclictest11721-21latency_hist19:10:022
1164199169,5cyclictest27740-21interrupts20:00:181
1164199168,6cyclictest20761-21irqcore21:00:171
1164199168,6cyclictest20761-21irqcore21:00:171
1164199162,11cyclictest19880-21kthreadcore20:25:181
1164199161,12cyclictest26311-21diskstats19:25:151
1164199161,10cyclictest7876-21kthreadcore20:45:201
11641991610,4cyclictest31326-21irqcore19:30:181
1164199160,11cyclictest28808-21cpuspeed20:35:131
1166099159,4cyclictest0-21swapper/320:05:293
1166099151,8cyclictest0-21swapper/319:55:153
11660991510,3cyclictest0-21swapper/320:20:283
11660991510,3cyclictest0-21swapper/320:20:283
1164199152,10cyclictest11678-21latency_hist19:10:021
1166099148,4cyclictest0-21swapper/319:15:453
1166099148,4cyclictest0-21swapper/319:15:013
1166099148,4cyclictest0-21swapper/319:10:023
1166099146,5cyclictest321rcuc/320:45:123
1166099142,7cyclictest0-21swapper/319:40:133
1166099141,9cyclictest0-21swapper/320:15:123
1166099141,8cyclictest0-21swapper/320:00:293
1166099141,11cyclictest0-21swapper/320:40:123
1162999141,10cyclictest11528-21latency_hist19:10:020
1166099136,5cyclictest33-21ksoftirqd/319:20:143
1166099131,10cyclictest0-21swapper/320:35:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional