You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 03:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Jul 01, 2026 00:45:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
994599623,48cyclictest5535-21ssh22:45:140
994599593,44cyclictest11551-21ssh00:00:130
995399582,38cyclictest18422-21ssh23:10:111
9945995634,9cyclictest12-21ksoftirqd/023:40:130
995399546,37cyclictest14147-21ssh21:20:141
996199522,8cyclictest0-21swapper/219:10:132
995399521,37cyclictest5525-21ssh22:45:131
996199511,4cyclictest0-21swapper/220:40:142
996199511,45cyclictest0-21swapper/200:10:092
996199511,45cyclictest0-21swapper/200:10:092
996199491,6cyclictest25773-21apt-get23:50:142
996199491,6cyclictest25773-21apt-get23:50:132
996199491,2cyclictest0-21swapper/222:55:122
995399494,39cyclictest10598-21ssh00:20:151
996199471,42cyclictest0-21swapper/221:20:132
995399474,34cyclictest28892-21ssh22:40:141
996999435,23cyclictest35966irq/58-eth023:40:123
996199432,6cyclictest29359-21apt-get19:30:122
996199423,35cyclictest27346irq/40-dwc2_hso22:35:132
996199421,4cyclictest0-21swapper/220:35:132
996199421,4cyclictest0-21swapper/220:35:132
9953994218,7cyclictest21-21ksoftirqd/121:10:141
9953994218,7cyclictest21-21ksoftirqd/121:10:141
996199413,5cyclictest27346irq/40-dwc2_hso20:10:132
996199411,3cyclictest0-21swapper/221:25:112
994599414,29cyclictest23560-21ssh22:55:110
996199401,4cyclictest13054-21apt-get23:25:142
996199401,4cyclictest0-21swapper/223:55:112
996199401,34cyclictest28877-21ssh22:40:132
9953994033,5cyclictest0-21swapper/123:50:141
9953994033,5cyclictest0-21swapper/123:50:141
995399401,33cyclictest5304-21ssh00:35:161
996999392,23cyclictest35966irq/58-eth022:40:143
996199393,3cyclictest0-21swapper/221:30:222
996199391,7cyclictest24974-21ssh22:20:022
996199391,6cyclictest0-21swapper/221:35:212
9953993921,13cyclictest21-21ksoftirqd/100:25:131
996999383,19cyclictest0-21swapper/300:00:123
996199382,33cyclictest0-21swapper/223:15:152
996199382,33cyclictest0-21swapper/223:15:152
996199382,19cyclictest31058-21ssh23:35:112
996199381,6cyclictest7804-21ssh23:03:432
996199381,5cyclictest5014-21grep00:15:242
996199381,4cyclictest0-21swapper/222:45:132
996199381,33cyclictest0-21swapper/223:05:132
996999376,19cyclictest35966irq/58-eth022:45:143
996199371,8cyclictest32044-21kthreadcore20:05:202
996199371,8cyclictest17875-21kthreadcore19:15:212
996199371,6cyclictest21609-21/usr/sbin/munin22:00:152
996199371,6cyclictest21609-21/usr/sbin/munin22:00:152
996199371,3cyclictest0-21swapper/220:20:142
996199371,33cyclictest3956-21acpi23:20:122
996999364,21cyclictest35966irq/58-eth021:25:113
996199366,5cyclictest12191-21cut00:20:242
996199361,9cyclictest11900-21if_eth100:00:172
996199361,7cyclictest23348-21ntp_states00:25:282
996199361,7cyclictest18593-21irqrtprio21:40:182
996199361,7cyclictest16423-21tail19:50:182
996199361,6cyclictest9325-21ps20:50:172
996199361,6cyclictest17385-21irqrtprio23:45:202
996199361,6cyclictest17385-21irqrtprio23:45:202
996199361,6cyclictest12350-21sh23:40:302
996999351,23cyclictest35966irq/58-eth021:15:133
996199358,24cyclictest12647-21ssh21:55:162
996199351,8cyclictest9466-21unixbench_singl00:35:322
996199351,8cyclictest23525-21timerwakeupswit19:20:292
995399353,21cyclictest7760-21ssh23:40:131
996999342,20cyclictest35966irq/58-eth022:15:113
996199342,5cyclictest8068-21head20:45:302
996199341,7cyclictest22485-21grep21:00:282
996199341,7cyclictest22485-21grep21:00:282
996199341,7cyclictest21459-21usage_Anybus20:25:302
996199341,7cyclictest18256-21ssh23:10:022
996199341,6cyclictest11503-21grep20:15:272
996199341,5cyclictest32388-21sh00:30:282
996199341,5cyclictest24309-21cat21:05:202
996199341,5cyclictest13928-21cut22:30:222
996199341,5cyclictest13928-21cut22:30:222
996199341,4cyclictest10306-21timerwakeupswit19:40:282
996199341,26cyclictest14560-21ssh22:50:152
996199340,8cyclictest24835-21tail19:55:282
996999331,15cyclictest35966irq/58-eth000:20:153
996199331,8cyclictest26247-21grep23:30:302
996199331,7cyclictest2837-21kthreadcore19:35:192
996199331,29cyclictest28518-21cut21:10:162
996199331,29cyclictest28518-21cut21:10:162
996199330,6cyclictest1892-21ssh22:25:132
996199330,5cyclictest15584-21ssh22:14:382
996199330,4cyclictest26940-21ssh21:45:132
9953993320,7cyclictest9937-21cyclictest22:15:101
995399331,23cyclictest23148-21ssh21:25:121
994599332,27cyclictest30667-21ssh22:05:130
9945993321,4cyclictest14128-21apt-get21:20:130
994599331,23cyclictest16849-21ssh23:45:150
994599331,23cyclictest16849-21ssh23:45:150
9969993210,14cyclictest35966irq/58-eth020:35:133
9969993210,14cyclictest35966irq/58-eth020:35:123
996199321,7cyclictest22189-21/usr/sbin/munin20:30:122
996199321,7cyclictest11059-21ssh21:53:462
996199321,5cyclictest25821-21kthreadcore19:25:192
995399322,5cyclictest9300-21ssh23:05:121
995399322,21cyclictest10913-21sh22:30:141
995399322,21cyclictest10913-21sh22:30:131
9969993115,11cyclictest35966irq/58-eth021:40:133
996199311,7cyclictest30864-21fschecks_time22:05:162
996199311,7cyclictest27799-21kthreadcore22:20:222
996199310,6cyclictest0-21swapper/220:00:122
996199310,5cyclictest4577-21apt-get21:15:042
995399316,18cyclictest7249-21ssh22:10:121
995399313,18cyclictest844-21munin-node00:15:151
996999304,18cyclictest0-21swapper/323:20:133
996199302,7cyclictest17287-21ps20:55:262
996199300,5cyclictest24186-21ssh00:09:332
995399302,22cyclictest30615-21ssh22:05:111
994599306,4cyclictest10590-21apt-get00:20:150
996999292,14cyclictest35966irq/58-eth000:35:153
996199293,3cyclictest9983-21latency_hist19:10:012
996199290,5cyclictest11682-21cut19:45:172
995399295,21cyclictest22064-21ssh23:30:141
996999284,16cyclictest35966irq/58-eth021:10:143
996999284,16cyclictest35966irq/58-eth021:10:143
996999283,18cyclictest0-21swapper/323:34:113
996999283,16cyclictest0-21swapper/323:35:113
996999282,17cyclictest35966irq/58-eth000:10:093
996999282,17cyclictest35966irq/58-eth000:10:083
996999282,15cyclictest35966irq/58-eth000:25:133
9953992810,9cyclictest9937-21cyclictest22:35:131
994599281,18cyclictest28680-21ssh00:30:130
996999273,16cyclictest0-21swapper/323:47:113
996999273,16cyclictest0-21swapper/323:47:113
996999273,16cyclictest0-21swapper/322:09:113
996999272,16cyclictest0-21swapper/323:50:143
996999272,16cyclictest0-21swapper/323:50:143
995399272,19cyclictest4451-21ssh23:36:321
9945992720,4cyclictest18416-21apt-get23:10:120
996999261,16cyclictest0-21swapper/322:25:143
995399263,17cyclictest1906-21ssh22:25:141
995399261,18cyclictest5059-21ssh21:15:141
994599268,9cyclictest0-21swapper/000:10:100
994599268,9cyclictest0-21swapper/000:10:100
994599264,6cyclictest12-21ksoftirqd/023:34:090
994599262,18cyclictest2082-21ssh22:41:230
9945992620,3cyclictest30659-21apt-get20:05:130
9969992514,8cyclictest35966irq/58-eth022:30:133
9969992514,8cyclictest35966irq/58-eth022:30:133
9969992512,9cyclictest35966irq/58-eth022:20:153
9969992512,8cyclictest0-21swapper/323:55:113
996999251,18cyclictest35966irq/58-eth020:45:123
995399252,18cyclictest31533-21ssh22:59:111
995399252,16cyclictest26965-21ssh21:45:141
994599253,6cyclictest12-21ksoftirqd/020:25:180
994599253,4cyclictest12-21ksoftirqd/023:05:020
9945992514,7cyclictest32150-21apt-get21:30:150
996999243,13cyclictest0-21swapper/322:02:113
996999243,13cyclictest0-21swapper/322:02:113
995399249,10cyclictest0-21swapper/119:30:121
994599248,11cyclictest31124-21ssh23:35:140
994599245,4cyclictest12-21ksoftirqd/023:59:080
994599245,14cyclictest0-21swapper/020:00:120
994599244,5cyclictest12-21ksoftirqd/022:03:380
994599244,5cyclictest12-21ksoftirqd/022:03:380
994599244,5cyclictest12-21ksoftirqd/021:26:140
994599243,6cyclictest12-21ksoftirqd/021:59:200
994599243,4cyclictest12-21ksoftirqd/023:51:530
994599243,4cyclictest12-21ksoftirqd/023:51:530
994599242,6cyclictest12-21ksoftirqd/022:31:340
994599242,6cyclictest12-21ksoftirqd/022:31:330
994599241,16cyclictest1599-21ssh00:15:150
996999234,14cyclictest0-21swapper/321:20:143
996999232,11cyclictest0-21swapper/319:25:123
995399232,13cyclictest17605-21cut20:25:151
995399231,5cyclictest53646usb-storage21:40:061
9953992315,6cyclictest11773-21kworker/1:223:25:131
995399231,18cyclictest19771-21apt-get19:20:131
995399231,17cyclictest32687-21ssh23:00:151
995399231,14cyclictest12518-21sh21:55:131
9953992310,7cyclictest21-21ksoftirqd/120:15:121
994599235,6cyclictest12-21ksoftirqd/021:51:060
994599234,9cyclictest0-21swapper/019:10:130
994599233,5cyclictest12-21ksoftirqd/021:38:540
994599233,4cyclictest29778-21tr23:15:230
994599233,4cyclictest29778-21tr23:15:220
994599233,4cyclictest12-21ksoftirqd/021:12:350
994599233,4cyclictest12-21ksoftirqd/021:12:350
994599232,9cyclictest0-21swapper/022:50:100
994599231,6cyclictest12-21ksoftirqd/000:26:310
994599231,19cyclictest7983-21cut20:15:160
994599231,15cyclictest23544-21cut21:05:170
995399222,11cyclictest8752-21sshd21:35:121
995399221,15cyclictest19563-21ls19:20:011
994599226,8cyclictest12-21ksoftirqd/022:24:360
994599225,5cyclictest12-21ksoftirqd/023:20:130
994599224,9cyclictest0-21swapper/020:30:120
994599224,4cyclictest12-21ksoftirqd/020:50:130
994599223,5cyclictest12-21ksoftirqd/021:16:560
994599223,4cyclictest12-21ksoftirqd/022:18:230
994599223,4cyclictest12-21ksoftirqd/000:36:150
994599222,5cyclictest20031-21df_inode22:35:160
994599222,5cyclictest12-21ksoftirqd/023:05:260
994599222,4cyclictest12-21ksoftirqd/022:12:470
994599222,15cyclictest17899-21ssh21:40:130
994599221,10cyclictest12-21ksoftirqd/023:27:510
9969992114,4cyclictest33-21ksoftirqd/319:50:303
9969992111,7cyclictest0-21swapper/321:55:173
995399212,14cyclictest21349-21ssh00:06:111
9953992112,6cyclictest1-21systemd19:50:491
995399211,15cyclictest30215-21ssh22:20:341
995399211,15cyclictest27454-21ssh23:15:151
995399211,15cyclictest27454-21ssh23:15:151
995399211,14cyclictest0-21swapper/123:20:141
995399211,13cyclictest15538-21ls00:00:291
994599214,4cyclictest12-21ksoftirqd/021:45:370
994599213,5cyclictest12-21ksoftirqd/020:55:150
994599213,4cyclictest12-21ksoftirqd/020:45:230
994599213,4cyclictest12-21ksoftirqd/020:10:290
994599213,4cyclictest12-21ksoftirqd/000:05:460
994599212,5cyclictest12-21ksoftirqd/020:35:150
994599212,5cyclictest12-21ksoftirqd/020:35:150
994599212,4cyclictest12-21ksoftirqd/022:26:210
996999209,8cyclictest0-21swapper/323:26:153
996999203,11cyclictest0-21swapper/322:55:113
995399200,15cyclictest3130-21cut20:10:161
994599204,4cyclictest12-21ksoftirqd/019:30:280
994599204,2cyclictest131rcu_preempt21:00:300
994599204,2cyclictest131rcu_preempt21:00:300
994599203,4cyclictest12-21ksoftirqd/019:45:240
994599200,4cyclictest24410-21ntp_192.168.11419:55:240
994599200,17cyclictest16090-21find19:50:120
996999193,9cyclictest0-21swapper/322:50:153
996999193,11cyclictest0-21swapper/322:14:113
996999193,11cyclictest0-21swapper/300:06:113
996999192,9cyclictest35966irq/58-eth023:10:123
996999192,11cyclictest0-21swapper/319:15:133
996999192,11cyclictest0-21swapper/300:15:113
996999192,10cyclictest0-21swapper/323:00:163
996999191,13cyclictest35966irq/58-eth021:45:143
996999191,13cyclictest0-21swapper/320:10:233
996999191,12cyclictest35966irq/58-eth023:15:143
996999191,12cyclictest35966irq/58-eth023:15:143
9969991910,6cyclictest35966irq/58-eth021:00:113
9969991910,6cyclictest35966irq/58-eth021:00:113
9969991910,6cyclictest0-21swapper/323:06:303
995399195,12cyclictest21-21ksoftirqd/100:30:321
995399192,13cyclictest27015-21apt-get20:35:121
995399192,13cyclictest27015-21apt-get20:35:121
995399192,11cyclictest19944-21ssh22:51:111
995399192,10cyclictest0-21swapper/122:02:111
995399192,10cyclictest0-21swapper/122:02:111
995399191,9cyclictest9131-21fschecks_count20:50:151
995399191,14cyclictest1323-21ssh00:15:011
995399191,14cyclictest1323-21ssh00:15:011
995399191,12cyclictest6201-21ssh23:55:251
995399190,9cyclictest11076-21sh19:45:011
994599192,4cyclictest12-21ksoftirqd/019:15:130
9945991913,4cyclictest151rcuc/019:25:250
9945991913,3cyclictest0-21swapper/020:40:130
996999189,6cyclictest35966irq/58-eth021:50:143
996999189,6cyclictest35966irq/58-eth000:30:143
996999182,10cyclictest0-21swapper/320:40:153
996999182,10cyclictest0-21swapper/320:30:133
996999182,10cyclictest0-21swapper/320:20:143
996999182,10cyclictest0-21swapper/320:15:133
996999182,10cyclictest0-21swapper/319:20:143
996999182,10cyclictest0-21swapper/319:10:163
996999181,13cyclictest0-21swapper/320:25:273
995399189,6cyclictest10554-21cut19:10:181
995399187,8cyclictest7118-21ntp_kernel_err21:50:271
995399183,7cyclictest4836-21kernelversion20:45:191
995399181,9cyclictest5130-21ntp_kernel_pll_19:35:241
9953991812,3cyclictest21-21ksoftirqd/119:45:211
994599182,5cyclictest12-21ksoftirqd/019:40:140
994599182,4cyclictest0-21swapper/019:35:120
9945991812,4cyclictest0-21swapper/019:20:140
994599181,13cyclictest0-21swapper/020:20:130
996999179,6cyclictest35966irq/58-eth019:55:143
996999179,6cyclictest0-21swapper/322:35:123
996999179,5cyclictest35966irq/58-eth021:35:113
996999172,10cyclictest0-21swapper/320:55:143
996999172,10cyclictest0-21swapper/319:40:133
9969991711,4cyclictest0-21swapper/321:31:083
996999171,10cyclictest0-21swapper/319:45:213
995399178,7cyclictest1688-21ntp_states20:05:261
995399178,6cyclictest28137-21cat19:25:261
995399178,6cyclictest18604-21sshd21:00:121
995399178,6cyclictest18604-21sshd21:00:111
995399175,9cyclictest21188-21fschecks_time19:55:161
995399174,10cyclictest13225-21irqrtprio20:20:191
995399172,12cyclictest17250-21ntp_states20:55:251
995399172,11cyclictest0-21swapper/121:30:141
995399171,12cyclictest26922-21grep20:35:011
995399171,11cyclictest27968-21awk21:10:011
995399171,11cyclictest16848-21sh23:45:161
995399171,11cyclictest16848-21sh23:45:151
996999162,10cyclictest0-21swapper/321:05:153
996999162,10cyclictest0-21swapper/320:50:133
996999161,11cyclictest0-21swapper/320:00:203
996999161,10cyclictest0-21swapper/319:30:173
995399161,13cyclictest32345-21irqrtprio20:40:191
995399161,10cyclictest28199-21kthreadcore20:00:211
996999149,3cyclictest0-21swapper/319:35:133
995399141,10cyclictest10003-21latency_hist19:10:011
994599141,10cyclictest10089-21latency_hist19:10:020
996999131,7cyclictest0-21swapper/319:10:013
996999129,2cyclictest33-21ksoftirqd/320:05:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional