You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 11:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jul 21, 2026 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24539996846,11cyclictest35966irq/58-eth000:15:123
2451999686,50cyclictest24258-21ssh23:40:111
2450799665,50cyclictest4564-21ssh00:05:100
2451999551,6cyclictest131rcu_preempt21:35:141
2450799504,33cyclictest18914-21ssh23:55:110
24507994823,21cyclictest12-21ksoftirqd/000:20:140
24539994612,30cyclictest0-21swapper/323:40:123
2451999453,35cyclictest26552-21ssh00:35:121
24519994525,7cyclictest21-21ksoftirqd/121:55:141
2450799455,32cyclictest29030-21ssh21:20:140
2450799455,32cyclictest29030-21ssh21:20:140
2450799434,31cyclictest2252-21ssh23:10:140
2453999425,23cyclictest35966irq/58-eth022:30:133
2453999421,28cyclictest35966irq/58-eth000:35:123
2451999421,35cyclictest887-21ssh23:45:141
2450799421,31cyclictest18768-21sh21:50:160
2452799403,28cyclictest29017-21ssh21:20:142
2452799403,28cyclictest29017-21ssh21:20:132
2452799382,30cyclictest11417-21rm00:07:542
2453999374,23cyclictest35966irq/58-eth023:45:133
2452799373,13cyclictest7433-21ssh22:55:142
2452799372,18cyclictest430-21ssh00:37:162
2452799371,32cyclictest564-21ssh21:40:152
2453999363,19cyclictest35966irq/58-eth021:50:163
24539993518,10cyclictest35966irq/58-eth022:00:133
2452799356,23cyclictest27-21ksoftirqd/223:05:142
2452799352,19cyclictest930-21snmpd20:35:142
2451999352,22cyclictest21-21ksoftirqd/120:15:141
2453999343,19cyclictest35966irq/58-eth022:50:123
2453999342,17cyclictest0-21swapper/300:27:163
2452799343,9cyclictest630-21Async09:36:312
24527993422,10cyclictest24496-21cyclictest23:10:122
24527993413,7cyclictest24278-21ssh23:40:122
2451999343,24cyclictest21754-21ssh22:45:141
24519993417,9cyclictest21-21ksoftirqd/121:10:131
2450799341,6cyclictest23920-21apt-get21:35:140
2453999334,19cyclictest0-21swapper/323:05:143
24539993319,9cyclictest35966irq/58-eth022:05:133
2452799333,14cyclictest10725-21ssh21:10:142
24519993321,8cyclictest0-21swapper/120:25:131
2451999331,24cyclictest532-21ssh21:40:141
24519993311,20cyclictest30844-21ssh22:50:141
2450799331,24cyclictest881-21ssh23:45:130
2450799331,21cyclictest12629-21sh22:40:130
2453999323,19cyclictest35966irq/58-eth021:15:133
24539993214,12cyclictest35966irq/58-eth021:35:133
24527993222,8cyclictest261rcuc/221:45:132
2452799322,21cyclictest27346irq/40-dwc2_hso23:55:132
2452799322,20cyclictest27346irq/40-dwc2_hso23:15:142
2450799328,17cyclictest664-21ntpd20:35:140
2450799323,7cyclictest12-21ksoftirqd/022:25:150
2450799323,23cyclictest9655-21ssh21:45:110
2452799315,16cyclictest21745-21ssh22:45:142
2452799313,19cyclictest27346irq/40-dwc2_hso19:35:142
2451999319,17cyclictest21-21ksoftirqd/122:20:141
2451999312,22cyclictest17713-21ssh23:17:171
2450799316,9cyclictest12-21ksoftirqd/022:45:150
24539993015,9cyclictest35966irq/58-eth022:45:153
2452799302,18cyclictest5637-21ssh22:17:142
2452799302,18cyclictest3449-21ssh21:59:142
24527993013,6cyclictest0-21swapper/223:00:132
2452799301,2cyclictest0-21swapper/223:20:132
2451999303,16cyclictest13592-21ssh22:05:121
24507993016,10cyclictest0-21swapper/022:30:110
2452799299,9cyclictest930-21snmpd23:34:172
2452799296,14cyclictest27346irq/40-dwc2_hso20:55:132
2452799293,20cyclictest0-21swapper/200:27:572
2452799292,16cyclictest4296-21ssh23:28:162
2452799292,11cyclictest4511-21ssh22:00:132
24527992920,6cyclictest27-21ksoftirqd/222:25:122
2451999291,23cyclictest12651-21ssh22:40:151
2450799292,21cyclictest5133-21ssh23:29:150
2450799292,17cyclictest555-21ssh21:40:150
2453999285,12cyclictest35966irq/58-eth021:55:153
2451999282,19cyclictest29170-21ssh22:12:281
2450799282,19cyclictest14731-21ssh00:27:150
24507992815,4cyclictest12-21ksoftirqd/022:10:020
2453999273,13cyclictest35966irq/58-eth022:25:143
2453999272,18cyclictest0-21swapper/323:22:443
2452799275,16cyclictest0-21swapper/200:15:132
2452799274,13cyclictest27346irq/40-dwc2_hso19:40:132
2452799272,21cyclictest0-21swapper/222:37:142
2452799272,20cyclictest16575-21ssh22:24:162
2452799272,20cyclictest0-21swapper/219:30:132
2452799272,12cyclictest31900-21df00:20:152
24527992717,8cyclictest27-21ksoftirqd/222:32:352
2450799273,18cyclictest19895-21diskmemload21:15:140
2450799272,6cyclictest12-21ksoftirqd/021:55:140
2450799272,19cyclictest6384-21ssh23:30:170
24507992711,8cyclictest12-21ksoftirqd/000:10:450
24507992710,10cyclictest0-21swapper/023:50:140
2453999268,12cyclictest35966irq/58-eth020:35:153
2453999263,14cyclictest0-21swapper/323:17:163
2452799264,10cyclictest2916-21kworker/2:019:55:132
2452799262,17cyclictest27346irq/40-dwc2_hso20:10:132
2451999262,19cyclictest22451-21ssh21:50:281
2451999262,18cyclictest11416-21sh00:07:541
2451999262,17cyclictest22697-21ssh00:15:131
2451999261,18cyclictest26685-21sh22:30:121
2450799265,16cyclictest25598-21kworker/u8:023:05:150
2453999253,14cyclictest0-21swapper/323:28:153
2453999252,15cyclictest35966irq/58-eth023:00:153
2452799253,14cyclictest27346irq/40-dwc2_hso19:25:152
2452799252,9cyclictest0-21swapper/221:50:162
2452799252,8cyclictest17908-21ssh23:54:172
24527992514,8cyclictest17380-21cron20:10:022
2452799251,19cyclictest13605-21ssh22:05:122
2451999251,7cyclictest14825-21latency_hist23:35:011
2450799253,16cyclictest4525-21ssh22:00:140
2450799252,19cyclictest11508-21ssh23:15:160
24507992514,9cyclictest0-21swapper/000:00:260
2450799251,3cyclictest131rcu_preempt20:00:130
2453999244,12cyclictest35966irq/58-eth023:55:113
24539992412,9cyclictest35966irq/58-eth000:30:133
2452799241,15cyclictest2084-21ntpq22:50:252
2452799241,14cyclictest356-21ls19:15:252
2451999242,17cyclictest13621-21ssh00:10:151
2451999242,15cyclictest201rcuc/121:30:141
2451999242,14cyclictest27957-21ssh00:00:111
24519992413,7cyclictest0-21swapper/120:10:121
2451999241,19cyclictest925-21find21:00:111
2451999241,18cyclictest9913-21ssh23:50:151
2450799242,6cyclictest0-21swapper/023:00:150
2450799242,3cyclictest131rcu_preempt22:15:190
2450799241,20cyclictest5728-21ssh21:25:140
2450799241,20cyclictest29989-21latency00:35:230
2450799241,19cyclictest22695-21ssh22:10:150
2450799241,16cyclictest3375-21ssh22:35:140
2453999233,13cyclictest0-21swapper/300:07:533
2452799238,7cyclictest1017-21df_inode23:45:172
2452799231,18cyclictest10326-21diskmemload22:13:152
2452799231,14cyclictest3179-21kthreadcore19:20:192
24527992310,9cyclictest19415-21kthreadcore00:30:212
2451999234,14cyclictest0-21swapper/119:40:151
2451999232,6cyclictest0-21swapper/122:01:141
2451999232,12cyclictest21-21ksoftirqd/119:15:141
2451999231,18cyclictest26032-21ls19:45:151
2451999231,18cyclictest19142-21cut23:55:161
2451999231,14cyclictest6680-21sshd19:25:151
24507992312,6cyclictest0-21swapper/020:55:140
24507992312,4cyclictest0-21swapper/021:05:130
24507992311,4cyclictest0-21swapper/023:38:330
2453999223,12cyclictest0-21swapper/322:12:283
24539992211,7cyclictest35966irq/58-eth022:55:153
2453999221,12cyclictest35966irq/58-eth023:50:143
2452799228,7cyclictest29254-21cron19:15:002
2452799228,5cyclictest28035-21kthreadcore20:20:202
2452799226,3cyclictest131rcu_preempt21:15:142
2452799222,16cyclictest27346irq/40-dwc2_hso19:45:142
2452799221,11cyclictest11021-21kthreadcore20:00:202
2451999221,4cyclictest131rcu_preempt21:15:131
24519992213,6cyclictest2390-21fschecks_count23:10:161
2450799229,4cyclictest12-21ksoftirqd/023:21:510
2450799225,5cyclictest12-21ksoftirqd/000:15:130
2450799221,18cyclictest4546-21cut20:30:150
24507992210,3cyclictest0-21swapper/000:32:390
2453999211,13cyclictest35966irq/58-eth019:10:123
2452799213,9cyclictest26683-21kthreadcore20:50:232
24527992113,5cyclictest27-21ksoftirqd/223:35:222
24527992111,8cyclictest19183-21sed20:45:162
24527992111,7cyclictest0-21swapper/221:05:262
2452799211,16cyclictest18630-21idleruntime-cro20:45:012
2452799211,11cyclictest13568-21/usr/sbin/munin00:10:152
2452799211,11cyclictest13510-21kthreadcore22:40:212
2451999212,15cyclictest27397-21fschecks_count20:20:151
2451999212,14cyclictest7446-21sshd22:55:141
2451999211,16cyclictest15239-21ssh23:35:151
2450799215,8cyclictest12-21ksoftirqd/021:10:160
2450799215,3cyclictest131rcu_preempt22:20:130
2450799212,6cyclictest25749-21munin-run19:45:020
2450799212,14cyclictest6677-21apt-get19:25:140
24507992114,4cyclictest12-21ksoftirqd/019:30:280
24507992113,5cyclictest0-21swapper/023:40:150
2450799211,17cyclictest22569-21timerandwakeup20:45:280
2450799211,16cyclictest30866-21cut19:50:170
2450799210,7cyclictest32443-21ssh22:50:200
2453999209,8cyclictest0-21swapper/321:33:393
2453999206,8cyclictest35966irq/58-eth000:20:163
2453999203,11cyclictest35966irq/58-eth021:45:123
24539992012,5cyclictest35966irq/58-eth022:40:143
2452799201,9cyclictest9782-21timerwakeupswit21:25:302
2452799201,9cyclictest26122-21wc20:15:292
2452799201,4cyclictest1131-21grep21:00:162
2452799201,4cyclictest0-21swapper/219:50:142
2452799201,16cyclictest28652-21ssh00:00:172
2452799200,11cyclictest32585-21munin-plugin-st21:40:022
2451999209,8cyclictest20842-21interrupts23:20:191
2451999201,7cyclictest20248-21ntp_states23:00:281
2451999201,16cyclictest28457-21apt-get20:55:141
2451999201,14cyclictest4115-21kthreadcore22:35:181
2451999201,10cyclictest7734-21kthreadcore21:25:221
2451999201,10cyclictest0-21swapper/121:20:541
2451999201,10cyclictest0-21swapper/121:20:531
2450799209,3cyclictest12-21ksoftirqd/019:15:150
2450799208,4cyclictest12-21ksoftirqd/019:20:220
2450799207,4cyclictest12-21ksoftirqd/019:55:210
2453999199,6cyclictest35966irq/58-eth022:15:133
2453999194,7cyclictest0-21swapper/321:25:133
2453999193,11cyclictest0-21swapper/323:33:173
2453999193,11cyclictest0-21swapper/323:10:163
2453999193,10cyclictest0-21swapper/322:20:143
2453999192,11cyclictest0-21swapper/320:00:133
2453999192,10cyclictest35966irq/58-eth000:10:153
2453999192,10cyclictest0-21swapper/321:05:153
24539991910,6cyclictest35966irq/58-eth019:15:143
2452799191,9cyclictest24643-21latency_hist19:10:022
2452799191,8cyclictest506-21kthreadcore20:25:182
2452799191,10cyclictest4769-21irqcore20:30:172
2451999196,10cyclictest29454-21grep23:05:281
2451999192,13cyclictest8391-21apt-get00:25:131
2451999192,10cyclictest0-21swapper/122:29:161
2451999191,15cyclictest607-21klogd20:55:011
2451999191,14cyclictest9279-21sort20:35:161
2451999191,13cyclictest10326-21diskmemload00:24:161
24519991910,6cyclictest0-21swapper/119:35:171
2450799197,2cyclictest131rcu_preempt21:30:260
2450799193,7cyclictest4255-21missed_timers21:00:230
24507991913,4cyclictest0-21swapper/022:55:160
2450799191,14cyclictest0-21swapper/020:15:120
2450799190,14cyclictest19528-21latency19:35:230
2453999187,8cyclictest0-21swapper/321:41:293
2453999183,10cyclictest0-21swapper/322:38:153
2453999182,11cyclictest0-21swapper/319:30:143
2453999182,10cyclictest0-21swapper/321:00:053
2453999181,11cyclictest35966irq/58-eth019:45:123
2453999181,10cyclictest35966irq/58-eth020:55:143
24539991810,6cyclictest0-21swapper/323:36:313
24539991810,6cyclictest0-21swapper/321:20:173
24539991810,6cyclictest0-21swapper/321:20:163
2451999189,6cyclictest0-21swapper/119:10:281
2451999186,3cyclictest131rcu_preempt22:15:151
2451999181,7cyclictest24576-21cat19:10:011
2451999181,7cyclictest21187-21ntpq00:30:281
2451999181,14cyclictest12907-21grep20:05:171
2451999181,13cyclictest5420-21sendmail_mailqu19:20:271
2450799188,8cyclictest17616-21apt-get20:10:130
2450799188,7cyclictest29312-21missed_timers19:45:220
2450799188,3cyclictest131rcu_preempt19:10:010
24507991812,4cyclictest0-21swapper/019:10:300
2450799181,11cyclictest2973-21missed_timers20:25:220
2453999176,7cyclictest35966irq/58-eth021:10:123
2453999173,7cyclictest0-21swapper/320:20:123
2453999172,10cyclictest0-21swapper/319:40:053
2453999171,9cyclictest35966irq/58-eth000:00:123
2451999179,6cyclictest2924-21sshd19:52:121
2451999178,7cyclictest8955-21users20:30:281
2451999178,6cyclictest3264-21sshd19:55:151
2451999177,7cyclictest10319-21cron21:10:021
2451999176,3cyclictest131rcu_preempt23:25:301
2451999171,13cyclictest11084-21tr20:00:211
24519991710,5cyclictest1-21systemd20:42:181
2450799173,8cyclictest12-21ksoftirqd/020:40:310
24507991711,4cyclictest0-21swapper/020:50:190
24507991710,5cyclictest0-21swapper/020:20:120
2453999163,9cyclictest0-21swapper/320:15:143
2453999162,9cyclictest0-21swapper/320:45:143
2453999162,9cyclictest0-21swapper/320:25:143
2451999168,5cyclictest12943-21kthreadcore19:30:201
2451999161,12cyclictest9942-21grep21:45:171
2451999161,12cyclictest20766-21tr20:45:201
2450799161,12cyclictest12866-21df_inode20:05:150
2453999159,4cyclictest0-21swapper/320:50:213
2453999149,3cyclictest0-21swapper/319:50:293
2453999149,3cyclictest0-21swapper/319:25:283
2453999143,6cyclictest0-21swapper/319:55:133
2453999138,3cyclictest0-21swapper/320:41:483
2453999138,3cyclictest0-21swapper/320:10:163
2453999121,8cyclictest0-21swapper/320:30:123
2453999121,8cyclictest0-21swapper/320:10:013
2453999121,7cyclictest0-21swapper/319:40:013
2453999121,7cyclictest0-21swapper/319:25:023
2453999111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional