You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 11:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Jul 31, 2026 00:45:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
585099661,51cyclictest31458-21ssh23:20:132
5858995442,5cyclictest35966irq/58-eth023:15:133
585099541,33cyclictest22526-21ssh21:45:142
583999491,38cyclictest862-21ssh21:15:151
585899486,24cyclictest35966irq/58-eth023:20:133
585099469,10cyclictest131rcu_preempt19:15:142
585899452,29cyclictest35966irq/58-eth000:25:143
585899451,26cyclictest35966irq/58-eth000:15:143
585899451,26cyclictest35966irq/58-eth000:15:133
583099454,34cyclictest8102-21ssh21:55:140
583099441,36cyclictest73-21kswapd023:35:120
583999431,32cyclictest6341-21ssh22:30:121
5858994222,12cyclictest35966irq/58-eth000:00:133
583999412,25cyclictest24158-21ssh00:10:171
585099402,16cyclictest2620-21ssh22:10:112
585099382,19cyclictest20635-21ssh22:20:092
583999381,4cyclictest131rcu_preempt21:55:141
585899379,18cyclictest35966irq/58-eth022:55:143
585899371,22cyclictest35966irq/58-eth022:35:133
583999365,7cyclictest131rcu_preempt20:35:121
5839993631,2cyclictest21-21ksoftirqd/122:05:131
5850993515,12cyclictest27-21ksoftirqd/200:15:122
5850993515,12cyclictest27-21ksoftirqd/200:15:112
583999353,28cyclictest27927-21ssh23:00:171
585899342,19cyclictest35966irq/58-eth020:20:133
5858993312,13cyclictest35966irq/58-eth021:20:133
585099332,15cyclictest18754-21ssh21:25:092
585099332,15cyclictest18754-21ssh21:25:092
5830993316,7cyclictest0-21swapper/000:20:140
585899323,19cyclictest35966irq/58-eth023:25:153
585099322,4cyclictest131rcu_preempt22:00:142
585099322,14cyclictest27030-21ssh23:52:272
5830993224,6cyclictest151rcuc/022:00:140
585899313,17cyclictest0-21swapper/300:35:133
5858993115,11cyclictest35966irq/58-eth021:15:163
585099312,23cyclictest9908-21ssh21:20:142
583999312,25cyclictest29550-21ssh23:55:131
583099313,19cyclictest9870-21ssh22:50:120
583099311,27cyclictest27918-21ssh23:00:160
585899303,15cyclictest35966irq/58-eth022:45:123
585899303,15cyclictest35966irq/58-eth022:45:123
585099302,19cyclictest27346irq/40-dwc2_hso20:50:122
585099302,19cyclictest27346irq/40-dwc2_hso19:55:142
585099302,19cyclictest27346irq/40-dwc2_hso19:55:142
583999309,18cyclictest15329-21ssh22:35:131
583999302,7cyclictest131rcu_preempt23:35:121
583999301,21cyclictest21-21ksoftirqd/121:20:131
585899294,15cyclictest35966irq/58-eth022:20:123
585099292,21cyclictest0-21swapper/219:35:142
585099292,20cyclictest0-21swapper/223:40:142
585099291,13cyclictest29701-21ssh22:25:122
583099294,16cyclictest24337-21ssh22:40:120
583099294,13cyclictest4425-21ssh00:35:130
5830992914,3cyclictest131rcu_preempt21:30:130
585899282,6cyclictest0-21swapper/300:20:143
585099289,12cyclictest0-21swapper/200:05:142
585099283,17cyclictest27346irq/40-dwc2_hso23:15:152
585099282,22cyclictest27833-21ssh21:30:142
583999284,7cyclictest131rcu_preempt21:00:131
5830992811,12cyclictest12-21ksoftirqd/023:06:080
585099272,14cyclictest858-21ssh21:15:152
585099271,20cyclictest18760-21ssh00:25:152
585099271,17cyclictest27346irq/40-dwc2_hso20:40:122
583999271,6cyclictest15624-21timerwakeupswit23:45:291
583999271,6cyclictest15624-21timerwakeupswit23:45:291
583099273,3cyclictest839-21apt-get21:15:150
583099273,19cyclictest6127-21ssh00:00:130
583099272,22cyclictest0-21swapper/019:55:140
583099272,22cyclictest0-21swapper/019:55:140
5858992611,9cyclictest35966irq/58-eth023:55:143
585099263,16cyclictest27346irq/40-dwc2_hso21:50:142
585099261,19cyclictest27915-21apt-get23:00:172
583999260,17cyclictest73-21kswapd022:25:121
583099263,4cyclictest131rcu_preempt22:10:130
583099262,21cyclictest693-21ssh00:15:120
583099262,21cyclictest693-21ssh00:15:120
583099262,18cyclictest0-21swapper/022:55:090
583099261,21cyclictest31443-21ssh23:20:120
585899254,9cyclictest321rcuc/320:50:113
585899253,15cyclictest0-21swapper/322:00:143
585099259,13cyclictest18341-21cron00:25:012
585099256,15cyclictest0-21swapper/223:30:132
585099252,8cyclictest30633-21ssh00:12:302
585099252,11cyclictest13496-21ssh23:10:132
5850992511,11cyclictest0-21swapper/220:30:172
583999252,18cyclictest22527-21ssh21:45:131
583999251,8cyclictest131rcu_preempt19:40:121
585899242,16cyclictest0-21swapper/300:05:143
5858992415,7cyclictest0-21swapper/321:30:153
5858992410,8cyclictest35966irq/58-eth023:50:133
585099242,16cyclictest0-21swapper/223:35:092
585099242,16cyclictest0-21swapper/221:40:092
585099242,10cyclictest18895-21ssh22:55:142
5850992413,9cyclictest20682-21ssh22:36:102
5850992412,10cyclictest10490-21sshd21:36:562
585099241,19cyclictest0-21swapper/223:45:132
585099241,19cyclictest0-21swapper/223:45:132
583999244,15cyclictest2654-21ssh22:10:131
583999242,8cyclictest4373-21ssh21:35:101
583099242,20cyclictest11554-21ssh23:45:120
583099242,20cyclictest11554-21ssh23:45:120
5830992417,4cyclictest26116-21ssh22:05:140
5858992315,5cyclictest0-21swapper/323:45:253
5858992315,5cyclictest0-21swapper/323:45:253
5858992314,6cyclictest0-21swapper/322:34:393
5858992314,6cyclictest0-21swapper/321:39:103
5858992312,7cyclictest35966irq/58-eth021:10:143
585099239,4cyclictest0-21swapper/222:15:212
585099233,14cyclictest27346irq/40-dwc2_hso20:25:132
585099232,16cyclictest8021-21ssh23:25:142
585099232,15cyclictest27346irq/40-dwc2_hso20:15:122
5850992312,6cyclictest27-21ksoftirqd/222:40:132
5850992312,5cyclictest9834-21ssh22:50:112
5850992311,5cyclictest29702-21ssh21:11:142
583099238,4cyclictest131rcu_preempt00:10:240
583099237,4cyclictest131rcu_preempt19:15:140
583099233,6cyclictest131rcu_preempt20:00:130
5830992317,3cyclictest0-21swapper/023:50:320
583099231,19cyclictest23910-21diskmemload22:20:120
585899229,8cyclictest35966irq/58-eth021:50:143
5858992215,5cyclictest0-21swapper/300:30:333
5858992213,7cyclictest0-21swapper/322:40:233
5858992213,6cyclictest0-21swapper/323:05:163
5858992212,7cyclictest0-21swapper/323:00:323
5858992212,7cyclictest0-21swapper/320:55:203
5858992212,7cyclictest0-21swapper/320:55:203
585099229,9cyclictest0-21swapper/220:00:262
585099228,11cyclictest27-21ksoftirqd/222:45:132
585099228,11cyclictest27-21ksoftirqd/222:45:132
585099227,12cyclictest13175-21latency_hist23:10:012
585099222,6cyclictest0-21swapper/200:04:112
585099222,17cyclictest0-21swapper/219:10:132
5850992211,5cyclictest1388-21ssh00:31:572
5839992215,4cyclictest0-21swapper/123:10:311
583999221,15cyclictest9796-21ssh00:20:151
5839992210,9cyclictest15340-21perl00:05:181
583099224,14cyclictest22474-21ssh00:25:290
583099223,15cyclictest29556-21ssh23:55:130
583099222,16cyclictest4460-21ssh21:35:150
583099221,6cyclictest856-21ntp_192.168.11422:25:240
585899213,12cyclictest0-21swapper/323:40:093
5858992115,4cyclictest0-21swapper/322:15:563
5858992115,4cyclictest0-21swapper/322:10:133
5858992115,4cyclictest0-21swapper/321:40:233
5858992112,6cyclictest0-21swapper/323:36:553
5858992112,6cyclictest0-21swapper/323:30:303
5858992112,6cyclictest0-21swapper/322:25:133
585099219,5cyclictest10769-21ssh00:37:162
585099218,10cyclictest22032-21kthreadcore19:25:202
585099216,10cyclictest0-21swapper/219:45:132
585099213,9cyclictest9980-21kthreadcore20:20:202
585099211,16cyclictest3380-21ls20:15:012
585099211,10cyclictest29289-21ssh22:05:242
583999212,14cyclictest24821-21ssh21:26:541
583999212,14cyclictest24821-21ssh21:26:541
5839992113,5cyclictest14206-21usage_Anybus22:50:301
583999211,18cyclictest13505-21ssh21:40:131
583999211,17cyclictest20579-21ssh23:50:131
583999211,13cyclictest91-21pr/ttyS019:40:011
5839992110,8cyclictest28409-21sed22:40:311
583099212,16cyclictest27795-21ssh00:30:130
583099212,14cyclictest2559-21ssh23:40:130
5830992116,3cyclictest31529-21ssh21:50:140
5830992115,3cyclictest0-21swapper/023:10:140
583099211,17cyclictest895-21ssh22:45:140
583099211,17cyclictest895-21ssh22:45:140
583099211,17cyclictest15304-21ssh22:35:110
585899202,12cyclictest35966irq/58-eth023:10:143
5858992014,4cyclictest0-21swapper/321:55:213
5858992014,4cyclictest0-21swapper/319:25:153
5858992014,4cyclictest0-21swapper/300:10:283
5858992013,4cyclictest0-21swapper/321:05:203
5858992013,4cyclictest0-21swapper/320:15:133
5858992012,5cyclictest0-21swapper/321:45:133
5858992012,5cyclictest0-21swapper/321:00:243
585099209,9cyclictest3606-21ssh23:57:302
585099203,14cyclictest23919-21latency_hist21:10:012
585099202,9cyclictest23395-21ps20:35:182
585099202,9cyclictest15043-21latency_hist22:35:022
585099201,9cyclictest32644-21fschecks_time20:45:152
585099201,9cyclictest31068-21latency_hist20:10:022
585099201,7cyclictest16004-21irqcore19:20:182
585099201,15cyclictest8202-21df_inode21:55:162
585099201,10cyclictest5888-21ntp_offset19:40:242
585099201,10cyclictest27485-21kthreadcore19:30:222
585099201,10cyclictest14748-21fschecks_time21:00:172
583999202,14cyclictest16822-21apt-get19:55:131
583999202,14cyclictest16822-21apt-get19:55:131
583999202,12cyclictest23910-21diskmemload00:35:141
5839992013,4cyclictest0-21swapper/119:50:201
5839992012,5cyclictest0-21swapper/120:15:151
583999201,16cyclictest31557-21df_inode23:20:141
5839992011,6cyclictest17651-21sed23:30:191
5839992011,6cyclictest0-21swapper/100:03:111
5839992011,5cyclictest2764-21ntp_states21:50:271
583099206,2cyclictest131rcu_preempt20:35:220
5830992015,3cyclictest0-21swapper/020:30:120
5830992013,4cyclictest20298-21latency_hist22:20:000
583099201,16cyclictest22525-21sh21:45:140
583099201,16cyclictest15104-21ssh00:05:150
5858991913,4cyclictest0-21swapper/322:50:173
5858991913,4cyclictest0-21swapper/322:05:203
5858991913,4cyclictest0-21swapper/320:30:173
5858991913,4cyclictest0-21swapper/320:10:143
5858991913,4cyclictest0-21swapper/319:15:193
5858991912,5cyclictest0-21swapper/321:25:143
5858991912,5cyclictest0-21swapper/321:25:143
5858991912,5cyclictest0-21swapper/319:50:283
5858991912,5cyclictest0-21swapper/319:30:173
5858991912,5cyclictest0-21swapper/319:10:273
585099191,8cyclictest12170-21fschecks_time19:50:152
5850991912,4cyclictest27-21ksoftirqd/220:55:142
5850991912,4cyclictest27-21ksoftirqd/220:55:142
583999199,3cyclictest131rcu_preempt23:00:021
583999199,2cyclictest0-21swapper/122:16:361
583999193,9cyclictest0-21swapper/122:20:081
5839991912,4cyclictest0-21swapper/123:25:301
583999191,16cyclictest20809-21kthreadcore21:05:191
5839991910,7cyclictest21625-21apt-get20:00:131
5839991910,6cyclictest11777-21cron19:50:011
583099192,14cyclictest13389-21memory21:20:250
583099192,13cyclictest18380-21users21:40:320
583099191,7cyclictest16606-21latency_hist23:30:020
5830991912,4cyclictest23448-21cut23:15:200
5830991911,6cyclictest6988-21sshd19:43:090
5830991910,6cyclictest0-21swapper/020:05:290
583099190,13cyclictest24405-21ls21:10:150
585899182,10cyclictest0-21swapper/320:00:143
5858991812,4cyclictest0-21swapper/320:45:263
5858991812,4cyclictest0-21swapper/320:35:213
5858991812,4cyclictest0-21swapper/320:25:173
5858991812,4cyclictest0-21swapper/319:45:253
5858991812,4cyclictest0-21swapper/319:40:273
5858991812,4cyclictest0-21swapper/319:35:203
5858991811,5cyclictest0-21swapper/319:20:273
585099181,8cyclictest5771-21latency_hist19:10:012
583999189,5cyclictest23126-21usage_Anybus00:25:331
583999188,8cyclictest10842-21apt-get19:15:131
583999187,8cyclictest26106-21cut21:10:211
583999185,4cyclictest25613-21kworker/1:100:30:141
583999182,13cyclictest32432-21latency_hist20:45:021
583999181,15cyclictest6164-21uniq19:10:151
583999181,15cyclictest32598-21fschecks_count20:45:151
583999181,14cyclictest27680-21kthreadcore20:05:191
583999181,13cyclictest23910-21diskmemload22:00:261
5839991810,6cyclictest12963-21cron20:25:011
583099183,7cyclictest0-21swapper/020:55:140
583099183,7cyclictest0-21swapper/020:55:140
583099183,10cyclictest0-21swapper/020:25:040
583099182,14cyclictest16958-21sh23:30:140
583099181,6cyclictest621-21kthreadcore20:10:210
583099181,14cyclictest8596-21fw_packets20:20:160
583099181,12cyclictest24192-21ssh21:26:090
583099181,12cyclictest24192-21ssh21:26:090
5830991810,6cyclictest30016-21sshd19:35:080
5830991810,6cyclictest22831-21pluginstate21:05:260
5858991711,4cyclictest0-21swapper/320:40:203
5858991711,4cyclictest0-21swapper/320:05:233
583999178,6cyclictest27794-21/usr/sbin/munin21:30:201
583999177,7cyclictest2585-21cut22:45:211
583999177,7cyclictest2585-21cut22:45:211
583999175,9cyclictest6285-21kthreadcore23:05:231
583999175,7cyclictest22479-21ssh23:15:141
583999171,6cyclictest20442-21/usr/sbin/munin19:25:121
5839991711,4cyclictest1440-21kernelversion00:15:191
5839991711,4cyclictest1440-21kernelversion00:15:191
5839991710,5cyclictest6059-21latency23:40:231
583099179,6cyclictest29071-21grep19:30:300
583099179,6cyclictest24982-21sshd19:28:070
583099175,9cyclictest8139-21kthreadcore19:45:190
583099174,10cyclictest28803-21kthreadcore20:40:180
583099171,4cyclictest19054-21ntp_192.168.11419:20:240
583099171,12cyclictest3681-21diskstats20:15:130
5830991710,4cyclictest19103-21idleruntime-cro21:05:010
5858991610,4cyclictest0-21swapper/319:55:153
5858991610,4cyclictest0-21swapper/319:55:153
583999168,5cyclictest21937-21unixbench_multi20:30:281
583999168,5cyclictest13491-21hwlatdetect20:25:161
583999162,9cyclictest0-21swapper/120:50:131
583999162,10cyclictest0-21swapper/119:20:141
583999161,13cyclictest25754-21sed19:30:191
583999161,12cyclictest3359-21munin-run20:15:011
583999161,12cyclictest10121-21irqcore20:55:191
583999161,12cyclictest10121-21irqcore20:55:181
583099165,8cyclictest12-21ksoftirqd/020:50:130
583099164,9cyclictest2015-21kthreadcore20:45:190
583099164,10cyclictest12176-21fschecks_time19:50:150
583099161,7cyclictest53646usb-storage22:30:040
583099161,11cyclictest6134-21cut19:10:150
583999141,9cyclictest5915-21latency_hist19:10:011
583099146,6cyclictest5926-21cat19:10:000
585899131,8cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional