You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 02:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Jun 11, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18323997224,9cyclictest12-21ksoftirqd/000:20:110
18339994614,29cyclictest27-21ksoftirqd/200:15:142
1832399461,33cyclictest32180-21ssh21:25:120
18339994410,25cyclictest27-21ksoftirqd/200:35:152
1834799439,20cyclictest35966irq/58-eth023:05:103
18339994316,16cyclictest27-21ksoftirqd/200:20:112
18347994227,7cyclictest35966irq/58-eth021:25:113
1834799413,21cyclictest35966irq/58-eth023:15:123
1833999401,18cyclictest26748-21ssh23:45:122
1832399400,31cyclictest29663-21ssh22:35:110
1833999393,25cyclictest261rcuc/219:15:122
1833999373,25cyclictest27-21ksoftirqd/223:50:122
1832399361,26cyclictest10223-21rm23:00:130
1834799352,18cyclictest35966irq/58-eth021:45:123
1832999352,27cyclictest19219-21ssh23:05:101
1832399352,24cyclictest12-21ksoftirqd/023:25:130
1832399352,18cyclictest11619-21ssh22:25:130
1834799341,19cyclictest35966irq/58-eth023:35:123
1833999345,20cyclictest630-21Async09:36:312
1834799333,19cyclictest35966irq/58-eth021:15:123
18339993324,4cyclictest8729-21ssh23:35:132
1833999332,19cyclictest24497-21ssh22:50:142
1833999332,19cyclictest24497-21ssh22:50:132
1833999331,15cyclictest27346irq/40-dwc2_hso23:30:112
1832399331,4cyclictest131rcu_preempt21:40:130
1834799324,18cyclictest35966irq/58-eth019:30:113
1834799322,17cyclictest35966irq/58-eth000:25:113
1832399322,23cyclictest4912-21ssh23:15:120
18323993221,8cyclictest12-21ksoftirqd/023:05:100
1832999311,22cyclictest7261-21ssh00:10:131
1834799303,14cyclictest35966irq/58-eth019:15:123
1833999309,16cyclictest14019-21sh23:20:132
1833999304,19cyclictest27346irq/40-dwc2_hso19:10:122
1833999304,18cyclictest0-21swapper/222:25:132
1834799292,18cyclictest35966irq/58-eth022:15:123
1833999294,14cyclictest27346irq/40-dwc2_hso21:50:122
1833999291,21cyclictest0-21swapper/220:50:132
1832999292,22cyclictest26752-21ssh23:45:121
1832399292,4cyclictest131rcu_preempt21:30:130
1834799283,13cyclictest35966irq/58-eth022:35:123
1834799282,17cyclictest35966irq/58-eth023:55:153
1833999283,15cyclictest27346irq/40-dwc2_hso19:55:132
1833999282,14cyclictest27346irq/40-dwc2_hso19:45:122
1833999281,22cyclictest32179-21sh21:25:132
1834799271,17cyclictest35966irq/58-eth021:10:133
1834799271,11cyclictest0-21swapper/320:45:133
1833999273,16cyclictest27346irq/40-dwc2_hso00:25:102
1832999272,20cyclictest21530-21apt-get00:00:111
1832399271,19cyclictest12497-21ssh23:55:150
1833999263,15cyclictest27346irq/40-dwc2_hso22:55:142
1833999262,18cyclictest7128-21ssh00:10:042
1833999262,15cyclictest27346irq/40-dwc2_hso20:20:122
1833999261,9cyclictest131rcu_preempt21:15:592
18339992617,3cyclictest0-21swapper/223:00:182
18339992612,11cyclictest0-21swapper/219:35:142
1832999264,19cyclictest6355-21ssh22:40:111
1832399266,14cyclictest30597-21ssh00:05:120
1832399262,19cyclictest16830-21ssh22:10:140
1834799251,15cyclictest35966irq/58-eth000:30:113
1834799251,15cyclictest35966irq/58-eth000:30:113
1833999252,17cyclictest3819-21ssh21:45:132
18339992516,6cyclictest28285-21ssh23:10:122
18339992514,5cyclictest0-21swapper/223:25:242
1833999251,21cyclictest6925-21ps22:40:182
1833999251,13cyclictest31116-21ls21:40:292
1833999251,12cyclictest30600-21sh00:05:122
1833999251,12cyclictest131rcu_preempt21:14:512
18329992516,6cyclictest0-21swapper/123:00:141
1834799243,12cyclictest35966irq/58-eth023:45:133
1834799242,13cyclictest35966irq/58-eth021:40:133
1833999247,9cyclictest1-21systemd00:33:442
1833999247,9cyclictest1-21systemd00:33:442
1833999243,10cyclictest0-21swapper/222:30:142
1833999242,13cyclictest21542-21ssh00:00:122
1833999241,3cyclictest0-21swapper/222:15:142
18339992413,9cyclictest2411-21ssh22:20:062
18329992419,3cyclictest0-21swapper/123:40:131
1832399246,6cyclictest10005-21ssh23:00:010
1832399241,19cyclictest4679-21ssh21:10:140
1832399241,15cyclictest12-21ksoftirqd/000:15:120
1834799232,13cyclictest35966irq/58-eth022:30:133
18347992311,8cyclictest35966irq/58-eth000:20:133
1833999236,8cyclictest18925-21unixbench_multi19:40:292
1833999232,17cyclictest21954-21ssh21:55:122
1833999231,16cyclictest26387-21kthreadcore21:20:212
1833999231,12cyclictest7334-21latency_hist22:05:012
1832999231,6cyclictest0-21swapper/122:05:131
1832999231,13cyclictest20866-21munin-run20:20:011
1834799223,12cyclictest0-21swapper/323:30:053
1834799222,13cyclictest36066irq/59-eth022:40:113
18347992211,7cyclictest35966irq/58-eth000:00:133
1833999227,12cyclictest1-21systemd22:36:302
1833999223,8cyclictest27-21ksoftirqd/223:15:032
1833999223,3cyclictest131rcu_preempt22:08:072
1833999222,17cyclictest19027-21grep22:45:242
1833999222,11cyclictest0-21swapper/219:20:122
18339992211,8cyclictest0-21swapper/223:59:252
1833999221,12cyclictest6841-21wc20:35:272
18339992210,9cyclictest0-21swapper/222:10:412
1832999221,17cyclictest22974-21ssh21:20:141
1832399221,18cyclictest28342-21ls23:10:120
1832399221,16cyclictest20656-21ssh22:30:130
1832399221,14cyclictest7742-21rm22:05:120
1832399220,16cyclictest32004-21ssh23:30:060
1834799213,13cyclictest0-21swapper/322:25:143
1834799213,11cyclictest0-21swapper/300:35:143
18347992114,4cyclictest0-21swapper/321:50:283
18347992111,7cyclictest35966irq/58-eth021:20:133
1834799211,12cyclictest35966irq/58-eth022:20:113
18347992110,8cyclictest0-21swapper/323:26:573
1833999219,3cyclictest27-21ksoftirqd/220:25:132
1833999217,5cyclictest12706-21cpu20:45:152
1833999214,14cyclictest24821-21fschecks_count19:50:152
1833999214,12cyclictest0-21swapper/221:35:132
1833999212,16cyclictest23693-21kthreadcore20:55:202
1833999212,15cyclictest27346irq/40-dwc2_hso21:00:122
1833999212,15cyclictest0-21swapper/223:40:142
1833999212,11cyclictest27346irq/40-dwc2_hso21:30:132
1833999211,17cyclictest21443-21cut23:05:202
1833999211,10cyclictest3010-21latency_hist20:35:012
1832999218,4cyclictest131rcu_preempt00:05:221
1832999215,8cyclictest131rcu_preempt00:20:271
1832999213,14cyclictest23075-21ssh23:25:141
1832999211,14cyclictest3505-21ssh23:50:131
1832399217,4cyclictest12-21ksoftirqd/021:05:190
18323992113,6cyclictest0-21swapper/021:37:370
18323992112,7cyclictest19643-21sshd19:43:370
1832399211,15cyclictest24480-21apt-get22:50:120
1832399211,15cyclictest24480-21apt-get22:50:120
1834799209,8cyclictest0-21swapper/322:08:503
1834799209,8cyclictest0-21swapper/300:08:223
18347992011,6cyclictest0-21swapper/321:35:133
18347992011,4cyclictest0-21swapper/322:48:283
1834799201,12cyclictest35966irq/58-eth022:50:143
1834799201,12cyclictest35966irq/58-eth022:50:133
1833999202,15cyclictest6804-21fschecks_count20:05:142
1833999201,8cyclictest0-21swapper/219:25:112
1833999201,15cyclictest16591-21ps20:15:182
1833999201,10cyclictest6422-21cron20:05:012
1833999201,10cyclictest12442-21kthreadcore20:10:202
1833999200,11cyclictest11174-21latency20:40:232
1832999207,3cyclictest87766lldpd22:25:581
1832999202,15cyclictest17952-21ssh21:35:121
1832999202,12cyclictest10269-21apt-get19:35:121
1832999201,8cyclictest1305-21cat22:35:291
1832999201,4cyclictest53646usb-storage21:05:041
18329992013,5cyclictest22476-21cron21:20:011
1832999201,17cyclictest21945-21ssh21:55:121
1832999201,16cyclictest31036-21ssh22:00:121
1832399202,3cyclictest0-21swapper/022:40:220
18323992015,3cyclictest19895-21ls00:35:020
18323992015,3cyclictest19895-21ls00:35:010
1832399201,16cyclictest16874-21ls21:50:290
1832399201,14cyclictest7244-21ssh00:10:100
1832399201,13cyclictest844-21munin-node19:30:120
18323992010,7cyclictest1-21systemd23:41:450
1832399200,16cyclictest30971-21ls21:00:280
1834799193,10cyclictest0-21swapper/320:05:133
1834799192,9cyclictest35966irq/58-eth023:10:113
1834799191,13cyclictest0-21swapper/319:50:023
1834799191,13cyclictest0-21swapper/319:25:143
18347991910,6cyclictest0-21swapper/323:52:013
18347991910,6cyclictest0-21swapper/323:03:473
1833999191,9cyclictest18396-21latency_hist19:10:022
1833999191,14cyclictest1420-21kthreadcore21:05:222
1832999195,3cyclictest131rcu_preempt21:35:011
1832999192,14cyclictest25931-21ssh22:15:141
1832999192,13cyclictest14129-21diskstats23:20:141
1832999191,9cyclictest0-21swapper/100:31:091
1832999191,9cyclictest0-21swapper/100:31:081
18329991913,3cyclictest0-21swapper/122:45:111
18329991911,5cyclictest20215-21sh00:35:141
1832999191,14cyclictest5489-21apt-get19:30:111
1832999191,13cyclictest16191-21ntp_states23:55:271
1832399199,7cyclictest11552-21df_inode20:10:150
1832399193,12cyclictest0-21swapper/020:30:100
1832399191,5cyclictest131rcu_preempt19:25:160
18323991912,4cyclictest7832-21usage_Anybus23:50:300
1832399191,15cyclictest7961-21grep21:45:290
1834799189,6cyclictest35966irq/58-eth022:00:123
1834799189,6cyclictest0-21swapper/323:42:373
1834799182,10cyclictest0-21swapper/322:55:133
1834799182,10cyclictest0-21swapper/322:10:143
1834799182,10cyclictest0-21swapper/321:05:043
1834799182,10cyclictest0-21swapper/320:35:123
1834799182,10cyclictest0-21swapper/320:30:123
1834799182,10cyclictest0-21swapper/320:15:153
1834799182,10cyclictest0-21swapper/319:50:133
1834799182,10cyclictest0-21swapper/319:40:133
1834799182,10cyclictest0-21swapper/319:20:123
1834799181,12cyclictest35966irq/58-eth021:30:143
18347991810,3cyclictest0-21swapper/321:56:433
1832999188,7cyclictest27151-21cut21:00:011
1832999188,7cyclictest14999-21cut21:50:201
1832999187,8cyclictest399-21tail23:30:181
1832999186,5cyclictest131rcu_preempt20:30:111
1832999184,4cyclictest21-21ksoftirqd/123:36:161
1832999183,12cyclictest32237-21sort21:25:131
1832999183,10cyclictest0-21swapper/122:10:141
1832999182,13cyclictest1183-21ssh22:55:131
1832999181,9cyclictest31100-21unixbench_singl21:00:291
1832999181,9cyclictest28525-21unixbench_singl19:50:271
1832999181,7cyclictest31250-21grep21:40:301
1832999181,7cyclictest2618-21ssh00:25:161
18329991812,4cyclictest0-21swapper/120:45:241
18329991812,3cyclictest0-21swapper/119:40:131
1832999181,14cyclictest23753-21grep19:45:281
18329991810,6cyclictest24909-21cat22:30:291
1832399186,3cyclictest131rcu_preempt19:15:010
1832399185,5cyclictest12-21ksoftirqd/023:40:010
1832399185,4cyclictest131rcu_preempt21:25:010
1832399181,8cyclictest28953-21kthreadcore23:45:210
1832399181,4cyclictest0-21swapper/023:20:140
1832399181,14cyclictest13769-21ssh21:15:120
18323991810,6cyclictest167-21systemd-network22:22:530
18323991810,5cyclictest2906-21usage_Anybus22:00:300
1834799179,3cyclictest0-21swapper/300:10:103
1834799172,10cyclictest0-21swapper/323:20:143
1834799172,10cyclictest0-21swapper/320:10:143
1834799172,10cyclictest0-21swapper/319:35:123
1834799171,13cyclictest0-21swapper/321:05:013
1834799171,13cyclictest0-21swapper/319:55:293
18347991711,3cyclictest0-21swapper/300:15:243
1834799171,12cyclictest0-21swapper/320:55:163
1832999179,5cyclictest3864-21cpu21:45:131
1832999176,7cyclictest18308-21cyclictest20:40:131
1832999174,10cyclictest0-21swapper/120:15:011
1832999173,2cyclictest4277-21diskmemload00:15:581
1832999172,12cyclictest31902-21ssh23:10:231
1832999172,12cyclictest2107-21grep20:00:161
1832999172,11cyclictest4787-21diskstats21:10:151
1832999171,7cyclictest22119-21cat20:55:011
1832999171,14cyclictest28755-21kthreadcore20:25:211
18329991711,4cyclictest0-21swapper/120:35:251
1832999171,13cyclictest6121-21kthreadcore23:15:181
1832999171,10cyclictest11416-21ls20:10:011
1832399178,7cyclictest25802-21awk20:55:260
1832399178,7cyclictest16367-21diskstats20:15:150
1832399178,6cyclictest14254-21cut20:45:200
1832399174,9cyclictest0-21swapper/020:35:120
1832399174,10cyclictest29819-21irqcore19:55:190
1832399172,10cyclictest0-21swapper/020:50:130
1832399171,8cyclictest23409-21tr21:55:190
1832399171,6cyclictest4277-21diskmemload22:15:310
1832399171,5cyclictest21512-21acpi00:00:110
1832399171,5cyclictest19042-21ntp_states22:45:250
1832399171,4cyclictest24794-21kthreadcore19:15:190
18323991712,3cyclictest536-21usb-storage19:20:100
1832399171,13cyclictest20390-21ssh00:35:160
18323991710,5cyclictest536-21usb-storage00:25:110
1832399170,4cyclictest131rcu_preempt19:45:160
1834799162,9cyclictest0-21swapper/320:20:123
1834799161,10cyclictest0-21swapper/320:50:133
1832999164,9cyclictest24597-21grep20:20:251
1832999164,9cyclictest18487-21apt-get19:10:111
1832999163,10cyclictest1345-21kthreadcore19:25:181
1832999161,3cyclictest131rcu_preempt19:20:101
1832999161,12cyclictest26797-21ntpq19:15:251
1832999161,11cyclictest6276-21ntp_states22:20:251
1832999161,11cyclictest29533-21ls19:55:141
1832999161,11cyclictest24028-21apt-get22:50:031
1832999161,11cyclictest24028-21apt-get22:50:031
1832399168,6cyclictest10767-21cut19:35:170
1832399168,5cyclictest131rcu_preempt20:40:120
1832399166,7cyclictest25601-21kthreadcore19:50:180
1832399163,10cyclictest5360-21grep20:00:270
1832399161,13cyclictest9532-21kthreadcore20:05:200
1832399161,12cyclictest29220-21wc20:25:230
1834799157,5cyclictest35966irq/58-eth020:25:133
18347991510,3cyclictest0-21swapper/319:13:373
1832399152,9cyclictest0-21swapper/020:20:120
1834799145,6cyclictest0-21swapper/320:40:163
1832399141,10cyclictest18356-21cat19:10:020
1832999136,5cyclictest18372-21sed19:10:031
1834799111,7cyclictest0-21swapper/320:00:203
1834799111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional