You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 12:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun May 03, 2026 00:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1493199672,51cyclictest27335-21ssh23:30:121
1493199603,45cyclictest18266-21ssh23:25:131
1493199603,45cyclictest18266-21ssh23:25:131
1492799573,41cyclictest23491-21ssh23:10:130
1493999541,37cyclictest28865-21ssh22:55:132
1493199523,40cyclictest9171-21ssh23:20:121
1493199491,38cyclictest12969-21ssh23:40:111
1493999482,26cyclictest21414-21ssh22:15:142
1493199481,37cyclictest26733-21ssh22:00:131
14927994725,12cyclictest131rcu_preempt21:00:140
1493199465,31cyclictest14746-21ssh00:35:131
1494899452,25cyclictest35966irq/58-eth000:15:133
1493999450,38cyclictest0-21swapper/219:30:142
1493199452,36cyclictest17661-21ssh21:55:131
1492799445,5cyclictest131rcu_preempt20:20:130
1493199432,30cyclictest31005-21ssh23:50:131
1492799436,23cyclictest12-21ksoftirqd/020:15:130
1492799431,33cyclictest12974-21ssh23:40:120
14948994217,16cyclictest35966irq/58-eth023:50:153
1493199421,35cyclictest1596-21ssh22:40:121
1493199402,26cyclictest21995-21ssh23:45:131
1492799384,27cyclictest12-21ksoftirqd/021:10:110
1493199374,18cyclictest22975-21ssh21:40:121
1492799374,26cyclictest13909-21ssh21:35:150
14927993724,8cyclictest16015-21sh22:30:140
1492799361,24cyclictest25617-21ssh00:05:130
1494899352,22cyclictest35966irq/58-eth022:55:133
14931993516,14cyclictest0-21swapper/120:30:131
1494899342,17cyclictest35966irq/58-eth023:55:133
1493999346,19cyclictest27346irq/40-dwc2_hso20:45:142
1493999346,19cyclictest27346irq/40-dwc2_hso20:45:142
1493999344,13cyclictest630-21Async09:36:312
1494899332,15cyclictest35966irq/58-eth022:30:123
1494899331,19cyclictest35966irq/58-eth021:40:113
1493999339,17cyclictest27-21ksoftirqd/223:55:132
1493999331,16cyclictest23507-21ssh23:10:142
1492799337,20cyclictest12-21ksoftirqd/021:05:130
1492799334,25cyclictest5785-21ssh00:30:140
1492799323,23cyclictest28125-21ssh21:25:130
14948993119,8cyclictest35966irq/58-eth021:50:153
14939993112,11cyclictest0-21swapper/219:10:112
1493199311,23cyclictest0-21swapper/119:20:121
1492799317,17cyclictest12-21ksoftirqd/019:10:130
1492799313,20cyclictest12-21ksoftirqd/022:20:130
1493999303,17cyclictest0-21swapper/219:15:132
1493999303,17cyclictest0-21swapper/219:15:122
1493999303,12cyclictest3315-21ssh22:05:132
1493199303,17cyclictest29222-21ssh00:25:131
1494899294,21cyclictest35966irq/58-eth021:20:123
1494899294,21cyclictest35966irq/58-eth021:20:113
1493999293,19cyclictest16009-21ssh22:30:122
14939992914,9cyclictest27-21ksoftirqd/220:25:102
1493999291,14cyclictest923-21sh21:10:132
1493199292,19cyclictest25024-21ssh22:35:141
1493199291,21cyclictest8634-21rm21:50:151
1492799292,19cyclictest0-21swapper/022:05:130
1492799291,23cyclictest0-21swapper/019:25:120
1494899283,16cyclictest35966irq/58-eth021:10:123
1494899282,18cyclictest35966irq/58-eth022:35:113
1493999281,21cyclictest13918-21ssh21:35:152
1493999281,20cyclictest19773-21ssh22:50:132
1493199284,16cyclictest5432-21ssh23:00:111
14931992817,6cyclictest3951-21ssh23:35:131
1492799281,21cyclictest2260-21rm00:10:130
1492799281,17cyclictest32491-21ssh23:15:100
1494899274,20cyclictest35966irq/58-eth000:10:133
1494899272,21cyclictest35966irq/58-eth022:20:143
1493999272,3cyclictest0-21swapper/223:35:312
1493999272,11cyclictest0-21swapper/200:05:152
1493999271,4cyclictest0-21swapper/220:15:152
1493999271,18cyclictest32510-21ssh23:15:132
14939992711,6cyclictest2256-21sh00:10:132
1493999271,14cyclictest11255-21ssh00:15:142
1493199275,14cyclictest11249-21ssh00:15:131
1493199273,19cyclictest32514-21sh23:15:131
1493199273,17cyclictest488-21diskmemload00:20:141
1492799276,16cyclictest12-21ksoftirqd/021:30:120
1494899262,20cyclictest35966irq/58-eth023:10:143
1494899261,20cyclictest35966irq/58-eth023:30:133
1493999267,11cyclictest12731-21irqcore22:10:172
1493999261,19cyclictest0-21swapper/219:25:122
1493999261,16cyclictest27346irq/40-dwc2_hso20:10:112
1493199261,19cyclictest7567-21rm23:55:141
1492799262,19cyclictest151rcuc/020:25:090
1494899256,15cyclictest35966irq/58-eth022:50:123
1494899253,13cyclictest35966irq/58-eth021:25:123
1493999258,14cyclictest507-21kthreadcore23:50:232
1493999255,14cyclictest8633-21sh21:50:142
14939992517,5cyclictest15865-21ssh00:35:192
1493999251,4cyclictest30535-21processes22:00:262
1493999251,15cyclictest0-21swapper/223:07:222
14927992520,2cyclictest12-21ksoftirqd/022:40:120
1494899242,14cyclictest35966irq/58-eth023:00:113
1494899241,10cyclictest35966irq/58-eth019:15:133
1494899241,10cyclictest35966irq/58-eth019:15:123
1493999248,2cyclictest131rcu_preempt23:25:112
1493999248,2cyclictest131rcu_preempt23:25:102
1493999247,2cyclictest21064-21cpu19:50:132
1493999242,9cyclictest0-21swapper/220:30:142
1493999242,18cyclictest0-21swapper/221:00:142
14939992412,9cyclictest24500-21ssh21:57:512
1493999241,16cyclictest27346irq/40-dwc2_hso21:05:142
14939992411,10cyclictest26534-21cat23:45:302
1493199243,15cyclictest0-21swapper/120:35:141
1494899230,14cyclictest35966irq/58-eth023:35:123
1493999235,12cyclictest27-21ksoftirqd/221:45:192
1493999232,10cyclictest19050-21ssh21:20:132
1493999232,10cyclictest19050-21ssh21:20:122
1493999231,7cyclictest11080-21grep22:25:292
1493999231,4cyclictest14250-21modprobe20:50:182
14939992312,9cyclictest19344-21ssh23:42:202
14939992312,8cyclictest20651-21interrupts00:20:182
14939992312,3cyclictest23119-21fschecks_count21:40:142
14939992311,9cyclictest11801-21ssh23:02:162
1493999231,15cyclictest0-21swapper/220:20:112
1493199232,16cyclictest32025-21ssh21:45:131
1493999222,15cyclictest5783-21sh00:30:132
1493999221,9cyclictest0-21swapper/222:20:132
14939992213,7cyclictest0-21swapper/219:20:162
1493999221,17cyclictest3126-21timerandwakeup20:35:292
1493999221,16cyclictest28662-21sh21:25:182
1493999221,16cyclictest17819-21ssh00:00:202
1493999221,14cyclictest0-21swapper/200:25:132
14939992210,9cyclictest23312-21cron21:00:022
1493199222,17cyclictest12359-21rm22:10:141
1492799227,12cyclictest12-21ksoftirqd/021:20:110
1492799227,12cyclictest12-21ksoftirqd/021:20:110
1492799222,16cyclictest12357-21ssh22:10:140
1492799221,8cyclictest27324-21apt-get23:30:130
14927992214,5cyclictest0-21swapper/020:50:230
1492799221,3cyclictest131rcu_preempt21:55:530
14927992212,4cyclictest121ksoftirqd/019:30:140
1492799221,17cyclictest9292-21grep20:45:170
1492799221,17cyclictest9292-21grep20:45:160
1492799221,15cyclictest32082-21ls20:35:160
1494899213,12cyclictest35966irq/58-eth000:30:123
1494899213,12cyclictest0-21swapper/323:20:123
14948992111,7cyclictest35966irq/58-eth023:25:123
14948992111,7cyclictest35966irq/58-eth023:25:113
1494899211,17cyclictest35966irq/58-eth019:45:143
1494899211,14cyclictest0-21swapper/320:55:153
1493999218,8cyclictest0-21swapper/219:35:122
1493999214,6cyclictest131rcu_preempt21:18:022
1493999211,12cyclictest25576-21ssh22:35:182
1493999211,12cyclictest1595-21ssh22:40:112
1493999211,11cyclictest9282-21diskstats23:20:152
1493999211,10cyclictest8529-21ntp_states21:30:272
1493199219,9cyclictest10731-21ssh21:15:191
1493199215,8cyclictest0-21swapper/120:40:101
1493199214,6cyclictest131rcu_preempt21:20:191
1493199214,6cyclictest131rcu_preempt21:20:191
1493199211,17cyclictest29038-21cut22:55:171
1493199211,15cyclictest25758-21cut00:05:151
1493199210,13cyclictest28359-21find21:05:121
1492799219,6cyclictest0-21swapper/019:15:140
1492799219,6cyclictest0-21swapper/019:15:140
1492799213,14cyclictest23700-21cat22:50:280
1492799212,16cyclictest25022-21sh22:35:130
1492799211,8cyclictest21903-21ntp_192.168.11423:25:240
1492799211,8cyclictest21903-21ntp_192.168.11423:25:230
1494899209,6cyclictest0-21swapper/300:25:483
1494899203,12cyclictest35966irq/58-eth021:05:133
1494899202,10cyclictest35966irq/58-eth022:05:123
1494899201,10cyclictest0-21swapper/321:35:153
1493999207,10cyclictest30753-21df20:00:132
1493999202,9cyclictest16406-21fschecks_time19:45:182
1493999201,9cyclictest14999-21find19:40:272
1493999201,9cyclictest14999-21find19:40:262
1493999201,6cyclictest488-21diskmemload22:47:302
1493199205,9cyclictest0-21swapper/121:25:111
1493199202,14cyclictest7000-21ssh22:25:131
1493199201,12cyclictest912-21ssh21:10:121
1492799203,12cyclictest12-21ksoftirqd/022:45:120
1492799202,13cyclictest488-21diskmemload22:55:040
1492799202,13cyclictest12424-21users23:55:310
14927992014,4cyclictest0-21swapper/000:20:160
14927992011,6cyclictest18995-21sh00:35:290
1492799201,16cyclictest11040-21ls22:25:280
1492799201,16cyclictest10001-21ssh21:15:140
1492799201,14cyclictest15042-21ps00:15:280
1492799201,11cyclictest1869-21cat20:00:280
1492799201,10cyclictest0-21swapper/023:51:430
14927992010,6cyclictest22013-21irqrtprio22:15:190
14927992010,6cyclictest12-21ksoftirqd/020:05:130
14927992010,6cyclictest12-21ksoftirqd/020:05:120
1494899199,6cyclictest0-21swapper/323:40:093
1494899192,6cyclictest0-21swapper/300:35:143
1494899192,11cyclictest0-21swapper/320:45:053
1494899192,11cyclictest0-21swapper/320:45:043
1494899191,11cyclictest35966irq/58-eth023:45:123
14948991910,6cyclictest0-21swapper/322:10:403
1493999196,5cyclictest14998-21cut19:10:022
1493999191,9cyclictest27934-21kthreadcore19:55:212
1493999191,9cyclictest135-21systemd-journal20:10:012
1493999191,9cyclictest135-21systemd-journal20:10:012
1493999191,8cyclictest5903-21kthreadcore20:40:182
1493199199,7cyclictest15092-21cat21:35:201
1493199193,11cyclictest12656-21apt-get20:15:101
14931991913,4cyclictest0-21swapper/122:55:011
1493199191,15cyclictest16610-21sh00:00:141
14931991910,6cyclictest17880-21kthreadcore23:05:231
1492799199,8cyclictest12390-21users20:10:290
1492799198,8cyclictest14554-21sh23:05:130
1492799194,11cyclictest12-21ksoftirqd/019:45:140
1492799191,6cyclictest22678-21irqstats23:45:170
1492799191,2cyclictest0-21swapper/023:35:040
14927991910,6cyclictest8029-21kthreadcore23:00:200
1494899189,6cyclictest0-21swapper/323:16:083
1494899188,8cyclictest0-21swapper/322:45:143
1494899182,10cyclictest0-21swapper/320:10:133
1494899182,10cyclictest0-21swapper/319:35:143
1494899181,8cyclictest35966irq/58-eth021:15:133
14948991812,4cyclictest0-21swapper/321:55:273
14948991811,5cyclictest0-21swapper/322:25:153
1494899181,14cyclictest35966irq/58-eth022:00:133
14948991811,4cyclictest0-21swapper/322:15:143
1494899181,13cyclictest0-21swapper/320:30:173
1494899181,12cyclictest0-21swapper/319:55:223
1494899181,11cyclictest0-21swapper/319:10:153
14948991810,6cyclictest0-21swapper/323:06:153
14948991810,6cyclictest0-21swapper/321:31:543
14948991810,6cyclictest0-21swapper/300:05:113
1493199189,6cyclictest11290-21ntpq20:10:241
1493199188,8cyclictest5883-21df_inode00:30:151
1493199187,8cyclictest19289-21cron22:50:011
1493199185,4cyclictest131rcu_preempt22:15:171
1493199182,14cyclictest763-21kthreadcore20:00:221
1493199182,10cyclictest24573-21ls19:20:021
1493199182,10cyclictest24573-21ls19:20:011
1492799187,8cyclictest2629-21cut21:45:220
14927991813,3cyclictest8873-21apt-config20:45:020
14927991811,5cyclictest0-21swapper/019:35:170
14927991811,4cyclictest29970-21kthreadcore00:25:190
1492799181,14cyclictest13972-21cut19:40:210
1492799181,14cyclictest13972-21cut19:40:210
1492799181,14cyclictest11597-21kthreadcore23:20:220
1492799181,13cyclictest30539-21awk22:00:270
1494899179,6cyclictest0-21swapper/300:00:073
1494899178,7cyclictest0-21swapper/300:24:133
1494899172,12cyclictest0-21swapper/321:45:133
1494899172,10cyclictest0-21swapper/322:40:133
1494899172,10cyclictest0-21swapper/320:15:123
1494899172,10cyclictest0-21swapper/320:00:143
1493199179,6cyclictest5706-21sh21:30:191
1493199178,6cyclictest6819-21expr22:05:221
1493199176,3cyclictest131rcu_preempt19:30:021
1493199175,8cyclictest1-21systemd22:32:511
1493199173,11cyclictest28297-21kthreadcore19:55:211
1493199172,10cyclictest0-21swapper/120:05:141
1493199172,10cyclictest0-21swapper/120:05:131
1493199171,3cyclictest0-21swapper/123:10:131
1493199171,13cyclictest30391-21apt-get22:20:131
1493199171,13cyclictest23341-21awk21:00:011
1492799179,5cyclictest25606-21cron19:55:010
1492799179,5cyclictest12248-21sh21:50:260
1492799173,11cyclictest23180-21kthreadcore19:50:210
1492799173,11cyclictest22297-21grep20:55:260
1492799172,12cyclictest23097-21diskstats21:40:140
1492799171,13cyclictest16601-21ssh00:00:120
1494899162,9cyclictest0-21swapper/319:40:143
1494899162,9cyclictest0-21swapper/319:40:133
14948991610,4cyclictest0-21swapper/320:35:293
1493199167,6cyclictest21-21ksoftirqd/119:10:191
1493199167,6cyclictest13010-21grep20:45:301
1493199167,6cyclictest13010-21grep20:45:291
1493199166,7cyclictest25109-21cut21:00:211
1493199163,10cyclictest24518-21kthreadcore20:25:181
1493199163,10cyclictest18493-21kthreadcore19:45:221
1493199162,11cyclictest2193-21irqcore19:30:171
1493199162,10cyclictest0-21swapper/119:40:141
1493199162,10cyclictest0-21swapper/119:40:131
1493199162,10cyclictest0-21swapper/119:35:141
1493199161,3cyclictest0-21swapper/120:50:141
1493199161,12cyclictest15034-21latency_hist19:10:011
1493199161,11cyclictest2473-21grep00:10:171
1493199161,10cyclictest21554-21irqrtprio19:50:181
1493199160,13cyclictest20177-21kthreadcore20:20:201
1492799162,11cyclictest27360-21kthreadcore19:20:220
1492799160,4cyclictest30614-21mailstats20:30:280
1492799160,13cyclictest14975-21latency_hist19:10:020
1494899152,11cyclictest0-21swapper/319:30:143
1494899151,7cyclictest0-21swapper/319:30:023
1494899149,3cyclictest0-21swapper/319:21:023
1494899144,6cyclictest35966irq/58-eth020:40:113
1494899138,3cyclictest0-21swapper/321:00:453
1494899138,3cyclictest0-21swapper/320:06:063
1494899138,3cyclictest0-21swapper/320:06:053
1494899131,5cyclictest0-21swapper/319:50:303
1494899131,4cyclictest0-21swapper/320:20:123
1494899125,5cyclictest0-21swapper/320:50:293
1494899121,7cyclictest0-21swapper/320:25:103
1494899101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional