You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 09:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Mar 31, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
770099614,44cyclictest14671-21ssh22:15:140
770099576,6cyclictest131rcu_preempt23:05:150
774299521,32cyclictest924-21snmpd21:05:142
770099515,40cyclictest30158-21ssh23:35:150
774299501,30cyclictest20859-21ssh21:25:132
770099492,33cyclictest5120-21ssh00:15:120
7742994732,10cyclictest26343-21apt-get23:15:122
770099471,39cyclictest13329-21ssh22:50:130
7700994642,2cyclictest0-21swapper/021:10:050
770099454,34cyclictest27592-21ssh22:40:120
770099433,31cyclictest2992-21sh23:20:100
774299422,30cyclictest23761-21ssh22:20:122
774299422,18cyclictest2852366ssh00:10:142
7742994220,12cyclictest22346irq/40-dwc2_hso19:40:142
7754994113,19cyclictest28366irq/58-eth021:15:153
770099412,27cyclictest0-21swapper/019:40:130
770099411,36cyclictest15802-21ssh21:40:150
774299402,21cyclictest16783-21ssh22:33:282
770099401,29cyclictest5608-21ssh22:10:170
7754993920,12cyclictest28366irq/58-eth021:20:153
773299391,34cyclictest384-21ssh22:25:131
773299391,34cyclictest384-21ssh22:25:131
770099392,28cyclictest24793-21sh23:50:130
774299382,30cyclictest21264-21ssh22:17:312
7742993819,10cyclictest10377-21apt-get00:00:122
7742993815,19cyclictest0-21swapper/220:55:122
7732993825,9cyclictest0-21swapper/121:30:121
773299381,26cyclictest26344-21ssh23:15:121
7754993721,10cyclictest28366irq/58-eth023:25:153
7742993714,18cyclictest0-21swapper/223:45:142
773299376,25cyclictest21-21ksoftirqd/121:10:141
7732993716,18cyclictest21-21ksoftirqd/120:35:131
773299371,29cyclictest10389-21ssh00:00:141
7700993714,17cyclictest12-21ksoftirqd/020:15:130
775499363,19cyclictest0-21swapper/322:50:133
7742993623,9cyclictest29457-21apt-get20:40:132
7742993611,12cyclictest22707-1kworker/u9:021:45:142
7742993610,19cyclictest0-21swapper/200:20:132
773299361,28cyclictest2993-21ssh23:20:111
770099363,28cyclictest1565-21ssh21:50:120
770099361,26cyclictest23759-21sh22:20:120
775499358,18cyclictest28366irq/58-eth022:25:153
775499358,18cyclictest28366irq/58-eth022:25:143
775499351,21cyclictest28366irq/58-eth000:00:133
774299353,25cyclictest0-21swapper/221:57:282
774299352,28cyclictest924-21snmpd22:08:272
7742993519,11cyclictest15801-21sh21:40:142
7742993512,19cyclictest0-21swapper/223:05:142
7754993414,11cyclictest28366irq/58-eth020:55:133
774299346,10cyclictest0-21swapper/222:25:142
774299346,10cyclictest0-21swapper/222:25:132
774299345,14cyclictest5948-21grep00:15:192
7742993413,17cyclictest0-21swapper/221:00:122
774299341,14cyclictest24779-21ssh23:50:122
7732993418,12cyclictest21-21ksoftirqd/123:55:161
7732993413,18cyclictest21-21ksoftirqd/120:55:131
770099342,24cyclictest11816-21ssh21:20:140
774299339,9cyclictest8972-21ssh21:17:122
774299335,12cyclictest0-21swapper/220:05:282
774299335,12cyclictest0-21swapper/220:05:282
774299334,14cyclictest31892-21cpu19:35:132
774299333,8cyclictest0-21swapper/222:00:282
774299332,26cyclictest26303-21ssh00:07:532
7742993315,13cyclictest0-21swapper/220:30:132
773299332,25cyclictest31469-21ssh23:00:121
770099334,21cyclictest19799-21ssh22:00:140
7700993319,9cyclictest25557-21diskmemload23:45:150
775499324,19cyclictest28366irq/58-eth021:45:133
774299329,7cyclictest21041-21mailstats19:20:272
774299323,21cyclictest1401-21ssh23:55:172
7742993211,17cyclictest14284-21irqcore19:50:182
770099329,9cyclictest12-21ksoftirqd/023:40:130
7742993120,8cyclictest0-21swapper/223:00:122
7742993111,15cyclictest924-21snmpd20:50:122
773299314,25cyclictest13319-21ssh22:50:131
7732993115,6cyclictest21-21ksoftirqd/121:52:321
7732993113,14cyclictest21-21ksoftirqd/121:25:121
770099318,7cyclictest131rcu_preempt21:27:250
770099312,23cyclictest9010-21ssh00:35:150
774299303,11cyclictest0-21swapper/221:20:332
7742993017,3cyclictest7861-21apt-get19:10:132
7742993014,10cyclictest0-21swapper/220:20:122
7742993013,13cyclictest0-21swapper/221:10:132
773299301,25cyclictest26151-21kworker/u8:221:45:131
770099303,19cyclictest19491-21ssh00:05:160
7700993023,3cyclictest12-21ksoftirqd/023:10:150
770099302,20cyclictest23228-21ssh00:25:130
770099302,20cyclictest16002-21ssh22:32:340
775499293,22cyclictest28366irq/58-eth022:30:143
7754992920,6cyclictest28366irq/58-eth000:05:163
7754992913,11cyclictest28366irq/58-eth021:10:263
774299296,9cyclictest5333-21/usr/sbin/munin20:45:262
774299294,13cyclictest154166/usr/sbin/munin21:50:152
774299292,20cyclictest0-21swapper/220:10:142
774299292,20cyclictest0-21swapper/200:25:132
7742992915,12cyclictest23235-21kthreadcore19:25:192
7742992911,9cyclictest27652-21irqrtprio19:30:192
7742992911,14cyclictest3672-21irqrtprio23:20:152
7742992910,14cyclictest2138-21kthreadcore00:30:192
773299293,19cyclictest201rcuc/122:30:131
773299292,21cyclictest1810-21ssh22:42:351
770099292,19cyclictest4260-21ssh22:45:140
770099291,20cyclictest28524-21ssh00:10:140
775499283,17cyclictest0-21swapper/322:37:343
775499283,16cyclictest0-21swapper/322:57:373
775499281,21cyclictest28366irq/58-eth022:05:143
774299288,7cyclictest7641-21cyclictest22:45:122
774299283,10cyclictest0-21swapper/223:12:292
774299282,12cyclictest25557-21diskmemload21:39:062
7742992814,12cyclictest12453-21ntp_192.168.11419:45:242
7742992813,8cyclictest1658-21/usr/sbin/munin23:35:312
7742992812,13cyclictest7726-21tr20:15:212
7742992812,12cyclictest15321-21/usr/sbin/munin20:25:182
7742992812,12cyclictest0-21swapper/200:35:132
7742992811,14cyclictest22626-21head22:35:292
7742992810,9cyclictest6084-21cut22:10:212
770099282,19cyclictest3712-21cat23:00:290
775499276,11cyclictest28366irq/58-eth022:40:133
774299279,16cyclictest1-21systemd22:52:372
774299276,7cyclictest27159-21sh23:31:542
774299275,11cyclictest0-21swapper/222:40:392
774299273,12cyclictest0-21swapper/223:40:132
7742992717,8cyclictest0-21swapper/222:55:142
7742992710,14cyclictest4954-21ssh21:33:302
7742992710,13cyclictest24719-21df_inode20:35:152
773299272,19cyclictest23811-21timerwakeupswit22:00:281
7732992710,12cyclictest21-21ksoftirqd/122:46:281
770099272,8cyclictest131rcu_preempt22:06:270
770099272,20cyclictest25084-21ssh22:37:340
770099272,19cyclictest6953-21ssh22:27:320
770099272,19cyclictest6953-21ssh22:27:320
770099272,19cyclictest29041-21ssh22:57:370
770099272,19cyclictest17231-21ssh21:57:280
770099272,14cyclictest12-21ksoftirqd/020:40:130
770099271,19cyclictest32320-21ssh00:30:130
774299268,13cyclictest12651-21/usr/sbin/munin19:15:112
774299263,14cyclictest22346irq/40-dwc2_hso20:00:122
7742992613,9cyclictest0-21swapper/223:25:142
7742992612,11cyclictest18645-21/usr/sbin/munin19:55:122
770099262,19cyclictest25557-21diskmemload21:15:160
7754992515,7cyclictest28366irq/58-eth023:00:123
773299253,6cyclictest131rcu_preempt19:15:131
773299253,18cyclictest17237-21ssh23:10:161
775499243,16cyclictest28366irq/58-eth021:55:133
775499242,14cyclictest28466irq/59-eth000:30:123
773299242,20cyclictest18529-21ssh22:15:291
773299241,19cyclictest12145-21ssh23:25:151
773299241,18cyclictest22402-21ssh22:55:141
770099242,16cyclictest26377-21ssh23:15:140
770099241,7cyclictest131rcu_preempt21:36:280
775499236,10cyclictest0-21swapper/321:25:123
775499233,16cyclictest0-21swapper/322:45:293
7754992319,2cyclictest0-21swapper/319:50:123
7754992310,6cyclictest28466irq/59-eth021:30:133
773299233,16cyclictest10714-21ssh21:55:151
773299233,10cyclictest0-21swapper/122:37:351
773299232,18cyclictest21148-21ssh23:30:151
770099231,7cyclictest30342-21if_eth121:30:180
7732992213,4cyclictest0-21swapper/100:20:131
773299221,18cyclictest31874-21find19:35:111
773299221,18cyclictest12737-21ls19:45:291
775499219,9cyclictest0-21swapper/321:44:383
775499219,10cyclictest0-21swapper/323:52:293
775499212,13cyclictest0-21swapper/321:52:283
7742992111,7cyclictest7762-21date19:10:012
7742992111,7cyclictest7762-21date19:10:012
773299216,3cyclictest21-21ksoftirqd/121:44:281
773299212,16cyclictest30140-21apt-get23:35:131
773299212,16cyclictest11789-21apt-get21:20:141
7732992114,5cyclictest0-21swapper/120:00:161
773299211,2cyclictest131rcu_preempt19:50:231
773299211,18cyclictest23432-21ssh00:05:311
7732992111,7cyclictest6360-21cron20:50:011
7732992111,7cyclictest28497-21apt-get00:10:131
7732992110,9cyclictest0-21swapper/123:07:591
770099211,16cyclictest9251-21cut19:45:170
775499209,8cyclictest0-21swapper/319:45:143
775499209,6cyclictest0-21swapper/300:23:253
775499209,5cyclictest0-21swapper/323:45:173
775499208,7cyclictest0-21swapper/322:15:283
775499205,9cyclictest0-21swapper/323:10:143
775499205,8cyclictest28366irq/58-eth022:00:143
775499202,14cyclictest0-21swapper/322:10:233
7754992010,8cyclictest0-21swapper/323:16:033
773299209,7cyclictest21-21ksoftirqd/121:35:131
773299207,3cyclictest131rcu_preempt21:00:261
773299207,10cyclictest802-21cut00:30:181
773299204,8cyclictest21-21ksoftirqd/120:50:151
773299202,15cyclictest22274-21apt-get19:25:131
7732992014,3cyclictest21-21ksoftirqd/122:20:161
773299201,15cyclictest5240-21ls22:10:021
773299201,14cyclictest0-21swapper/119:55:141
7732992010,7cyclictest9891-21cut00:35:221
770099202,15cyclictest11260-21netstat19:10:240
7700992012,6cyclictest11131-21sshd20:51:270
770099201,16cyclictest14006-21ssh00:00:250
775499199,8cyclictest33-21ksoftirqd/300:10:133
775499199,8cyclictest0-21swapper/323:35:023
775499198,6cyclictest0-21swapper/300:19:363
775499193,11cyclictest0-21swapper/323:07:293
775499191,14cyclictest0-21swapper/322:20:283
773299196,10cyclictest28167-21tr23:50:231
773299192,13cyclictest6312-21irqstats22:10:211
773299191,8cyclictest201rcuc/123:40:221
7732991910,6cyclictest9251-21grep20:15:291
773299190,17cyclictest8091-21ls19:10:161
770099193,6cyclictest12-21ksoftirqd/023:25:330
770099193,11cyclictest0-21swapper/020:45:130
770099191,9cyclictest21853-21sh23:30:190
7700991912,5cyclictest0-21swapper/023:55:330
7700991911,6cyclictest22582-21interrupts19:25:170
775499189,6cyclictest0-21swapper/323:40:593
775499189,6cyclictest0-21swapper/323:20:043
775499184,9cyclictest0-21swapper/319:15:123
775499183,10cyclictest0-21swapper/321:37:253
775499182,10cyclictest0-21swapper/320:35:153
775499182,10cyclictest0-21swapper/320:00:133
775499182,10cyclictest0-21swapper/319:30:143
7754991812,4cyclictest0-21swapper/323:30:253
7754991812,4cyclictest0-21swapper/320:30:263
7754991811,5cyclictest28366irq/58-eth019:35:133
7754991811,4cyclictest0-21swapper/300:25:233
775499181,10cyclictest0-21swapper/320:25:183
773299189,6cyclictest19239-21cat23:45:231
773299189,6cyclictest10575-21irqcore20:20:171
773299188,7cyclictest4770-21irqrtprio19:40:181
773299183,14cyclictest0-21swapper/121:15:131
7732991811,5cyclictest27614-21cat00:25:311
773299181,14cyclictest0-21swapper/120:40:141
770099182,11cyclictest31399-21ssh21:47:270
7700991812,4cyclictest10552-21if_eth120:20:170
770099181,14cyclictest13076-21irqcore19:15:170
770099181,10cyclictest29902-21tr20:05:210
770099181,10cyclictest29902-21tr20:05:200
7700991810,5cyclictest18880-21usage_EthcTask00:20:320
775499179,3cyclictest0-21swapper/300:38:243
775499173,9cyclictest28366irq/58-eth020:10:123
775499172,9cyclictest0-21swapper/323:55:173
773299179,6cyclictest4041-21nfs_client20:10:231
773299179,6cyclictest22023-21sshd19:21:531
773299176,3cyclictest21-21ksoftirqd/100:19:221
773299172,13cyclictest27412-21ps19:30:181
773299171,9cyclictest28487-21fschecks_time20:05:171
773299171,9cyclictest28487-21fschecks_time20:05:161
773299171,3cyclictest131rcu_preempt20:30:311
773299171,13cyclictest21119-21fschecks_count21:05:171
770099179,5cyclictest26105-21kthreadcore20:00:200
770099178,6cyclictest25612-21latency_hist21:10:020
770099173,4cyclictest17166systemd-network19:23:020
7700991710,5cyclictest4217-21mailstats20:10:260
7754991611,3cyclictest0-21swapper/319:20:123
7732991611,3cyclictest0-21swapper/120:25:171
770099169,5cyclictest23237-21cron20:00:010
770099165,9cyclictest24959-21tail20:35:190
770099163,9cyclictest0-21swapper/020:25:140
770099162,9cyclictest0-21swapper/019:50:130
770099162,2cyclictest131rcu_preempt20:30:120
770099161,12cyclictest28406-21kthreadcore19:30:200
770099161,12cyclictest16213-21apt-get21:00:130
770099161,12cyclictest13017-21kthreadcore20:55:210
775499151,11cyclictest0-21swapper/320:50:153
770099153,10cyclictest7712-21cp19:10:020
770099153,10cyclictest7712-21cp19:10:010
770099151,11cyclictest31931-21cpu19:35:130
775499149,3cyclictest0-21swapper/320:20:133
775499141,7cyclictest0-21swapper/320:00:023
775499141,11cyclictest0-21swapper/321:00:283
7754991410,2cyclictest0-21swapper/319:40:123
775499138,3cyclictest0-21swapper/319:10:283
775499136,5cyclictest0-21swapper/320:45:203
775499133,7cyclictest0-21swapper/319:25:173
775499131,8cyclictest0-21swapper/321:10:023
773299131,9cyclictest7811-21ls19:10:021
773299131,9cyclictest7811-21ls19:10:011
775499122,6cyclictest0-21swapper/320:15:153
775499121,9cyclictest0-21swapper/320:10:013
775499121,9cyclictest0-21swapper/320:10:013
775499113,6cyclictest0-21swapper/320:40:163
775499111,7cyclictest0-21swapper/319:10:023
775499111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional