You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 14:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Mon Jun 01, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19939997521,8cyclictest12-21ksoftirqd/012:05:130
1994899612,46cyclictest18666-21ssh09:35:141
19948995111,10cyclictest131rcu_preempt09:00:141
1994899503,6cyclictest131rcu_preempt10:40:131
19948994817,24cyclictest21-21ksoftirqd/108:20:121
1994899471,36cyclictest13402-21ssh11:20:141
19948994512,25cyclictest21-21ksoftirqd/108:25:141
1993999445,30cyclictest27741-21ssh09:40:120
19954994323,13cyclictest0-21swapper/210:05:122
1994899433,33cyclictest9321-21ssh12:30:131
1996299422,24cyclictest35966irq/58-eth007:15:133
1995499422,32cyclictest13434-21sh11:00:232
1996299413,24cyclictest35966irq/58-eth009:40:133
19954994115,4cyclictest11644-21sshd11:35:242
19962994020,11cyclictest35966irq/58-eth011:55:133
1994899402,30cyclictest21-21ksoftirqd/111:40:111
1993999401,33cyclictest7857-21rm11:35:110
1996299393,22cyclictest35966irq/58-eth007:35:133
1995499391,9cyclictest9799-21fschecks_count09:30:152
1994899395,4cyclictest131rcu_preempt08:05:151
19954993822,10cyclictest628-21tcf-agent11:25:242
1994899381,30cyclictest23816-21ssh12:20:141
1996299362,21cyclictest35966irq/58-eth007:40:123
1995499361,8cyclictest2972-21cut10:20:162
19954993613,4cyclictest0-21swapper/209:20:102
1993999363,23cyclictest13509-21ssh09:50:140
1996299353,26cyclictest35966irq/58-eth010:10:103
1995499351,9cyclictest19738-21ssh11:05:182
1993999351,7cyclictest22296-21ssh11:25:130
19939993516,6cyclictest131rcu_preempt10:25:140
1995499346,5cyclictest0-21swapper/211:15:222
1995499342,7cyclictest23356-21usage_EthcTask09:35:312
1995499341,8cyclictest23076-21kthreadcore12:00:212
1995499341,8cyclictest23076-21kthreadcore12:00:212
1994899341,24cyclictest619-21ssh09:25:151
1993999342,23cyclictest5848-21ssh12:10:130
1994899331,26cyclictest24249-21ssh10:50:131
1996299321,20cyclictest35966irq/58-eth011:30:153
1994899322,24cyclictest27765-21ssh09:40:131
1993999321,18cyclictest767-21ssh10:55:130
1995499313,21cyclictest27-21ksoftirqd/207:35:142
1995499311,15cyclictest734-21/usr/sbin/munin10:55:242
1995499311,10cyclictest6079-21ssh09:10:132
1996299307,7cyclictest35966irq/58-eth012:25:143
1995499309,9cyclictest28799-21ssh12:20:422
1995499308,3cyclictest18040-21latency_hist12:35:022
1995499304,15cyclictest27346irq/40-dwc2_hso08:10:122
1995499302,21cyclictest9857-21ntp_192.168.11410:40:232
1994899303,18cyclictest21-21ksoftirqd/107:55:121
1993999302,20cyclictest13435-21rm11:00:240
1995499295,11cyclictest27-21ksoftirqd/212:15:262
1995499293,15cyclictest27346irq/40-dwc2_hso08:35:132
1994899293,19cyclictest15269-21ssh10:45:131
1996299283,11cyclictest0-21swapper/311:05:123
1994899284,18cyclictest9844-21ssh11:00:131
1994899281,20cyclictest14834-21ssh12:15:141
1993999284,6cyclictest131rcu_preempt09:15:120
1993999282,21cyclictest23561-21ssh09:35:330
1996299273,14cyclictest0-21swapper/311:10:133
1995499272,10cyclictest0-21swapper/209:50:052
1994899273,20cyclictest27856-21ssh11:10:131
1994899272,19cyclictest6145-21ssh11:33:241
19939992716,6cyclictest0-21swapper/010:30:140
1996299262,12cyclictest35966irq/58-eth011:00:133
1996299261,18cyclictest0-21swapper/309:15:133
1995499263,12cyclictest314-21ssh12:25:132
1995499262,10cyclictest0-21swapper/208:05:172
1994899262,18cyclictest11655-21ssh11:35:251
19948992612,11cyclictest29459-21cron07:20:011
1993999263,17cyclictest73-21kswapd010:00:130
1993999261,20cyclictest9651-21rm09:30:140
1993999261,15cyclictest12-21ksoftirqd/008:45:140
1996299253,14cyclictest0-21swapper/311:35:243
1995499253,17cyclictest27346irq/40-dwc2_hso10:00:132
1995499252,17cyclictest27346irq/40-dwc2_hso12:10:112
1995499252,16cyclictest0-21swapper/207:40:052
19954992513,9cyclictest24523-21kthreadcore08:20:202
19954992512,11cyclictest0-21swapper/208:45:192
1995499251,15cyclictest32260-21apt-get08:30:122
1994899251,19cyclictest2716-21ssh10:20:141
1996299242,14cyclictest35966irq/58-eth012:20:123
1996299241,13cyclictest35966irq/58-eth010:35:133
19954992413,9cyclictest0-21swapper/208:15:182
1995499240,21cyclictest31743-21ls11:10:282
1994899244,13cyclictest0-21swapper/108:30:121
1994899242,17cyclictest21-21ksoftirqd/107:10:141
1994899241,18cyclictest25971-21ssh11:45:131
1993999242,17cyclictest20552-21kworker/u8:012:25:140
1996299238,10cyclictest0-21swapper/310:50:133
1996299233,11cyclictest0-21swapper/309:25:153
1996299231,17cyclictest35966irq/58-eth010:45:133
1995499238,13cyclictest0-21swapper/211:55:272
1995499233,11cyclictest5960-21grep07:25:282
1995499233,11cyclictest5960-21grep07:25:282
1995499232,6cyclictest0-21swapper/210:54:232
1995499232,17cyclictest27346irq/40-dwc2_hso09:25:132
1995499232,16cyclictest0-21swapper/210:48:242
1995499232,15cyclictest0-21swapper/210:15:042
1995499232,15cyclictest0-21swapper/209:40:132
19954992312,9cyclictest0-21swapper/210:10:122
1995499231,18cyclictest0-21swapper/211:20:142
1994899236,4cyclictest131rcu_preempt12:25:231
1994899232,14cyclictest0-21swapper/107:25:131
1994899232,14cyclictest0-21swapper/107:25:121
1994899232,12cyclictest15035-21diskmemload09:15:131
1994899231,19cyclictest21459-21find07:45:121
1994899231,16cyclictest26108-21ssh09:55:261
1996299225,14cyclictest33-21ksoftirqd/312:35:133
19962992214,5cyclictest35966irq/58-eth009:20:133
1996299221,14cyclictest0-21swapper/312:15:133
19962992210,9cyclictest0-21swapper/309:38:053
1995499226,7cyclictest131rcu_preempt12:35:242
1995499223,9cyclictest25327-21head07:45:292
1995499222,15cyclictest27346irq/40-dwc2_hso09:45:132
1995499221,5cyclictest16766systemd-network10:32:212
1995499221,4cyclictest0-21swapper/208:40:212
19954992210,6cyclictest91-21pr/ttyS011:45:012
1994899229,11cyclictest21-21ksoftirqd/110:28:441
1994899226,11cyclictest0-21swapper/108:40:121
1994899221,2cyclictest131rcu_preempt09:47:381
1994899221,18cyclictest26415-21ls10:15:171
1993999222,16cyclictest28682-21apt-get09:00:120
1993999221,14cyclictest73-21kswapd010:10:110
1993999221,14cyclictest18828-21find11:05:110
1996299219,9cyclictest0-21swapper/310:00:253
1996299219,9cyclictest0-21swapper/309:12:503
19962992115,4cyclictest0-21swapper/310:25:263
19962992112,6cyclictest35966irq/58-eth011:15:123
1995499218,10cyclictest31405-21df11:30:152
1995499213,8cyclictest14795-21kthreadcore10:25:232
1995499212,16cyclictest16692-21kworker/u8:107:20:112
1995499212,15cyclictest27652-21kthreadcore11:45:192
19954992114,4cyclictest0-21swapper/209:15:172
1995499211,11cyclictest25512-21kthreadcore08:55:222
1995499211,11cyclictest25512-21kthreadcore08:55:212
1994899212,4cyclictest131rcu_preempt08:00:181
1994899211,8cyclictest3510-21kthreadcore09:05:211
19948992113,5cyclictest0-21swapper/107:50:231
1994899211,16cyclictest19034-21find08:50:111
1994899211,16cyclictest18250-21ssh12:35:131
1994899211,15cyclictest31507-21cpuspeed10:00:131
1994899211,15cyclictest2482-21ssh11:50:121
1994899211,15cyclictest17118-21ssh09:50:251
1993999218,10cyclictest6191-21diskmemload09:10:150
1993999217,3cyclictest131rcu_preempt10:15:040
1993999217,11cyclictest26132-21ssh11:45:150
1993999212,14cyclictest4413-21ssh09:45:130
1993999211,4cyclictest0-21swapper/012:15:090
19939992113,5cyclictest13051-21processes12:30:260
1993999211,17cyclictest24049-21ssh09:20:140
1996299209,9cyclictest0-21swapper/310:15:313
1996299203,10cyclictest35966irq/58-eth011:40:113
1996299203,10cyclictest0-21swapper/310:05:123
19962992013,4cyclictest0-21swapper/310:20:313
19962992011,6cyclictest35966irq/58-eth009:50:143
19962992011,4cyclictest0-21swapper/310:30:313
19962992011,3cyclictest0-21swapper/311:50:123
1995499205,8cyclictest1126-21apt-get09:05:122
1995499205,11cyclictest26604-21irqcore07:50:172
1995499204,3cyclictest8818-21kthreadcore07:30:212
1995499202,9cyclictest19046-21cpu08:50:132
1995499202,14cyclictest29377-21ssh12:05:152
1995499201,9cyclictest3205-21munin-plugin-st08:00:012
1995499201,9cyclictest23760-21awk07:10:272
1995499201,9cyclictest1032-21latency_hist09:05:012
1995499201,8cyclictest3060-21ps11:50:182
1995499201,6cyclictest19966-21date07:10:012
1995499201,10cyclictest24034-21ssh09:55:222
1995499200,9cyclictest32092-21latency_hist08:30:012
1994899207,2cyclictest131rcu_preempt12:02:251
1994899207,2cyclictest131rcu_preempt12:02:241
1994899202,15cyclictest11931-21cut11:55:201
1994899201,2cyclictest131rcu_preempt10:09:541
19948992011,7cyclictest20720-21apt-get10:30:131
1994899201,14cyclictest9467-21ssh12:10:241
1994899201,14cyclictest16351-21apt-get07:40:011
1994899200,14cyclictest6944-21find07:30:111
1993999209,8cyclictest2736-21swap11:30:290
1993999208,9cyclictest24973-21cut10:50:180
1993999202,15cyclictest27478-21ntpq12:20:260
19939992013,4cyclictest3209-21sendmail08:00:010
1993999201,16cyclictest23987-21cut08:55:160
1993999201,16cyclictest23987-21cut08:55:160
1993999201,15cyclictest8058-21sh10:05:130
1993999201,14cyclictest29842-21ssh10:35:170
1996299196,7cyclictest0-21swapper/312:00:123
1996299196,7cyclictest0-21swapper/312:00:123
1996299192,11cyclictest0-21swapper/309:05:133
19962991912,4cyclictest0-21swapper/312:05:193
19962991911,6cyclictest0-21swapper/309:55:273
19962991911,3cyclictest0-21swapper/310:56:133
1996299191,12cyclictest0-21swapper/308:50:153
19962991910,6cyclictest0-21swapper/311:21:393
1995499191,9cyclictest3563-21fschecks_count08:00:152
1995499191,9cyclictest28394-21wc07:15:272
1995499191,7cyclictest30103-21grep10:35:192
1994899194,5cyclictest131rcu_preempt11:30:021
1994899192,2cyclictest131rcu_preempt09:31:501
1994899192,14cyclictest641-21kthreadcore10:35:231
1994899191,2cyclictest131rcu_preempt12:07:511
19948991912,5cyclictest0-21swapper/108:35:191
19948991912,4cyclictest8048-21kthreadcore09:10:221
19948991912,4cyclictest21-21ksoftirqd/111:15:191
1994899191,15cyclictest894-21ssh10:55:161
1994899191,15cyclictest29835-21cpu07:20:141
19948991910,6cyclictest21012-21grep10:10:261
1993999199,7cyclictest4150-21cat11:15:010
1993999192,15cyclictest22510-21kthreadcore12:00:200
1993999192,15cyclictest22510-21kthreadcore12:00:200
1993999192,14cyclictest16488-21apt-get11:40:050
1993999191,7cyclictest2455-21sort11:50:120
1993999191,14cyclictest23215-21irqstats09:55:200
1996299189,6cyclictest35966irq/58-eth012:10:123
1996299189,6cyclictest0-21swapper/309:31:133
1996299187,4cyclictest0-21swapper/309:49:183
1996299182,10cyclictest0-21swapper/309:00:153
1996299182,10cyclictest0-21swapper/307:10:143
19962991812,4cyclictest0-21swapper/312:32:103
1996299181,13cyclictest0-21swapper/311:25:213
1996299181,10cyclictest0-21swapper/307:20:053
19962991810,6cyclictest0-21swapper/310:41:453
19948991811,4cyclictest24435-21irqcore09:20:171
1994899181,14cyclictest21203-21kthreadcore11:05:211
19948991810,6cyclictest21212-21sshd07:40:451
1993999183,12cyclictest8169-21mailstats11:15:260
1993999181,5cyclictest53646usb-storage09:05:040
1993999181,15cyclictest15259-21ssh10:45:130
19939991811,4cyclictest4850-21usage_Anybus09:25:320
19939991811,4cyclictest20873-21kthreadcore12:35:210
1993999181,13cyclictest2823-21diskstats10:20:150
1993999181,13cyclictest13507-21df_inode11:20:160
19939991810,6cyclictest32015-21sshd08:27:200
1996299179,5cyclictest0-21swapper/311:45:183
1996299172,8cyclictest0-21swapper/307:30:123
1996299172,10cyclictest0-21swapper/308:55:143
1996299172,10cyclictest0-21swapper/308:55:143
1996299172,10cyclictest0-21swapper/308:40:133
1996299172,10cyclictest0-21swapper/307:55:133
1996299172,10cyclictest0-21swapper/307:25:133
1996299172,10cyclictest0-21swapper/307:25:133
1996299171,13cyclictest0-21swapper/308:20:283
1996299171,12cyclictest0-21swapper/307:50:153
1994899172,3cyclictest131rcu_preempt08:10:121
1994899172,10cyclictest0-21swapper/108:55:151
1994899172,10cyclictest0-21swapper/108:55:141
1994899171,9cyclictest20099-21cut08:15:211
1993999178,6cyclictest25016-21sendmail_mailqu07:45:270
1993999175,9cyclictest3124-21wc08:30:230
1993999174,10cyclictest21126-21kthreadcore07:10:190
1993999173,11cyclictest0-21swapper/008:20:130
1993999172,11cyclictest20133-21grep12:00:010
1993999171,9cyclictest2317-21fschecks_time07:25:160
1993999171,9cyclictest2317-21fschecks_time07:25:160
1993999171,8cyclictest13317-21sshd08:10:180
1993999171,14cyclictest6272-21ssh10:40:120
1993999171,13cyclictest8813-21kthreadcore07:30:210
1993999171,12cyclictest16583-21latency_hist07:40:020
1996299161,7cyclictest0-21swapper/308:00:133
1994899161,12cyclictest14255-21apt-get08:45:161
1993999164,9cyclictest29161-21kthreadcore07:50:210
1993999164,9cyclictest10892-21kthreadcore08:40:190
1993999163,10cyclictest6424-21kthreadcore08:35:210
1993999163,10cyclictest19716-21grep08:50:170
1993999163,10cyclictest18037-21kthreadcore07:40:200
1993999163,10cyclictest11109-21kthreadcore08:05:240
1993999162,9cyclictest0-21swapper/008:15:150
1993999162,11cyclictest27723-21tr07:15:220
1993999161,4cyclictest53646usb-storage07:20:040
1993999161,4cyclictest0-21swapper/007:10:020
1993999151,11cyclictest3454-21sshd08:00:120
1996299149,3cyclictest0-21swapper/308:17:453
1996299147,4cyclictest33-21ksoftirqd/308:10:313
1994899141,10cyclictest19961-21latency_hist07:10:011
1996299138,3cyclictest33-21ksoftirqd/307:47:423
1996299138,3cyclictest0-21swapper/308:45:303
1996299137,4cyclictest33-21ksoftirqd/308:10:013
1996299132,6cyclictest0-21swapper/308:30:143
1996299131,10cyclictest0-21swapper/308:35:213
1996299131,10cyclictest0-21swapper/308:25:233
1996299101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional