You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Dec 20, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
171302840,3sleep217133-21grep10:20:352
1209399834,24cyclictest3139-21chrt08:15:183
1209099801,50cyclictest0-21swapper/011:15:190
268742760,2sleep20-21swapper/210:50:052
272192740,5sleep01209099cyclictest07:50:280
12090997014,35cyclictest12236-21grep10:10:050
12090996949,13cyclictest101ktimersoftd/010:35:340
12090996914,44cyclictest1125-21runrttasks09:15:230
111982690,2sleep00-21swapper/008:35:320
1209099673,61cyclictest0-21swapper/010:30:180
1209399661,43cyclictest19901-21df10:30:213
1209099667,52cyclictest0-21swapper/012:30:100
12090996610,45cyclictest0-21swapper/007:10:340
12090996610,45cyclictest0-21swapper/007:10:340
12090996543,14cyclictest101ktimersoftd/012:10:340
12090996543,14cyclictest101ktimersoftd/012:10:340
1209399642,39cyclictest5998-21ntp_states11:20:283
1209399641,44cyclictest24412-21sshd09:12:173
1209399641,44cyclictest24412-21sshd09:12:173
1209399641,41cyclictest28701-21munin-node10:55:183
1209399641,40cyclictest24899-21apt-get09:15:173
1209099642,18cyclictest0-21swapper/009:10:190
1209099642,18cyclictest0-21swapper/009:10:190
1209399631,43cyclictest13520-21awk10:10:323
1209099636,41cyclictest0-21swapper/012:00:180
1209099634,42cyclictest0-21swapper/010:00:340
1209399621,44cyclictest913-21/usr/sbin/munin12:35:303
1209299622,15cyclictest0-21swapper/212:30:192
1209399611,56cyclictest10472-21apt-get11:35:103
1209399611,41cyclictest15608-21users11:45:353
1209099614,32cyclictest0-21swapper/010:40:360
1209099614,32cyclictest0-21swapper/010:40:360
1209099606,29cyclictest9-21ksoftirqd/007:20:340
12090996016,26cyclictest29346-21kworker/u8:007:20:040
12090996012,14cyclictest0-21swapper/008:30:180
1209399592,41cyclictest4394-21unixbench_multi09:45:333
1209399592,34cyclictest7950irq/56-dwc_otg_12:30:053
1209399591,41cyclictest15049-21ls08:45:333
1209399591,37cyclictest8896-21/usr/sbin/munin11:30:333
12090995930,6cyclictest7950irq/56-dwc_otg_07:30:360
1209399581,46cyclictest13277-21nfs4_client11:40:293
1209399581,42cyclictest30993-21df12:30:223
1209399581,40cyclictest29621-21taskset07:56:263
12090995835,9cyclictest101ktimersoftd/012:35:250
12090995810,28cyclictest789-21unixbench_multi09:35:320
1209399571,53cyclictest22943-21apt07:40:173
1209399571,52cyclictest4705-21apt-get08:20:083
1209399571,52cyclictest4705-21apt-get08:20:083
1209399571,52cyclictest16443-21df11:50:243
1209399571,51cyclictest7107-21apt-get11:25:203
1209399571,46cyclictest22581-21/usr/sbin/munin12:05:303
1209099576,33cyclictest9-21ksoftirqd/007:25:340
12090995727,8cyclictest19152-21unixbench_multi11:55:330
1209099572,31cyclictest2988-21latency_hist08:15:050
1209099572,29cyclictest18181-21fschecks_time10:25:230
1209099571,22cyclictest0-21swapper/009:43:200
1209399561,51cyclictest19347-21apt-get07:30:193
1209399561,41cyclictest27810-21taskset07:51:203
12093995610,43cyclictest7028-21sort08:25:213
1209299562,4cyclictest0-21swapper/207:20:182
1209099564,33cyclictest13-21rcuc/011:20:230
1209399551,51cyclictest7033-21aten_rbpower_cu09:55:193
1209399551,39cyclictest27169-21irqstats09:20:263
1209399551,35cyclictest12266-21apt-get08:40:173
1209099556,30cyclictest9-21ksoftirqd/011:45:310
1209099552,6cyclictest28407-21latency_hist09:25:050
1209099551,9cyclictest9397-21latency11:30:260
1209099551,31cyclictest22794-21grep12:05:320
1209399541,50cyclictest22903-21latency_hist07:40:063
1209399541,38cyclictest4877-21taskset11:18:003
1209399541,38cyclictest26755-21sshd12:19:173
1209399541,38cyclictest10427-21chrt08:35:073
1209299541,10cyclictest7750irq/56-dwc_otg12:35:192
1209099549,12cyclictest0-21swapper/012:25:180
12090995435,11cyclictest0-21swapper/010:55:370
1209399531,34cyclictest4851-21apt-get09:50:093
1209399531,34cyclictest4851-21apt-get09:50:093
1209399531,34cyclictest24579-21taskset10:41:373
1209399531,34cyclictest24579-21taskset10:41:373
12093995312,38cyclictest15855-21cpu07:20:213
12090995336,5cyclictest101ktimersoftd/011:40:050
12090995329,5cyclictest7950irq/56-dwc_otg_08:25:310
1209099532,33cyclictest0-21swapper/011:40:320
1209099532,28cyclictest0-21swapper/008:45:260
12090995311,22cyclictest17934-21fschecks_count08:55:240
1209399526,16cyclictest0-21swapper/308:50:213
1209399522,46cyclictest17844-21apt-get10:25:193
1209399521,48cyclictest32329-21df_inode09:35:203
1209399521,47cyclictest17544-21apt-get07:25:223
1209399521,47cyclictest14850-21ls10:15:283
1209299524,9cyclictest0-21swapper/211:25:202
1209099521,26cyclictest7392-21fschecks_time11:25:250
1209099521,24cyclictest0-21swapper/007:45:170
1209099521,23cyclictest0-21swapper/009:30:310
1209399519,39cyclictest32505-21cpuspeed11:05:213
1209399511,47cyclictest27642-21/usr/sbin/munin12:20:313
1209399511,47cyclictest13914-21apt-get07:15:173
1209399511,46cyclictest29587-21/usr/sbin/munin09:25:343
1209399511,46cyclictest12120-21apt-get07:10:183
1209399511,46cyclictest12120-21apt-get07:10:183
1209399511,36cyclictest7850irq/56-dwc_otg_10:20:353
12093995113,35cyclictest8783-21/usr/sbin/munin10:00:203
1209399511,32cyclictest26401-21taskset10:46:523
1209299513,14cyclictest0-21swapper/210:25:192
12091995116,23cyclictest0-21swapper/111:35:121
1209099511,27cyclictest0-21swapper/011:50:320
1209099511,25cyclictest6053-21switchtime09:50:350
1209099511,25cyclictest6053-21switchtime09:50:350
1209399501,46cyclictest30518-21apt-get11:00:203
1209399501,45cyclictest21012-21/usr/sbin/munin12:00:333
1209399501,42cyclictest1001-21seq08:09:243
1209099505,27cyclictest13-21rcuc/011:03:350
12090995025,23cyclictest9-21ksoftirqd/010:10:270
12090995024,3cyclictest0-21swapper/011:05:180
1209099502,28cyclictest0-21swapper/007:55:190
1209099501,26cyclictest0-21swapper/009:00:160
1209399492,36cyclictest7850irq/56-dwc_otg_09:06:563
1209399491,45cyclictest23180-21latency_hist10:40:063
1209399491,44cyclictest9178-21latency08:30:273
1209399491,44cyclictest30887-21/usr/sbin/munin08:00:303
1209399491,44cyclictest19553-21/usr/sbin/munin09:00:193
12092994916,17cyclictest12089-21cyclictest11:15:192
1209099494,23cyclictest0-21swapper/012:15:300
12090994927,15cyclictest101ktimersoftd/008:21:520
12090994927,15cyclictest101ktimersoftd/008:21:520
12090994927,13cyclictest101ktimersoftd/009:45:240
1209399482,43cyclictest19086-21ls11:55:333
1209399481,44cyclictest2037-21ntp_states08:10:303
1209399481,44cyclictest1703-21/usr/sbin/munin09:40:243
1209399481,30cyclictest0-21swapper/309:31:333
12092994822,7cyclictest10819-21apt-get11:35:202
12091994818,16cyclictest221ktimersoftd/112:16:081
12090994825,5cyclictest7850irq/56-dwc_otg_09:58:230
1209399471,43cyclictest2335-21netstat11:10:273
1209399471,43cyclictest19072-21sshd08:58:173
1209399471,42cyclictest26911-21apt-get10:50:183
1209399471,32cyclictest26008-21runrttasks07:46:173
1209299477,25cyclictest0-21swapper/207:45:202
1209099476,25cyclictest9-21ksoftirqd/011:15:050
1209099476,20cyclictest12325-21aten_rbpower_en08:40:190
12090994723,5cyclictest32009-21aten_rbpower_cu08:05:190
1209099472,26cyclictest0-21swapper/009:27:480
1209099472,24cyclictest0-21swapper/010:20:180
1209099471,23cyclictest31906-21latency_hist08:05:050
1209299467,25cyclictest0-21swapper/211:00:202
1209299466,25cyclictest0-21swapper/212:20:232
1209099468,19cyclictest27513-21iwlist12:20:280
12090994627,7cyclictest101ktimersoftd/007:41:030
1209299457,24cyclictest0-21swapper/208:45:202
1209199456,25cyclictest0-21swapper/109:50:101
1209199456,25cyclictest0-21swapper/109:50:101
1209199453,19cyclictest0-21swapper/112:05:161
1209399441,40cyclictest23453-21apt-get12:10:163
1209399441,40cyclictest23453-21apt-get12:10:163
1209399441,40cyclictest12002-21sshd10:08:173
1209199449,25cyclictest0-21swapper/111:15:191
1209199447,25cyclictest0-21swapper/109:35:181
1209199441,36cyclictest0-21swapper/108:50:211
12090994416,5cyclictest16307-21if_wlan008:50:260
1209299437,24cyclictest0-21swapper/207:30:212
12090994320,13cyclictest0-21swapper/010:53:320
1209099431,25cyclictest0-21swapper/010:20:050
1209299427,24cyclictest0-21swapper/208:20:102
1209299427,24cyclictest0-21swapper/208:20:102
1209299427,24cyclictest0-21swapper/207:55:192
1209199427,22cyclictest0-21swapper/109:00:181
12090994230,5cyclictest4636-21taskset08:19:220
1209099422,20cyclictest0-21swapper/010:45:220
1209299417,22cyclictest0-21swapper/208:25:202
1209299411,8cyclictest0-21swapper/207:50:192
1209199416,25cyclictest0-21swapper/112:20:231
1209199416,22cyclictest11-21rcu_preempt10:00:191
12091994118,4cyclictest2884-21sh08:15:051
1209299407,21cyclictest0-21swapper/207:15:182
1209199407,23cyclictest0-21swapper/111:40:211
1209199404,28cyclictest7950irq/56-dwc_otg_08:25:181
12091994020,6cyclictest1543-21aten_rbpower_en09:40:221
12090994015,5cyclictest22765-21cat07:40:050
12090994015,16cyclictest0-21swapper/009:05:090
1209299391,6cyclictest0-21swapper/210:50:182
1209199392,24cyclictest7750irq/56-dwc_otg10:20:051
12091993919,10cyclictest23-21ksoftirqd/109:15:201
1209299386,18cyclictest0-21swapper/212:15:172
1209199386,22cyclictest0-21swapper/112:30:101
1209199386,21cyclictest0-21swapper/112:13:391
1209199386,21cyclictest0-21swapper/112:13:391
12091993814,16cyclictest8917-21taskset11:30:191
229792370,5sleep222977-21cut07:40:202
1209199378,19cyclictest7850irq/56-dwc_otg_10:10:061
1209199377,21cyclictest0-21swapper/110:30:201
1209199373,23cyclictest29061-21ls10:55:251
12091993724,3cyclictest0-21swapper/111:25:201
1209199371,27cyclictest0-21swapper/109:24:011
1209199371,26cyclictest25306-21ls10:45:201
1209299369,15cyclictest7950irq/56-dwc_otg_09:50:302
1209299369,15cyclictest7950irq/56-dwc_otg_09:50:302
1209299367,15cyclictest0-21swapper/210:15:202
12092993616,5cyclictest9087-21cat11:30:212
1209299361,23cyclictest20708-21switchtime12:00:312
1209299361,23cyclictest14007-21ls10:15:052
1209199365,20cyclictest0-21swapper/109:55:221
1209199364,16cyclictest7850irq/56-dwc_otg_07:15:251
1209199363,25cyclictest7950irq/56-dwc_otg_12:35:201
1209199361,5cyclictest0-21swapper/107:30:181
1209199361,3cyclictest0-21swapper/111:45:191
12091993610,16cyclictest7750irq/56-dwc_otg11:15:051
1209299351,24cyclictest25462-21missed_timers09:15:272
1209199357,20cyclictest31647-21grep11:00:361
1209199356,4cyclictest7750irq/56-dwc_otg10:25:231
1209199353,12cyclictest0-21swapper/112:25:191
1209199352,20cyclictest0-21swapper/107:26:531
12091993519,13cyclictest0-21swapper/109:10:051
1209199351,6cyclictest4386-21grep09:45:331
12091993514,13cyclictest0-21swapper/111:20:181
12091993513,6cyclictest5896-21switchtime08:20:321
12091993513,6cyclictest5896-21switchtime08:20:321
1209199351,23cyclictest7750irq/56-dwc_otg08:05:311
12091993512,15cyclictest0-21swapper/111:55:331
12091993511,15cyclictest32407-21wget11:05:201
1209299344,12cyclictest7950irq/56-dwc_otg_08:40:052
12092993412,5cyclictest28726-21/usr/sbin/munin09:25:222
1209299341,22cyclictest13474-21ls11:40:332
1209199343,24cyclictest0-21swapper/108:35:211
1209199341,5cyclictest0-21swapper/108:15:161
12091993415,4cyclictest435-21cron07:15:041
12091993415,4cyclictest435-21cron07:15:041
12091993413,5cyclictest14104-21cut08:45:201
109822346,14sleep00-21swapper/007:05:300
182602330,3sleep2301ktimersoftd/211:55:222
1209299337,15cyclictest0-21swapper/209:00:182
1209299333,13cyclictest7850irq/56-dwc_otg_10:30:222
1209299331,12cyclictest0-21swapper/212:30:052
1209199335,19cyclictest7750irq/56-dwc_otg09:13:141
1209199335,19cyclictest7750irq/56-dwc_otg09:13:141
1209199332,20cyclictest0-21swapper/110:35:201
1209199331,6cyclictest128-21systemd-journal11:53:451
1209199331,2cyclictest0-21swapper/107:45:181
1209199331,25cyclictest7950irq/56-dwc_otg_07:20:181
1209199331,22cyclictest28600-21fschecks_count07:55:231
1174623319,6sleep10-21swapper/107:09:161
1209299327,5cyclictest7950irq/56-dwc_otg_11:20:162
1209299327,11cyclictest7850irq/56-dwc_otg_09:35:252
1209299325,17cyclictest7950irq/56-dwc_otg_08:10:202
12092993226,4cyclictest26824-21aten_rbpower_vo09:20:212
1209299321,8cyclictest31272-21tail08:00:342
1209299321,20cyclictest13893-21ls08:45:052
1209199326,10cyclictest7850irq/56-dwc_otg_07:35:261
1209199324,5cyclictest7750irq/56-dwc_otg10:40:361
1209199324,5cyclictest7750irq/56-dwc_otg10:40:361
1209199323,4cyclictest7750irq/56-dwc_otg08:30:341
1209199322,20cyclictest7750irq/56-dwc_otg12:00:191
1209199321,22cyclictest19238-21latency_hist09:00:061
1209199321,22cyclictest0-21swapper/110:10:071
1209299315,10cyclictest7950irq/56-dwc_otg_10:35:272
1209299314,12cyclictest7950irq/56-dwc_otg_10:40:302
1209299314,12cyclictest7950irq/56-dwc_otg_10:40:302
1209299313,5cyclictest7750irq/56-dwc_otg09:55:302
1209299313,18cyclictest24835-21latency_hist09:15:062
1209299313,18cyclictest24835-21latency_hist09:15:062
1209299312,4cyclictest0-21swapper/208:55:202
1209299311,7cyclictest7850irq/56-dwc_otg_11:05:312
1209299311,6cyclictest7850irq/56-dwc_otg_10:00:302
1209299311,6cyclictest3968-21ntpq09:45:282
1209299311,6cyclictest29888-21users10:55:362
12092993114,9cyclictest7750irq/56-dwc_otg07:15:042
12092993114,9cyclictest7750irq/56-dwc_otg07:15:042
1209299311,14cyclictest7750irq/56-dwc_otg07:35:212
1209199314,6cyclictest7850irq/56-dwc_otg_09:30:311
1209199314,18cyclictest7750irq/56-dwc_otg07:40:191
1209199312,4cyclictest7750irq/56-dwc_otg09:25:201
1209199311,5cyclictest26881-21fschecks_time07:50:231
1209199311,24cyclictest7750irq/56-dwc_otg10:25:051
1209199311,14cyclictest0-21swapper/108:40:321
1209299307,11cyclictest7750irq/56-dwc_otg08:30:302
1209299306,15cyclictest0-21swapper/210:05:202
1209299306,11cyclictest7850irq/56-dwc_otg_11:50:252
1209299302,5cyclictest7750irq/56-dwc_otg08:15:332
1209299302,3cyclictest7750irq/56-dwc_otg12:05:262
1209299302,13cyclictest7950irq/56-dwc_otg_09:05:222
1209299301,15cyclictest7950irq/56-dwc_otg_08:05:432
12091993019,4cyclictest7850irq/56-dwc_otg_08:00:291
12091993010,11cyclictest12089-21cyclictest10:50:301
1209299292,7cyclictest17760-21diskstats07:25:252
1209299291,6cyclictest7850irq/56-dwc_otg_11:10:322
1209299291,24cyclictest0-21swapper/208:50:252
1209299286,4cyclictest7950irq/56-dwc_otg_12:10:272
1209299286,4cyclictest7950irq/56-dwc_otg_12:10:272
1209299284,6cyclictest7950irq/56-dwc_otg_11:45:262
1209299282,15cyclictest0-21swapper/209:30:572
1209299281,21cyclictest0-21swapper/209:41:212
109692277,10sleep30-21swapper/307:05:293
106222275,9sleep20-21swapper/207:05:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional