You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 15:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Jun 19, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2010421760,5sleep31839099cyclictest07:15:053
1838799986,85cyclictest0-21swapper/010:05:170
26402900,1sleep20-21swapper/212:25:302
325552890,2sleep20-21swapper/210:29:432
18387998726,44cyclictest8050irq/56-dwc_otg_12:30:280
1838799858,36cyclictest0-21swapper/009:25:120
1838799842,62cyclictest25471-21/usr/sbin/munin12:15:310
18387998218,15cyclictest0-21swapper/011:07:480
326612800,2sleep10-21swapper/109:50:331
326612800,2sleep10-21swapper/109:50:331
1838799781,20cyclictest0-21swapper/011:57:490
182712780,3sleep20-21swapper/207:10:052
18387997725,32cyclictest7850irq/56-dwc_otg11:40:050
1838799771,24cyclictest26389-21proc_pri07:30:300
1838799742,27cyclictest0-21swapper/009:30:070
18387997411,14cyclictest25885-21aten_rbpower_po09:00:200
175092740,5sleep01838799cyclictest11:27:140
142722730,2sleep00-21swapper/012:01:000
1838799721,21cyclictest0-21swapper/009:05:260
1838799721,20cyclictest125-21systemd-journal08:02:460
1838799721,16cyclictest0-21swapper/011:16:120
18387997140,18cyclictest9676-21fschecks_count08:15:210
18387997135,20cyclictest0-21swapper/008:45:160
18387997021,2cyclictest0-21swapper/007:55:050
1838799701,22cyclictest7874-21fschecks_count10:00:220
1838799692,24cyclictest0-21swapper/010:25:200
18387996912,5cyclictest17013-21cat10:10:310
18387996912,20cyclictest113212chrt10:41:570
1838799691,16cyclictest0-21swapper/009:16:180
1838799687,15cyclictest0-21swapper/011:54:350
1838799681,5cyclictest6987-21unixbench_singl08:05:330
1838799681,5cyclictest6987-21unixbench_singl08:05:330
1838799681,5cyclictest17308-21grep08:35:250
1838799681,21cyclictest19032-21ssh10:50:330
1838799681,21cyclictest19032-21ssh10:50:330
1838799676,22cyclictest19516-21ssh11:30:180
18387996720,5cyclictest30991-21grep12:20:330
1838799671,34cyclictest0-21swapper/007:55:150
1839099666,22cyclictest0-21swapper/311:25:113
1838799667,14cyclictest0-21swapper/010:46:420
18387996613,3cyclictest0-21swapper/010:22:280
18387996613,3cyclictest0-21swapper/010:22:280
1838799661,20cyclictest0-21swapper/011:45:320
18387996532,16cyclictest7850irq/56-dwc_otg09:40:340
1838799651,20cyclictest4005-21chrt12:27:290
1838799651,17cyclictest0-21swapper/010:30:260
1838799649,12cyclictest23827-21awk07:25:130
1838799641,3cyclictest0-21swapper/012:39:180
1838799641,21cyclictest0-21swapper/012:06:380
1838799641,18cyclictest0-21swapper/010:39:140
1838799641,15cyclictest29187-21diskmemload10:15:490
289252630,2sleep20-21swapper/212:19:142
1838799632,21cyclictest0-21swapper/009:10:530
18387996319,4cyclictest7950irq/56-dwc_otg_11:20:220
1838799631,15cyclictest0-21swapper/010:55:560
1838799626,14cyclictest0-21swapper/009:56:390
18387996220,4cyclictest7850irq/56-dwc_otg07:17:120
18387996220,4cyclictest7850irq/56-dwc_otg07:17:120
18387996216,8cyclictest0-21swapper/009:38:210
1838799611,23cyclictest0-21swapper/011:02:020
18387996112,11cyclictest0-21swapper/008:57:360
154292610,1sleep10-21swapper/110:09:331
1838799605,14cyclictest29187-21diskmemload11:13:250
1838799601,22cyclictest0-21swapper/008:24:330
1838799601,18cyclictest0-21swapper/011:40:260
1838799598,13cyclictest0-21swapper/009:52:550
1838799598,13cyclictest0-21swapper/009:52:550
1838799598,12cyclictest0-21swapper/008:30:220
1838799591,16cyclictest0-21swapper/009:46:340
180812593,48sleep00-21swapper/007:09:440
53162580,2sleep35287-21ssh11:50:333
203692570,2sleep20-21swapper/210:15:182
18390995721,24cyclictest0-21swapper/310:40:183
18387995713,40cyclictest101ktimersoftd/008:40:190
18387995616,3cyclictest7950irq/56-dwc_otg_09:21:530
18387995612,14cyclictest0-21swapper/008:13:430
120292560,22sleep18050irq/56-dwc_otg_09:26:381
1839099553,8cyclictest16077-21ssh10:10:163
18387995534,12cyclictest32175-21unixbench_multi07:45:330
1838899543,8cyclictest7850irq/56-dwc_otg08:50:161
1838799541,40cyclictest23766-21modprobe08:55:040
1838799521,36cyclictest22309-21fschecks_count07:20:220
1838999513,13cyclictest8050irq/56-dwc_otg_10:45:152
1838899514,13cyclictest7850irq/56-dwc_otg07:25:141
1838799512,35cyclictest0-21swapper/007:35:190
1839099505,32cyclictest8050irq/56-dwc_otg_10:35:173
18387994825,5cyclictest7850irq/56-dwc_otg07:15:050
18389994720,5cyclictest7347-21latency_hist10:00:052
1839099466,23cyclictest11834-21ssh10:05:173
1839099462,8cyclictest7850irq/56-dwc_otg08:15:153
1838899464,9cyclictest7850irq/56-dwc_otg08:20:181
18388994621,5cyclictest727-21runrttasks08:35:251
18387994626,5cyclictest7850irq/56-dwc_otg08:25:080
18387994626,5cyclictest7850irq/56-dwc_otg08:25:080
24422450,2sleep30-21swapper/309:54:103
24422450,2sleep30-21swapper/309:54:103
1839099457,25cyclictest0-21swapper/307:20:183
1839099452,2cyclictest0-21swapper/309:25:163
1839099452,12cyclictest8050irq/56-dwc_otg_07:40:173
1838899457,24cyclictest0-21swapper/111:20:181
1838899455,18cyclictest7850irq/56-dwc_otg12:35:191
1839099447,25cyclictest0-21swapper/309:30:183
1839099447,23cyclictest0-21swapper/308:25:113
1839099447,23cyclictest0-21swapper/308:25:113
1839099444,9cyclictest7950irq/56-dwc_otg_12:20:163
1838899446,24cyclictest0-21swapper/109:15:111
1838899442,9cyclictest17703-21ssh10:50:161
1838899442,9cyclictest17703-21ssh10:50:161
18388994421,5cyclictest22271-21fschecks_count07:20:221
1838799442,18cyclictest0-21swapper/012:15:050
1839099431,5cyclictest8050irq/56-dwc_otg_08:05:163
1839099431,5cyclictest8050irq/56-dwc_otg_08:05:163
1838999431,34cyclictest2356-21ssh11:10:182
1838899438,25cyclictest0-21swapper/109:40:181
1838899437,6cyclictest7950irq/56-dwc_otg_12:00:171
1838899437,27cyclictest0-21swapper/108:10:181
1839099427,25cyclictest0-21swapper/309:45:183
18390994219,8cyclictest20698-21fschecks_time10:15:223
18388994219,5cyclictest765650wget10:00:191
1838899421,7cyclictest7850irq/56-dwc_otg07:40:151
174812426,24sleep117576-21sed07:05:321
118202421,3sleep30-21swapper/311:59:113
1838999412,24cyclictest8050irq/56-dwc_otg_09:35:172
18388994124,4cyclictest27504-21aten_rbpower_en07:35:191
18388994111,7cyclictest8050irq/56-dwc_otg_12:15:171
18387994125,5cyclictest0-21swapper/007:43:230
1839099407,23cyclictest0-21swapper/308:00:173
1839099402,30cyclictest8050irq/56-dwc_otg_08:10:163
18389994013,17cyclictest0-21swapper/211:15:172
1838899402,4cyclictest7850irq/56-dwc_otg11:15:171
1838899402,25cyclictest14521-21tr09:30:191
18388994017,6cyclictest18686-21aten_rbpower_vo08:40:201
1838899401,3cyclictest0-21swapper/109:20:181
18389993913,10cyclictest767-21cut10:30:192
18388993916,6cyclictest27587-21fschecks_time09:45:221
1838899391,28cyclictest26055-21ls11:00:051
18388993911,4cyclictest0-21swapper/109:55:161
1838999386,21cyclictest24873-21timerwakeupswit11:35:312
18389993824,5cyclictest11866-21ssh11:59:152
1838999381,25cyclictest1307-21ls12:25:052
18388993819,5cyclictest27251-21seq09:03:461
1839099379,6cyclictest7850irq/56-dwc_otg11:45:043
1839099374,5cyclictest7950irq/56-dwc_otg_10:20:193
1839099374,5cyclictest7950irq/56-dwc_otg_10:20:193
18390993715,4cyclictest0-21swapper/311:10:043
1838999374,20cyclictest0-21swapper/211:20:182
18389993725,5cyclictest13701-21ssh12:39:342
18389993723,5cyclictest2287450rm11:34:102
1838999372,11cyclictest7950irq/56-dwc_otg_09:40:182
18389993717,10cyclictest61002sleep209:20:232
1838999371,23cyclictest771-21ls07:50:232
1838999371,19cyclictest30190-21munin-run11:05:052
1838999371,17cyclictest3220-21ssh11:49:122
1838899374,11cyclictest0-21swapper/112:05:171
1838899373,4cyclictest0-21swapper/111:30:161
1838899372,3cyclictest0-21swapper/110:20:171
1838899372,3cyclictest0-21swapper/110:20:171
18388993711,13cyclictest0-21swapper/111:50:211
179772372,26sleep30-21swapper/307:08:253
1839099367,14cyclictest8050irq/56-dwc_otg_09:00:053
1839099363,24cyclictest14086-21ls10:45:253
1838999367,24cyclictest0-21swapper/209:25:122
1838999366,12cyclictest7850irq/56-dwc_otg09:15:302
18389993617,7cyclictest29810-21fschecks_count09:10:222
18389993614,12cyclictest301ktimersoftd/212:12:062
18389993613,13cyclictest0-21swapper/209:30:222
1838999361,23cyclictest21664-21latency_hist10:55:062
1838999361,23cyclictest21664-21latency_hist10:55:062
1838999361,23cyclictest13493-21uniq08:25:222
1838999361,23cyclictest13493-21uniq08:25:222
1838899365,22cyclictest5845-21ssh12:30:171
18388993620,7cyclictest5241-21fschecks_count10:35:211
18388993615,6cyclictest23-21ksoftirqd/111:26:091
18388993610,21cyclictest8050irq/56-dwc_otg_11:35:151
1839099357,13cyclictest7950irq/56-dwc_otg_10:50:213
1839099357,13cyclictest7950irq/56-dwc_otg_10:50:213
1839099352,7cyclictest11486-21munin-run10:05:043
18390993520,4cyclictest24055-21fschecks_count11:35:213
18390993511,10cyclictest0-21swapper/312:09:563
18389993519,4cyclictest17647-21missed_timers12:05:282
1838999351,25cyclictest4382-21ls08:00:222
1838999351,23cyclictest8211-21missed_timers08:10:262
1838999351,17cyclictest0-21swapper/211:25:242
1838899357,15cyclictest0-21swapper/109:35:201
1838899355,5cyclictest7850irq/56-dwc_otg07:55:331
18388993518,5cyclictest26320-21wget11:00:181
1838899351,27cyclictest1022-21ls07:50:261
18388993511,5cyclictest8050irq/56-dwc_otg_09:10:051
1838899351,11cyclictest21514-21grep08:45:321
18388993510,5cyclictest7850irq/56-dwc_otg07:35:051
1839099349,5cyclictest7850irq/56-dwc_otg09:40:053
1839099346,16cyclictest0-21swapper/311:30:183
1839099346,13cyclictest7950irq/56-dwc_otg_09:10:273
18390993411,7cyclictest0-21swapper/310:30:223
18390993410,4cyclictest0-21swapper/312:15:303
1838999346,5cyclictest31-21ksoftirqd/210:35:252
1838999342,11cyclictest16328-21df_inode10:10:212
18389993418,4cyclictest31742-21sh11:44:482
18389993416,8cyclictest8786-21sh12:33:252
18389993416,5cyclictest13173-21iwlist12:00:252
18389993416,4cyclictest28018-21sh09:45:282
18389993412,6cyclictest0-21swapper/210:56:162
18389993412,4cyclictest0-21swapper/208:55:182
1838999341,23cyclictest31839-21uniq09:50:232
1838999341,23cyclictest31839-21uniq09:50:232
1838999341,22cyclictest20724-21fschecks_time08:45:222
18389993411,6cyclictest32568-21sh11:06:492
18389993411,15cyclictest0-21swapper/210:07:512
1838899341,5cyclictest7411-21latency_hist08:10:051
1838899341,5cyclictest7411-21latency_hist08:10:051
18388993415,10cyclictest4021-21/usr/sbin/munin08:00:211
1839099337,10cyclictest8050irq/56-dwc_otg_08:40:273
1839099335,8cyclictest0-21swapper/307:45:333
1839099332,11cyclictest39-21ksoftirqd/309:40:173
18390993314,5cyclictest23305-21vmstat10:55:343
18390993311,9cyclictest0-21swapper/312:25:253
18390993310,11cyclictest0-21swapper/312:10:193
1838999334,13cyclictest8050irq/56-dwc_otg_07:10:342
18389993312,4cyclictest31788-21/usr/sbin/munin07:45:302
1838999331,14cyclictest0-21swapper/209:02:222
1838899339,11cyclictest0-21swapper/110:43:261
1838899338,10cyclictest7850irq/56-dwc_otg11:40:221
1838899333,8cyclictest518-21rs:main1
18388993313,6cyclictest29187-21diskmemload11:49:461
1838899331,23cyclictest21600-21uniq12:10:221
1838899331,21cyclictest21375-21timerandwakeup10:15:311
1839099327,9cyclictest8050irq/56-dwc_otg_09:17:353
1839099327,16cyclictest0-21swapper/311:45:173
1839099326,6cyclictest8050irq/56-dwc_otg_11:25:053
1839099326,5cyclictest0-21swapper/312:00:323
1839099326,16cyclictest0-21swapper/307:15:183
1839099326,16cyclictest0-21swapper/307:15:183
1839099326,15cyclictest0-21swapper/308:20:193
18390993215,5cyclictest26873-21grep09:00:343
1838999328,14cyclictest0-21swapper/211:50:182
1838999327,7cyclictest29336-21ssh09:10:062
1838999322,13cyclictest7850irq/56-dwc_otg08:50:232
18389993212,11cyclictest18416-21latency_hist08:40:052
1838899329,10cyclictest0-21swapper/111:12:021
1838899328,12cyclictest0-21swapper/110:34:101
1838899326,13cyclictest31749-21grep11:05:321
1838899324,12cyclictest8050irq/56-dwc_otg_10:10:221
18388993215,5cyclictest7850irq/56-dwc_otg10:45:191
1838899321,17cyclictest32062-21ssh09:12:401
1838899321,16cyclictest0-21swapper/111:57:141
18388993211,4cyclictest0-21swapper/112:25:301
1838899321,12cyclictest0-21swapper/107:12:561
1839099319,5cyclictest0-21swapper/311:02:203
1839099318,5cyclictest8050irq/56-dwc_otg_09:58:483
1839099316,13cyclictest7659-21timerandwakeup11:15:323
1839099312,6cyclictest17935-21unixbench_singl08:35:333
1838999319,12cyclictest0-21swapper/207:15:182
1838999319,12cyclictest0-21swapper/207:15:182
1838999317,14cyclictest16140-21sshd08:30:372
1838999311,14cyclictest25814-21sed10:20:342
1838999311,14cyclictest25814-21sed10:20:342
1838899316,5cyclictest8050irq/56-dwc_otg_08:55:211
1838899315,6cyclictest7850irq/56-dwc_otg12:20:271
1838899311,6cyclictest21773-21cron07:20:051
1838899311,6cyclictest21773-21cron07:20:051
1839099308,4cyclictest0-21swapper/312:30:143
18390993022,6cyclictest3189-21ssh11:10:293
1839099301,5cyclictest28716-21grep09:05:343
1838999306,15cyclictest0-21swapper/207:25:152
1838999304,13cyclictest8050irq/56-dwc_otg_10:05:052
1838999302,10cyclictest0-21swapper/210:44:262
1838999301,8cyclictest25967-21fschecks_time07:30:232
1838999301,16cyclictest27475-21aten_rbpower_cu07:35:192
1838999301,12cyclictest0-21swapper/207:57:052
1839099299,4cyclictest0-21swapper/310:29:333
1839099291,5cyclictest6877-21sort09:20:333
1839099291,4cyclictest14020-21munin-run12:40:053
1838999293,17cyclictest0-21swapper/208:15:222
1838999292,17cyclictest0-21swapper/207:20:242
18388992912,7cyclictest32091-21cat07:45:331
1839099287,4cyclictest7950irq/56-dwc_otg_07:30:233
1839099285,6cyclictest0-21swapper/308:30:283
1838999286,12cyclictest0-21swapper/208:40:232
1838999282,17cyclictest0-21swapper/208:20:262
1838999282,12cyclictest0-21swapper/207:40:212
1838999281,11cyclictest0-21swapper/208:06:192
1838999281,11cyclictest0-21swapper/208:06:192
1838899284,11cyclictest7950irq/56-dwc_otg_08:30:361
18388992811,4cyclictest0-21swapper/108:27:181
18388992811,4cyclictest0-21swapper/108:27:181
18390992718,6cyclictest20669-21fschecks_count08:45:213
1839099271,4cyclictest7850irq/56-dwc_otg07:30:043
1839099263,6cyclictest7850irq/56-dwc_otg07:35:243
18390992619,5cyclictest2583-21idleruntime07:55:233
1839099261,5cyclictest8050irq/56-dwc_otg_07:50:193
1838899264,5cyclictest9678-21sort08:15:211
1839099251,6cyclictest23016-21sensors_temp08:50:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional