You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 14:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat Apr 25, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
320422990,5sleep11690199cyclictest17:11:121
125272920,10sleep00-21swapper/018:50:500
72212870,1sleep30-21swapper/318:44:203
247992860,13sleep37950irq/56-dwc_otg_19:42:503
247992860,13sleep37950irq/56-dwc_otg_19:42:503
47612840,2sleep00-21swapper/017:26:080
16902998216,19cyclictest7750irq/56-dwc_otg17:50:562
16902998216,19cyclictest7750irq/56-dwc_otg17:50:562
16901997830,25cyclictest7336-21apt-get20:41:021
16901997830,25cyclictest7336-21apt-get20:41:021
40372750,2sleep2301ktimersoftd/219:18:052
1690099746,37cyclictest9-21ksoftirqd/017:31:000
16901997020,18cyclictest17898-21ssh21:32:041
1690099672,23cyclictest0-21swapper/021:46:020
19822660,2sleep10-21swapper/119:55:201
1690099661,50cyclictest25730-21fschecks_count19:06:050
1690099661,43cyclictest12058-21unixbench_multi20:06:170
1690099653,37cyclictest13-21rcuc/018:16:010
1690099651,50cyclictest23770-21timerwakeupswit19:41:160
1690099651,50cyclictest23770-21timerwakeupswit19:41:160
1690099651,37cyclictest4827-21fschecks_time21:56:080
1690099651,37cyclictest4827-21fschecks_time21:56:080
16900996446,6cyclictest7950irq/56-dwc_otg_20:26:020
1690099641,48cyclictest32484-21grep17:11:180
16900996412,30cyclictest13356-21missed_timers18:51:110
176822630,1sleep30-21swapper/320:52:433
1690099635,46cyclictest0-21swapper/019:56:020
1690099632,53cyclictest0-21swapper/020:25:490
1690099632,42cyclictest26408-21fschecks_count18:26:070
16900996318,26cyclictest13-21rcuc/019:15:560
1690099622,46cyclictest8719-21proc_pri17:36:130
1690099621,39cyclictest3242-21aten_rbpower_en20:36:040
1690099621,36cyclictest0-21swapper/019:36:180
166182620,54sleep30-21swapper/316:30:413
1690099612,48cyclictest0-21swapper/021:01:150
1690099612,45cyclictest29861-21fschecks_time17:06:080
1690099612,42cyclictest0-21swapper/018:36:180
202542600,1sleep20-21swapper/221:35:582
16901996022,18cyclictest5113-21ps17:26:141
16901996021,21cyclictest21302-21latency18:11:101
16900996033,4cyclictest30917-21ssh19:51:020
1690099602,40cyclictest17683-21grep18:56:110
1690099602,40cyclictest0-21swapper/020:01:130
157692600,2sleep10-21swapper/118:54:251
23742590,3sleep0101ktimersoftd/017:20:490
1690099598,35cyclictest20138-21latency_hist21:35:500
1690099595,36cyclictest0-21swapper/021:20:490
1690099592,42cyclictest0-21swapper/020:51:020
1690099592,42cyclictest0-21swapper/016:46:120
169882580,2sleep20-21swapper/218:56:032
16901995824,15cyclictest18807-21/usr/sbin/munin19:36:071
1690099581,44cyclictest19238-21wc16:36:090
1690099581,38cyclictest8421-21ssh21:21:080
16901995726,19cyclictest24545-21netstat21:01:111
16901995723,19cyclictest2768-21df_inode17:21:061
16901995720,17cyclictest29111-21/usr/sbin/munin21:46:081
16901995717,28cyclictest7714-21grep21:20:491
16901995717,21cyclictest13431-21ssh21:26:281
16900995735,6cyclictest7850irq/56-dwc_otg_21:40:560
1690099571,41cyclictest2869-21fschecks_time17:21:070
54002560,2sleep20-21swapper/221:56:162
54002560,2sleep20-21swapper/221:56:162
1690299564,13cyclictest29-21rcuc/221:01:002
1690099567,28cyclictest9-21ksoftirqd/020:31:070
16900995626,7cyclictest657-21sort21:51:050
1690099561,42cyclictest0-21swapper/020:41:180
1690099561,42cyclictest0-21swapper/020:41:180
1690099561,40cyclictest0-21swapper/018:31:010
1690099561,40cyclictest0-21swapper/018:31:010
16901995517,26cyclictest10479-21sh21:23:381
16901995516,25cyclictest14117-21sh20:09:541
16900995527,5cyclictest15094-21apt17:55:490
16900995527,5cyclictest15094-21apt17:55:490
1690099551,40cyclictest10168-21aten_rbpower_cu19:26:020
1690099551,22cyclictest5166-21switchtime18:41:160
273132540,2sleep00-21swapper/019:46:090
221142540,2sleep30-21swapper/319:01:153
16901995420,16cyclictest20167-21ssh18:59:401
16901995419,16cyclictest23165-21cat18:16:111
16901995415,12cyclictest29439-21ssh21:06:351
16900995429,14cyclictest21153-21grep16:41:110
1690099542,41cyclictest0-21swapper/021:07:570
1690299536,41cyclictest0-21swapper/217:21:032
1690199538,9cyclictest0-21swapper/117:56:011
16901995322,13cyclictest11030-21ssh18:48:341
16901995321,14cyclictest1925-21ssh21:51:451
16901995320,18cyclictest7850irq/56-dwc_otg_19:33:031
16901995320,17cyclictest12019-21ssh19:27:201
16901995320,14cyclictest23768-21kernelversion20:21:091
16901995318,20cyclictest21750-21ssh20:18:431
16901995317,21cyclictest3793-21ssh18:40:461
16901995315,22cyclictest6152-21/usr/sbin/munin17:31:121
16901995315,19cyclictest32484-21ssh19:13:431
16901995310,24cyclictest7750irq/56-dwc_otg20:32:281
1690099532,39cyclictest0-21swapper/020:18:180
1690099532,37cyclictest0-21swapper/021:26:030
1690099531,40cyclictest0-21swapper/017:46:150
1690099531,39cyclictest0-21swapper/018:01:070
16902995212,8cyclictest7850irq/56-dwc_otg_21:50:592
16901995218,21cyclictest9382-21ssh20:03:471
16901995217,11cyclictest2318-21munin-run17:20:481
16901995211,20cyclictest7950irq/56-dwc_otg_20:55:491
16900995217,16cyclictest27146-21users16:56:200
1690099521,42cyclictest0-21swapper/021:35:500
16901995122,16cyclictest5313-21ssh19:58:531
16901995120,21cyclictest25570-21grep19:06:031
16901995120,13cyclictest7937-21ssh18:45:361
16901995118,12cyclictest1755-21/usr/sbin/munin19:16:041
16901995116,12cyclictest12995-21ssh20:46:431
1690099511,40cyclictest1289-21latency_hist19:15:490
1690099511,39cyclictest21515-21proc_pri18:11:140
1690099511,35cyclictest27759-21/usr/sbin/munin17:00:590
1690099511,35cyclictest27759-21/usr/sbin/munin17:00:590
1690099511,34cyclictest16254-21unixbench_singl20:11:170
1690099511,34cyclictest16254-21unixbench_singl20:11:170
181412500,21sleep37950irq/56-dwc_otg_19:35:503
1690399507,18cyclictest1683-21ssh19:16:023
1690399503,5cyclictest0-21swapper/320:56:013
16901995020,14cyclictest3226-21sh21:15:271
16901995020,14cyclictest3226-21sh21:15:271
16901995019,14cyclictest18543-21ssh20:15:251
16901995019,14cyclictest18543-21ssh20:15:251
16901995010,27cyclictest7950irq/56-dwc_otg_18:32:191
16901995010,27cyclictest7950irq/56-dwc_otg_18:32:191
1690099503,30cyclictest0-21swapper/016:51:170
1690099502,36cyclictest0-21swapper/019:31:080
1690099501,38cyclictest0-21swapper/021:11:140
1690099501,38cyclictest0-21swapper/021:11:140
1690399491,4cyclictest7950irq/56-dwc_otg_21:41:003
16903994910,8cyclictest7950irq/56-dwc_otg_19:21:013
1690199499,26cyclictest22366-21runrttasks16:46:031
16901994918,19cyclictest8136-21rm22:00:341
16901994918,19cyclictest8136-21rm22:00:341
1690199491,7cyclictest7750irq/56-dwc_otg16:56:041
16901994915,13cyclictest9797-21sh19:25:491
16901994913,10cyclictest0-21swapper/120:26:471
16901994911,13cyclictest24099-21latency_hist21:40:491
1690099493,37cyclictest0-21swapper/019:01:100
1690099491,35cyclictest20274-21grep20:56:100
1690399488,25cyclictest0-21swapper/319:46:033
1690299488,25cyclictest0-21swapper/221:41:022
1690299485,23cyclictest7950irq/56-dwc_otg_21:31:022
16901994821,11cyclictest29909-21ssh19:49:381
16901994816,14cyclictest18843-21seq18:05:491
1690099482,35cyclictest0-21swapper/019:24:540
16900994816,3cyclictest0-21swapper/017:41:020
16900994814,15cyclictest12645-21unixbench-2d20:46:170
1690099481,32cyclictest0-21swapper/017:56:050
1690299477,25cyclictest0-21swapper/220:26:032
16901994721,14cyclictest27179-21ssh21:43:431
16901994718,21cyclictest23159-21sh19:02:261
16901994714,11cyclictest7750irq/56-dwc_otg20:39:521
162892472,35sleep00-21swapper/016:26:210
1690399467,25cyclictest0-21swapper/318:16:043
1690399461,5cyclictest7950irq/56-dwc_otg_17:16:033
1690199463,17cyclictest0-21swapper/118:21:021
16901994612,12cyclictest20851-21grep20:56:171
1690199457,25cyclictest0-21swapper/116:51:021
1690199453,8cyclictest7750irq/56-dwc_otg19:41:001
1690199453,8cyclictest7750irq/56-dwc_otg19:41:001
16901994527,14cyclictest19028-21aten_rbpower_po18:06:041
16900994532,5cyclictest7750irq/56-dwc_otg16:33:160
1690399447,24cyclictest0-21swapper/321:51:013
1690399447,24cyclictest0-21swapper/317:26:033
1690099442,28cyclictest0-21swapper/018:21:090
1690099441,35cyclictest0-21swapper/018:08:350
16902994318,6cyclictest6726-21sh20:01:012
1690299431,10cyclictest0-21swapper/216:31:022
16901994312,11cyclictest0-21swapper/117:01:031
16901994312,11cyclictest0-21swapper/117:01:031
16901994311,17cyclictest7950irq/56-dwc_otg_17:06:151
16901994311,10cyclictest0-21swapper/118:30:491
1690299427,24cyclictest0-21swapper/216:41:032
1690199427,25cyclictest0-21swapper/117:51:031
1690199427,25cyclictest0-21swapper/117:51:031
16901994213,15cyclictest20236-21chrt16:40:041
16901994212,11cyclictest7850irq/56-dwc_otg_16:42:231
1690399416,24cyclictest0-21swapper/321:46:033
16901994118,12cyclictest18207-21chrt16:32:191
16901994111,11cyclictest0-21swapper/117:41:371
1690399407,7cyclictest7850irq/56-dwc_otg_20:36:173
1690399406,4cyclictest7950irq/56-dwc_otg_21:11:013
1690399406,4cyclictest7950irq/56-dwc_otg_21:11:013
1690299406,25cyclictest0-21swapper/219:31:022
304892390,2sleep30-21swapper/320:29:363
206192390,7sleep320621-21grep21:36:083
16903993920,5cyclictest25739-21fschecks_count19:06:053
16902993916,5cyclictest27739-21diskmemload18:40:032
16901993911,12cyclictest9188-21chrt17:36:441
16901993910,9cyclictest0-21swapper/117:46:551
16903993828,5cyclictest381ktimersoftd/316:36:033
16903993819,12cyclictest5366-21ssh19:59:003
16903993817,13cyclictest39-21ksoftirqd/320:21:193
16903993812,13cyclictest18638-21sh21:33:263
1690299385,7cyclictest0-21swapper/217:56:022
1690299381,24cyclictest6513-21ls17:31:072
1690299381,23cyclictest19249-21cut18:06:072
1690399371,25cyclictest4108-21cut21:16:073
16902993717,4cyclictest13282-21sh20:08:212
16902993716,5cyclictest32358-21sh21:50:392
1690299371,5cyclictest0-21swapper/219:01:032
1690299371,23cyclictest26036-21ls18:25:492
16903993618,4cyclictest39-21ksoftirqd/318:11:013
1690399361,21cyclictest0-21swapper/317:41:083
16903993612,13cyclictest39-21ksoftirqd/318:46:083
1690299366,25cyclictest0-21swapper/219:51:032
1690299364,21cyclictest0-21swapper/218:01:022
16902993618,4cyclictest27534-21latency_hist17:00:502
16902993614,4cyclictest7992-21wget21:21:022
300702350,5sleep21690299cyclictest19:11:092
1690399358,15cyclictest0-21swapper/316:46:033
1690399356,4cyclictest0-21swapper/318:34:103
1690399356,4cyclictest0-21swapper/318:34:103
1690299357,4cyclictest7750irq/56-dwc_otg18:46:072
1690299355,15cyclictest7950irq/56-dwc_otg_20:45:492
1690299355,15cyclictest7950irq/56-dwc_otg_20:45:492
1690299351,29cyclictest4952-21missed_timers17:26:112
16902993510,5cyclictest5497-21sh19:59:142
16902993510,4cyclictest0-21swapper/218:51:082
1690399348,14cyclictest0-21swapper/321:09:113
1690399345,4cyclictest7850irq/56-dwc_otg_17:15:483
16903993425,5cyclictest520-21cron18:30:493
16903993420,5cyclictest32574-21sh19:51:583
16903993413,5cyclictest20276-21unixbench-2d20:16:163
1690399341,21cyclictest26989-21ls16:56:183
1690299348,11cyclictest7750irq/56-dwc_otg20:21:152
1690299347,12cyclictest7850irq/56-dwc_otg_17:11:112
16902993424,7cyclictest31855-21grep20:31:072
16902993417,5cyclictest11937-21if_wlan020:46:082
16902993414,5cyclictest1569750wget20:51:032
1690299341,26cyclictest12427-21ls21:26:072
1690399337,9cyclictest7950irq/56-dwc_otg_20:10:523
1690399337,9cyclictest7950irq/56-dwc_otg_20:10:523
1690399335,9cyclictest7950irq/56-dwc_otg_18:21:003
1690399331,5cyclictest7850irq/56-dwc_otg_21:25:323
16903993312,4cyclictest15725-21sh18:54:203
1690299337,6cyclictest7850irq/56-dwc_otg_19:10:502
1690299337,16cyclictest0-21swapper/217:06:042
1690299336,7cyclictest7850irq/56-dwc_otg_19:21:172
1690299335,19cyclictest18429-21ssh19:35:552
16902993319,4cyclictest20728-21sh20:16:502
1690299331,26cyclictest26432-21grep18:26:072
1690399326,10cyclictest0-21swapper/317:50:063
1690399322,6cyclictest0-21swapper/316:31:003
16903993226,4cyclictest29714-21ssh19:10:553
16903993224,6cyclictest25475-21rm21:01:543
1690299329,11cyclictest0-21swapper/218:35:172
1690299329,11cyclictest0-21swapper/218:35:172
1690299326,5cyclictest7950irq/56-dwc_otg_16:51:122
1690299325,5cyclictest7950irq/56-dwc_otg_16:36:032
1690299325,11cyclictest0-21swapper/221:14:302
1690299325,11cyclictest0-21swapper/221:14:302
1690299322,3cyclictest7850irq/56-dwc_otg_19:48:272
1690299321,4cyclictest28967-21sed21:06:152
56382311,26sleep20-21swapper/221:17:192
23992310,2sleep30-21swapper/318:38:213
1690399318,4cyclictest7950irq/56-dwc_otg_21:30:153
1690399317,4cyclictest0-21swapper/320:41:043
1690399317,4cyclictest0-21swapper/320:41:043
1690399316,8cyclictest7950irq/56-dwc_otg_20:01:043
1690399316,15cyclictest0-21swapper/318:56:033
1690399313,12cyclictest6372-21kworker/u8:020:31:073
1690399312,14cyclictest10043-21rm19:25:523
16903993115,6cyclictest4502-21ssh21:56:023
16903993115,6cyclictest4502-21ssh21:56:023
16903993112,3cyclictest0-21swapper/319:40:463
16903993111,4cyclictest15771-21if_wlan017:56:083
1690299317,6cyclictest7850irq/56-dwc_otg_16:46:072
1690299317,15cyclictest0-21swapper/219:26:022
1690299317,14cyclictest0-21swapper/220:36:042
1690299312,13cyclictest7850irq/56-dwc_otg_17:36:062
1690299311,25cyclictest0-21swapper/218:11:072
16902993111,4cyclictest6973-21sh18:43:552
16902993110,4cyclictest1013-21cut17:16:072
1690399306,6cyclictest0-21swapper/320:09:453
16903993022,6cyclictest381ktimersoftd/316:51:003
16903993012,5cyclictest908-21runrttasks17:04:043
16903993012,5cyclictest908-21runrttasks17:04:043
16903993012,11cyclictest13471-21dpkg17:50:513
16903993012,11cyclictest13471-21dpkg17:50:513
1690299308,4cyclictest0-21swapper/220:55:522
1690299308,4cyclictest0-21swapper/219:40:532
1690299308,4cyclictest0-21swapper/219:40:532
1690299307,14cyclictest0-21swapper/220:11:032
1690299307,14cyclictest0-21swapper/220:11:032
1690299304,13cyclictest7850irq/56-dwc_otg_17:03:572
1690299304,13cyclictest7850irq/56-dwc_otg_17:03:572
1690399299,7cyclictest0-21swapper/320:46:163
1690399285,8cyclictest7850irq/56-dwc_otg_17:37:403
1690399284,4cyclictest7950irq/56-dwc_otg_18:06:183
1690399283,10cyclictest0-21swapper/316:41:023
1690399282,12cyclictest0-21swapper/317:23:033
1690399275,9cyclictest7850irq/56-dwc_otg_17:10:073
1690399271,7cyclictest7750irq/56-dwc_otg17:31:063
1690299272,3cyclictest7950irq/56-dwc_otg_18:16:152
1690299271,23cyclictest0-21swapper/217:41:042
164732273,14sleep20-21swapper/216:28:462
160782277,9sleep10-21swapper/116:26:181
1690399262,6cyclictest7950irq/56-dwc_otg_18:01:023
1690299262,5cyclictest0-21swapper/217:46:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional