You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-13 - 09:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Sep 13, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2702021220,4sleep0101ktimersoftd/023:04:580
1978021010,4sleep219343-21/usr/sbin/munin22:55:422
1852521000,2sleep10-21swapper/120:40:341
1852521000,2sleep10-21swapper/120:40:341
1787899996,64cyclictest0-21swapper/021:45:340
1787899996,64cyclictest0-21swapper/021:45:340
172322940,4sleep30-21swapper/320:35:453
1787899841,58cyclictest21428-21apt-config19:20:210
17878998224,42cyclictest7850irq/56-dwc_otg_22:45:420
1787899791,28cyclictest7147-21cron23:20:200
1787899791,26cyclictest0-21swapper/022:05:340
262422770,2sleep31788199cyclictest00:21:263
1787899772,21cyclictest0-21swapper/019:25:440
1787899753,33cyclictest0-21swapper/021:25:300
1787899752,19cyclictest0-21swapper/000:25:470
1787899752,19cyclictest0-21swapper/000:25:470
1787899741,9cyclictest0-21swapper/019:55:340
17878997415,10cyclictest7750irq/56-dwc_otg21:10:330
57892730,3sleep00-21swapper/023:56:440
17878997315,11cyclictest0-21swapper/023:43:130
1787899723,55cyclictest21306-21ssh22:57:000
1787899722,26cyclictest0-21swapper/000:20:350
1787899721,22cyclictest23715-21/usr/sbin/munin20:55:310
3962710,2sleep20-21swapper/221:52:412
1787899712,19cyclictest0-21swapper/021:57:060
1787899711,12cyclictest0-21swapper/022:25:330
1787899711,12cyclictest0-21swapper/022:25:330
17878997110,14cyclictest18873-21open_inodes21:35:460
46852700,2sleep10-21swapper/122:36:451
17878997049,6cyclictest6976-21ssh22:40:320
17878997024,3cyclictest0-21swapper/023:25:200
1787899702,26cyclictest0-21swapper/021:40:320
17878997017,5cyclictest0-21swapper/000:03:250
1787899701,15cyclictest21020-21sed23:35:480
210022690,3sleep221005-21unixbench-2d23:35:482
17878996923,24cyclictest0-21swapper/023:10:350
1787899691,12cyclictest0-21swapper/022:51:180
1787899682,19cyclictest0-21swapper/000:17:440
17878996815,5cyclictest0-21swapper/022:11:150
17878996812,4cyclictest7850irq/56-dwc_otg_19:20:240
184482670,2sleep30-21swapper/322:15:173
1787899677,15cyclictest0-21swapper/021:15:350
1787899677,15cyclictest0-21swapper/021:15:350
1787899672,38cyclictest6608-21sed20:05:480
1787899672,38cyclictest6608-21sed20:05:480
1787899671,7cyclictest15730-21grep20:30:490
1787899671,25cyclictest0-21swapper/021:31:430
1787899661,20cyclictest0-21swapper/021:20:460
1787899657,13cyclictest0-21swapper/023:27:440
1787899657,13cyclictest0-21swapper/023:27:440
1787899651,18cyclictest0-21swapper/022:22:450
17878996511,12cyclictest0-21swapper/019:15:200
1787899649,4cyclictest18712chrt22:34:100
1787899643,35cyclictest22022-21grep20:50:350
17878996414,12cyclictest16203-21systemd-journal00:13:270
17878996414,12cyclictest16203-21systemd-journal00:13:270
1787899641,16cyclictest0-21swapper/000:37:410
1787899641,15cyclictest0-21swapper/000:31:360
296042630,2sleep20-21swapper/222:28:042
296042630,2sleep20-21swapper/222:28:042
1787899635,17cyclictest0-21swapper/021:05:320
1787899632,22cyclictest0-21swapper/020:41:430
1787899632,22cyclictest0-21swapper/020:41:430
17878996312,12cyclictest0-21swapper/022:19:540
1787899631,20cyclictest0-21swapper/023:08:480
1787899629,10cyclictest0-21swapper/020:25:200
1787899622,24cyclictest0-21swapper/020:00:560
1787899622,22cyclictest0-21swapper/000:09:090
1787899621,4cyclictest0-21swapper/021:51:280
1787899621,25cyclictest21837-21latency_hist20:50:200
1787899621,19cyclictest0-21swapper/022:02:450
1787899614,14cyclictest17569-21sh23:31:490
17878996112,28cyclictest3638-21basename23:55:190
1787899611,15cyclictest4859-21sh22:37:040
230612600,2sleep10-21swapper/123:00:161
1787899602,23cyclictest0-21swapper/020:17:140
1787899601,22cyclictest0-21swapper/019:50:260
1787899597,18cyclictest0-21swapper/020:25:460
1787899591,18cyclictest29680-21ssh23:46:100
1787899589,11cyclictest0-21swapper/020:14:320
1787899582,28cyclictest28804-21latency_hist19:40:200
17878995816,4cyclictest7950irq/56-dwc_otg_19:45:420
17878995815,6cyclictest0-21swapper/019:41:590
1787899581,32cyclictest25365-21aten_rbpower_vo19:30:350
17878995611,9cyclictest26939-21chrt21:03:480
17878995427,6cyclictest17235-21proc_pri20:35:450
301622530,2sleep20-21swapper/223:47:032
1788099493,4cyclictest7850irq/56-dwc_otg_22:15:322
17881994820,6cyclictest14363-21idleruntime-cro22:10:193
17881994716,7cyclictest4907-21fschecks_count00:35:383
1788099477,25cyclictest0-21swapper/219:15:342
17879994713,5cyclictest18840-21ssh22:15:341
17881994620,4cyclictest7240-21sshd22:00:443
17879994619,5cyclictest60-21kswapd022:30:311
176042463,10sleep00-21swapper/019:10:180
1788099457,24cyclictest0-21swapper/223:15:352
1788099457,24cyclictest0-21swapper/219:40:342
1787999445,24cyclictest31673-21/usr/sbin/munin23:10:401
17879994418,11cyclictest28886-21diskmemload21:35:461
1787999437,5cyclictest0-21swapper/123:25:331
1787999437,5cyclictest0-21swapper/123:25:331
1787999431,32cyclictest0-21swapper/100:25:331
1787999431,32cyclictest0-21swapper/100:25:331
1788199427,21cyclictest0-21swapper/319:10:353
17881994213,13cyclictest0-21swapper/323:00:483
5692410,2sleep30-21swapper/322:31:483
1788199415,26cyclictest0-21swapper/320:15:353
17881994121,4cyclictest1180-21runrttasks00:15:103
17881994121,4cyclictest1180-21runrttasks00:15:103
17881994120,14cyclictest7699-21ssh00:00:163
1787999416,22cyclictest0-21swapper/120:35:271
17879994120,6cyclictest23-21ksoftirqd/123:40:301
17879994115,7cyclictest23-21ksoftirqd/119:20:241
1788199409,17cyclictest0-21swapper/321:25:423
1788199405,17cyclictest0-21swapper/322:15:423
17881994022,10cyclictest4817-21ssh21:57:593
17881994017,5cyclictest8565-21/usr/sbin/munin00:00:403
1788099402,26cyclictest23071-21unixbench_singl20:50:482
1788099401,9cyclictest0-21swapper/200:25:252
1788099401,9cyclictest0-21swapper/200:25:252
17880994017,5cyclictest24674-21sh00:20:352
17879994022,4cyclictest21-21rcuc/121:45:341
17879994022,4cyclictest21-21rcuc/121:45:341
164302400,1sleep20-21swapper/223:30:412
131372400,2sleep20-21swapper/223:26:142
131372400,2sleep20-21swapper/223:26:142
1788199391,4cyclictest7950irq/56-dwc_otg_20:40:293
1788199391,4cyclictest7950irq/56-dwc_otg_20:40:293
17881993914,4cyclictest2151-21sed21:55:213
17880993917,5cyclictest18246-21fschecks_time19:10:402
1787999398,7cyclictest23-21ksoftirqd/122:05:201
1788199385,6cyclictest20797-21unixbench_multi19:15:493
17881993832,4cyclictest381ktimersoftd/321:10:333
1788099386,20cyclictest7750irq/56-dwc_otg20:05:202
1787999387,11cyclictest19608-21ssh23:35:201
1787999381,4cyclictest7750irq/56-dwc_otg19:45:331
1787999381,27cyclictest31406-21ls23:10:201
1787999381,15cyclictest21-21rcuc/122:45:331
1788199371,25cyclictest27549-21grep21:05:373
1788199371,24cyclictest0-21swapper/322:24:463
1788099371,4cyclictest7850irq/56-dwc_otg_20:05:332
1788099371,4cyclictest7850irq/56-dwc_otg_20:05:332
1788099371,3cyclictest0-21swapper/200:10:362
1788099371,3cyclictest0-21swapper/200:10:362
17880993713,7cyclictest0-21swapper/222:48:122
1787999377,21cyclictest0-21swapper/123:15:351
1787999373,8cyclictest1911-21latency_hist21:55:201
1787999373,4cyclictest7850irq/56-dwc_otg_00:40:201
17879993727,6cyclictest23828-21wget20:55:351
17879993718,11cyclictest23-21ksoftirqd/123:23:171
17879993716,11cyclictest23-21ksoftirqd/120:30:361
17879993714,4cyclictest21-21rcuc/100:20:201
1788199362,22cyclictest0-21swapper/321:22:453
1788099363,15cyclictest7750irq/56-dwc_otg23:05:432
17880993623,7cyclictest7850irq/56-dwc_otg_00:17:452
1788099361,5cyclictest2575650ssh22:23:342
17880993615,14cyclictest301ktimersoftd/200:00:462
1787999369,5cyclictest7750irq/56-dwc_otg21:20:201
1787999369,5cyclictest7750irq/56-dwc_otg21:20:201
1787999366,13cyclictest15551-21grep22:10:451
17879993616,5cyclictest13178-21if_eth020:25:401
1787999361,20cyclictest14070-21ssh21:30:401
1788199359,3cyclictest0-21swapper/321:41:493
1788199357,20cyclictest0-21swapper/319:50:343
1788199355,17cyclictest0-21swapper/320:50:393
17881993520,6cyclictest692850ssh22:40:293
17881993512,5cyclictest4047-21if_eth020:00:373
1788199351,23cyclictest20416-21ssh23:35:403
1788199351,18cyclictest381ktimersoftd/319:25:513
1788199351,18cyclictest16064-21ssh23:30:363
17881993510,5cyclictest7850irq/56-dwc_otg_23:45:203
1788099354,17cyclictest7750irq/56-dwc_otg20:55:462
1788099352,5cyclictest7850irq/56-dwc_otg_19:20:362
17880993520,8cyclictest8632-21ssh22:02:262
1788099351,7cyclictest3999-21seq23:55:352
17880993515,5cyclictest32162-21grep22:30:462
17880993513,6cyclictest11998-21ssh22:06:002
17880993513,12cyclictest149432sleep221:30:522
1787999359,7cyclictest28154-21unixbench_multi19:35:461
1787999355,11cyclictest7750irq/56-dwc_otg19:25:391
1787999354,5cyclictest720650ssh22:40:361
17879993518,9cyclictest25116-21kernelversion00:20:411
17879993518,10cyclictest21-21rcuc/121:06:171
17879993514,5cyclictest21-21rcuc/100:00:361
58522340,2sleep30-21swapper/323:17:573
1788199341,3cyclictest0-21swapper/300:35:073
1788199341,17cyclictest24421-21latency_hist00:20:203
1788099348,13cyclictest19925-21latency_hist20:45:212
1788099348,13cyclictest19925-21latency_hist20:45:212
1788099342,23cyclictest0-21swapper/222:13:352
1788099341,5cyclictest10806-21grep21:25:472
1788099341,24cyclictest11978-21timerwakeupswit20:20:492
1788099341,23cyclictest7750irq/56-dwc_otg22:35:402
1788099341,20cyclictest27885-21missed_timers21:05:422
1788099341,17cyclictest13090-21fschecks_time20:25:392
17880993410,12cyclictest31-21ksoftirqd/221:20:212
17880993410,12cyclictest31-21ksoftirqd/221:20:212
1787999348,6cyclictest23-21ksoftirqd/123:50:401
1787999345,17cyclictest21-21rcuc/119:55:401
17879993428,4cyclictest23-21ksoftirqd/123:35:481
1787999342,4cyclictest11882-21ssh22:05:491
17879993421,4cyclictest21-21rcuc/123:02:031
1787999341,4cyclictest7950irq/56-dwc_otg_21:10:321
1788199332,16cyclictest39-21ksoftirqd/323:48:463
17881993316,4cyclictest39-21ksoftirqd/323:05:353
1788199331,5cyclictest0-21swapper/322:37:563
17881993315,5cyclictest7227-21/usr/sbin/munin20:10:443
1788099339,5cyclictest4750irq/17-3f00b88020:35:222
1788099337,23cyclictest0-21swapper/220:15:262
1788099334,18cyclictest7750irq/56-dwc_otg22:40:342
1788099332,20cyclictest0-21swapper/200:34:062
17880993314,4cyclictest26983-21latency21:45:412
17880993314,4cyclictest26983-21latency21:45:412
17880993312,5cyclictest32300-21sh21:13:532
1788099331,23cyclictest0-21swapper/219:30:052
1788099331,21cyclictest0-21swapper/223:10:482
1788099331,19cyclictest0-21swapper/223:21:532
1787999338,6cyclictest7850irq/56-dwc_otg_22:50:401
1787999338,5cyclictest7950irq/56-dwc_otg_22:25:481
1787999338,5cyclictest7950irq/56-dwc_otg_22:25:481
1787999335,13cyclictest7950irq/56-dwc_otg_00:30:391
1787999331,7cyclictest21681-21basename20:50:191
1787999331,17cyclictest7750irq/56-dwc_otg23:57:251
1787999331,15cyclictest0-21swapper/121:25:471
1788199326,15cyclictest0-21swapper/300:25:333
1788199326,15cyclictest0-21swapper/300:25:333
1788199326,11cyclictest7950irq/56-dwc_otg_20:55:393
1788199324,16cyclictest7850irq/56-dwc_otg_19:20:443
1788199323,17cyclictest0-21swapper/322:47:173
1788199321,16cyclictest7950irq/56-dwc_otg_22:50:433
17881993211,13cyclictest14122-21cut21:30:413
17881993210,4cyclictest0-21swapper/322:55:453
1788099328,14cyclictest0-21swapper/222:50:372
1788099324,14cyclictest7850irq/56-dwc_otg_19:50:212
1788099322,13cyclictest0-21swapper/219:47:052
17880993214,5cyclictest8818-21basename20:15:192
1788099321,22cyclictest0-21swapper/221:44:582
1788099321,22cyclictest0-21swapper/221:35:212
1788099321,14cyclictest2955-21grep21:55:422
1787999327,15cyclictest0-21swapper/121:40:321
1787999327,13cyclictest7850irq/56-dwc_otg_21:00:491
1787999326,6cyclictest0-21swapper/120:00:291
1787999325,13cyclictest7950irq/56-dwc_otg_20:55:191
1787999324,8cyclictest22992-21grep22:20:331
1787999323,16cyclictest7950irq/56-dwc_otg_20:20:381
1787999321,6cyclictest7950irq/56-dwc_otg_21:25:201
1787999321,17cyclictest0-21swapper/119:17:121
17879993210,4cyclictest20315-21grep00:15:211
17879993210,4cyclictest20315-21grep00:15:211
17881993116,4cyclictest381ktimersoftd/321:00:343
1788199311,20cyclictest12126-21ssh23:25:393
1788199311,20cyclictest12126-21ssh23:25:393
1788199311,19cyclictest0-21swapper/322:28:343
1788199311,19cyclictest0-21swapper/322:28:343
1788099318,14cyclictest0-21swapper/223:52:432
1788099317,7cyclictest27853-21ssh23:45:212
1788099316,12cyclictest7850irq/56-dwc_otg_23:00:452
1788099315,12cyclictest7950irq/56-dwc_otg_00:35:402
1788099312,19cyclictest0-21swapper/221:00:432
1788099312,15cyclictest0-21swapper/219:59:532
1788099311,19cyclictest0-21swapper/219:35:422
1788099311,16cyclictest5861-21cat21:20:412
1787999318,10cyclictest23-21ksoftirqd/119:55:211
1787999316,14cyclictest7950irq/56-dwc_otg_19:15:201
1787999313,13cyclictest7750irq/56-dwc_otg21:55:351
1787999311,22cyclictest0-21swapper/119:45:201
17879993112,10cyclictest0-21swapper/100:08:061
1788199306,15cyclictest0-21swapper/319:40:343
1788199304,8cyclictest7750irq/56-dwc_otg20:30:463
1788199304,5cyclictest7950irq/56-dwc_otg_19:35:463
1788199304,17cyclictest9937-21rm23:22:553
1788199303,3cyclictest7850irq/56-dwc_otg_23:52:453
1788199301,6cyclictest443-21unixbench_multi23:10:513
17881993013,5cyclictest1180-21runrttasks20:24:543
1788199301,19cyclictest13478-21grep00:05:493
1788199301,10cyclictest0-21swapper/321:40:153
1788099307,15cyclictest518-21in:imuxsock19:32:142
1788099302,16cyclictest0-21swapper/220:48:562
1788099301,12cyclictest12253-21ssh00:05:342
1787999305,10cyclictest7950irq/56-dwc_otg_20:10:311
1787999304,6cyclictest7850irq/56-dwc_otg_20:05:381
1787999304,6cyclictest7850irq/56-dwc_otg_20:05:381
1787999302,10cyclictest7750irq/56-dwc_otg20:17:331
17879993015,4cyclictest7750irq/56-dwc_otg23:45:211
162712305,13sleep20-21swapper/219:05:342
1788199294,13cyclictest1913-21wget19:55:373
1788199291,7cyclictest21652-21cron20:50:193
17881992913,5cyclictest381ktimersoftd/321:47:463
17881992913,5cyclictest381ktimersoftd/321:47:463
1788199291,18cyclictest0-21swapper/321:18:483
1788199291,18cyclictest0-21swapper/321:18:483
1788099292,13cyclictest0-21swapper/220:33:342
17879992919,8cyclictest0-21swapper/119:35:191
170412294,14sleep117140-21unixbench_multi19:05:461
17881992818,7cyclictest7096-21latency_hist20:10:213
17881992818,7cyclictest7096-21latency_hist20:10:213
1788199274,5cyclictest7950irq/56-dwc_otg_20:25:413
1788199262,5cyclictest0-21swapper/319:50:093
172592263,13sleep30-21swapper/319:05:563
1788199257,9cyclictest0-21swapper/319:30:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional