You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 07:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Jul 30, 2026 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2195922560,2sleep30-21swapper/319:05:433
23510991243,91cyclictest0-21swapper/022:50:480
235109912353,51cyclictest7950irq/56-dwc_otg_22:25:520
235109912353,51cyclictest7950irq/56-dwc_otg_22:25:520
23510991112,10cyclictest6599-21ntp_states00:30:510
23510991065,67cyclictest0-21swapper/021:20:560
2351299972,8cyclictest7850irq/56-dwc_otg22:25:512
2351299972,8cyclictest7850irq/56-dwc_otg22:25:512
2351299955,55cyclictest0-21swapper/222:50:482
2351099952,10cyclictest5851-21memory23:50:490
285492940,2sleep30-21swapper/321:40:563
285492940,2sleep30-21swapper/321:40:563
2351099925,13cyclictest0-21swapper/022:30:370
2351299904,20cyclictest7850irq/56-dwc_otg00:30:532
2351099891,23cyclictest24038-21switchtime21:35:510
271422850,2sleep10-21swapper/122:19:391
201262850,6sleep02351099cyclictest22:10:440
2351299833,15cyclictest5771-21grep23:50:462
2351299833,14cyclictest0-21swapper/221:20:582
2351099801,33cyclictest0-21swapper/023:45:540
2351099801,25cyclictest22480-21awk00:10:490
2351099781,56cyclictest30520-21munin-plugin-st19:30:250
2351099781,56cyclictest30520-21munin-plugin-st19:30:250
23510997735,7cyclictest7950irq/56-dwc_otg_21:50:350
140652760,2sleep10-21swapper/121:24:391
295052750,2sleep229504-21grep19:25:452
295052750,2sleep229504-21grep19:25:452
293892740,1sleep30-21swapper/300:20:003
291002740,1sleep00-21swapper/022:20:550
2351099726,58cyclictest0-21swapper/020:20:390
23513997115,30cyclictest7950irq/56-dwc_otg_00:30:523
318992680,5sleep339-21ksoftirqd/319:30:543
49952660,2sleep20-21swapper/221:13:172
49952660,2sleep20-21swapper/221:13:172
2351299663,18cyclictest0-21swapper/200:30:262
23513996516,28cyclictest7850irq/56-dwc_otg21:20:583
2351199658,21cyclictest0-21swapper/122:25:551
2351199658,21cyclictest0-21swapper/122:25:551
2351099656,43cyclictest22250-21grep23:30:530
2351099653,47cyclictest0-21swapper/000:25:390
2351099642,32cyclictest7147-21grep21:15:450
196232640,2sleep20-21swapper/200:07:142
196232640,2sleep20-21swapper/200:07:142
23510996310,38cyclictest0-21swapper/021:40:440
23510996310,38cyclictest0-21swapper/021:40:440
2351299628,16cyclictest7950irq/56-dwc_otg_22:55:402
2351299621,5cyclictest22348-21ssh00:10:482
23510996231,4cyclictest8050irq/56-dwc_otg_22:40:380
23510996228,5cyclictest12505-21unixbench_multi22:00:540
2351099621,33cyclictest32753-21wget19:35:400
229772623,52sleep20-21swapper/219:07:012
128832620,7sleep22351299cyclictest23:20:402
23513996116,26cyclictest7850irq/56-dwc_otg22:25:543
23513996116,26cyclictest7850irq/56-dwc_otg22:25:543
23510996126,6cyclictest0-21swapper/021:00:360
2351099611,46cyclictest637-21ssh23:05:440
23510996111,36cyclictest30392-21/usr/sbin/munin23:40:500
9752600,4sleep00-21swapper/000:24:480
2351299609,4cyclictest0-21swapper/223:36:022
2351299607,17cyclictest7950irq/56-dwc_otg_22:30:592
2351299602,6cyclictest0-21swapper/220:30:532
2351299602,6cyclictest0-21swapper/220:30:532
2351299601,8cyclictest1964-21ssh23:45:512
2351199602,11cyclictest20969-21ssh22:50:471
2351099605,31cyclictest0-21swapper/020:25:310
2351099605,26cyclictest0-21swapper/021:30:540
2351099602,32cyclictest16308-21latency_hist20:20:260
2351099601,46cyclictest11723-21ntp_kernel_pll_20:05:480
2351099601,43cyclictest17763-21proc_pri23:25:500
2351399593,11cyclictest18963-21ssh21:30:373
2351399592,8cyclictest0-21swapper/319:40:373
2351299594,4cyclictest0-21swapper/222:05:412
2351299592,16cyclictest19659-21ls21:30:472
2351299592,11cyclictest8050irq/56-dwc_otg_22:36:302
2351199593,26cyclictest0-21swapper/100:30:531
23510995930,5cyclictest8050irq/56-dwc_otg_23:00:440
23510995927,14cyclictest0-21swapper/023:55:420
23510995915,21cyclictest15854-21sh23:24:060
23510995914,42cyclictest101ktimersoftd/000:00:530
2351299587,12cyclictest7950irq/56-dwc_otg_22:16:502
23510995831,6cyclictest8050irq/56-dwc_otg_22:45:390
23510995831,6cyclictest8050irq/56-dwc_otg_22:45:390
2351099581,5cyclictest28405-21ssh23:00:270
2351099581,29cyclictest5403-21sh19:50:260
2351099581,29cyclictest15353-21ssh21:25:450
23510995812,33cyclictest26200-21fschecks_time00:15:440
178412580,2sleep20-21swapper/222:46:422
178412580,2sleep20-21swapper/222:46:422
2351299572,11cyclictest7950irq/56-dwc_otg_21:52:322
23512995719,24cyclictest0-21swapper/220:50:392
2351299571,5cyclictest0-21swapper/222:10:522
2351199572,32cyclictest0-21swapper/121:55:361
23511995717,24cyclictest21-21rcuc/123:50:461
2351099578,35cyclictest0-21swapper/019:15:460
2351299563,15cyclictest7950irq/56-dwc_otg_00:19:032
2351299562,9cyclictest0-21swapper/219:45:382
2351299562,10cyclictest651-21ssh21:46:432
2351299561,6cyclictest2196-21ssh23:06:492
2351099564,32cyclictest0-21swapper/021:45:400
2351299553,14cyclictest28622-21ssh23:00:412
2351299552,6cyclictest23871-21ssh21:35:482
2351299551,45cyclictest0-21swapper/221:00:362
2351099555,28cyclictest0-21swapper/021:10:520
2351099555,28cyclictest0-21swapper/021:10:520
23510995525,5cyclictest15474-21/usr/sbin/munin22:05:490
2351099552,27cyclictest0-21swapper/020:35:450
2351099552,23cyclictest0-21swapper/020:55:410
23510995510,3cyclictest0-21swapper/019:35:260
2351099542,29cyclictest0-21swapper/023:15:460
2351099541,39cyclictest18267-21grep00:05:510
2351099541,39cyclictest18267-21grep00:05:510
2351399534,7cyclictest7950irq/56-dwc_otg_23:45:363
2351299537,4cyclictest7850irq/56-dwc_otg23:10:562
2351299537,4cyclictest7850irq/56-dwc_otg23:10:562
2351099532,34cyclictest0-21swapper/019:55:440
2351099532,34cyclictest0-21swapper/019:55:440
2351099532,26cyclictest0-21swapper/023:35:520
2351099531,29cyclictest24317-21kernelversion22:15:450
285522520,2sleep128550-21sh23:39:431
2351399525,22cyclictest20979-21missed_timers22:50:473
23512995212,8cyclictest11213-21ssh23:56:572
2351299521,17cyclictest0-21swapper/223:40:402
2351099521,28cyclictest21338-21grep20:30:540
2351099521,28cyclictest21338-21grep20:30:540
186022520,3sleep123508-21cyclictest23:26:351
2351099511,4cyclictest5801-21head23:10:560
2351099511,4cyclictest5801-21head23:10:560
2351399507,19cyclictest9550irq/48-DMA15:26:473
2351299507,28cyclictest0-21swapper/221:15:382
2351099491,26cyclictest25478-21cat20:45:260
2351199481,29cyclictest1013-21ssh23:45:331
2351099482,5cyclictest0-21swapper/022:35:410
23510994825,5cyclictest0-21swapper/021:58:430
2351299477,37cyclictest0-21swapper/220:55:382
2351299477,26cyclictest0-21swapper/223:25:382
2351199473,13cyclictest0-21swapper/123:00:391
2351099472,30cyclictest0-21swapper/000:36:410
2351099472,30cyclictest0-21swapper/000:36:410
2351099472,23cyclictest0-21swapper/019:43:330
231252473,35sleep00-21swapper/019:08:570
35302460,2sleep10-21swapper/100:26:541
2351399467,26cyclictest0-21swapper/319:10:393
2351399462,19cyclictest8050irq/56-dwc_otg_00:05:423
2351399462,19cyclictest8050irq/56-dwc_otg_00:05:423
2351199469,24cyclictest0-21swapper/120:45:391
2351199467,25cyclictest0-21swapper/123:05:311
2351099462,33cyclictest0-21swapper/019:54:080
23510994613,5cyclictest250692chrt19:15:130
2351299457,23cyclictest0-21swapper/219:35:312
2351299451,15cyclictest0-21swapper/222:03:242
2351099456,6cyclictest13-21rcuc/020:00:430
95892440,2sleep30-21swapper/323:55:443
23513994412,10cyclictest0-21swapper/323:07:103
2351199449,4cyclictest0-21swapper/100:08:151
2351199449,4cyclictest0-21swapper/100:08:151
2351199446,29cyclictest0-21swapper/123:55:421
2351199444,29cyclictest0-21swapper/122:45:381
2351199444,29cyclictest0-21swapper/122:45:381
23510994414,18cyclictest0-21swapper/020:50:390
23513994320,21cyclictest2350-21ssh21:10:413
23513994320,21cyclictest2350-21ssh21:10:413
2351399431,3cyclictest8050irq/56-dwc_otg_22:35:383
23513994312,5cyclictest8050irq/56-dwc_otg_23:40:363
2351299433,26cyclictest0-21swapper/220:10:372
2351199437,24cyclictest0-21swapper/122:05:311
2351099433,4cyclictest0-21swapper/020:10:380
23510994317,14cyclictest0-21swapper/020:47:370
2351299427,21cyclictest0-21swapper/220:15:392
2351199427,26cyclictest0-21swapper/120:30:391
2351199427,26cyclictest0-21swapper/120:30:391
23513994118,6cyclictest1970-21idleruntime-cro21:10:263
2351299416,25cyclictest0-21swapper/221:25:382
23512994116,18cyclictest7950irq/56-dwc_otg_21:40:442
23512994116,18cyclictest7950irq/56-dwc_otg_21:40:442
2351199417,22cyclictest0-21swapper/120:20:391
2351399406,22cyclictest0-21swapper/319:50:413
2351299406,23cyclictest0-21swapper/220:25:322
2351199407,26cyclictest0-21swapper/119:10:391
2351199406,25cyclictest0-21swapper/119:50:411
23511994020,5cyclictest6564-21sh21:15:361
23511994020,4cyclictest11258-21/usr/sbin/munin22:00:441
23511994010,22cyclictest0-21swapper/119:35:401
2351099402,22cyclictest0-21swapper/019:20:480
2351099401,25cyclictest0-21swapper/021:09:480
2351199397,24cyclictest0-21swapper/123:40:361
2351199397,21cyclictest0-21swapper/119:15:411
23511993911,18cyclictest221ktimersoftd/119:30:391
90292380,3sleep38618-21/usr/sbin/munin23:15:423
2351399382,7cyclictest7434-21cut21:55:423
23513993816,6cyclictest25605-21fschecks_count19:15:453
2351399381,5cyclictest37-21rcuc/322:55:423
2351399381,27cyclictest31589-21ls21:45:413
2351299384,23cyclictest29994-21ssh00:20:402
2351199386,24cyclictest0-21swapper/122:35:411
23511993812,8cyclictest0-21swapper/119:40:371
81072370,2sleep10-21swapper/119:55:471
81072370,2sleep10-21swapper/119:55:471
289372370,2sleep30-21swapper/323:40:263
257792371,30sleep30-21swapper/320:45:423
2351399376,3cyclictest0-21swapper/323:20:393
2351399372,6cyclictest7850irq/56-dwc_otg20:40:393
23513993715,13cyclictest22604-21sh23:31:213
23513993714,5cyclictest20063-21cat22:10:443
2351199377,8cyclictest0-21swapper/121:50:341
2351399362,15cyclictest8050irq/56-dwc_otg_20:05:383
23513993619,5cyclictest4658-21/usr/sbin/munin23:10:413
23513993619,5cyclictest4658-21/usr/sbin/munin23:10:413
23513993610,13cyclictest0-21swapper/322:20:403
2351299364,9cyclictest7950irq/56-dwc_otg_21:55:362
23512993624,3cyclictest25545-21latency_hist20:45:262
23512993620,4cyclictest22617-21sh23:31:232
2351299361,26cyclictest9309-21ls23:15:452
23511993615,4cyclictest8736-21aten_rbpower_po23:15:381
23511993613,5cyclictest0-21swapper/100:25:261
23511993612,4cyclictest28302-21unixbench_singl19:20:541
2351399358,17cyclictest22259-21fschecks_count20:35:443
2351399356,13cyclictest0-21swapper/322:30:393
2351399353,4cyclictest31302-21/usr/sbin/munin00:20:563
23513993513,4cyclictest12117-21/usr/sbin/munin22:00:503
23513993510,4cyclictest0-21swapper/322:05:423
2351299351,20cyclictest25972-21ls20:45:442
2351199358,9cyclictest12174-21unixbench_multi20:05:541
23511993519,4cyclictest1114-21/usr/sbin/munin21:05:511
2351199351,8cyclictest14906-21aten_rbpower_en21:25:391
23511993518,4cyclictest25955-21basename00:15:411
23511993516,5cyclictest1591-21sh21:48:261
2351399344,8cyclictest0-21swapper/321:50:343
2351399343,25cyclictest7950irq/56-dwc_otg_22:15:453
2351399343,14cyclictest7950irq/56-dwc_otg_22:45:503
2351399343,14cyclictest7950irq/56-dwc_otg_22:45:503
2351399341,23cyclictest10812-21latency_hist20:05:263
2351399341,16cyclictest7850irq/56-dwc_otg23:00:453
2351299344,21cyclictest14409-21ssh00:00:552
2351299343,5cyclictest7850irq/56-dwc_otg22:40:462
2351299341,22cyclictest2231-21ls19:40:402
2351199347,16cyclictest0-21swapper/120:00:381
2351199346,18cyclictest0-21swapper/123:30:381
2351199343,7cyclictest12498-21ssh22:40:471
2351199342,13cyclictest7850irq/56-dwc_otg21:40:541
2351199342,13cyclictest7850irq/56-dwc_otg21:40:541
23511993413,4cyclictest13730-21idleruntime-cro00:40:261
23511993413,4cyclictest13730-21idleruntime-cro00:40:261
2351199341,23cyclictest5587-21ls19:50:271
2351199341,20cyclictest0-21swapper/121:15:271
2351199341,20cyclictest0-21swapper/121:15:271
23511993411,4cyclictest0-21swapper/121:30:521
23511993411,4cyclictest0-21swapper/100:00:541
2351199340,15cyclictest3627-21ssh22:30:291
2351399334,12cyclictest0-21swapper/320:50:383
2351399332,4cyclictest7850irq/56-dwc_otg20:10:373
23513993311,4cyclictest0-21swapper/300:10:423
2351299331,20cyclictest32395-21apt-config19:35:262
2351199339,6cyclictest7850irq/56-dwc_otg00:10:561
2351199337,22cyclictest0-21swapper/120:25:391
2351199335,5cyclictest7850irq/56-dwc_otg21:35:401
2351199334,14cyclictest8050irq/56-dwc_otg_19:25:531
2351199334,14cyclictest8050irq/56-dwc_otg_19:25:531
23511993312,8cyclictest24570-21aten_rbpower_en22:55:431
2351199331,24cyclictest31982-21ls21:00:521
2351399328,5cyclictest0-21swapper/321:25:383
2351399326,14cyclictest0-21swapper/319:35:313
2351399322,20cyclictest0-21swapper/320:28:573
2351399322,20cyclictest0-21swapper/319:46:553
2351399321,27cyclictest15020-21ls20:15:443
2351399321,22cyclictest30789-21latency_hist21:00:263
2351299325,18cyclictest0-21swapper/200:35:472
2351299325,18cyclictest0-21swapper/200:35:472
2351199321,21cyclictest8356-21ls23:15:261
2351199321,21cyclictest8356-21ls23:15:261
23511993211,4cyclictest0-21swapper/123:20:321
2351399312,9cyclictest0-21swapper/300:35:483
2351399312,9cyclictest0-21swapper/300:35:483
2351399311,5cyclictest8050irq/56-dwc_otg_22:40:393
2351399311,20cyclictest1974-21diskmemload00:05:273
2351399311,12cyclictest7950irq/56-dwc_otg_21:17:373
2351299316,12cyclictest7950irq/56-dwc_otg_19:15:472
2351299314,20cyclictest8050irq/56-dwc_otg_22:20:442
2351299312,19cyclictest7850irq/56-dwc_otg19:59:132
2351299312,19cyclictest7850irq/56-dwc_otg19:59:132
2351199317,12cyclictest0-21swapper/122:15:251
2351199311,27cyclictest14606-21ls20:15:261
2351399307,4cyclictest0-21swapper/300:26:213
2351399301,4cyclictest7850irq/56-dwc_otg21:00:463
23513993010,5cyclictest0-21swapper/323:26:063
2351299308,6cyclictest31-21ksoftirqd/221:10:262
2351299306,17cyclictest7950irq/56-dwc_otg_20:20:402
2351299304,7cyclictest22251-21fschecks_count20:35:442
2351299301,3cyclictest7950irq/56-dwc_otg_20:05:392
2351199304,7cyclictest30802-21modprobe21:00:251
2351199304,5cyclictest7850irq/56-dwc_otg22:20:521
2351199302,19cyclictest0-21swapper/120:54:101
2351399297,5cyclictest7950irq/56-dwc_otg_19:20:423
2351299292,5cyclictest8050irq/56-dwc_otg_19:10:392
2351199291,4cyclictest7950irq/56-dwc_otg_20:35:551
2351399282,12cyclictest7950irq/56-dwc_otg_21:35:513
2351399281,4cyclictest0-21swapper/320:30:433
2351399281,4cyclictest0-21swapper/320:30:433
2351299283,11cyclictest5634-21/usr/sbin/munin19:50:562
2351299281,19cyclictest0-21swapper/220:00:512
2351199287,4cyclictest8050irq/56-dwc_otg_20:40:441
216652284,15sleep121350-21apt-get19:05:301
2351199272,16cyclictest0-21swapper/120:19:121
2351399266,3cyclictest7850irq/56-dwc_otg20:20:373
2351399265,14cyclictest29317-21basename19:25:433
2351399265,14cyclictest29317-21basename19:25:433
2351399262,10cyclictest7950irq/56-dwc_otg_19:55:443
2351399262,10cyclictest7950irq/56-dwc_otg_19:55:443
2351299261,19cyclictest0-21swapper/219:20:482
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional