You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 03:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Sep 20, 2026 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2860499142102,25cyclictest0-21swapper/022:20:540
2648721160,2sleep10-21swapper/121:35:041
286049910523,56cyclictest30186-21apt-get19:15:240
286049910523,56cyclictest30186-21apt-get19:15:240
286049910430,58cyclictest19479-21fschecks_count21:25:430
286049910428,56cyclictest28569-21runrttasks00:22:040
286049910329,57cyclictest18891-21ps22:05:510
610521020,2sleep00-21swapper/023:12:080
2869821020,2sleep10-21swapper/122:18:531
286049910227,61cyclictest11036-21tune2fs19:50:410
28604991014,51cyclictest0-21swapper/020:00:350
286049910127,49cyclictest0-21swapper/022:36:260
2640121010,2sleep20-21swapper/200:20:252
286049910028,52cyclictest7249-21grep00:35:570
286049910027,57cyclictest11764-21ntpq00:00:500
286049910020,69cyclictest16203-21systemd-journal23:37:110
28604999832,42cyclictest2597950ssh00:19:430
28604999828,66cyclictest11683-21basename23:20:230
28604999724,50cyclictest0-21swapper/019:17:380
28604999722,61cyclictest14188-21basename22:00:370
28604999528,46cyclictest4105-21ssh22:28:530
28604999526,53cyclictest1702-21runrttasks21:44:420
28604999523,58cyclictest355-21aten_rbpower_vo23:05:400
29182930,3sleep20-21swapper/222:26:322
28604999328,56cyclictest14844-21rm00:05:320
28604999325,47cyclictest0-21swapper/020:32:020
28604999325,47cyclictest0-21swapper/020:32:020
28604999223,53cyclictest0-21swapper/020:50:430
28604999223,53cyclictest0-21swapper/020:50:430
28604999222,54cyclictest0-21swapper/020:42:470
28604999210,15cyclictest0-21swapper/023:45:380
28604999210,15cyclictest0-21swapper/023:45:380
28604999128,49cyclictest9829-21ssh21:55:160
28604999126,49cyclictest28674-21taskset21:36:360
28604999125,50cyclictest15887-21ssh22:43:570
28604999125,48cyclictest0-21swapper/021:31:120
28604999120,55cyclictest0-21swapper/019:23:430
28604999120,55cyclictest0-21swapper/019:23:430
28604999116,59cyclictest0-21swapper/021:23:040
28604999032,54cyclictest14109-21ls20:00:240
28604999032,54cyclictest14109-21ls20:00:240
28604999026,48cyclictest21978-21rm23:32:220
28604999026,48cyclictest0-21swapper/020:45:470
28604999024,49cyclictest0-21swapper/021:55:380
28604999022,56cyclictest0-21swapper/021:20:150
28604999022,56cyclictest0-21swapper/021:20:150
28604998933,36cyclictest8487-21ssh23:56:070
28604998928,48cyclictest4432-21rm21:47:200
28604998831,32cyclictest32303-21sh00:26:430
28604998828,46cyclictest22578-21ssh22:51:300
28604998828,45cyclictest28726-21ssh22:18:560
28604998827,44cyclictest18137-21ssh23:27:310
28604998826,46cyclictest8055-21ssh22:33:560
28604998824,50cyclictest21000-21rm00:12:380
28604998821,54cyclictest0-21swapper/019:46:110
28604998734,26cyclictest696750diskmemload23:00:380
28604998729,44cyclictest26444-21ssh22:56:250
28604998727,41cyclictest0-21swapper/023:20:390
28604998724,49cyclictest0-21swapper/019:28:140
28604998626,43cyclictest3943-21/usr/sbin/munin19:30:440
28604998625,49cyclictest23068-21seq20:24:440
28604998623,52cyclictest0-21swapper/022:49:030
28604998622,45cyclictest0-21swapper/023:51:280
28604998622,45cyclictest0-21swapper/021:01:130
2860499861,50cyclictest0-21swapper/022:10:380
2860499861,50cyclictest0-21swapper/022:10:380
28604998425,45cyclictest0-21swapper/020:29:350
28604998424,49cyclictest9138-21ssh21:11:570
28604998421,50cyclictest0-21swapper/023:42:030
28604998321,51cyclictest0-21swapper/000:30:360
28604998024,39cyclictest0-21swapper/021:09:290
28604998024,39cyclictest0-21swapper/021:09:290
28604998023,47cyclictest0-21swapper/020:07:300
28604997925,43cyclictest0-21swapper/019:39:280
28604997923,43cyclictest0-21swapper/020:55:590
14102780,3sleep3381ktimersoftd/323:05:543
28604997620,43cyclictest0-21swapper/020:17:210
28604997423,34cyclictest0-21swapper/020:37:260
326802730,4sleep229-21rcuc/220:50:412
326802730,4sleep229-21rcuc/220:50:412
28604997325,32cyclictest19487-21chrt20:15:170
28604997323,38cyclictest0-21swapper/019:41:460
241612670,3sleep12860599cyclictest22:54:391
299382620,2sleep20-21swapper/223:42:362
28302620,2sleep20-21swapper/200:30:482
2860599548,29cyclictest0-21swapper/119:20:381
2860599548,29cyclictest0-21swapper/119:20:381
2860699531,12cyclictest2294-21ls19:25:472
2860599538,30cyclictest0-21swapper/122:25:371
269522530,35sleep326956-21fschecks_count00:20:423
2860599528,30cyclictest0-21swapper/119:25:391
28605995217,22cyclictest7950irq/56-dwc_otg_20:55:421
2860599518,28cyclictest0-21swapper/122:45:381
2860599506,29cyclictest6014-21ssh00:35:401
28605995011,27cyclictest0-21swapper/123:45:391
28605995011,27cyclictest0-21swapper/123:45:391
203492500,2sleep10-21swapper/100:11:211
2860599499,25cyclictest0-21swapper/121:05:381
2860599499,25cyclictest0-21swapper/121:05:381
2860599488,28cyclictest0-21swapper/123:20:391
2860599488,26cyclictest0-21swapper/123:10:391
2860599488,26cyclictest0-21swapper/120:40:371
2860599487,30cyclictest0-21swapper/123:25:381
2860599481,4cyclictest7950irq/56-dwc_otg_22:55:391
177502480,2sleep20-21swapper/223:26:452
168562470,13sleep37950irq/56-dwc_otg_22:03:433
28607994623,6cyclictest13016-21/usr/sbin/munin22:40:453
2860699461,20cyclictest0-21swapper/220:45:462
2860699460,37cyclictest24480-21ls23:35:462
274152460,3sleep30-21swapper/322:58:203
2860799455,3cyclictest0-21swapper/322:35:383
2860799451,4cyclictest7950irq/56-dwc_otg_20:45:383
2860599454,12cyclictest0-21swapper/119:10:261
2860599454,12cyclictest0-21swapper/119:10:261
2860799444,6cyclictest7950irq/56-dwc_otg_19:30:373
28607994430,6cyclictest3223-21wget23:50:383
28607994418,13cyclictest0-21swapper/322:25:353
2860699442,15cyclictest7850irq/56-dwc_otg_21:45:342
2860599446,24cyclictest5219-21ssh22:30:371
2860599441,28cyclictest32606-21ls20:50:381
2860599441,28cyclictest32606-21ls20:50:381
2860799434,18cyclictest0-21swapper/319:10:263
2860799434,18cyclictest0-21swapper/319:10:263
2860799433,23cyclictest0-21swapper/320:57:443
2860599438,20cyclictest0-21swapper/123:40:371
28605994326,14cyclictest14100-21/usr/sbin/munin22:00:491
38202420,2sleep10-21swapper/121:00:431
2860699422,32cyclictest0-21swapper/222:22:322
2860599422,7cyclictest14914-21ssh00:05:381
28605994219,18cyclictest0-21swapper/122:40:371
2860599421,35cyclictest6872-21ls21:50:451
211102420,2sleep30-21swapper/322:09:243
2860799417,17cyclictest7950irq/56-dwc_otg_23:40:253
2860799411,30cyclictest7320-21ls19:40:403
2860799411,16cyclictest23251-21ssh00:15:463
2860699417,16cyclictest7950irq/56-dwc_otg_00:05:242
28606994119,12cyclictest16203-21systemd-journal22:10:232
2860599412,4cyclictest7850irq/56-dwc_otg_23:55:551
2860599411,30cyclictest27517-21ls20:35:451
28605994111,6cyclictest7750irq/56-dwc_otg23:05:431
306562400,11sleep1221ktimersoftd/100:25:401
2860799404,5cyclictest7950irq/56-dwc_otg_00:25:393
2860699403,4cyclictest7850irq/56-dwc_otg_21:10:362
28606994024,8cyclictest7950irq/56-dwc_otg_22:14:342
28606994024,8cyclictest7950irq/56-dwc_otg_22:14:342
2860699402,26cyclictest25778-21missed_timers20:30:472
2860699402,26cyclictest25778-21missed_timers20:30:472
28606994022,10cyclictest27491-21ssh22:16:302
28606994014,6cyclictest28156-21grep20:35:542
2860599407,9cyclictest0-21swapper/121:19:551
2860599407,9cyclictest0-21swapper/121:19:551
2860599405,20cyclictest7950irq/56-dwc_otg_21:40:411
2860599403,10cyclictest7750irq/56-dwc_otg00:30:381
2860599402,26cyclictest12828-21cut19:55:431
2860599402,26cyclictest12828-21cut19:55:431
28605994019,6cyclictest13201-21sh00:02:431
2860599401,23cyclictest6878-21ssh23:55:241
2860599401,23cyclictest18482-21ls20:10:441
267752406,24sleep226788-21apt19:05:362
2860799394,6cyclictest7950irq/56-dwc_otg_21:35:363
28607993918,5cyclictest2407-21ntp_offset19:25:493
28607993913,9cyclictest39-21ksoftirqd/323:45:383
28607993913,9cyclictest39-21ksoftirqd/323:45:383
2860799391,25cyclictest5154-21latency_hist21:05:243
2860699397,16cyclictest7950irq/56-dwc_otg_21:55:382
2860699391,27cyclictest0-21swapper/220:05:182
2860699391,23cyclictest0-21swapper/219:40:372
2860699391,10cyclictest12130-21ssh22:39:162
2860599393,11cyclictest0-21swapper/119:50:421
2860599392,26cyclictest0-21swapper/100:15:471
2860599391,27cyclictest0-21swapper/120:34:151
2860599391,27cyclictest0-21swapper/120:34:151
307332380,7sleep22860699cyclictest19:15:402
2860799388,14cyclictest7950irq/56-dwc_otg_00:35:503
2860799387,16cyclictest0-21swapper/321:57:033
2860799382,5cyclictest7950irq/56-dwc_otg_19:20:383
2860799382,5cyclictest7950irq/56-dwc_otg_19:20:383
28607993823,5cyclictest2161350wget20:20:413
2860799381,32cyclictest30800-21cut19:15:423
2860799381,31cyclictest6321-21ls21:05:523
2860799381,31cyclictest6321-21ls21:05:523
2860799381,23cyclictest19911-21/usr/sbin/munin23:30:473
2860799381,17cyclictest0-21swapper/300:12:493
28607993810,14cyclictest8039-21ssh21:10:483
28606993817,11cyclictest21769-21sh23:31:582
28606993814,14cyclictest888-21sh23:05:472
28606993810,19cyclictest0-21swapper/222:30:452
2860599384,10cyclictest8066-21/usr/sbin/munin23:15:501
2860599383,21cyclictest0-21swapper/122:06:371
2860599381,7cyclictest1994950ssh23:30:391
28605993812,5cyclictest7750irq/56-dwc_otg21:26:021
2860599381,22cyclictest10209-21/usr/sbin/munin21:55:511
2860599381,21cyclictest7750irq/56-dwc_otg22:20:431
2860799376,7cyclictest39-21ksoftirqd/321:40:443
2860799376,15cyclictest7950irq/56-dwc_otg_22:15:413
2860799375,17cyclictest7950irq/56-dwc_otg_21:50:433
2860799373,20cyclictest7950irq/56-dwc_otg_22:30:483
2860799372,5cyclictest7950irq/56-dwc_otg_20:05:413
28607993716,11cyclictest5623-21ssh23:11:103
2860799371,24cyclictest8394-21cut23:15:423
2860799371,24cyclictest3040-21ls21:45:453
2860699374,5cyclictest7950irq/56-dwc_otg_21:15:382
2860699374,5cyclictest7950irq/56-dwc_otg_21:15:382
2860699374,20cyclictest31907-21awk23:45:412
2860699374,20cyclictest31907-21awk23:45:412
2860699374,20cyclictest24940-21cron20:30:242
2860699372,26cyclictest0-21swapper/223:20:392
2860699372,26cyclictest0-21swapper/219:32:052
2860699372,24cyclictest0-21swapper/223:50:502
28606993719,6cyclictest31-21ksoftirqd/221:55:232
2860699371,14cyclictest7750irq/56-dwc_otg21:20:562
2860599378,17cyclictest0-21swapper/123:35:371
2860599378,17cyclictest0-21swapper/121:20:301
2860599373,24cyclictest12385-21ssh22:39:471
2860599373,16cyclictest7750irq/56-dwc_otg19:45:471
28605993713,6cyclictest21-21rcuc/121:35:431
2860599371,25cyclictest7384-21cut19:40:431
2860799369,10cyclictest0-21swapper/322:47:043
2860799367,14cyclictest15593-21apt-get23:25:283
2860799366,15cyclictest7950irq/56-dwc_otg_20:10:373
2860799364,18cyclictest7950irq/56-dwc_otg_19:50:243
2860799362,17cyclictest7950irq/56-dwc_otg_00:00:433
2860799362,16cyclictest7950irq/56-dwc_otg_19:35:443
2860799362,16cyclictest7750irq/56-dwc_otg22:20:493
2860799362,14cyclictest25731-21grep22:15:253
2860799362,14cyclictest25731-21grep22:15:253
2860799361,25cyclictest15855-21latency21:20:473
2860799361,22cyclictest19508-21cut21:25:433
2860799360,18cyclictest30984-21rm23:44:423
2860699365,14cyclictest7850irq/56-dwc_otg_23:15:512
2860699362,21cyclictest0-21swapper/221:36:572
2860699362,15cyclictest0-21swapper/220:10:452
2860699362,13cyclictest0-21swapper/221:44:392
28606993616,5cyclictest7950irq/56-dwc_otg_21:31:002
2860699361,4cyclictest7950irq/56-dwc_otg_23:10:372
2860699361,23cyclictest0-21swapper/222:40:462
2860699361,22cyclictest0-21swapper/222:55:442
2860699361,21cyclictest0-21swapper/220:57:562
2860699361,18cyclictest0-21swapper/219:23:322
2860699361,18cyclictest0-21swapper/219:23:322
2860599367,14cyclictest7850irq/56-dwc_otg_20:25:441
28605993616,6cyclictest2915-21sh21:45:441
2803223622,7sleep00-21swapper/019:06:350
2860799358,13cyclictest0-21swapper/323:59:563
2860799354,14cyclictest0-21swapper/300:09:123
2860799353,28cyclictest12285-21uniq23:20:433
2860799351,21cyclictest28003-21timerwakeupswit20:35:533
28607993510,11cyclictest7950irq/56-dwc_otg_20:30:543
28607993510,11cyclictest7950irq/56-dwc_otg_20:30:543
2860699358,13cyclictest7950irq/56-dwc_otg_19:35:422
2860699356,14cyclictest7950irq/56-dwc_otg_19:55:552
2860699356,14cyclictest7950irq/56-dwc_otg_19:55:552
2860699356,10cyclictest31-21ksoftirqd/222:00:412
28606993523,5cyclictest101302chrt19:46:022
2860699352,23cyclictest0-21swapper/221:06:432
2860699352,23cyclictest0-21swapper/221:06:432
2860699352,12cyclictest0-21swapper/200:20:452
2860699351,6cyclictest7950irq/56-dwc_otg_20:15:372
28606993515,11cyclictest6967-21diskmemload00:35:492
2860699351,26cyclictest0-21swapper/221:27:422
2860699351,25cyclictest0-21swapper/222:45:482
2860699351,24cyclictest0-21swapper/221:01:312
2860699351,17cyclictest0-21swapper/223:01:562
2860699351,13cyclictest29-21rcuc/200:10:232
2860599357,17cyclictest0-21swapper/121:10:381
2860599356,19cyclictest0-21swapper/122:14:011
2860599356,19cyclictest0-21swapper/122:14:011
2860599355,6cyclictest7750irq/56-dwc_otg20:45:421
2860599355,14cyclictest7950irq/56-dwc_otg_19:35:381
2860599353,6cyclictest30025-21ssh23:00:551
272342355,17sleep327306-21latency19:05:453
2860799349,11cyclictest32054-21ssh23:04:473
2860799347,13cyclictest7950irq/56-dwc_otg_22:50:523
2860799344,14cyclictest7950irq/56-dwc_otg_20:00:413
28607993410,10cyclictest7950irq/56-dwc_otg_21:15:383
28607993410,10cyclictest7950irq/56-dwc_otg_21:15:383
2860699342,21cyclictest0-21swapper/200:29:052
2860699341,26cyclictest0-21swapper/200:13:432
2860699341,22cyclictest0-21swapper/223:58:132
2860699341,17cyclictest0-21swapper/220:44:102
2860599342,14cyclictest7750irq/56-dwc_otg20:00:451
28605993419,5cyclictest27034-21fschecks_time00:20:431
2860799334,16cyclictest7950irq/56-dwc_otg_21:35:243
2860799334,13cyclictest7950irq/56-dwc_otg_00:35:243
2860799332,15cyclictest12405-21ls19:55:243
2860799331,2cyclictest0-21swapper/320:25:333
28606993321,4cyclictest7950irq/56-dwc_otg_22:51:202
28606993318,7cyclictest31-21ksoftirqd/219:15:252
28606993318,7cyclictest31-21ksoftirqd/219:15:252
2860699331,16cyclictest0-21swapper/219:55:162
2860699331,15cyclictest16141-21/usr/sbin/munin20:05:482
2860599332,18cyclictest0-21swapper/120:09:301
2860599331,17cyclictest0-21swapper/119:15:311
280432334,17sleep128044-21sshd19:06:441
2860799328,10cyclictest7950irq/56-dwc_otg_20:40:393
2860699321,15cyclictest7850irq/56-dwc_otg_20:20:552
2860799312,17cyclictest20259-21grep20:15:443
2860599314,20cyclictest7950irq/56-dwc_otg_20:15:371
2860799302,8cyclictest7850irq/56-dwc_otg_20:00:243
2860799302,8cyclictest7850irq/56-dwc_otg_20:00:243
2860599302,4cyclictest7750irq/56-dwc_otg20:20:381
2860599301,6cyclictest7750irq/56-dwc_otg19:30:421
2860799292,20cyclictest0-21swapper/320:54:473
2860799292,20cyclictest0-21swapper/320:54:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional