You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 05:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Sep 24, 2026 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1729721150,5sleep117304-21timerandwakeup21:45:541
9546991047,79cyclictest0-21swapper/023:30:390
954699987,32cyclictest12261-21ssh00:25:330
954699982,6cyclictest4110-21http21:30:260
226402970,2sleep10-21swapper/100:38:021
954699952,6cyclictest24080-21/usr/sbin/munin21:55:410
954699952,6cyclictest24080-21/usr/sbin/munin21:55:410
954699941,78cyclictest22023-21ls23:55:540
9546999412,53cyclictest1200-21sshd22:38:370
954699932,75cyclictest0-21swapper/021:00:420
954699932,5cyclictest11896-21latency_hist00:25:260
954699932,5cyclictest11896-21latency_hist00:25:260
954699931,6cyclictest0-21swapper/022:10:450
954699922,5cyclictest14999-21munin-run20:55:260
954699921,80cyclictest29674-21sh00:05:530
954699921,5cyclictest7724-21ssh22:15:450
954699921,5cyclictest7724-21ssh22:15:450
954699921,5cyclictest22618-21sh21:53:090
954699912,5cyclictest5126-21ntp_states00:15:500
954699911,7cyclictest27207-21wget22:40:410
954699911,68cyclictest16925-21gpgv21:00:260
954699911,5cyclictest11445-21cat20:45:270
9546999045,33cyclictest32720-21latency_hist21:25:270
954699901,5cyclictest3469-21sed20:20:550
283952890,10sleep0101ktimersoftd/021:19:540
954699882,74cyclictest0-21swapper/022:20:430
954699882,5cyclictest31064-21grep22:45:400
954899874,62cyclictest9730-21apt-get20:40:362
954699874,69cyclictest0-21swapper/023:40:470
9546998730,43cyclictest7850irq/56-dwc_otg_22:50:440
954699872,71cyclictest0-21swapper/023:05:210
9546998711,63cyclictest5296-21/usr/sbin/munin23:35:510
954699867,5cyclictest10077-21latency_hist23:00:270
9546998634,39cyclictest15296-21/usr/sbin/munin22:25:360
954699862,5cyclictest7974-21aten_rbpower_en20:35:380
954699862,3cyclictest0-21swapper/022:00:420
954699861,5cyclictest19551-21ssh22:30:440
9546998613,61cyclictest21308-21hostname21:10:430
954699853,70cyclictest0-21swapper/019:35:470
954699852,5cyclictest32386-21grep22:05:440
954699842,4cyclictest16235-21apt-get00:30:370
954699841,74cyclictest20188-21snmpd23:20:480
954699822,4cyclictest0-21swapper/020:30:430
954699821,5cyclictest8111-21munin-run21:35:250
954699821,5cyclictest20342-21ssh23:12:420
954699821,5cyclictest12286-21ssh21:40:290
954699821,5cyclictest12286-21ssh21:40:290
9546998210,56cyclictest17396-21timerwakeupswit21:45:550
954699817,60cyclictest1008-21/usr/sbin/munin00:10:450
954699817,60cyclictest1008-21/usr/sbin/munin00:10:450
954899801,59cyclictest28022-21/usr/sbin/munin22:00:472
9546998034,35cyclictest0-21swapper/000:00:400
9546998030,35cyclictest25677-21idleruntime-cro23:20:250
954699802,5cyclictest0-21swapper/019:15:460
954699802,5cyclictest0-21swapper/019:15:460
9546998020,45cyclictest7850irq/56-dwc_otg_21:05:450
954699801,5cyclictest8812-21fschecks_time21:35:450
954699801,5cyclictest17634-21kernelversion23:50:480
954699792,3cyclictest0-21swapper/020:03:490
954699791,6cyclictest13216-21modprobe20:50:260
954899781,48cyclictest16913-21idleruntime-cro21:00:262
954699782,62cyclictest0-21swapper/019:32:430
9546997823,45cyclictest13899-21ntpq23:45:490
954699781,7cyclictest13176-21/usr/sbin/munin19:20:450
9548997723,45cyclictest14828-21latency_hist19:25:262
9548997717,45cyclictest13429-21/usr/sbin/munin20:50:482
954699773,5cyclictest4696-21cat20:25:460
954699771,5cyclictest0-21swapper/019:51:420
954899761,63cyclictest28709-21latency_hist21:20:262
9546997616,50cyclictest23770-21rm00:40:170
9546997610,52cyclictest1180-21runrttasks23:05:400
954899756,56cyclictest19477-21fschecks_time21:05:462
954899754,53cyclictest572-21grep22:05:532
9547997511,47cyclictest7950irq/56-dwc_otg_23:00:381
954699751,70cyclictest20188-21snmpd23:27:270
954699751,6cyclictest14997-21/usr/sbin/munin19:25:360
954899741,60cyclictest16494-21/usr/sbin/munin00:30:432
954799746,50cyclictest7950irq/56-dwc_otg_00:30:441
314662740,2sleep30-21swapper/322:05:113
954899731,7cyclictest13081-21ssh00:25:512
954699731,6cyclictest0-21swapper/020:18:300
954899721,8cyclictest13914-21awk23:05:262
954899721,40cyclictest5664-21users00:15:572
9546997217,26cyclictest0-21swapper/019:40:410
954699721,5cyclictest29408-21/usr/sbin/munin20:05:470
954899718,47cyclictest25636-21sh22:38:162
954899715,52cyclictest12495-21switchtime20:45:522
954899712,49cyclictest8728-21ssh22:57:592
954899711,52cyclictest9992-21chrt22:18:192
954899711,52cyclictest9992-21chrt22:18:192
954699711,5cyclictest0-21swapper/019:49:370
954999702,7cyclictest7850irq/56-dwc_otg_23:45:363
954899709,46cyclictest17613-21basename21:00:442
954899706,51cyclictest24833-21ssh21:55:482
954899706,51cyclictest24833-21ssh21:55:482
954899705,51cyclictest6511-21latency20:30:492
954899701,50cyclictest8169-21grep20:35:422
954899701,50cyclictest0-21swapper/223:54:332
9548997013,37cyclictest3945-21fschecks_time22:10:502
9547997016,42cyclictest15133-21sh23:47:131
954699701,5cyclictest0-21swapper/020:13:360
954699701,5cyclictest0-21swapper/020:13:360
954699701,5cyclictest0-21swapper/019:59:190
954899692,6cyclictest14587-21perl23:05:452
954899691,6cyclictest12042-21grep19:15:502
954899691,6cyclictest12042-21grep19:15:502
954899691,54cyclictest5072-21ssh21:30:512
954899691,47cyclictest12192-21ssh22:20:512
954899691,43cyclictest0-21swapper/221:43:082
954899691,43cyclictest0-21swapper/221:43:082
954799699,8cyclictest7850irq/56-dwc_otg_21:55:471
954799699,8cyclictest7850irq/56-dwc_otg_21:55:471
9547996913,23cyclictest7850irq/56-dwc_otg_22:25:401
954899687,49cyclictest8444-21ssh21:35:382
954899685,50cyclictest2518-21basename22:50:442
954899681,53cyclictest8532-21ssh23:39:432
954899681,40cyclictest1799-21ssh22:49:482
954899674,51cyclictest23639-21ssh00:40:022
954899673,51cyclictest18149-21seq23:10:402
954899672,40cyclictest0-21swapper/222:25:502
954899671,54cyclictest31725-21sh00:09:032
954899671,45cyclictest899-21ssh23:30:102
954799672,5cyclictest7750irq/56-dwc_otg22:50:451
234262670,3sleep10-21swapper/123:58:091
954899661,50cyclictest16286-21aten_rbpower_cu21:45:402
954899661,49cyclictest16317-21ssh23:49:352
954899661,49cyclictest15051-21aten_rbpower_en19:25:392
954899661,44cyclictest5147-21/usr/sbin/munin20:25:542
954799668,5cyclictest7850irq/56-dwc_otg_21:30:331
954799666,4cyclictest7750irq/56-dwc_otg21:35:451
954899655,44cyclictest20083-21ps22:30:522
954899653,47cyclictest0-21swapper/221:15:262
954899652,47cyclictest0-21swapper/221:25:262
954899652,44cyclictest11325-21sh23:42:272
954899651,44cyclictest0-21swapper/223:35:092
954899651,43cyclictest0-21swapper/223:15:322
954799656,4cyclictest0-21swapper/100:07:571
954799655,6cyclictest7750irq/56-dwc_otg22:15:591
954799655,6cyclictest7750irq/56-dwc_otg22:15:591
954699652,50cyclictest10823-21taskset19:11:390
954899646,39cyclictest7750irq/56-dwc_otg21:50:432
954899642,52cyclictest0-21swapper/200:20:472
954899642,52cyclictest0-21swapper/200:20:472
954899641,46cyclictest6041-21kworker/u8:223:24:592
954799648,5cyclictest7750irq/56-dwc_otg22:20:521
954799644,15cyclictest28901-21latency_hist21:20:271
954799641,4cyclictest0-21swapper/122:05:161
954799641,16cyclictest0-21swapper/121:20:461
954899631,50cyclictest27085-21taskset19:56:572
954899631,50cyclictest22824-21kworker/u8:300:13:342
954899631,50cyclictest22824-21kworker/u8:300:13:342
954799638,6cyclictest0-21swapper/120:45:511
954799637,9cyclictest7950irq/56-dwc_otg_21:40:391
954799637,9cyclictest7950irq/56-dwc_otg_21:40:391
954799633,4cyclictest0-21swapper/100:11:181
954799633,4cyclictest0-21swapper/100:11:181
9547996318,42cyclictest7155-21sort22:15:261
954799631,11cyclictest17147-21cat21:00:271
954899626,41cyclictest22686-21sshd23:56:402
954899622,49cyclictest32663-21taskset20:14:002
954899622,49cyclictest32663-21taskset20:14:002
954899622,46cyclictest0-21swapper/222:40:472
954799628,9cyclictest7950irq/56-dwc_otg_22:30:511
954799623,4cyclictest0-21swapper/122:55:431
954799623,15cyclictest7950irq/56-dwc_otg_19:25:431
954799621,12cyclictest12285-21threads19:15:541
954799621,12cyclictest12285-21threads19:15:541
4422620,2sleep10-21swapper/122:47:081
954899611,48cyclictest21884-21taskset19:44:332
954899611,44cyclictest0-21swapper/219:32:482
954899611,42cyclictest3013-21ssh21:28:182
954799614,6cyclictest19473-21grep21:05:461
954799614,12cyclictest7850irq/56-dwc_otg_00:18:231
954799614,12cyclictest1180-21runrttasks00:21:051
954799614,12cyclictest1180-21runrttasks00:21:051
954799613,13cyclictest7950irq/56-dwc_otg_22:35:511
954799612,5cyclictest10882-21unixbench_singl20:40:551
9547996121,37cyclictest7602-21latency_hist20:35:271
954799611,7cyclictest5349-21head20:25:561
954799611,5cyclictest7950irq/56-dwc_otg_20:50:551
954999606,32cyclictest39-21ksoftirqd/323:40:403
954899605,40cyclictest0-21swapper/200:01:482
954799605,10cyclictest7950irq/56-dwc_otg_21:10:281
954799602,8cyclictest27883-21ssh00:04:121
954799602,44cyclictest0-21swapper/120:35:431
954799601,8cyclictest12268-21ssh00:25:341
954799601,6cyclictest7950irq/56-dwc_otg_23:10:441
954799601,15cyclictest0-21swapper/123:25:051
954799595,7cyclictest7950irq/56-dwc_otg_21:50:481
954799592,5cyclictest27710-21memory22:40:481
954799592,17cyclictest0-21swapper/123:44:551
954799592,15cyclictest0-21swapper/123:27:501
954799591,5cyclictest514-21/usr/sbin/munin22:05:531
954799591,4cyclictest0-21swapper/121:00:491
954899584,44cyclictest25296-21runrttasks19:52:072
954799584,4cyclictest0-21swapper/123:32:361
954799583,3cyclictest0-21swapper/123:39:001
954799583,10cyclictest7950irq/56-dwc_otg_19:20:381
954799581,13cyclictest24449-21rm23:18:011
954799571,5cyclictest0-21swapper/123:05:391
954899567,37cyclictest7950irq/56-dwc_otg_19:35:492
954899562,39cyclictest0-21swapper/220:01:482
954899561,6cyclictest16203-21systemd-journal20:20:582
954899561,44cyclictest10997-21taskset19:13:552
954899561,36cyclictest30864-21taskset20:09:072
954799561,6cyclictest20077-21ssh23:54:161
954799561,11cyclictest25276-21chrt19:51:511
954899554,46cyclictest666-21/usr/sbin/munin20:15:412
954899552,35cyclictest0-21swapper/219:46:592
954799553,5cyclictest1003-21uname21:25:471
954799551,16cyclictest0-21swapper/120:16:281
954999548,29cyclictest0-21swapper/320:25:393
87812540,1sleep20-21swapper/219:05:532
954999539,28cyclictest8301-21ssh00:20:393
954999539,28cyclictest8301-21ssh00:20:393
954799535,5cyclictest7750irq/56-dwc_otg19:47:181
954799531,6cyclictest10975-21taskset19:13:381
91252532,44sleep00-21swapper/019:08:350
954999528,27cyclictest0-21swapper/320:35:383
954999528,27cyclictest0-21swapper/320:30:423
954799522,4cyclictest7950irq/56-dwc_otg_20:06:261
954799522,4cyclictest0-21swapper/119:37:201
954799521,13cyclictest0-21swapper/120:01:331
954999519,30cyclictest0-21swapper/323:00:403
954999518,27cyclictest0-21swapper/322:40:403
954799512,3cyclictest0-21swapper/119:30:491
954999508,26cyclictest0-21swapper/322:05:403
954799508,25cyclictest0-21swapper/119:55:401
954799501,6cyclictest0-21swapper/119:40:381
954999499,29cyclictest0-21swapper/321:30:323
954999496,28cyclictest337-21ssh00:10:333
954999496,28cyclictest337-21ssh00:10:333
954799491,10cyclictest0-21swapper/120:14:011
954799491,10cyclictest0-21swapper/120:14:011
954999488,27cyclictest0-21swapper/321:55:383
954999488,27cyclictest0-21swapper/321:55:383
9549994825,12cyclictest7750irq/56-dwc_otg21:45:423
954999478,28cyclictest0-21swapper/321:05:413
9547994712,5cyclictest3768-21seq20:23:051
954999468,27cyclictest0-21swapper/320:45:333
954999468,27cyclictest0-21swapper/319:30:423
954999449,29cyclictest0-21swapper/319:25:393
954999441,24cyclictest22709-21ls19:45:493
954999432,26cyclictest19152-21switchtime23:10:583
954999431,26cyclictest19567-21cut22:30:443
9549994310,5cyclictest7850irq/56-dwc_otg_21:35:483
954999428,17cyclictest0-21swapper/319:40:413
954999426,24cyclictest21088-21ssh21:10:323
954999411,32cyclictest12771-21ls21:40:443
954999411,32cyclictest12771-21ls21:40:443
954999411,30cyclictest29585-21latency_hist23:25:263
954999411,24cyclictest15686-21cut22:25:443
954999408,20cyclictest0-21swapper/320:55:323
954999405,8cyclictest7850irq/56-dwc_otg_20:50:463
954999401,6cyclictest7950irq/56-dwc_otg_20:00:403
9549994015,11cyclictest20708-21diskmemload00:36:213
954999398,28cyclictest0-21swapper/320:15:403
954999393,29cyclictest7950irq/56-dwc_otg_23:30:373
954999392,24cyclictest0-21swapper/319:20:393
954999391,27cyclictest11945-21latency22:20:463
954999391,26cyclictest26024-21timerwakeupswit00:00:553
954999388,18cyclictest0-21swapper/323:50:413
954999384,17cyclictest7950irq/56-dwc_otg_19:15:563
954999384,17cyclictest7950irq/56-dwc_otg_19:15:563
954999384,16cyclictest7850irq/56-dwc_otg_22:50:283
954999381,7cyclictest29042-21ls00:05:423
9549993817,6cyclictest9859-21fschecks_time19:10:443
954999381,25cyclictest30069-21ls21:20:573
9549993810,4cyclictest7750irq/56-dwc_otg22:15:393
9549993810,4cyclictest7750irq/56-dwc_otg22:15:393
954999374,9cyclictest0-21swapper/320:40:363
954999374,7cyclictest7850irq/56-dwc_otg_23:15:473
954999374,16cyclictest7850irq/56-dwc_otg_23:05:413
954999373,4cyclictest0-21swapper/300:30:503
954999372,17cyclictest7850irq/56-dwc_otg_19:55:563
954999371,4cyclictest7850irq/56-dwc_otg_00:15:423
954999367,6cyclictest7850irq/56-dwc_otg_23:55:563
954999367,16cyclictest7850irq/56-dwc_otg_19:40:273
954999359,10cyclictest39-21ksoftirqd/321:20:263
954999355,6cyclictest7850irq/56-dwc_otg_23:25:473
9549993526,7cyclictest24122-21ssh22:35:533
9549993514,7cyclictest0-21swapper/322:50:523
954999351,26cyclictest6494-21basename22:55:393
954999345,20cyclictest0-21swapper/319:50:413
954999342,18cyclictest15897-21ssh00:30:263
954999341,16cyclictest7750irq/56-dwc_otg21:51:003
9549993411,4cyclictest0-21swapper/321:25:553
954999338,13cyclictest7950irq/56-dwc_otg_22:10:563
954999331,6cyclictest0-21swapper/323:35:583
954999331,14cyclictest7850irq/56-dwc_otg_20:20:473
954999324,13cyclictest7850irq/56-dwc_otg_20:10:493
954999324,13cyclictest7850irq/56-dwc_otg_20:10:493
954999321,9cyclictest17528-21fschecks_time21:00:423
954999321,14cyclictest7750irq/56-dwc_otg20:08:233
91262293,17sleep10-21swapper/119:08:361
89262283,16sleep30-21swapper/319:05:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional