You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 15:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Jul 30, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
53989917536,24cyclictest0-21swapper/012:10:450
53989915018,25cyclictest0-21swapper/009:20:510
53989915018,25cyclictest0-21swapper/009:20:510
53989914615,19cyclictest1605-21sh11:30:270
53989914560,62cyclictest101ktimersoftd/012:00:420
53989914154,10cyclictest10370-21unixbench_multi10:20:560
53989913847,16cyclictest0-21swapper/012:20:450
5398991259,50cyclictest1329-21ssh10:10:440
53989912132,22cyclictest0-21swapper/010:05:380
5398991103,26cyclictest0-21swapper/012:15:500
53989910974,15cyclictest5450-21uniq10:15:440
53989910974,15cyclictest5450-21uniq10:15:440
53989910854,24cyclictest0-21swapper/012:05:520
53989910756,28cyclictest5397-21cyclictest10:46:000
5398991073,84cyclictest2055-21dhclient-script11:30:450
54009910611,70cyclictest1521-21latency_hist11:30:262
5400991031,45cyclictest0-21swapper/208:05:392
5400991022,64cyclictest25723-21sshd09:20:522
5400991022,64cyclictest25723-21sshd09:20:522
53989910170,18cyclictest101ktimersoftd/010:00:470
53989910053,20cyclictest21400-21netstat09:15:480
5398991003,73cyclictest0-21swapper/010:50:470
540099993,62cyclictest31445-21ssh12:05:502
540099982,60cyclictest2761-21/usr/sbin/munin12:10:462
539899983,68cyclictest0-21swapper/009:10:440
539899975,71cyclictest0-21swapper/011:20:530
5398999449,24cyclictest0-21swapper/011:45:540
5398999359,9cyclictest7850irq/56-dwc_otg11:55:540
540099923,56cyclictest26581-21apt-get12:00:412
540099923,54cyclictest2197-21perl11:30:472
5398999234,30cyclictest727-21runrttasks12:25:300
5398999234,30cyclictest727-21runrttasks12:25:300
5398999231,40cyclictest0-21swapper/009:43:310
540099913,67cyclictest5594-21irqstats10:15:472
540099913,67cyclictest5594-21irqstats10:15:472
539899915,66cyclictest24073-21timerandwakeup12:35:520
540099905,65cyclictest0-21swapper/212:15:542
15122870,3sleep10-21swapper/108:29:051
15122870,3sleep10-21swapper/108:29:051
60582860,1sleep00-21swapper/007:10:460
540099864,73cyclictest26136-21ps10:00:512
540099863,77cyclictest24377-21ssh12:35:592
5398998650,10cyclictest0-21swapper/010:40:470
540099853,74cyclictest21623-21smartpower09:15:542
540099853,74cyclictest15464-21users11:45:532
540099853,60cyclictest10376-21ssh10:20:562
5400998511,58cyclictest28279-21ssh10:43:412
540099843,74cyclictest14372-21apt-get12:25:302
540099843,74cyclictest14372-21apt-get12:25:302
540199838,39cyclictest7950irq/56-dwc_otg_12:10:443
540099837,57cyclictest22460-21/usr/sbin/munin11:55:542
540099833,73cyclictest21435-21apt-get11:15:322
540099823,72cyclictest27081-21ssh11:20:512
540099823,72cyclictest11154-21ssh12:20:452
540099823,71cyclictest16858-21fschecks_count09:10:452
5398998257,9cyclictest7850irq/56-dwc_otg11:15:320
286782820,2sleep30-21swapper/310:04:503
232902820,4sleep312403-21kworker/3:109:18:143
539899813,53cyclictest0-21swapper/012:30:450
540099803,70cyclictest11717-21ssh09:43:312
318442800,5sleep0539899cyclictest08:20:550
540099793,69cyclictest30835-21users10:46:002
5398997948,10cyclictest0-21swapper/011:10:400
540099781,59cyclictest14418-21/usr/sbin/munin11:05:522
539999782,59cyclictest5409-21cut10:15:431
539999782,59cyclictest5409-21cut10:15:431
540199763,35cyclictest0-21swapper/310:15:433
540199763,35cyclictest0-21swapper/310:15:433
540099763,65cyclictest1669-21cat10:50:472
540099762,67cyclictest17634-21ssh11:10:382
540099761,59cyclictest19669-21ssh11:51:022
540099753,44cyclictest0-21swapper/210:10:442
540099752,67cyclictest19184-21ssh12:30:442
540099721,50cyclictest0-21swapper/209:35:372
5401997129,22cyclictest24348-21ssh12:35:563
539999714,23cyclictest0-21swapper/112:10:441
5401997011,25cyclictest8050irq/56-dwc_otg_11:30:263
5401997010,35cyclictest8050irq/56-dwc_otg_10:20:563
540099703,61cyclictest29374-21sh10:05:382
540099691,52cyclictest10029-21/usr/sbin/munin11:40:452
5399996913,20cyclictest7237-21/usr/sbin/munin12:15:511
72612680,1sleep30-21swapper/309:38:003
540199685,8cyclictest0-21swapper/311:45:513
540099681,50cyclictest0-21swapper/208:31:462
540099681,45cyclictest1975150ssh09:53:212
540099681,43cyclictest531-21latency_hist09:30:262
539999688,39cyclictest0-21swapper/110:43:411
320142680,1sleep30-21swapper/310:48:083
540199679,43cyclictest0-21swapper/309:10:463
540099671,51cyclictest668-21ssh09:30:342
540099671,50cyclictest11940-21ssh11:03:092
5399996754,6cyclictest0-21swapper/109:10:461
539999673,16cyclictest1310-21fschecks_count10:10:441
539999673,15cyclictest23-21ksoftirqd/111:30:281
539999673,10cyclictest18904-21/usr/sbin/munin12:30:421
540199663,11cyclictest2053-21ssh11:30:453
5401996610,9cyclictest8050irq/56-dwc_otg_09:20:493
5401996610,9cyclictest8050irq/56-dwc_otg_09:20:493
540099662,37cyclictest8050irq/56-dwc_otg_07:40:272
540099661,50cyclictest31904-21/usr/sbin/munin08:20:562
540099661,44cyclictest13911-21taskset09:00:562
540099661,44cyclictest13911-21taskset09:00:562
539999663,13cyclictest1-21systemd09:20:501
539999663,13cyclictest1-21systemd09:20:501
540199654,30cyclictest8050irq/56-dwc_otg_12:15:513
540099651,52cyclictest23838-21ssh09:58:242
540099651,41cyclictest18332-21cut07:45:432
540099651,40cyclictest24695-21ssh10:39:392
540199643,10cyclictest26785-21cpuspeed12:00:443
5399996432,12cyclictest23-21ksoftirqd/110:00:501
5398996416,36cyclictest0-21swapper/010:55:550
539899641,46cyclictest0-21swapper/009:25:480
540199639,11cyclictest11087-21ssh12:20:453
540199635,19cyclictest0-21swapper/312:30:433
540099631,51cyclictest8409-21ssh10:59:132
540099631,38cyclictest17977-21missed_timers10:30:472
539999635,17cyclictest0-21swapper/112:05:501
539999633,10cyclictest17616-21apt-get11:10:381
5399996240,12cyclictest15370-21grep11:45:521
5399996220,20cyclictest30777-21sh10:46:001
539899626,30cyclictest6001-21sh11:35:390
539899621,17cyclictest0-21swapper/008:05:390
540199618,32cyclictest0-21swapper/309:05:323
540199614,11cyclictest13157-21kworker/u8:011:20:533
539999614,38cyclictest0-21swapper/112:00:411
539999613,12cyclictest23505-21ssh11:55:551
539999612,39cyclictest19763-21kworker/1:112:25:281
539999612,39cyclictest19763-21kworker/1:112:25:281
5399996122,20cyclictest10258-21timerwakeupswit10:20:541
5399996120,11cyclictest21592-21rt-features09:15:531
306562610,2sleep10-21swapper/109:26:301
540099601,43cyclictest13721-21irqstats10:25:442
5399996026,10cyclictest3846-21kworker/1:212:20:451
540099591,50cyclictest24957-21chrt08:04:302
540099591,42cyclictest8418-21ssh11:38:042
540099591,41cyclictest0-21swapper/209:45:592
540099591,41cyclictest0-21swapper/209:45:592
5398995933,9cyclictest8050irq/56-dwc_otg_07:15:550
49142592,13sleep20-21swapper/207:07:382
540099581,41cyclictest6889-21chrt07:14:272
540099581,41cyclictest0-21swapper/207:40:372
540099581,41cyclictest0-21swapper/207:40:372
539999582,14cyclictest7950irq/56-dwc_otg_11:30:461
5401995710,8cyclictest7850irq/56-dwc_otg10:10:463
540099572,45cyclictest8050irq/56-dwc_otg_08:26:042
540099572,45cyclictest8050irq/56-dwc_otg_08:26:042
540099571,40cyclictest0-21swapper/208:55:382
539999579,28cyclictest0-21swapper/111:15:321
540199567,29cyclictest0-21swapper/307:55:403
5401995617,9cyclictest0-21swapper/311:15:323
5399995611,13cyclictest0-21swapper/107:55:401
178212560,3sleep317802-21ssh09:50:493
540199553,28cyclictest0-21swapper/311:10:403
5401995513,9cyclictest7950irq/56-dwc_otg_12:25:293
5401995513,9cyclictest7950irq/56-dwc_otg_12:25:293
539999553,17cyclictest0-21swapper/110:05:401
539999553,10cyclictest27228-21awk11:20:541
5398995523,17cyclictest0-21swapper/010:32:080
540099544,46cyclictest14863-21fschecks_count09:05:432
540099541,49cyclictest10279-21uptime08:50:572
540099541,42cyclictest0-21swapper/207:24:462
540099541,37cyclictest0-21swapper/208:18:142
5398995431,6cyclictest7950irq/56-dwc_otg_10:35:550
5398995427,15cyclictest5945-21sed09:35:550
5398995424,16cyclictest18587-21sort11:50:450
540199533,13cyclictest25961-21ssh10:40:453
5401995316,9cyclictest8050irq/56-dwc_otg_12:05:503
540099531,48cyclictest21677-21apt-get07:55:372
540099531,3cyclictest0-21swapper/207:50:402
540099531,36cyclictest0-21swapper/207:19:192
539999533,16cyclictest0-21swapper/109:43:311
5398995318,5cyclictest9650irq/80-mmc009:05:260
5398995318,5cyclictest9650irq/80-mmc009:05:260
540199524,11cyclictest8050irq/56-dwc_otg_08:30:403
540199522,23cyclictest8050irq/56-dwc_otg_11:55:473
540099521,40cyclictest0-21swapper/208:39:302
540099521,35cyclictest7950irq/56-dwc_otg_08:15:182
539999525,13cyclictest0-21swapper/112:35:531
5398995223,19cyclictest0-21swapper/010:25:390
5398995210,12cyclictest21535-21uniq09:55:440
540199518,18cyclictest8050irq/56-dwc_otg_10:50:503
540099512,33cyclictest7950irq/56-dwc_otg_08:45:542
540099511,33cyclictest0-21swapper/207:30:012
539999513,13cyclictest1752-21irqstats10:50:501
539899512,41cyclictest0-21swapper/008:50:450
5398995123,14cyclictest0-21swapper/008:35:540
540099501,37cyclictest0-21swapper/207:32:192
222242500,3sleep322225-21/usr/sbin/munin10:35:513
540099491,38cyclictest6997-21taskset08:44:482
5398994912,23cyclictest0-21swapper/008:10:400
540199489,10cyclictest11718-21ssh09:43:313
5401994810,13cyclictest7950irq/56-dwc_otg_11:40:403
539999489,7cyclictest7850irq/56-dwc_otg09:50:391
539899485,17cyclictest4511-21modprobe09:35:260
5398994828,7cyclictest8050irq/56-dwc_otg_08:50:260
5398994822,5cyclictest13214-21grep09:45:430
5398994822,5cyclictest13214-21grep09:45:430
5398994820,12cyclictest18182-21timerandwakeup09:50:540
540199477,24cyclictest0-21swapper/307:25:393
540199473,11cyclictest29362-21apt-get10:05:393
539899471,2cyclictest0-21swapper/011:00:420
539999463,4cyclictest7950irq/56-dwc_otg_07:40:381
539999463,4cyclictest7950irq/56-dwc_otg_07:40:381
5399994616,6cyclictest494350wget09:35:411
539899462,36cyclictest0-21swapper/008:28:410
539899462,36cyclictest0-21swapper/008:28:410
539899462,28cyclictest0-21swapper/007:32:180
50672463,12sleep00-21swapper/007:09:380
539999459,5cyclictest0-21swapper/110:55:351
539999455,5cyclictest8050irq/56-dwc_otg_11:40:401
539999453,4cyclictest7850irq/56-dwc_otg08:30:381
539899457,29cyclictest0-21swapper/009:05:320
539899451,35cyclictest14433-21seq11:05:530
5399994427,5cyclictest27330-21fschecks_count08:10:431
539899442,21cyclictest0-21swapper/011:40:490
5398994420,7cyclictest14882-21if_eth007:35:450
5399994311,17cyclictest0-21swapper/109:05:321
5398994314,19cyclictest0-21swapper/007:21:030
539899431,26cyclictest24501-21/usr/sbin/munin08:00:530
539899431,22cyclictest11391-21/usr/sbin/munin08:55:470
539899431,15cyclictest0-21swapper/007:45:470
539999428,20cyclictest8050irq/56-dwc_otg_08:20:421
539999423,3cyclictest7950irq/56-dwc_otg_11:35:371
539899422,23cyclictest5761-21fschecks_count08:40:440
539899421,19cyclictest0-21swapper/007:50:360
540199417,10cyclictest17705-21ssh10:30:443
539899412,20cyclictest2585-21ntp_states08:30:500
5398994116,18cyclictest0-21swapper/008:16:310
539899411,2cyclictest0-21swapper/007:40:390
539899411,2cyclictest0-21swapper/007:40:390
5398994110,21cyclictest0-21swapper/007:25:390
539999401,7cyclictest17327-21/usr/sbin/munin10:30:391
539999393,21cyclictest7950irq/56-dwc_otg_10:40:271
539899392,18cyclictest0-21swapper/007:55:520
540199382,5cyclictest7950irq/56-dwc_otg_08:25:363
540199382,5cyclictest7950irq/56-dwc_otg_08:25:363
539999386,8cyclictest9650irq/80-mmc009:00:271
540199375,16cyclictest8050irq/56-dwc_otg_09:34:573
539999372,7cyclictest1905950ssh11:50:511
5399993716,10cyclictest7850irq/56-dwc_otg11:00:491
540199364,5cyclictest0-21swapper/307:50:373
540199362,20cyclictest5896-21latency10:55:453
540199362,10cyclictest8050irq/56-dwc_otg_11:50:393
539999367,6cyclictest7950irq/56-dwc_otg_09:45:461
539999367,6cyclictest7950irq/56-dwc_otg_09:45:461
5399993610,8cyclictest0-21swapper/111:05:561
5399993610,16cyclictest0-21swapper/109:30:561
540199351,7cyclictest31410-21ntp_states08:20:493
539999354,8cyclictest0-21swapper/107:35:371
539999354,15cyclictest8050irq/56-dwc_otg_08:05:391
539999353,14cyclictest8050irq/56-dwc_otg_09:05:271
539999353,14cyclictest8050irq/56-dwc_otg_09:05:271
539999352,7cyclictest7850irq/56-dwc_otg10:25:481
540199348,5cyclictest6418-21/usr/sbin/munin11:35:443
540199348,12cyclictest8050irq/56-dwc_otg_11:09:133
540199347,6cyclictest0-21swapper/311:04:063
540199347,18cyclictest0-21swapper/307:10:383
540199343,4cyclictest7850irq/56-dwc_otg08:50:413
540199342,7cyclictest7950irq/56-dwc_otg_09:25:483
5401993411,14cyclictest0-21swapper/309:48:453
5401993411,14cyclictest0-21swapper/309:48:453
539999343,4cyclictest7950irq/56-dwc_otg_08:20:261
540199339,5cyclictest8050irq/56-dwc_otg_09:55:503
540199334,12cyclictest8050irq/56-dwc_otg_08:05:513
540199333,5cyclictest7950irq/56-dwc_otg_07:40:393
540199333,5cyclictest7950irq/56-dwc_otg_07:40:393
5401993315,4cyclictest12704-21/usr/sbin/munin09:00:433
5401993315,4cyclictest12704-21/usr/sbin/munin09:00:433
5401993313,14cyclictest727-21runrttasks08:44:343
5401993310,4cyclictest0-21swapper/310:29:083
539999339,19cyclictest18156-21wget07:45:401
539999339,13cyclictest125-21systemd-journal08:43:191
539999333,17cyclictest8050irq/56-dwc_otg_07:50:501
539999332,15cyclictest0-21swapper/107:15:491
539999331,22cyclictest23716-21ssh09:58:101
540199325,5cyclictest8050irq/56-dwc_otg_08:55:563
5401993225,4cyclictest27410-21fschecks_time08:10:443
539999324,17cyclictest8050irq/56-dwc_otg_07:20:381
539999324,17cyclictest5736-21tune2fs07:10:421
5399993214,8cyclictest3995-21fschecks_time08:35:431
539999321,19cyclictest0-21swapper/107:25:471
540199317,22cyclictest0-21swapper/308:15:393
540199311,5cyclictest19107-21timerwakeupswit07:45:533
540199311,11cyclictest8050irq/56-dwc_otg_07:30:523
539999318,15cyclictest9705-21missed_timers08:50:501
539999314,20cyclictest0-21swapper/108:47:171
539999305,3cyclictest0-21swapper/108:00:461
540199297,5cyclictest7850irq/56-dwc_otg07:15:553
540199296,4cyclictest7850irq/56-dwc_otg07:35:433
5401992913,4cyclictest727-21runrttasks07:20:553
539999292,4cyclictest0-21swapper/107:35:271
37042290,3sleep30-21swapper/307:05:403
540199282,17cyclictest0-21swapper/308:47:063
49112282,14sleep10-21swapper/107:07:371
5401992710,4cyclictest381ktimersoftd/308:00:463
5401992518,5cyclictest4525-21smartpower08:35:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional