You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 01:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Jul 29, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6060991389,104cyclictest0-21swapper/012:05:450
60609913822,85cyclictest13580-21rm09:44:100
3166521290,1sleep10-21swapper/112:03:291
60609912720,84cyclictest0-21swapper/010:00:460
60609912631,68cyclictest9210-21cut11:35:540
60629911635,30cyclictest8050irq/56-dwc_otg_10:00:452
6060991164,83cyclictest0-21swapper/012:25:550
6060991164,83cyclictest0-21swapper/012:25:550
60609911517,77cyclictest0-21swapper/011:05:540
6063991143,87cyclictest16828-21rt-features11:05:513
6060991143,95cyclictest0-21swapper/010:45:520
6063991082,68cyclictest9084-21ssh11:35:503
6060991084,84cyclictest0-21swapper/011:30:560
6063991063,69cyclictest20064-21memory10:30:473
6063991063,69cyclictest20064-21memory10:30:473
60629910343,33cyclictest16956-21rm11:05:532
6060991037,76cyclictest12790-21grep11:40:450
60609910314,82cyclictest0-21swapper/011:50:460
60629910240,26cyclictest971-21tune2fs12:05:432
6060991004,17cyclictest3477-21grep10:10:490
6062999945,24cyclictest17627-21df_inode12:25:432
6062999945,24cyclictest17627-21df_inode12:25:432
606399983,72cyclictest22315-21rm12:30:533
606399982,61cyclictest0-21swapper/312:05:583
6062999640,24cyclictest9136-21rm11:35:532
6062999542,21cyclictest750-21snmpd09:44:092
6062999541,20cyclictest8050irq/56-dwc_otg_12:30:522
606099956,45cyclictest0-21swapper/008:00:310
6060999430,36cyclictest22297-21ntp_states12:30:510
6062999337,34cyclictest19415-21/usr/sbin/munin10:30:482
6062999337,34cyclictest19415-21/usr/sbin/munin10:30:482
6060999360,9cyclictest25902-21ssh11:55:540
6060999360,9cyclictest25902-21ssh11:55:540
6060999354,20cyclictest20109-21wc10:30:490
6060999354,20cyclictest20109-21wc10:30:490
6060998813,62cyclictest23851-21taskset10:35:430
606099848,55cyclictest32418-21unixbench-2d10:05:540
606099848,55cyclictest32418-21unixbench-2d10:05:540
6060998314,45cyclictest0-21swapper/012:14:340
606399813,71cyclictest26030-21grep11:55:563
606399813,71cyclictest26030-21grep11:55:563
6062998134,25cyclictest16946-21diskmemload12:14:332
606399804,69cyclictest5427-21grep11:30:563
606399803,71cyclictest27271-21ssh10:00:443
6061998036,27cyclictest7850irq/56-dwc_otg11:40:441
6060998011,56cyclictest2238-21fschecks_time09:30:440
606399796,50cyclictest7950irq/56-dwc_otg_12:25:433
606399796,50cyclictest7950irq/56-dwc_otg_12:25:433
6060997916,54cyclictest19318-21cut07:45:450
606199788,42cyclictest3242-21ssh10:10:441
606399773,68cyclictest3201-21fschecks_count10:10:433
6062997648,22cyclictest3354-21cat10:10:452
6060997612,48cyclictest26641-21rm12:35:520
606399753,66cyclictest8171-21ssh12:14:333
606399752,67cyclictest492-21ssh10:45:543
6060997515,45cyclictest17587-21ssh10:27:410
129062750,2sleep20-21swapper/212:20:262
606399742,65cyclictest12684-21/usr/sbin/munin11:40:433
6062997432,36cyclictest12697-21grep11:40:432
606399733,64cyclictest21297-21/usr/sbin/munin11:50:473
6062997345,23cyclictest372-21ssh10:45:522
6062997341,26cyclictest26079-21grep11:55:572
6062997341,26cyclictest26079-21grep11:55:572
6060997311,46cyclictest30122-21aten_rbpower_cu09:25:390
6062997244,22cyclictest5493-21grep11:30:572
6061997211,39cyclictest23-21ksoftirqd/111:35:531
606199712,50cyclictest1031-21ls12:05:451
606299703,19cyclictest7950irq/56-dwc_otg_07:15:322
606199708,23cyclictest21309-21ssh11:50:491
606099708,46cyclictest14636-21taskset10:24:520
606399692,61cyclictest13568-21ssh09:44:093
606099694,48cyclictest0-21swapper/012:16:310
6062996838,23cyclictest21347-21ls11:50:492
606199683,48cyclictest17367-21/usr/sbin/munin12:25:441
606199683,48cyclictest17367-21/usr/sbin/munin12:25:441
606099687,38cyclictest0-21swapper/008:15:380
6060996811,7cyclictest32741-21munin-run12:05:250
138652680,2sleep10-21swapper/107:30:461
6061996714,30cyclictest7850irq/56-dwc_otg11:30:581
606099678,41cyclictest2732-21/usr/sbin/munin08:30:420
606099676,45cyclictest20441-21fschecks_time11:10:450
6060996721,38cyclictest0-21swapper/011:16:420
606099668,44cyclictest1046-21aten_rbpower_vo08:25:410
606099667,56cyclictest16856-21ls07:40:250
6060996616,47cyclictest14380-21sensors_temp09:00:510
6060996610,36cyclictest8979-21ssh09:38:320
165652660,2sleep00-21swapper/011:45:410
606399651,48cyclictest13581-1kworker/u9:211:28:463
606399651,48cyclictest13581-1kworker/u9:211:28:463
6061996519,20cyclictest6314-21kworker/1:209:44:081
606099658,45cyclictest0-21swapper/010:15:530
606099656,43cyclictest0-21swapper/007:40:500
606099655,44cyclictest0-21swapper/011:30:030
606099655,44cyclictest0-21swapper/011:30:030
606399641,7cyclictest19105-21munin-run10:30:263
6062996427,20cyclictest11612-21latency_hist08:55:252
6062996426,14cyclictest26000-21rm11:16:502
606199649,34cyclictest7950irq/56-dwc_otg_10:45:541
606099645,46cyclictest15175-21runrttasks12:22:030
6061996312,27cyclictest750-21snmpd11:55:571
6061996312,27cyclictest750-21snmpd11:55:571
606399623,42cyclictest7850irq/56-dwc_otg12:03:353
606399621,4cyclictest2480-21cat09:30:463
606199627,33cyclictest0-21swapper/108:25:401
606099627,52cyclictest25132-21taskset09:19:490
606099623,43cyclictest0-21swapper/009:14:020
606399611,46cyclictest0-21swapper/310:54:563
6061996115,13cyclictest21-21rcuc/110:30:471
6061996115,13cyclictest21-21rcuc/110:30:471
606099619,41cyclictest17509-21ssh09:48:470
606099619,41cyclictest17509-21ssh09:48:470
606099619,39cyclictest11728-21latency_hist08:55:260
606099617,43cyclictest0-21swapper/009:55:250
606099614,38cyclictest0-21swapper/007:14:360
606099611,56cyclictest22916-21aten_rbpower_en09:55:390
606399602,41cyclictest0-21swapper/310:55:373
606399601,44cyclictest28852-21ssh12:39:253
6061996012,14cyclictest0-21swapper/107:50:401
6060996031,3cyclictest0-21swapper/008:35:500
6060996013,34cyclictest15446-21rm11:05:110
6060996012,45cyclictest30654-21ssh10:43:560
6060996010,39cyclictest7160-21ssh10:55:110
606399592,46cyclictest8050irq/56-dwc_otg_10:25:023
606399591,47cyclictest20946-21ssh09:52:313
606199593,7cyclictest0-21swapper/111:05:511
6061995921,8cyclictest23-21ksoftirqd/110:00:441
606099599,38cyclictest0-21swapper/007:25:450
606099597,35cyclictest12565-21rt-features08:55:490
606099592,40cyclictest30011-21ssh11:21:330
6060995914,41cyclictest16617-21chrt09:06:370
6060995911,35cyclictest11308-21runrttasks07:24:490
6060995911,35cyclictest11308-21runrttasks07:24:490
606399585,40cyclictest8050irq/56-dwc_otg_11:49:163
606399582,41cyclictest20043-21/usr/sbin/munin11:10:533
606399581,44cyclictest125-21systemd-journal07:48:453
606199587,14cyclictest0-21swapper/111:25:381
606199587,14cyclictest0-21swapper/111:25:381
606099587,37cyclictest0-21swapper/009:24:030
606099585,40cyclictest0-21swapper/008:09:500
606099585,38cyclictest0-21swapper/007:50:440
606399573,36cyclictest0-21swapper/310:37:043
606399571,4cyclictest934-21apt-get08:25:403
606399571,42cyclictest8050irq/56-dwc_otg_11:00:443
606399571,40cyclictest31337-21aten_rbpower_po10:05:403
606399571,40cyclictest31337-21aten_rbpower_po10:05:403
6062995726,19cyclictest23621-21ssh09:17:132
606399563,40cyclictest7850irq/56-dwc_otg10:40:533
606399561,46cyclictest30067-21apt-get09:25:363
606399561,3cyclictest0-21swapper/312:19:023
6063995613,40cyclictest8050irq/56-dwc_otg_09:10:383
6062995626,9cyclictest24336-21ntp_states10:35:502
6062995626,13cyclictest32561-21ssh09:27:572
6062995618,12cyclictest31297-21taskset08:20:102
606399551,5cyclictest18028-21ssh09:49:443
606399551,5cyclictest18028-21ssh09:49:443
606399551,5cyclictest12460-21ntp_states08:55:473
6062995526,12cyclictest10016-21ssh10:57:502
6062995525,18cyclictest22657-21ssh09:55:262
6062995524,14cyclictest11756-21/usr/sbin/munin08:55:502
6062995523,13cyclictest7605-21ssh09:36:072
6062995516,17cyclictest8050irq/56-dwc_otg_09:58:192
606199559,23cyclictest221ktimersoftd/112:30:511
6061995511,22cyclictest6367-21kworker/1:112:14:331
606099555,37cyclictest0-21swapper/011:00:160
606399542,38cyclictest0-21swapper/309:36:233
606399542,38cyclictest0-21swapper/307:53:463
606399542,36cyclictest0-21swapper/311:15:483
606399541,49cyclictest6673-21fschecks_time08:40:453
606399541,49cyclictest6673-21fschecks_time08:40:453
6062995425,11cyclictest17761-21/usr/sbin/munin07:40:502
6062995423,12cyclictest7850irq/56-dwc_otg08:25:432
6062995421,13cyclictest29634-21sh09:25:162
6062995419,12cyclictest19848-21rm11:49:342
6062995415,14cyclictest9829-21ls08:50:252
606099548,43cyclictest92832sleep007:17:080
606399531,4cyclictest8050irq/56-dwc_otg_09:55:393
606399531,3cyclictest0-21swapper/312:23:423
606399531,35cyclictest0-21swapper/311:21:073
6062995322,8cyclictest7125-21ssh10:55:072
606399521,48cyclictest4439-21latency_hist08:35:263
606399521,47cyclictest9161-21unixbench_singl07:15:563
606399521,42cyclictest0-21swapper/309:21:553
6062995225,13cyclictest19290-21irqstats07:45:452
6062995222,14cyclictest18203-21sshd09:50:032
6062995222,14cyclictest18203-21sshd09:50:032
6062995221,13cyclictest14183-21ssh11:02:532
6062995220,20cyclictest21702-21ssh11:11:312
6062995219,13cyclictest30205-21ssh11:21:542
606399512,45cyclictest8299-21grep10:15:553
606399511,5cyclictest27982-21cut08:10:393
606399511,46cyclictest7250-21grep07:10:533
606399511,39cyclictest0-21swapper/309:20:173
6062995124,11cyclictest2313-21if_err_eth009:30:442
6062995123,13cyclictest15225-21apt-get07:35:372
6062995122,11cyclictest19109-21idleruntime-cro10:30:262
6062995120,12cyclictest27180-21ssh12:36:212
6062995119,11cyclictest2252650kworker/u8:310:21:312
6062995116,12cyclictest8813-21sh10:16:462
606099512,23cyclictest0-21swapper/007:55:410
606399502,44cyclictest14222-21/usr/sbin/munin09:00:473
606399501,2cyclictest0-21swapper/308:23:023
6062995027,8cyclictest31575-21apt-get08:20:382
6062995021,12cyclictest16716-21ssh12:24:512
606199503,6cyclictest0-21swapper/110:40:351
6060995010,33cyclictest31561-21/usr/sbin/munin08:20:360
79742490,4sleep17653-21apt-get07:15:321
606399494,42cyclictest0-21swapper/307:35:373
606399491,45cyclictest10096-21fschecks_time07:20:443
606399491,45cyclictest10096-21fschecks_time07:20:443
606399491,43cyclictest10078-21wget08:50:413
6062994924,10cyclictest14770-21seq07:32:542
6062994922,14cyclictest326022sleep210:06:012
6062994922,14cyclictest326022sleep210:06:012
6062994921,12cyclictest29362-21ssh10:41:332
6062994920,11cyclictest2343-21ssh11:27:432
6062994920,11cyclictest2343-21ssh11:27:432
6062994916,10cyclictest0-21swapper/212:00:282
6062994913,24cyclictest11340-21taskset07:25:122
6062994913,24cyclictest11340-21taskset07:25:122
606399483,42cyclictest29812-21aten_rbpower_en08:15:393
606399481,7cyclictest4527-21/usr/sbin/munin08:35:463
606399481,5cyclictest24004-21apt-get08:00:323
606399481,44cyclictest26250-21aten_rbpower_vo08:05:413
606399481,43cyclictest11897-21fschecks_time07:25:443
606399481,3cyclictest0-21swapper/307:55:383
606399481,32cyclictest0-21swapper/307:40:333
6062994823,11cyclictest29478-21runrttasks08:15:092
6062994821,13cyclictest16946-21diskmemload09:14:302
606199487,26cyclictest0-21swapper/108:10:391
606099487,31cyclictest0-21swapper/008:50:240
606099481,32cyclictest0-21swapper/008:10:360
606399471,43cyclictest15361-21/usr/sbin/munin09:05:523
606399471,2cyclictest0-21swapper/307:30:383
6062994718,14cyclictest4154-21chrt08:33:462
606199477,26cyclictest0-21swapper/108:35:391
606199471,8cyclictest20076-21ssh11:10:401
606099478,25cyclictest7660-21chrt08:42:180
606099478,25cyclictest7660-21chrt08:42:180
6060994710,23cyclictest0-21swapper/007:35:150
606399461,42cyclictest8460-21grep08:45:433
6062994625,17cyclictest7427-21grep08:40:562
6062994625,17cyclictest7427-21grep08:40:562
6062994623,7cyclictest26145-21apt-get08:05:372
6062994620,10cyclictest125-21systemd-journal07:51:022
606199467,25cyclictest0-21swapper/107:10:391
6062994530,12cyclictest6640-21/usr/sbin/munin07:10:462
606199458,25cyclictest0-21swapper/110:05:331
606199458,25cyclictest0-21swapper/110:05:331
606199457,26cyclictest0-21swapper/108:30:401
55092452,35sleep00-21swapper/007:06:530
6062994417,10cyclictest23685-21chrt07:56:392
606199447,24cyclictest0-21swapper/108:00:321
6062994322,8cyclictest14742-21users09:00:542
6062994317,13cyclictest12949-21runrttasks07:27:432
6062994316,10cyclictest8050irq/56-dwc_otg_09:06:432
606199436,23cyclictest0-21swapper/109:45:391
606199436,23cyclictest0-21swapper/109:45:391
6062994226,13cyclictest5684-21grep08:35:532
6062994216,10cyclictest8050irq/56-dwc_otg_08:02:032
606199417,25cyclictest0-21swapper/108:50:401
606199417,24cyclictest0-21swapper/108:20:391
606199417,10cyclictest0-21swapper/111:05:261
606199415,7cyclictest7850irq/56-dwc_otg08:55:361
6061994025,5cyclictest2840050wget11:20:421
606199396,24cyclictest0-21swapper/112:15:401
606199395,19cyclictest8050irq/56-dwc_otg_10:35:391
606199393,5cyclictest0-21swapper/108:05:381
606199393,17cyclictest16544-21ssh11:45:381
6061993922,5cyclictest17190-21latency_hist12:25:261
6061993920,5cyclictest1548650wget09:05:411
606199389,9cyclictest0-21swapper/109:35:401
6061993817,12cyclictest5291-21sh09:34:331
606199381,27cyclictest8442-21cut08:45:431
606199376,16cyclictest18744-21ssh09:50:371
606199372,5cyclictest8050irq/56-dwc_otg_11:15:381
6061993713,10cyclictest0-21swapper/109:20:441
6061993712,9cyclictest0-21swapper/109:20:261
6061993710,4cyclictest8050irq/56-dwc_otg_10:50:371
606199367,15cyclictest0-21swapper/109:55:381
606199361,23cyclictest17264-21cut07:40:431
606199356,5cyclictest7850irq/56-dwc_otg10:25:481
606199356,18cyclictest0-21swapper/110:55:361
606199356,17cyclictest0-21swapper/110:15:371
606199353,5cyclictest13570-21apt-get09:00:381
6061993512,5cyclictest0-21swapper/107:25:381
606199349,4cyclictest0-21swapper/109:15:261
606199346,15cyclictest10042-21cut07:20:431
606199346,15cyclictest10042-21cut07:20:431
606199342,4cyclictest7850irq/56-dwc_otg10:20:531
606199341,22cyclictest6675-21cut08:40:451
606199341,22cyclictest6675-21cut08:40:451
6061993411,11cyclictest7850irq/56-dwc_otg12:35:411
49402347,12sleep30-21swapper/307:05:513
606199333,20cyclictest0-21swapper/109:30:131
606199333,19cyclictest8050irq/56-dwc_otg_07:35:531
606199323,5cyclictest23312-21/usr/sbin/munin07:55:531
606199296,4cyclictest7850irq/56-dwc_otg08:15:411
606199283,5cyclictest0-21swapper/107:45:381
54862283,16sleep20-21swapper/207:06:342
57742273,14sleep10-21swapper/107:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional