You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 07:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Apr 18, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1157721740,2sleep13134599cyclictest09:07:221
1707121720,3sleep03134499cyclictest09:14:540
3112821680,5sleep03134499cyclictest08:12:500
644421660,5sleep23134699cyclictest09:01:112
3057921310,4sleep13134599cyclictest08:11:501
31344991123,54cyclictest4005-21sh07:40:550
636221110,4sleep03134499cyclictest09:40:590
31344991001,78cyclictest0-21swapper/008:01:030
22822960,5sleep13134599cyclictest08:16:491
3134499952,78cyclictest0-21swapper/004:31:070
208842930,5sleep13134599cyclictest09:19:181
3134499912,75cyclictest0-21swapper/007:11:000
3134499911,74cyclictest22624-21cron06:45:450
3134499911,74cyclictest22624-21cron06:45:450
3134499911,4cyclictest28416-21grep06:51:120
31344999049,18cyclictest7750irq/56-dwc_otg09:55:560
3134499902,74cyclictest0-21swapper/009:08:120
3134499882,70cyclictest0-21swapper/008:45:500
3134499875,66cyclictest0-21swapper/008:37:130
3134499875,66cyclictest0-21swapper/008:37:130
3134499872,71cyclictest0-21swapper/005:25:460
3134499872,71cyclictest0-21swapper/005:25:460
3134499871,7cyclictest16091-21apt-get07:55:470
3134499871,71cyclictest23315-21fschecks_time06:46:020
3134499862,73cyclictest0-21swapper/009:05:450
3134499846,61cyclictest0-21swapper/009:36:130
3134499832,66cyclictest0-21swapper/009:49:420
3134499831,69cyclictest0-21swapper/005:26:090
3134499831,5cyclictest10116-21sed08:26:140
3134499822,70cyclictest0-21swapper/008:43:240
3134499822,68cyclictest0-21swapper/007:33:010
3134499821,64cyclictest0-21swapper/009:19:190
3134499821,4cyclictest11301-21idleruntime-cro07:10:450
3134499821,4cyclictest10080-21fschecks_time05:01:020
3134499812,66cyclictest0-21swapper/006:21:130
3134499801,65cyclictest0-21swapper/009:30:460
3134499801,64cyclictest5137-21grep08:21:030
31344998012,57cyclictest9802-21diskmemload06:59:310
31344998012,57cyclictest9802-21diskmemload06:59:310
3134499791,37cyclictest0-21swapper/007:45:580
201882790,6sleep23134699cyclictest09:18:022
3134499781,4cyclictest2522-21ssh07:38:390
3134499777,37cyclictest0-21swapper/005:55:580
3134499772,64cyclictest0-21swapper/009:52:280
3134499771,6cyclictest27839-21sh08:09:240
3134499762,63cyclictest0-21swapper/008:17:510
3134499762,60cyclictest0-21swapper/008:57:240
3134499761,61cyclictest16153-21dpkg07:55:460
3134499761,60cyclictest4525-21latency_hist06:15:460
3134499761,60cyclictest4525-21latency_hist06:15:460
3134499761,5cyclictest30643-21ssh08:51:110
3134499752,70cyclictest0-21swapper/006:07:060
3134499751,5cyclictest0-21swapper/006:26:230
3134499751,4cyclictest23265-21ssh07:25:000
3134499741,63cyclictest27595-21ssh07:30:190
3134499741,63cyclictest27595-21ssh07:30:190
3134499741,58cyclictest6764-21sh07:05:050
3134499731,5cyclictest0-21swapper/004:36:130
3134499731,57cyclictest0-21swapper/006:32:150
3134499731,4cyclictest25434-21ssh09:24:530
3134499727,57cyclictest0-21swapper/004:50:580
3134499722,58cyclictest0-21swapper/005:51:000
3134499722,56cyclictest0-21swapper/007:19:070
31344997210,49cyclictest24313-21/usr/sbin/munin05:41:100
3134499712,57cyclictest15237-21grep05:15:590
3134499712,56cyclictest0-21swapper/009:33:090
3134499712,55cyclictest5457-21grep04:46:130
31347997028,15cyclictest0-21swapper/306:50:553
3134499708,44cyclictest0-21swapper/006:35:550
3134499703,51cyclictest0-21swapper/005:40:450
31344997013,39cyclictest16929-21latency_hist08:35:460
50292690,2sleep1221ktimersoftd/104:46:071
3134499694,4cyclictest8054-21aten_rbpower_vo04:55:590
3134499692,54cyclictest0-21swapper/005:06:010
31346996835,18cyclictest7750irq/56-dwc_otg08:20:432
31346996825,22cyclictest14229-21sh09:11:032
3134499682,48cyclictest0-21swapper/004:44:300
31346996725,26cyclictest23775-21/usr/sbin/munin06:46:092
31346996626,17cyclictest12273-21missed_timers07:11:072
31346996528,18cyclictest12213-21ssh09:08:332
31346996524,21cyclictest1023-21ssh06:57:112
31346996524,21cyclictest1023-21ssh06:57:112
3134499657,6cyclictest0-21swapper/005:35:440
3134499653,12cyclictest0-21swapper/005:10:560
3134499651,43cyclictest664-21chrt06:04:470
31346996428,21cyclictest14929-21rm08:32:152
31346996427,27cyclictest16091-21apt-get07:55:512
31346996422,20cyclictest7491-21ssh08:23:472
31346996328,22cyclictest14404-21ssh07:52:422
31346996328,19cyclictest27730-21ssh07:30:352
31346996328,19cyclictest27730-21ssh07:30:352
31346996323,23cyclictest25610-21ssh09:25:122
3134499631,4cyclictest0-21swapper/005:46:300
3134499631,4cyclictest0-21swapper/005:46:300
44982610,2sleep20-21swapper/207:41:032
31346996133,11cyclictest21041-21rm06:42:592
31346996133,11cyclictest21041-21rm06:42:592
31346996129,17cyclictest31505-21ssh08:13:312
31346996125,13cyclictest20997-21fschecks_time08:01:022
31346996124,19cyclictest32514-21ssh08:53:582
31346996123,13cyclictest11407-21ssh07:49:472
31346996024,14cyclictest7850irq/56-dwc_otg_08:45:552
31346995930,19cyclictest20999-21rm09:58:082
31346995928,12cyclictest23275-21ssh07:25:012
31346995927,19cyclictest17144-21ssh07:16:222
31346995924,27cyclictest2348-21df_inode09:36:022
31346995924,20cyclictest30378-21sh06:54:232
3134499591,44cyclictest4612-21/usr/sbin/munin06:16:030
31346995832,18cyclictest28090-21cpu05:51:002
31346995829,15cyclictest32448-21/usr/sbin/munin04:31:122
31346995828,11cyclictest6423-21/usr/sbin/munin06:20:562
31346995827,18cyclictest30747-21ls05:56:112
31346995827,13cyclictest13636-21ssh06:35:002
31346995826,12cyclictest472-21ssh09:33:282
31346995825,16cyclictest16690-21ssh09:52:492
31346995823,14cyclictest9889-21ssh08:26:122
3134799565,13cyclictest7850irq/56-dwc_otg_08:05:553
3134699564,13cyclictest1360-21ssh08:55:582
31346995629,12cyclictest10237-21/usr/sbin/munin05:01:052
31346995626,9cyclictest5236-21ssh07:02:172
31346995624,11cyclictest7881-21sh09:41:492
31346995528,20cyclictest32293-21apt-get07:35:552
31346995525,12cyclictest12058-21ssh09:46:542
31346995522,16cyclictest7950irq/56-dwc_otg_06:13:052
31346995522,16cyclictest7950irq/56-dwc_otg_06:13:052
31346995518,11cyclictest0-21swapper/209:30:212
155752550,2sleep30-21swapper/308:33:263
54362540,2sleep30-21swapper/307:02:393
31346995425,12cyclictest5837-21chrt04:49:592
31346995421,12cyclictest18623-21sed05:25:462
31346995421,12cyclictest18623-21sed05:25:462
214452540,27sleep3381ktimersoftd/307:21:403
31346995327,12cyclictest30495-21ssh07:33:052
31346995326,18cyclictest9758-21ssh07:07:592
31346995325,17cyclictest26579-21ssh08:07:052
31346995323,11cyclictest20160-21rm08:39:032
31346995323,11cyclictest20160-21rm08:39:032
31346995322,12cyclictest14848-21taskset05:14:472
31346995322,12cyclictest0-21swapper/206:40:192
31346995322,11cyclictest25845-21seq05:45:272
305112533,13sleep029516-21/usr/sbin/munin04:26:120
31346995228,11cyclictest1930-21sshd04:36:172
31346995138,11cyclictest22452-21/usr/sbin/munin08:41:112
31346995124,17cyclictest19965-21users05:26:132
31346995124,14cyclictest2271-21sshd06:07:062
31346995117,16cyclictest7950irq/56-dwc_otg_04:55:562
31346995026,7cyclictest16501-21chrt05:17:562
31346995023,18cyclictest13010-21runrttasks05:09:212
31346995022,10cyclictest9652-21taskset06:29:102
31346994915,11cyclictest7950irq/56-dwc_otg_05:50:452
31346994915,11cyclictest7950irq/56-dwc_otg_05:50:452
31346994720,13cyclictest7850irq/56-dwc_otg_05:36:542
31346994719,15cyclictest412-21runrttasks06:01:412
31346994715,22cyclictest0-21swapper/204:52:512
3134799468,25cyclictest0-21swapper/309:20:503
3134799467,25cyclictest0-21swapper/305:05:573
3134799467,24cyclictest20676-21ssh08:00:583
31346994621,14cyclictest4000-21taskset04:44:332
31346994619,20cyclictest22047-21chrt05:34:262
3134599457,23cyclictest0-21swapper/105:05:571
300592450,4sleep121-21rcuc/104:26:041
3134599447,24cyclictest17225-21ssh08:35:521
3134599447,24cyclictest17225-21ssh08:35:521
3134599435,5cyclictest7750irq/56-dwc_otg08:41:121
31345994320,7cyclictest10658-21seq07:48:261
31345994313,23cyclictest0-21swapper/104:55:581
3134799426,24cyclictest0-21swapper/307:55:513
205482420,2sleep30-21swapper/308:39:463
205482420,2sleep30-21swapper/308:39:463
153452420,2sleep30-21swapper/309:11:433
3134799417,21cyclictest0-21swapper/306:10:563
3134799417,21cyclictest0-21swapper/306:10:563
31346994119,13cyclictest0-21swapper/206:18:332
31345994018,11cyclictest7850irq/56-dwc_otg_07:55:501
3134799397,22cyclictest0-21swapper/306:20:583
3134799395,20cyclictest31544-21ssh06:55:513
3134799395,20cyclictest31544-21ssh06:55:513
3134599399,12cyclictest31343-21cyclictest09:26:051
6352380,5sleep13134599cyclictest08:55:061
3134799387,21cyclictest0-21swapper/309:15:583
31347993811,5cyclictest0-21swapper/309:36:033
3134599384,7cyclictest0-21swapper/106:50:551
3134599382,18cyclictest0-21swapper/106:27:211
31345993818,5cyclictest15429-21fschecks_count05:16:021
31345993817,6cyclictest221ktimersoftd/106:56:031
31345993817,6cyclictest221ktimersoftd/106:56:031
31345993814,12cyclictest0-21swapper/107:08:261
31345993812,19cyclictest0-21swapper/108:23:161
263192381,33sleep10-21swapper/106:49:461
3134799371,25cyclictest16717-21switchtime07:16:103
3134799371,16cyclictest7750irq/56-dwc_otg05:45:453
31345993723,4cyclictest1117-21grep07:36:131
3134599371,6cyclictest21058-21sed07:21:101
3134599371,33cyclictest7950irq/56-dwc_otg_09:50:571
3134799363,22cyclictest3433-21ls04:41:093
31347993613,12cyclictest0-21swapper/306:40:583
31347993613,12cyclictest0-21swapper/306:40:583
3134599367,5cyclictest7750irq/56-dwc_otg09:46:141
31345993615,6cyclictest21-21rcuc/105:38:571
3134599361,24cyclictest19494-21ssh06:41:091
3134599361,24cyclictest19494-21ssh06:41:091
3134599361,18cyclictest10166-21ssh06:30:581
3134599358,6cyclictest7850irq/56-dwc_otg_09:35:451
3134599358,15cyclictest0-21swapper/105:21:021
3134599358,15cyclictest0-21swapper/105:21:021
3134599357,15cyclictest0-21swapper/104:40:571
31345993513,5cyclictest29340-21unixbench-2d07:31:131
31345993512,4cyclictest0-21swapper/107:41:121
3134599351,21cyclictest22413-21ssh10:00:441
82352341,4sleep30-21swapper/306:25:573
3134799342,22cyclictest0-21swapper/309:42:503
31347993420,5cyclictest18099-21ssh06:40:193
31347993413,11cyclictest32276-21/usr/sbin/munin07:35:583
3134599344,23cyclictest0-21swapper/107:15:571
31345993415,12cyclictest0-21swapper/105:01:091
31345993410,6cyclictest14963-21latency_hist05:15:451
31345993410,4cyclictest0-21swapper/108:06:011
88332331,2sleep30-21swapper/308:25:503
3134799337,6cyclictest7950irq/56-dwc_otg_05:21:123
3134799337,6cyclictest7950irq/56-dwc_otg_05:21:123
3134799336,13cyclictest7950irq/56-dwc_otg_08:51:143
31347993318,8cyclictest5410-21unixbench-2d04:46:133
3134799331,24cyclictest16174-21ssh07:55:463
3134799331,22cyclictest10030-21cut05:01:023
3134599339,8cyclictest7850irq/56-dwc_otg_08:31:071
3134599336,12cyclictest7850irq/56-dwc_otg_09:02:281
3134599336,11cyclictest7950irq/56-dwc_otg_07:26:131
3134599336,11cyclictest7950irq/56-dwc_otg_07:26:131
3134599333,7cyclictest7950irq/56-dwc_otg_09:20:571
31345993312,5cyclictest14851-21ssh06:36:041
3134599331,20cyclictest13674-21rm09:10:481
31345993311,5cyclictest292850wget06:10:571
31345993311,5cyclictest292850wget06:10:571
3134799329,5cyclictest0-21swapper/307:11:363
3134799329,3cyclictest0-21swapper/307:42:313
3134799328,12cyclictest0-21swapper/306:31:133
3134799327,15cyclictest0-21swapper/308:20:583
3134799325,4cyclictest7750irq/56-dwc_otg08:15:453
3134799321,6cyclictest7850irq/56-dwc_otg_09:05:463
3134799321,6cyclictest10318-21ssh09:45:523
31347993216,8cyclictest4867-21fschecks_count06:16:013
3134799321,5cyclictest0-21swapper/308:16:113
3134599327,12cyclictest7850irq/56-dwc_otg_07:11:071
3134599322,5cyclictest0-21swapper/106:01:031
31345993221,8cyclictest28189-21fschecks_count05:51:021
3134599322,15cyclictest0-21swapper/106:24:361
3134599321,7cyclictest6253-21aten_rbpower_po04:50:581
3134599321,25cyclictest0-21swapper/108:26:591
31345993212,5cyclictest0-21swapper/105:31:071
3134599321,21cyclictest19051-21uniq05:26:011
31345993210,4cyclictest0-21swapper/107:50:571
258002320,6sleep17750irq/56-dwc_otg08:46:001
3134799318,11cyclictest0-21swapper/308:40:583
3134799317,4cyclictest0-21swapper/309:10:263
3134799317,14cyclictest0-21swapper/306:45:583
3134599316,14cyclictest21-21rcuc/105:55:591
3134599311,7cyclictest6290-21latency_hist06:20:451
3134599311,7cyclictest21-21rcuc/109:36:041
31345993116,4cyclictest973-21grep04:36:021
3134599311,20cyclictest1500-21ls06:06:041
31345993110,5cyclictest21-21rcuc/108:01:131
102102310,3sleep30-21swapper/307:47:363
38752300,3sleep30-21swapper/308:58:263
3134799308,5cyclictest0-21swapper/307:27:093
3134799308,5cyclictest0-21swapper/307:27:093
3134799301,3cyclictest0-21swapper/307:32:103
3134799301,10cyclictest9478-21ssh07:07:283
3134599302,5cyclictest0-21swapper/107:00:561
31345993016,4cyclictest24638-21fschecks_time05:41:021
3134799296,11cyclictest7850irq/56-dwc_otg_04:36:433
3134799295,4cyclictest7750irq/56-dwc_otg04:56:093
3134799293,4cyclictest7750irq/56-dwc_otg04:31:063
3134799293,12cyclictest25560-21ssh08:45:563
3134799292,13cyclictest7950irq/56-dwc_otg_09:51:143
31347992914,3cyclictest19462-21ntpq05:26:063
31347992913,6cyclictest908-21runrttasks09:55:513
3134599299,13cyclictest0-21swapper/109:44:001
3134599298,4cyclictest0-21swapper/108:56:091
3134599294,5cyclictest0-21swapper/105:45:561
3134599294,5cyclictest0-21swapper/105:45:561
307952298,12sleep30-21swapper/304:27:123
3134799287,5cyclictest7750irq/56-dwc_otg09:31:043
3134799286,4cyclictest0-21swapper/309:26:323
3134799283,15cyclictest0-21swapper/305:11:483
31347992819,5cyclictest381ktimersoftd/305:30:593
3134799281,5cyclictest615250/usr/sbin/munin04:51:033
3134799281,20cyclictest0-21swapper/305:58:253
3134799281,19cyclictest0-21swapper/306:01:223
31345992810,9cyclictest0-21swapper/104:30:571
3134799271,4cyclictest0-21swapper/305:36:123
3134799267,5cyclictest7950irq/56-dwc_otg_06:06:103
3134799263,17cyclictest7950irq/56-dwc_otg_05:46:513
3134799263,17cyclictest7950irq/56-dwc_otg_05:46:513
3134799261,5cyclictest28781-21grep05:51:113
31347992611,4cyclictest908-21runrttasks05:15:593
307572264,15sleep20-21swapper/204:26:432
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional