You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 00:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 22, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4416999913,69cyclictest23185-21aten_rbpower_cu21:16:020
4416999913,69cyclictest23185-21aten_rbpower_cu21:16:020
96112960,4sleep2301ktimersoftd/221:38:192
10362880,2sleep20-21swapper/220:48:432
441699862,41cyclictest0-21swapper/020:10:590
298032850,3sleep228483-21/usr/sbin/munin18:46:162
298032850,3sleep228483-21/usr/sbin/munin18:46:162
4416998414,57cyclictest23781-21wget21:56:000
441699821,60cyclictest0-21swapper/016:45:590
4416998212,49cyclictest763-21basename19:30:470
4416998014,50cyclictest4660-21latency_hist18:00:480
441699796,60cyclictest0-21swapper/018:05:470
4416997815,50cyclictest6402-21fschecks_time20:16:050
4416997813,47cyclictest29293-21taskset20:04:440
4416997714,45cyclictest23255-21head19:56:160
4416997613,47cyclictest6976-21sort18:06:040
441699756,51cyclictest0-21swapper/018:40:580
4416997513,48cyclictest8044-21ssh21:36:120
59982740,5sleep2441899cyclictest18:56:262
441699748,47cyclictest908-21runrttasks21:04:090
441999733,14cyclictest7750irq/56-dwc_otg19:55:583
4416997310,50cyclictest0-21swapper/020:55:480
4416997310,50cyclictest0-21swapper/020:55:480
4416997223,45cyclictest17731-21apt-get19:50:580
4416997211,51cyclictest8316-21chrt19:39:160
441699718,48cyclictest0-21swapper/021:06:110
441699718,45cyclictest0-21swapper/019:41:120
4416997022,7cyclictest6676-21ssh20:56:000
4416997013,49cyclictest22524-21sh20:35:550
4416997010,50cyclictest29097-21awk19:25:470
441699698,50cyclictest0-21swapper/020:10:280
441699697,53cyclictest0-21swapper/021:21:530
4416996911,41cyclictest4889-21cut18:56:040
441699689,50cyclictest24409-21ssh19:19:470
441699689,50cyclictest24409-21ssh19:19:470
441699689,37cyclictest8910-21wget19:01:010
441699686,52cyclictest0-21swapper/018:54:470
441699686,46cyclictest0-21swapper/016:31:170
441699683,46cyclictest20310-21ssh18:36:050
441699678,48cyclictest0-21swapper/021:26:080
441699674,50cyclictest16270-21runrttasks20:27:230
441699674,50cyclictest16270-21runrttasks20:27:230
4416996712,38cyclictest15103-21munin-run21:45:470
4416996710,48cyclictest14151-21sh19:06:140
68412660,2sleep26840-21df_inode18:06:022
441699669,49cyclictest0-21swapper/020:21:400
441699668,45cyclictest0-21swapper/018:49:160
441699668,45cyclictest0-21swapper/018:49:160
4416996617,26cyclictest2116-21ls19:31:130
119372660,2sleep211940-21timerwakeupswit21:01:152
4417996512,9cyclictest7750irq/56-dwc_otg18:55:591
441699657,46cyclictest0-21swapper/020:33:070
441699657,42cyclictest31212-21taskset17:42:240
441699657,42cyclictest31212-21taskset17:42:240
441699654,53cyclictest0-21swapper/020:41:400
441699647,47cyclictest0-21swapper/021:51:080
441699647,41cyclictest0-21swapper/019:11:230
4416996415,36cyclictest17606-21rm21:47:510
4416996415,36cyclictest17606-21rm21:47:510
4416996413,35cyclictest9843-21taskset18:14:050
4416996413,35cyclictest9843-21taskset18:14:050
4416996410,40cyclictest0-21swapper/019:47:430
40042642,15sleep30-21swapper/316:28:593
213832640,2sleep30-21swapper/321:13:123
205522640,2sleep20-21swapper/220:32:412
441699639,39cyclictest0-21swapper/021:33:200
441699638,36cyclictest0-21swapper/021:11:120
441699635,48cyclictest0-21swapper/020:47:020
4416996313,37cyclictest2731-21wc17:55:470
4416996214,28cyclictest12040-21cut16:51:040
441699618,43cyclictest0-21swapper/017:27:220
4416996114,37cyclictest17466-21seq18:31:560
4416996114,25cyclictest12288-21fschecks_count18:21:040
441699601,30cyclictest11388-21grep18:16:160
441699598,37cyclictest0-21swapper/016:56:040
441699597,40cyclictest0-21swapper/017:50:090
441699592,6cyclictest0-21swapper/017:16:140
441799586,6cyclictest0-21swapper/118:35:591
441699586,45cyclictest0-21swapper/017:22:110
4416995829,14cyclictest15428-21munin-run18:30:470
4416995813,23cyclictest26609-21fschecks_time17:31:030
441699576,42cyclictest0-21swapper/016:38:590
4416995727,19cyclictest0-21swapper/016:41:010
35152570,6sleep10-21swapper/116:26:121
441699566,43cyclictest0-21swapper/017:01:380
4416995624,20cyclictest20165-21grep17:11:130
294392560,4sleep329429-21ssh18:46:133
294392560,4sleep329429-21ssh18:46:133
441899556,19cyclictest0-21swapper/216:41:012
441799552,19cyclictest7950irq/56-dwc_otg_18:05:531
441699556,37cyclictest0-21swapper/017:36:100
186712550,2sleep10-21swapper/119:51:131
441799537,30cyclictest7750irq/56-dwc_otg19:35:581
441699536,36cyclictest0-21swapper/017:09:340
441699536,36cyclictest0-21swapper/017:09:340
441999514,22cyclictest7850irq/56-dwc_otg_20:16:053
441899502,12cyclictest0-21swapper/220:41:002
26332502,7sleep00-21swapper/016:26:000
118702500,1sleep30-21swapper/319:43:133
441799492,9cyclictest0-21swapper/119:25:581
4419994816,20cyclictest0-21swapper/318:36:013
441899487,26cyclictest0-21swapper/216:35:592
441899487,25cyclictest0-21swapper/220:20:532
441899481,8cyclictest7950irq/56-dwc_otg_19:25:582
441899477,25cyclictest0-21swapper/221:05:592
441899477,25cyclictest0-21swapper/217:41:012
441899477,25cyclictest0-21swapper/217:41:012
191952470,2sleep20-21swapper/221:11:012
441999467,25cyclictest0-21swapper/317:21:003
441999467,24cyclictest0-21swapper/317:36:003
441999467,24cyclictest0-21swapper/317:26:003
441899461,10cyclictest7750irq/56-dwc_otg21:25:572
441999457,25cyclictest0-21swapper/317:10:593
441999457,24cyclictest0-21swapper/319:51:013
441899457,26cyclictest0-21swapper/219:06:472
441899457,23cyclictest0-21swapper/218:01:012
441899456,25cyclictest0-21swapper/217:31:002
441899452,7cyclictest0-21swapper/218:35:592
441799457,23cyclictest0-21swapper/116:31:011
441799452,4cyclictest0-21swapper/119:00:591
441999447,24cyclictest0-21swapper/320:56:013
441999443,4cyclictest7850irq/56-dwc_otg_18:30:583
441899447,25cyclictest0-21swapper/219:51:012
441899447,23cyclictest0-21swapper/218:26:002
441799448,5cyclictest7950irq/56-dwc_otg_18:51:011
441799447,7cyclictest0-21swapper/119:45:581
441799444,8cyclictest0-21swapper/117:21:001
441999437,23cyclictest0-21swapper/320:06:003
441999435,9cyclictest7750irq/56-dwc_otg21:16:003
441999435,9cyclictest7750irq/56-dwc_otg21:16:003
441899438,21cyclictest7950irq/56-dwc_otg_19:36:002
441899431,10cyclictest7850irq/56-dwc_otg_20:00:592
441899430,26cyclictest1698-21ls19:31:062
441799437,24cyclictest0-21swapper/117:10:531
4419994229,4cyclictest7950irq/56-dwc_otg_20:11:083
441899427,23cyclictest0-21swapper/216:55:592
441899423,8cyclictest7950irq/56-dwc_otg_19:15:592
441899423,8cyclictest7950irq/56-dwc_otg_19:15:592
441899423,25cyclictest0-21swapper/217:56:002
441899421,11cyclictest17304-21ssh19:50:372
441799422,7cyclictest0-21swapper/119:16:001
441799422,7cyclictest0-21swapper/119:16:001
4417994215,11cyclictest14885-21ssh21:05:581
4419994119,6cyclictest13183-21sh21:42:183
441899416,21cyclictest3041-21ssh21:31:002
441899411,18cyclictest0-21swapper/219:05:112
441899410,14cyclictest11882-21/usr/sbin/munin16:51:012
4417994128,6cyclictest18252-21latency_hist20:30:481
4417994128,6cyclictest18252-21latency_hist20:30:481
181182410,18sleep37950irq/56-dwc_otg_17:06:143
181182410,18sleep37950irq/56-dwc_otg_17:06:143
441999405,21cyclictest14348-21ssh20:26:003
441999405,21cyclictest14348-21ssh20:26:003
441999404,23cyclictest0-21swapper/316:46:013
441999403,28cyclictest0-21swapper/316:35:573
4418994019,5cyclictest22637-21aten_rbpower_en20:36:012
441899401,24cyclictest0-21swapper/221:42:312
441899401,24cyclictest0-21swapper/217:46:012
441899401,13cyclictest23112-21/usr/sbin/munin21:16:102
441899401,13cyclictest23112-21/usr/sbin/munin21:16:102
441799409,4cyclictest0-21swapper/121:38:531
441799404,17cyclictest7850irq/56-dwc_otg_22:00:481
441799402,8cyclictest7919-21grep20:56:161
4417994019,5cyclictest1384-21sh20:10:291
4417994017,5cyclictest29768-21sh20:05:371
441899396,18cyclictest7950irq/56-dwc_otg_18:40:592
441899395,20cyclictest9878-21sshd16:46:012
441899393,10cyclictest7750irq/56-dwc_otg18:54:292
441899392,4cyclictest7950irq/56-dwc_otg_17:20:582
441899391,22cyclictest0-21swapper/219:56:202
16052390,6sleep1441799cyclictest21:28:481
441999388,20cyclictest0-21swapper/319:19:073
441999388,20cyclictest0-21swapper/319:19:073
4419993827,4cyclictest7950irq/56-dwc_otg_20:49:563
4419993826,7cyclictest381ktimersoftd/320:21:003
441899388,15cyclictest0-21swapper/217:11:012
4417993819,6cyclictest115550wget17:51:021
4417993815,7cyclictest9493-21sh19:40:591
441999375,20cyclictest14897-21ssh21:05:593
4419993718,5cyclictest30698-21latency_hist20:45:483
441899377,25cyclictest20895-21taskset17:16:002
441899377,25cyclictest0-21swapper/217:36:002
441899375,13cyclictest7950irq/56-dwc_otg_21:58:522
441899370,17cyclictest6633-21ls20:16:082
441799373,17cyclictest7850irq/56-dwc_otg_21:11:161
441799372,9cyclictest7750irq/56-dwc_otg20:10:591
229912370,5sleep3441999cyclictest21:55:113
441899367,23cyclictest0-21swapper/217:00:592
441899361,9cyclictest0-21swapper/220:50:582
441899361,9cyclictest0-21swapper/220:50:582
4418993616,5cyclictest15760-21ssh18:30:592
441899360,10cyclictest7950irq/56-dwc_otg_21:22:192
441799369,13cyclictest7950irq/56-dwc_otg_16:41:161
4417993617,4cyclictest22557-21/usr/sbin/munin20:36:121
441799361,23cyclictest12632-21ls16:51:131
4417993610,18cyclictest0-21swapper/121:50:591
25302360,3sleep12513-21apt-get20:51:001
25302360,3sleep12513-21apt-get20:51:001
441999358,3cyclictest0-21swapper/320:04:313
441999358,15cyclictest0-21swapper/319:36:073
4419993517,5cyclictest11862-21aten_rbpower_po16:51:013
4419993510,4cyclictest0-21swapper/318:51:133
441899357,14cyclictest0-21swapper/221:46:012
441899357,14cyclictest0-21swapper/221:46:012
441899355,11cyclictest0-21swapper/221:55:402
4418993521,11cyclictest12064-21aten_rbpower_cu18:21:012
441899351,13cyclictest18653-21ssh19:11:452
4418993511,10cyclictest0-21swapper/220:25:592
4418993511,10cyclictest0-21swapper/220:25:592
441899350,12cyclictest1791-21ssh20:10:492
441799357,6cyclictest7850irq/56-dwc_otg_18:46:121
441799357,6cyclictest7850irq/56-dwc_otg_18:46:121
441799356,5cyclictest25659-21fschecks_time19:21:051
441799352,5cyclictest0-21swapper/119:11:051
4417993515,6cyclictest16534-21grep17:01:141
4417993510,13cyclictest0-21swapper/121:16:501
4417993510,13cyclictest0-21swapper/121:16:501
441999347,6cyclictest7850irq/56-dwc_otg_21:30:473
441999342,18cyclictest31013-21cron21:25:473
4419993412,5cyclictest37-21rcuc/319:01:083
441899348,15cyclictest7950irq/56-dwc_otg_19:22:572
441899346,14cyclictest7950irq/56-dwc_otg_20:56:042
441899340,11cyclictest0-21swapper/220:07:412
441799349,4cyclictest7750irq/56-dwc_otg20:31:071
441799348,4cyclictest7097-21grep16:36:121
441799345,8cyclictest7750irq/56-dwc_otg18:26:151
441799345,14cyclictest7850irq/56-dwc_otg_18:11:141
441799345,14cyclictest7850irq/56-dwc_otg_18:11:141
441799342,18cyclictest7750irq/56-dwc_otg21:34:461
4417993421,4cyclictest23339-21sh19:56:231
441799341,6cyclictest11566-21fschecks_count21:41:031
4417993416,6cyclictest30554-21ssh21:24:571
4417993414,4cyclictest12750-21sh21:02:291
4417993413,4cyclictest0-21swapper/120:46:101
441799341,23cyclictest24762-21cut17:26:041
4417993411,11cyclictest0-21swapper/119:33:411
441799341,10cyclictest25188-21/usr/sbin/munin18:41:121
112882340,8sleep1441799cyclictest18:16:161
441999339,12cyclictest0-21swapper/320:34:553
441999333,4cyclictest7950irq/56-dwc_otg_21:31:073
441999332,6cyclictest7850irq/56-dwc_otg_19:31:003
441999331,5cyclictest13774-21df16:56:013
441999331,22cyclictest14643-21timerandwakeup19:46:143
441999331,18cyclictest7950irq/56-dwc_otg_19:27:413
441999331,17cyclictest7750irq/56-dwc_otg19:15:323
441899336,15cyclictest0-21swapper/219:41:002
441899333,6cyclictest7950irq/56-dwc_otg_16:31:012
441899331,9cyclictest2185-21grep17:51:162
441899330,20cyclictest917-21snmpd18:14:292
441899330,20cyclictest917-21snmpd18:14:292
441799338,5cyclictest7950irq/56-dwc_otg_17:36:011
441799336,7cyclictest7750irq/56-dwc_otg17:46:161
441799335,15cyclictest7950irq/56-dwc_otg_18:35:481
441799335,15cyclictest7950irq/56-dwc_otg_17:56:051
4417993319,4cyclictest18752-21idleruntime-cro17:10:471
4417993319,4cyclictest18752-21idleruntime-cro17:10:471
441799331,4cyclictest0-21swapper/121:46:151
441799331,4cyclictest0-21swapper/121:46:151
441799331,3cyclictest0-21swapper/120:15:591
36482335,16sleep22587-21/usr/sbin/munin16:26:142
441999329,4cyclictest37-21rcuc/317:46:163
441999328,4cyclictest0-21swapper/317:16:153
441999327,22cyclictest0-21swapper/318:56:013
441999323,5cyclictest26458-21ssh20:40:483
441999321,4cyclictest0-21swapper/317:55:583
4419993213,5cyclictest6717-21aten_rbpower_vo18:06:013
441899323,13cyclictest7950irq/56-dwc_otg_17:10:482
441899323,13cyclictest7950irq/56-dwc_otg_17:10:482
441799328,13cyclictest0-21swapper/119:09:101
441799322,7cyclictest27957-21grep20:41:161
441999315,5cyclictest7950irq/56-dwc_otg_19:21:043
441999315,13cyclictest7950irq/56-dwc_otg_21:01:053
441999312,6cyclictest15380-21apt-get17:00:593
441999311,7cyclictest25490-21sed18:41:153
441799319,5cyclictest7750irq/56-dwc_otg17:16:041
441799315,14cyclictest7850irq/56-dwc_otg_20:21:161
441799313,4cyclictest7750irq/56-dwc_otg17:00:471
441799311,13cyclictest7950irq/56-dwc_otg_18:21:151
441999309,4cyclictest0-21swapper/320:51:093
441999309,4cyclictest0-21swapper/320:51:093
441999307,4cyclictest0-21swapper/321:58:413
441999302,18cyclictest0-21swapper/317:42:023
441999302,18cyclictest0-21swapper/317:42:023
4419993016,5cyclictest4709-21/usr/sbin/munin18:01:123
4419993013,6cyclictest908-21runrttasks18:18:023
441999301,13cyclictest7950irq/56-dwc_otg_16:41:123
441899304,4cyclictest0-21swapper/218:16:092
441999294,5cyclictest7850irq/56-dwc_otg_19:06:053
4419992910,4cyclictest0-21swapper/321:46:073
4419992910,4cyclictest0-21swapper/321:46:073
4419992910,3cyclictest0-21swapper/321:40:263
441899295,5cyclictest7750irq/56-dwc_otg17:26:042
441799294,4cyclictest7750irq/56-dwc_otg18:01:051
441999286,18cyclictest0-21swapper/316:31:013
441999281,12cyclictest7950irq/56-dwc_otg_17:55:403
441799286,6cyclictest7750irq/56-dwc_otg17:41:081
441799286,6cyclictest7750irq/56-dwc_otg17:41:081
441799284,6cyclictest10966-21tail16:46:161
4417992811,4cyclictest26938-21/usr/sbin/munin17:31:081
4419992718,6cyclictest12135-21sshd18:21:013
4419992717,7cyclictest14274-21if_wlan018:26:053
441999261,3cyclictest7750irq/56-dwc_otg17:31:053
441999261,12cyclictest7950irq/56-dwc_otg_18:13:193
441999261,12cyclictest7950irq/56-dwc_otg_18:13:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional