You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 19:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Apr 23, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2695225270,5sleep30-21swapper/316:26:053
629521730,4sleep22836699cyclictest21:05:122
629521730,4sleep22836699cyclictest21:05:122
88652990,2sleep30-21swapper/320:28:303
125632960,2sleep30-21swapper/320:32:403
310782940,5sleep02836499cyclictest20:55:520
28364998337,33cyclictest0-21swapper/021:06:000
311392790,6sleep32836799cyclictest19:36:203
215692770,4sleep231-21ksoftirqd/218:46:152
2836499769,52cyclictest101ktimersoftd/021:26:150
28364997628,15cyclictest0-21swapper/019:21:010
28367997424,7cyclictest7850irq/56-dwc_otg_21:00:583
28367997424,7cyclictest7850irq/56-dwc_otg_21:00:583
2836499747,36cyclictest0-21swapper/018:55:550
136682720,6sleep32836799cyclictest19:55:493
136682720,6sleep32836799cyclictest19:55:493
2836499713,54cyclictest4985-21aten_rbpower_en19:06:020
28365997014,10cyclictest7950irq/56-dwc_otg_20:26:001
28364997048,10cyclictest11698-21grep17:11:150
28364997016,9cyclictest0-21swapper/019:41:100
28364996916,49cyclictest101ktimersoftd/017:00:480
151892690,2sleep10-21swapper/119:56:151
2836499682,57cyclictest10929-21sed19:51:150
2836499682,57cyclictest10929-21sed19:51:150
279452680,3sleep30-21swapper/320:51:123
2836499668,44cyclictest25013-21munin-plugin-st17:50:480
2836499666,53cyclictest0-21swapper/020:41:010
2836499662,51cyclictest0-21swapper/019:11:040
28364996610,46cyclictest3858-21sh21:01:130
28364996610,46cyclictest3858-21sh21:01:130
28365996522,16cyclictest23816-21sh18:50:031
2836499657,39cyclictest0-21swapper/018:31:050
28364996518,36cyclictest7014-21diskmemload20:06:110
28364996515,46cyclictest101ktimersoftd/019:31:110
121562650,2sleep30-21swapper/319:14:343
28365996416,12cyclictest30698-21ssh20:55:351
2836499649,40cyclictest0-21swapper/018:36:090
2836499648,45cyclictest30788-21ssh21:34:290
2836499642,50cyclictest14713-21aten_rbpower_po20:36:020
2836499642,50cyclictest14713-21aten_rbpower_po20:36:020
28364996412,38cyclictest101ktimersoftd/017:56:040
28364996412,38cyclictest101ktimersoftd/017:56:040
312792630,2sleep20-21swapper/219:36:352
2836799632,9cyclictest11080-21ssh21:10:583
2836499638,38cyclictest0-21swapper/020:11:060
28364996321,26cyclictest5955-21latency_hist20:25:480
2836499631,28cyclictest0-21swapper/017:25:540
2836499629,44cyclictest12037-21df21:51:030
2836499628,26cyclictest0-21swapper/016:46:000
28364996238,5cyclictest0-21swapper/017:05:590
28364996213,32cyclictest101ktimersoftd/020:16:170
277582623,14sleep20-21swapper/216:26:262
28365996026,12cyclictest11052-21awk19:51:161
28365996026,12cyclictest11052-21awk19:51:161
28365996010,32cyclictest0-21swapper/121:01:371
28365996010,32cyclictest0-21swapper/121:01:371
2836499602,37cyclictest25870-21unixbench_singl18:51:170
28365995923,22cyclictest12288-21ssh19:14:491
28365995923,18cyclictest2970-21ssh21:40:221
2836499598,39cyclictest0-21swapper/020:02:180
2836499596,50cyclictest0-21swapper/019:39:340
2836499595,21cyclictest0-21swapper/021:16:000
28364995919,25cyclictest20245-21sh18:45:520
28364995910,4cyclictest101ktimersoftd/016:55:480
28364995910,36cyclictest0-21swapper/019:16:150
2836499587,43cyclictest0-21swapper/021:56:090
28364995831,12cyclictest0-21swapper/021:46:590
28364995831,12cyclictest0-21swapper/021:46:590
28364995815,33cyclictest101ktimersoftd/019:25:580
28364995811,31cyclictest101ktimersoftd/021:45:250
28364995810,30cyclictest2622250rm20:49:580
2836499576,27cyclictest0-21swapper/017:51:050
28364995713,28cyclictest101ktimersoftd/021:11:100
2836699567,29cyclictest0-21swapper/217:41:012
28365995624,18cyclictest7850irq/56-dwc_otg_21:21:071
28365995622,11cyclictest27396-21fschecks_time17:56:051
28365995622,11cyclictest27396-21fschecks_time17:56:051
2836499567,27cyclictest0-21swapper/018:16:150
2836499563,23cyclictest0-21swapper/017:20:590
2836499563,23cyclictest0-21swapper/017:20:590
2836499561,45cyclictest0-21swapper/019:48:050
2836499561,26cyclictest6421-21ls18:26:140
68012550,2sleep10-21swapper/121:44:491
301242550,2sleep30-21swapper/321:33:153
28365995521,21cyclictest27047-21cat20:11:091
28365995517,18cyclictest20093-21ssh20:02:381
2836499555,29cyclictest7475-21cut17:01:070
2836499554,40cyclictest29219-21diskmemload20:52:520
2836499554,27cyclictest29530-21unixbench_singl16:31:140
28364995537,10cyclictest6280-21ssh20:25:540
2836499551,30cyclictest3697-21aten_rbpower_en18:21:010
2836699547,25cyclictest0-21swapper/217:51:002
28365995419,12cyclictest26420-21ntp_states19:31:101
28365995414,12cyclictest12063-21/usr/sbin/munin21:51:041
28364995430,6cyclictest13865-21/usr/sbin/munin19:56:020
2836499541,7cyclictest32176-21wget16:41:010
28365995324,15cyclictest23207-21idleruntime-cro17:45:481
28365995324,11cyclictest8991-21/usr/sbin/munin17:06:001
28365995323,19cyclictest8730-21ssh21:07:051
28365995320,18cyclictest24025-21ssh19:28:261
28365995317,13cyclictest25903-21/usr/sbin/munin17:51:081
2836499533,37cyclictest0-21swapper/019:00:530
2836499533,37cyclictest0-21swapper/019:00:530
28364995331,9cyclictest7850irq/56-dwc_otg_20:30:580
28364995331,12cyclictest0-21swapper/018:06:000
2836499531,40cyclictest19464-21cut21:21:010
28366995220,8cyclictest6357-21runrttasks20:26:002
28365995220,18cyclictest26454-21sh20:50:231
28365995216,19cyclictest7634-21ssh18:31:021
2836499528,12cyclictest13999-21grep17:20:480
2836499525,23cyclictest0-21swapper/021:40:170
2836499522,34cyclictest23308-21latency_hist17:45:480
28365995122,17cyclictest14538-21ssh21:15:101
28365995121,10cyclictest23552-21/usr/sbin/munin21:26:121
28365995120,18cyclictest5534-21ls19:06:091
28365995120,16cyclictest17610-21ssh21:18:151
28365995120,11cyclictest20300-21ssh20:41:401
28365995115,12cyclictest22479-21cut20:06:071
279282513,10sleep00-21swapper/016:28:380
228932510,1sleep3381ktimersoftd/320:06:113
2836699503,31cyclictest0-21swapper/221:36:002
2836699501,16cyclictest0-21swapper/219:05:492
2836699501,16cyclictest0-21swapper/219:05:492
28365995023,17cyclictest30975-21ssh21:34:501
28365995023,11cyclictest12369-21memory18:36:101
28365995022,18cyclictest3651-21ssh19:42:481
28365995020,18cyclictest17342-21sh20:38:491
28365995020,18cyclictest17342-21sh20:38:491
28365995019,19cyclictest15041-21rm19:17:171
28365995017,19cyclictest31123-21ssh20:55:581
28365995012,23cyclictest11811-21mkdir17:11:301
28365995011,10cyclictest0-21swapper/119:00:531
28365995011,10cyclictest0-21swapper/119:00:531
28364995024,5cyclictest18773-21ps17:31:120
28365994921,15cyclictest10082-21ssh21:48:151
28365994921,15cyclictest10082-21ssh21:48:151
28365994920,13cyclictest5674-21sed16:56:081
28365994917,17cyclictest17652-21sh18:41:521
28365994914,24cyclictest31559-21ssh19:37:061
28365994913,24cyclictest0-21swapper/121:59:201
2836499494,38cyclictest0-21swapper/018:41:330
28365994821,17cyclictest7014-21diskmemload18:55:071
28365994816,23cyclictest31207-21sh18:57:541
28364994829,7cyclictest0-21swapper/018:12:040
2836499481,26cyclictest20929-21grep17:36:170
2836699477,26cyclictest0-21swapper/216:41:012
2836599477,25cyclictest0-21swapper/117:21:011
2836599477,25cyclictest0-21swapper/117:21:011
2836599473,8cyclictest7750irq/56-dwc_otg19:45:591
28365994713,12cyclictest7551-21load17:01:081
28365994711,11cyclictest7950irq/56-dwc_otg_19:23:191
2836599469,24cyclictest0-21swapper/116:36:021
28365994620,12cyclictest5523-21sh20:25:071
28365994612,15cyclictest13010-21ssh20:33:301
2836499461,32cyclictest30867-21ntpq16:36:100
28364994610,33cyclictest101ktimersoftd/018:01:260
2836799455,25cyclictest0-21swapper/317:11:013
2836699457,24cyclictest0-21swapper/217:20:542
2836699457,24cyclictest0-21swapper/217:20:542
2836699457,24cyclictest0-21swapper/216:51:012
2836699456,25cyclictest22913-21ssh20:46:012
2836599458,24cyclictest0-21swapper/116:40:531
2836599457,24cyclictest0-21swapper/118:26:001
28367994416,6cyclictest7014-21diskmemload20:56:163
2836699447,23cyclictest0-21swapper/218:21:012
2836599447,25cyclictest0-21swapper/116:46:011
2836599442,6cyclictest0-21swapper/117:26:001
2836699437,24cyclictest0-21swapper/219:51:002
2836699437,24cyclictest0-21swapper/219:51:002
2836699431,2cyclictest0-21swapper/217:05:582
2836599439,12cyclictest0-21swapper/117:48:271
2836599435,24cyclictest0-21swapper/118:21:011
28365994325,4cyclictest31364-21sshd18:06:111
28365994317,8cyclictest862-21ssh20:19:101
28365994311,11cyclictest1515-21seq18:15:051
2836699427,25cyclictest0-21swapper/217:26:012
2836699427,25cyclictest0-21swapper/217:00:532
2836699426,7cyclictest29-21rcuc/217:56:092
2836699426,7cyclictest29-21rcuc/217:56:092
28366994212,4cyclictest7750irq/56-dwc_otg21:51:022
2836599422,8cyclictest0-21swapper/117:36:021
28365994216,11cyclictest19534-21seq17:35:161
60052411,34sleep26103-21if_eth019:46:062
2836799416,21cyclictest1508-21ssh19:41:013
2836699417,7cyclictest29-21rcuc/219:11:442
2836699417,30cyclictest0-21swapper/221:21:012
2836699417,23cyclictest0-21swapper/219:20:532
2836699415,6cyclictest31-21ksoftirqd/221:11:512
28365994111,9cyclictest7950irq/56-dwc_otg_16:53:041
28365994111,12cyclictest7750irq/56-dwc_otg18:01:491
2836799407,20cyclictest0-21swapper/317:00:543
2836699407,24cyclictest0-21swapper/218:01:012
2836699405,29cyclictest7950irq/56-dwc_otg_19:05:592
28366994014,8cyclictest13263-21aten_rbpower_po19:16:012
255762400,2sleep30-21swapper/320:48:473
85132391,3sleep20-21swapper/218:31:142
2836799391,18cyclictest7750irq/56-dwc_otg21:50:593
2836699399,4cyclictest7750irq/56-dwc_otg20:36:022
2836699399,4cyclictest7750irq/56-dwc_otg20:36:022
2836699397,25cyclictest0-21swapper/220:51:002
28366993920,10cyclictest301ktimersoftd/218:55:352
2836699391,9cyclictest0-21swapper/221:16:002
28366993918,9cyclictest301ktimersoftd/219:31:152
28366993918,11cyclictest301ktimersoftd/219:45:392
2836599397,14cyclictest0-21swapper/116:30:581
2836799384,16cyclictest7950irq/56-dwc_otg_19:31:003
28367993820,5cyclictest23218-21idleruntime-cro17:45:483
2836799381,25cyclictest25483-21cut17:51:043
2836699385,17cyclictest31-21ksoftirqd/221:41:032
28366993822,4cyclictest31819-21apt-key16:40:482
28366993815,14cyclictest301ktimersoftd/219:26:052
2836699381,26cyclictest301ktimersoftd/220:08:212
2836599389,10cyclictest0-21swapper/118:20:241
28365993825,10cyclictest13111-21/usr/sbin/munin17:16:111
28367993727,7cyclictest3397-21latency_hist21:40:493
2836799372,6cyclictest13571-21grep17:16:163
28367993718,5cyclictest24809-21cpuspeed18:51:033
28367993715,5cyclictest908-21runrttasks21:23:103
2836799371,24cyclictest7850irq/56-dwc_otg_19:25:583
2836699379,19cyclictest301ktimersoftd/217:45:562
2836699379,15cyclictest31-21ksoftirqd/220:05:222
2836699377,8cyclictest301ktimersoftd/220:11:072
2836699376,3cyclictest7750irq/56-dwc_otg20:19:122
2836699376,22cyclictest0-21swapper/216:56:012
2836699376,17cyclictest301ktimersoftd/220:22:022
2836699374,6cyclictest31-21ksoftirqd/221:30:482
28366993718,10cyclictest29798-21proc_pri18:56:122
2836699371,24cyclictest17065-21ls18:41:152
28366993711,7cyclictest301ktimersoftd/220:44:262
2836699369,8cyclictest7750irq/56-dwc_otg21:30:572
2836699365,19cyclictest29-21rcuc/218:17:372
2836699363,6cyclictest29-21rcuc/221:06:072
2836799355,21cyclictest17328-21ssh19:20:533
2836799353,5cyclictest0-21swapper/318:05:593
28367993522,3cyclictest7950irq/56-dwc_otg_19:16:013
2836799351,23cyclictest16735-21latency17:26:073
2836799351,22cyclictest5348-21cut16:56:043
2836799351,20cyclictest4846-21latency_hist16:55:483
28367993510,16cyclictest6146-21kworker/3:121:44:493
2836699355,17cyclictest7850irq/56-dwc_otg_18:36:062
2836699354,18cyclictest301ktimersoftd/221:59:202
2836699352,9cyclictest0-21swapper/218:11:152
2836699351,5cyclictest29-21rcuc/217:11:152
28366993515,8cyclictest301ktimersoftd/220:56:062
2836699351,24cyclictest0-21swapper/219:59:302
2836799345,15cyclictest7950irq/56-dwc_otg_18:56:013
28367993419,4cyclictest39-21ksoftirqd/321:56:143
28367993414,6cyclictest7950irq/56-dwc_otg_18:40:593
2836799341,3cyclictest0-21swapper/318:46:153
2836799341,23cyclictest2703-21switchtime18:16:133
2836799341,21cyclictest16005-21dpkg17:25:493
2836799341,21cyclictest16005-21dpkg17:25:493
2836699346,15cyclictest0-21swapper/217:36:032
2836699344,12cyclictest7701-21ssh21:46:012
2836699344,12cyclictest7701-21ssh21:46:012
2836699343,21cyclictest29-21rcuc/217:16:082
2836699341,26cyclictest7850irq/56-dwc_otg_20:32:562
2836799338,12cyclictest7850irq/56-dwc_otg_20:16:073
2836799337,5cyclictest0-21swapper/320:21:073
2836799332,6cyclictest1586-21grep16:46:023
2836799331,25cyclictest31182-21timerwakeupswit16:36:153
2836799331,21cyclictest10790-21ls21:10:483
2836699335,15cyclictest29-21rcuc/216:50:482
2836799326,5cyclictest7950irq/56-dwc_otg_19:46:163
2836799324,4cyclictest0-21swapper/319:01:003
2836799324,4cyclictest0-21swapper/319:01:003
2836799322,21cyclictest0-21swapper/317:07:093
28367993220,5cyclictest18810-21wget20:41:013
28367993213,6cyclictest14921-21fschecks_count20:36:053
28367993213,6cyclictest14921-21fschecks_count20:36:053
2836699321,26cyclictest31185-21latency18:06:082
2836799317,11cyclictest7950irq/56-dwc_otg_17:31:023
2836799316,5cyclictest7750irq/56-dwc_otg17:36:183
2836799312,7cyclictest28626-21fschecks_count16:31:033
28367993124,5cyclictest6329-21ssh19:06:273
2836799311,8cyclictest17872-21ssh21:18:443
2836799311,17cyclictest7014-21diskmemload20:11:023
2836799311,13cyclictest7750irq/56-dwc_otg18:31:033
28367993110,3cyclictest0-21swapper/321:25:523
2836699319,4cyclictest7750irq/56-dwc_otg16:32:202
2836699315,17cyclictest7950irq/56-dwc_otg_18:30:482
2836799307,12cyclictest0-21swapper/318:39:383
2836799302,21cyclictest0-21swapper/318:04:543
2836799302,19cyclictest0-21swapper/317:58:523
2836799302,19cyclictest0-21swapper/317:58:523
28367993015,5cyclictest1733-21latency_hist18:15:483
2836799301,25cyclictest32378-21cut16:41:053
2836699305,17cyclictest301ktimersoftd/217:32:212
278352306,18sleep10-21swapper/116:27:261
67042290,2sleep30-21swapper/318:27:123
2836799299,9cyclictest7014-21diskmemload20:02:213
2836799291,22cyclictest0-21swapper/321:46:063
2836799291,22cyclictest0-21swapper/321:46:063
28367992910,9cyclictest16188-21ssh19:58:023
2836799272,8cyclictest0-21swapper/318:21:483
2836799272,16cyclictest0-21swapper/317:47:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional