You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 17:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Apr 15, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
145539910033,47cyclictest7178-21ls20:40:430
14553999828,59cyclictest14367-21ntpq21:26:060
14553999730,46cyclictest23868-21grep16:56:000
14553999728,47cyclictest22566-21latency_hist20:20:440
14553999725,56cyclictest0-21swapper/018:31:040
237552960,2sleep10-21swapper/120:59:451
14553999634,28cyclictest0-21swapper/016:30:570
14553999634,28cyclictest0-21swapper/016:30:570
14553999631,51cyclictest9173-21cron17:45:430
14553999626,48cyclictest30353-21munin-plugin-st21:45:430
14553999432,12cyclictest20648-21ssh19:40:440
14553999424,53cyclictest0-21swapper/019:57:490
14553999232,44cyclictest20068-21aten_rbpower_en16:45:560
14553999226,47cyclictest14145-21grep17:56:100
14553999225,55cyclictest14968-21egrep18:55:570
14553999224,55cyclictest26363-21grep17:01:100
14553999133,41cyclictest25595-21ssh19:46:010
14553999133,41cyclictest25595-21ssh19:46:010
14553998931,46cyclictest19471-21sh21:31:420
321032880,2sleep00-21swapper/020:31:050
14553998833,38cyclictest26837-21ssh19:09:570
14553998833,38cyclictest26837-21ssh19:09:570
14553998823,52cyclictest11180-21rm19:29:220
14553998820,54cyclictest5798-21grep20:00:590
14553998727,45cyclictest0-21swapper/019:21:040
14553998725,40cyclictest7287-21idleruntime-cro17:40:440
14553998725,40cyclictest7287-21idleruntime-cro17:40:440
14553998721,49cyclictest0-21swapper/020:41:000
14553998628,37cyclictest10160-21latency_hist18:50:440
14553998626,41cyclictest23755-21latency_hist18:25:450
14553998626,41cyclictest23755-21latency_hist18:25:450
14553998623,47cyclictest11462-21fschecks_count17:51:010
14553998528,44cyclictest27292-21ssh21:03:280
14553998429,41cyclictest28555-21ssh20:26:240
14553998427,41cyclictest11840-21rm21:23:180
14553998423,48cyclictest0-21swapper/017:48:390
14553998325,43cyclictest7208-21taskset21:17:390
14553998324,41cyclictest0-21swapper/020:20:590
14553998227,43cyclictest24144-21fschecks_count18:26:010
14553998225,42cyclictest22623-21ssh20:57:470
14553998224,45cyclictest0-21swapper/019:52:080
14553998224,44cyclictest22422-21rm19:04:430
14553998222,45cyclictest0-21swapper/021:51:410
14553998129,30cyclictest31316-21ls21:46:050
14553998127,39cyclictest10271-21ssh21:59:560
14553998122,37cyclictest21811-21grep19:41:060
14553998029,35cyclictest12867-21ssh18:53:080
14553998029,12cyclictest19533-21ssh20:54:550
14553998025,43cyclictest30514-21ssh21:06:390
14553998025,43cyclictest30514-21ssh21:06:390
14553998025,40cyclictest31843-21ssh19:15:490
14553998024,41cyclictest24092-21ssh21:37:180
14553997927,32cyclictest22484-21ntpq16:51:040
14553997922,53cyclictest10835-21rt-features20:06:100
14553997920,46cyclictest2549-21chrt21:12:000
14553997824,40cyclictest14386-21ssh19:32:310
14553997824,40cyclictest14386-21ssh19:32:310
14553997821,43cyclictest0-21swapper/018:36:130
14553997821,42cyclictest0-21swapper/020:14:550
14553997821,42cyclictest0-21swapper/017:27:160
14553997817,43cyclictest0-21swapper/020:49:140
32432770,2sleep30-21swapper/319:20:273
14553997726,34cyclictest1927-21cut18:40:560
14553997716,53cyclictest0-21swapper/018:12:140
269792760,4sleep10-21swapper/117:05:441
1455399766,54cyclictest0-21swapper/018:16:010
14553997625,38cyclictest17931-21taskset16:40:300
14553997525,32cyclictest28624-21taskset17:08:590
14553997524,34cyclictest31443-21ssh19:15:310
14553997424,38cyclictest16267-21chrt18:04:070
210402730,3sleep231-21ksoftirqd/219:02:182
14553997225,31cyclictest19746-21chrt16:45:380
325992710,5sleep314552-21cyclictest21:47:133
1455499717,26cyclictest0-21swapper/121:15:571
14553997129,25cyclictest32376-21taskset17:19:260
14553997123,9cyclictest0-21swapper/018:09:300
14553997120,39cyclictest908-21runrttasks17:32:310
118292710,2sleep211831-21basename20:45:572
14553997017,45cyclictest0-21swapper/017:24:400
78982690,7sleep1221ktimersoftd/119:25:441
1455499697,22cyclictest0-21swapper/120:20:581
14553996921,30cyclictest0-21swapper/017:11:530
14553996921,30cyclictest0-21swapper/017:11:530
156742660,2sleep20-21swapper/221:27:322
57622580,2sleep10-21swapper/121:54:321
134122580,5sleep31455699cyclictest20:09:593
1455499575,13cyclictest1380-21ssh19:55:561
295222550,2sleep20-21swapper/218:35:252
1455499497,26cyclictest0-21swapper/116:45:561
14554994913,23cyclictest0-21swapper/117:40:501
141002493,37sleep00-21swapper/016:28:230
1455599481,6cyclictest7850irq/56-dwc_otg_19:55:562
1455499487,26cyclictest0-21swapper/119:45:571
1455499487,26cyclictest0-21swapper/119:45:571
1455699477,25cyclictest0-21swapper/317:15:583
1455599477,26cyclictest0-21swapper/217:55:562
1455499477,26cyclictest3239-21ssh20:35:551
1455499476,27cyclictest0-21swapper/118:15:571
14554994722,5cyclictest26408-21/usr/sbin/munin18:31:041
94372460,5sleep31455699cyclictest18:49:353
14556994623,5cyclictest17907-21grep21:30:563
1455699461,35cyclictest0-21swapper/321:01:023
1455499467,25cyclictest0-21swapper/121:10:571
1455499466,24cyclictest0-21swapper/117:05:581
1455599458,22cyclictest0-21swapper/221:15:572
1455499456,26cyclictest0-21swapper/117:55:561
14554994520,5cyclictest8580-21sh20:04:001
1455499449,25cyclictest0-21swapper/121:20:571
1455499447,22cyclictest0-21swapper/119:10:571
204922430,3sleep220495-21fschecks_count18:16:012
1455699437,11cyclictest0-21swapper/319:50:573
1455699431,7cyclictest7950irq/56-dwc_otg_18:00:563
1455599436,16cyclictest7950irq/56-dwc_otg_18:00:572
1455499437,23cyclictest0-21swapper/118:05:561
1455499432,25cyclictest7750irq/56-dwc_otg18:51:001
126332430,2sleep10-21swapper/120:46:081
1455599427,24cyclictest0-21swapper/216:45:562
1455499426,24cyclictest0-21swapper/119:35:501
1455699418,19cyclictest0-21swapper/317:30:563
14556994112,5cyclictest0-21swapper/321:10:583
14556994112,16cyclictest39-21ksoftirqd/320:15:543
14555994126,7cyclictest14581-21fschecks_count20:11:002
14555994117,7cyclictest20018-21latency_hist18:15:442
14555994115,6cyclictest32621-21taskset17:20:442
14554994124,5cyclictest2397950wget18:25:581
1455699401,7cyclictest7706-21ssh21:56:093
1455599407,22cyclictest0-21swapper/219:35:502
1455499407,23cyclictest0-21swapper/118:20:561
1455499407,23cyclictest0-21swapper/118:20:561
1455499402,9cyclictest7750irq/56-dwc_otg21:00:571
1455499401,29cyclictest23131-21ls19:05:441
45562390,2sleep20-21swapper/217:31:052
190122390,3sleep339-21ksoftirqd/318:11:033
1455699397,22cyclictest0-21swapper/319:55:583
1455499393,14cyclictest7850irq/56-dwc_otg_17:35:441
14554993911,13cyclictest0-21swapper/120:10:471
1455699385,20cyclictest0-21swapper/320:41:323
1455599386,20cyclictest23367-21ssh19:05:562
1455599386,20cyclictest23367-21ssh19:05:562
14555993821,4cyclictest8628-21timerwakeupswit20:41:102
1455499381,4cyclictest19993-21cron20:55:431
14556993718,10cyclictest25688-21fschecks_count17:01:003
14556993716,13cyclictest0-21swapper/320:31:563
14556993712,4cyclictest7750irq/56-dwc_otg19:30:543
14556993712,4cyclictest7750irq/56-dwc_otg19:30:543
14555993719,9cyclictest4888-21sh20:36:472
1455699368,13cyclictest7950irq/56-dwc_otg_16:56:113
14556993624,4cyclictest7950irq/56-dwc_otg_20:21:043
1455699361,4cyclictest22267-21ssh19:41:103
14556993614,5cyclictest25427-21diskmemload19:25:543
1455599369,6cyclictest7850irq/56-dwc_otg_18:55:442
1455599367,21cyclictest7950irq/56-dwc_otg_18:45:442
1455599366,15cyclictest7950irq/56-dwc_otg_20:21:022
1455599362,24cyclictest0-21swapper/219:28:042
1455499369,5cyclictest0-21swapper/120:41:021
1455499367,13cyclictest7950irq/56-dwc_otg_17:16:021
1455699357,15cyclictest0-21swapper/319:20:563
1455699355,20cyclictest13605-21ssh21:25:573
1455699355,15cyclictest7850irq/56-dwc_otg_16:46:013
14556993518,4cyclictest908-21runrttasks21:55:403
14556993515,5cyclictest25427-21diskmemload18:36:053
14556993513,13cyclictest0-21swapper/317:40:573
14556993510,6cyclictest10547-21sed21:21:123
1455599357,23cyclictest16751-21ls20:51:052
1455599354,12cyclictest7850irq/56-dwc_otg_17:46:022
1455599353,16cyclictest7850irq/56-dwc_otg_21:50:552
14555993520,9cyclictest2892-21kworker/u8:321:22:502
1455599351,23cyclictest25550-21cut19:46:012
1455599351,23cyclictest25550-21cut19:46:012
1455599351,16cyclictest7850irq/56-dwc_otg_21:06:082
1455599351,16cyclictest7850irq/56-dwc_otg_21:06:082
1455599351,16cyclictest7850irq/56-dwc_otg_19:31:122
1455599351,16cyclictest7850irq/56-dwc_otg_19:31:122
1455499359,6cyclictest7950irq/56-dwc_otg_19:00:441
1455499356,5cyclictest7750irq/56-dwc_otg21:56:011
1455499355,3cyclictest7750irq/56-dwc_otg17:35:571
1455499355,3cyclictest7750irq/56-dwc_otg17:35:571
14554993512,5cyclictest2466-21sh19:19:061
1455699346,7cyclictest7850irq/56-dwc_otg_20:01:043
1455699346,6cyclictest3199-21/usr/sbin/munin18:41:113
1455699346,5cyclictest7950irq/56-dwc_otg_18:30:453
1455699345,6cyclictest7950irq/56-dwc_otg_21:20:433
14556993423,4cyclictest7950irq/56-dwc_otg_21:43:543
1455599347,6cyclictest7750irq/56-dwc_otg18:46:232
1455599345,20cyclictest27655-21ssh19:10:572
1455599343,14cyclictest7850irq/56-dwc_otg_17:40:452
1455599343,14cyclictest7850irq/56-dwc_otg_17:40:452
1455599341,19cyclictest7950irq/56-dwc_otg_19:51:102
1455499348,4cyclictest7950irq/56-dwc_otg_17:51:121
1455499344,5cyclictest7750irq/56-dwc_otg21:41:011
1455499343,18cyclictest7750irq/56-dwc_otg19:05:571
1455499343,18cyclictest7750irq/56-dwc_otg19:05:571
1455499343,14cyclictest7850irq/56-dwc_otg_18:36:111
1455499342,5cyclictest7850irq/56-dwc_otg_16:55:431
1455499342,18cyclictest11499-21ssh19:29:561
14554993419,4cyclictest21265-21sh20:18:351
14554993416,5cyclictest9918-21acpi20:05:561
1455499341,4cyclictest0-21swapper/118:40:531
1455499341,3cyclictest0-21swapper/116:35:541
1455499341,23cyclictest17533-21latency_hist21:30:441
317402330,4sleep337-21rcuc/319:15:443
1455699338,16cyclictest0-21swapper/321:35:573
14556993322,9cyclictest7950irq/56-dwc_otg_19:00:593
1455699332,19cyclictest0-21swapper/319:47:223
1455699332,19cyclictest0-21swapper/319:47:223
1455699331,7cyclictest37-21rcuc/321:06:113
1455699331,7cyclictest37-21rcuc/321:06:113
1455599339,13cyclictest24525-21ssh19:45:052
1455599338,13cyclictest0-21swapper/220:16:042
1455599332,19cyclictest7950irq/56-dwc_otg_21:11:012
1455599332,16cyclictest0-21swapper/221:57:492
1455599332,11cyclictest25425-21apt18:30:432
14555993312,9cyclictest25427-21diskmemload21:42:282
1455599331,22cyclictest0-21swapper/219:21:242
1455599331,20cyclictest18529-21ls21:31:032
14555993310,11cyclictest0-21swapper/220:30:562
1455499339,10cyclictest0-21swapper/121:08:101
1455499339,10cyclictest0-21swapper/121:08:101
1455499338,15cyclictest0-21swapper/117:45:581
1455499335,20cyclictest61-21kswapd021:45:551
1455499333,11cyclictest7850irq/56-dwc_otg_21:40:451
14554993312,6cyclictest27409-21fschecks_count20:26:011
1455699329,12cyclictest0-21swapper/320:54:083
14556993224,5cyclictest3492-21fschecks_count20:35:593
1455699322,14cyclictest7950irq/56-dwc_otg_17:51:103
14556993217,5cyclictest908-21runrttasks17:10:553
14556993217,5cyclictest908-21runrttasks17:10:553
1455699321,23cyclictest0-21swapper/318:26:013
1455599329,11cyclictest0-21swapper/219:16:072
1455599327,6cyclictest9558-21cron20:05:432
1455599326,5cyclictest7850irq/56-dwc_otg_16:36:102
1455599323,20cyclictest22515-21cut21:36:012
1455599321,27cyclictest15465-21ls18:56:042
1455599321,26cyclictest23721-21ls18:25:452
1455599321,26cyclictest23721-21ls18:25:452
14555993212,4cyclictest11839-21missed_timers17:51:052
14555993210,6cyclictest7950irq/56-dwc_otg_17:23:572
14554993213,4cyclictest6155-21sh18:45:541
324302310,1sleep10-21swapper/120:31:101
2955123116,8sleep10-21swapper/119:50:581
1455699318,6cyclictest13000-21unixbench-2d20:46:123
1455699317,14cyclictest0-21swapper/319:35:503
1455699316,14cyclictest14917-21ssh20:11:033
1455699313,15cyclictest22720-21ntp_kernel_err18:21:053
1455699313,15cyclictest22720-21ntp_kernel_err18:21:053
14556993127,2cyclictest0-21swapper/318:55:583
1455699311,5cyclictest5689-21/usr/sbin/munin17:36:053
1455699311,5cyclictest5689-21/usr/sbin/munin17:36:053
1455699311,4cyclictest7850irq/56-dwc_otg_18:06:123
14556993113,9cyclictest25427-21diskmemload19:09:333
14556993113,9cyclictest25427-21diskmemload19:09:333
1455699311,23cyclictest11105-21missed_timers18:51:043
14556993110,6cyclictest1-21systemd20:57:023
1455599317,12cyclictest0-21swapper/221:00:512
1455599314,6cyclictest7750irq/56-dwc_otg16:51:062
1455599313,14cyclictest7950irq/56-dwc_otg_20:26:112
1455599311,6cyclictest7850irq/56-dwc_otg_21:46:062
1455599311,5cyclictest0-21swapper/216:40:592
14555993110,10cyclictest24483-21grep21:00:442
1455499315,13cyclictest7950irq/56-dwc_otg_19:40:551
1455499311,4cyclictest0-21swapper/116:55:531
1455499311,15cyclictest0-21swapper/117:25:441
1455499311,13cyclictest7750irq/56-dwc_otg21:30:531
287842300,3sleep228780-21awk17:10:442
1455699305,6cyclictest7850irq/56-dwc_otg_17:56:053
1455699303,11cyclictest7950irq/56-dwc_otg_20:26:033
1455699301,21cyclictest0-21swapper/316:41:543
1455599303,16cyclictest0-21swapper/217:13:282
1455599303,16cyclictest0-21swapper/217:13:282
1455599303,14cyclictest7950irq/56-dwc_otg_18:05:542
1455599303,13cyclictest7950irq/56-dwc_otg_20:06:052
1455499308,4cyclictest0-21swapper/119:32:541
1455499308,4cyclictest0-21swapper/119:32:541
1455499304,4cyclictest7850irq/56-dwc_otg_16:31:081
1455499304,4cyclictest7850irq/56-dwc_otg_16:31:081
1455499301,8cyclictest2160-21aten_rbpower_en17:25:581
1455699298,12cyclictest7950irq/56-dwc_otg_17:46:053
1455699296,10cyclictest7950irq/56-dwc_otg_16:35:443
1455699296,10cyclictest7950irq/56-dwc_otg_16:35:443
1455699295,6cyclictest7850irq/56-dwc_otg_17:30:393
1455699294,8cyclictest9550irq/80-mmc017:25:443
1455699294,6cyclictest7950irq/56-dwc_otg_16:50:593
1455699293,5cyclictest7950irq/56-dwc_otg_18:20:443
1455699293,15cyclictest7950irq/56-dwc_otg_17:10:443
1455599296,14cyclictest0-21swapper/217:40:502
1455599294,4cyclictest7750irq/56-dwc_otg18:36:072
1455599293,15cyclictest0-21swapper/217:26:532
1455499294,5cyclictest7750irq/56-dwc_otg18:01:121
134752294,12sleep313563-21/usr/sbin/munin16:26:073
1455699282,18cyclictest0-21swapper/316:39:073
1455599282,13cyclictest7950irq/56-dwc_otg_16:30:532
1455599282,13cyclictest7950irq/56-dwc_otg_16:30:532
14555992811,5cyclictest908-21runrttasks17:01:482
1455499287,6cyclictest18444-21uniq16:41:001
1455499282,12cyclictest7850irq/56-dwc_otg_17:11:071
1455499282,12cyclictest7850irq/56-dwc_otg_17:11:071
1455499282,11cyclictest7950irq/56-dwc_otg_18:10:561
136532286,11sleep10-21swapper/116:26:091
1455599271,6cyclictest7850irq/56-dwc_otg_16:56:062
142592279,12sleep20-21swapper/216:30:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional