You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 14:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Thu Apr 16, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
80832910,3sleep3381ktimersoftd/319:02:033
129412810,1sleep00-21swapper/018:30:520
81202770,3sleep1175499cyclictest20:21:001
306222700,2sleep1221ktimersoftd/120:47:171
1752997023,11cyclictest13-21rcuc/017:45:570
107642690,1sleep30-21swapper/319:45:393
175299666,50cyclictest0-21swapper/019:50:590
175299653,38cyclictest5533-21grep16:40:570
1752996444,12cyclictest0-21swapper/021:51:110
175299643,16cyclictest27115-21unixbench-2d18:46:120
175299636,42cyclictest0-21swapper/021:25:580
175299631,6cyclictest28268-21grep17:41:120
175699628,13cyclictest25926-21ssh18:45:563
1752996117,15cyclictest21630-21acpi18:40:540
1752996117,15cyclictest21630-21acpi18:40:540
175299601,9cyclictest28256-21unixbench_multi19:26:150
107122601,2sleep00-21swapper/020:23:500
175299595,23cyclictest1668-21kworker/u8:119:20:550
175299591,7cyclictest2631-21grep16:31:080
13222590,50sleep10-21swapper/116:28:491
1752995839,8cyclictest0-21swapper/021:11:020
1752995839,8cyclictest0-21swapper/021:11:020
175299581,28cyclictest0-21swapper/020:05:570
13562583,13sleep00-21swapper/016:29:160
55672560,2sleep00-21swapper/019:38:440
1752995632,6cyclictest7950irq/56-dwc_otg_17:05:450
175299551,10cyclictest0-21swapper/017:10:540
1754995419,13cyclictest7850irq/56-dwc_otg_19:35:571
175299544,28cyclictest8775-21kworker/0:218:38:230
175299542,31cyclictest5265-21ssh20:56:070
175299542,27cyclictest6345-21aten_rbpower_cu19:00:580
175299534,14cyclictest9-21ksoftirqd/020:40:570
175299534,14cyclictest9-21ksoftirqd/020:40:570
175299531,35cyclictest25688-21apt-get22:00:440
175299525,5cyclictest0-21swapper/018:51:050
175299522,29cyclictest22394-21ntpq17:26:070
1752995217,5cyclictest5968-21fschecks_time18:11:020
175299521,5cyclictest10272-21timerwakeupswit18:21:090
175299517,40cyclictest2447-21runrttasks21:31:120
175299512,31cyclictest11390-21wget19:45:580
1752995118,16cyclictest21407-21sh21:16:000
175299511,27cyclictest9558-21wget21:40:590
175499502,19cyclictest0-21swapper/118:30:551
175299507,22cyclictest0-21swapper/020:14:260
175299501,27cyclictest4222-21sh20:16:040
1752995011,20cyclictest15128-21/usr/sbin/munin19:11:020
1752995011,20cyclictest15128-21/usr/sbin/munin19:11:020
7942490,6sleep20-21swapper/216:26:102
1752994926,5cyclictest7950irq/56-dwc_otg_17:40:440
1752994925,5cyclictest7850irq/56-dwc_otg_16:50:440
175299492,32cyclictest101ktimersoftd/017:51:050
175299491,27cyclictest10727-21cut19:06:010
175499487,28cyclictest0-21swapper/116:55:581
175499482,5cyclictest0-21swapper/118:55:561
175499481,19cyclictest3519-21latency_hist16:35:451
175299484,15cyclictest20867-21timerandwakeup17:21:100
175299482,27cyclictest0-21swapper/017:55:490
175299481,7cyclictest14093-21/usr/sbin/munin21:46:020
1752994813,19cyclictest9450irq/48-DMA09:12:250
175299481,14cyclictest12644-21diskmemload20:51:120
175499478,23cyclictest7750irq/56-dwc_otg19:56:141
175299476,29cyclictest29830-21ls20:46:130
1752994723,6cyclictest7950irq/56-dwc_otg_19:40:570
1752994717,7cyclictest25624-21grep21:21:010
175699465,14cyclictest15447-21ssh19:50:583
175699463,27cyclictest0-21swapper/317:35:553
175599467,25cyclictest12894-21ssh21:05:582
175599467,25cyclictest0-21swapper/217:05:572
1752994626,9cyclictest101ktimersoftd/020:05:340
1752994623,2cyclictest0-21swapper/018:56:010
1752994614,18cyclictest16238-21latency_hist17:10:450
1752994614,18cyclictest0-21swapper/019:31:140
175299461,26cyclictest0-21swapper/021:09:290
175299461,26cyclictest0-21swapper/020:35:450
1755994527,5cyclictest23570-21wget17:30:582
1755994526,7cyclictest22434-21ssh18:41:072
1755994526,7cyclictest22434-21ssh18:41:072
175299458,20cyclictest8607-21unixbench_singl18:16:110
175299452,8cyclictest11392-21fschecks_time18:26:020
175299452,23cyclictest0-21swapper/016:56:110
1752994520,3cyclictest0-21swapper/018:01:040
175299451,24cyclictest0-21swapper/019:57:460
175299451,23cyclictest0-21swapper/017:16:110
175299451,23cyclictest0-21swapper/017:16:110
1752994511,16cyclictest10190-21unixbench_multi16:51:120
175699447,23cyclictest0-21swapper/321:50:583
175699445,5cyclictest7950irq/56-dwc_otg_20:25:583
175499448,23cyclictest0-21swapper/117:55:511
175299449,16cyclictest0-21swapper/018:06:050
175299442,23cyclictest0-21swapper/021:36:060
175299442,23cyclictest0-21swapper/021:36:060
1752994418,12cyclictest7750irq/56-dwc_otg19:16:000
73522430,2sleep20-21swapper/221:37:342
73522430,2sleep20-21swapper/221:37:342
242682431,3sleep10-21swapper/117:31:091
175599437,24cyclictest0-21swapper/217:55:582
175499436,23cyclictest0-21swapper/118:10:581
1754994320,5cyclictest21837-21aten_rbpower_vo17:26:001
207682420,3sleep11750-21cyclictest19:17:351
175699427,23cyclictest0-21swapper/317:30:573
175599427,25cyclictest0-21swapper/220:20:592
175599427,25cyclictest0-21swapper/216:40:502
175499427,23cyclictest0-21swapper/117:10:571
175499426,24cyclictest0-21swapper/121:20:571
175499426,24cyclictest0-21swapper/117:05:571
175499423,5cyclictest0-21swapper/119:50:571
175299429,15cyclictest9472-21ssh21:01:070
175299422,25cyclictest0-21swapper/020:36:570
1752994211,14cyclictest0-21swapper/020:25:590
175699417,22cyclictest0-21swapper/320:05:593
175599415,17cyclictest0-21swapper/221:40:582
175299413,19cyclictest0-21swapper/017:31:120
175699408,5cyclictest7950irq/56-dwc_otg_19:05:563
175699405,5cyclictest7750irq/56-dwc_otg18:40:573
175699405,5cyclictest7750irq/56-dwc_otg18:40:573
175599407,23cyclictest14060-21ls18:31:112
1755994018,5cyclictest2701-21sh21:31:392
1755994018,5cyclictest16907-21sh19:51:312
175599401,24cyclictest7950irq/56-dwc_otg_21:11:072
175599401,24cyclictest7950irq/56-dwc_otg_21:11:072
1754994012,7cyclictest0-21swapper/121:10:591
1754994012,7cyclictest0-21swapper/121:10:591
166592400,1sleep30-21swapper/321:49:233
148152400,2sleep10-21swapper/119:50:161
175699398,6cyclictest7950irq/56-dwc_otg_20:40:573
175699398,6cyclictest7950irq/56-dwc_otg_20:40:573
175699397,19cyclictest0-21swapper/321:15:583
175699397,18cyclictest7950irq/56-dwc_otg_20:00:583
1755993924,7cyclictest12644-21diskmemload20:16:032
1755993920,4cyclictest8451-21latency_hist21:00:442
175499391,24cyclictest32473-21apt-config17:55:451
175499391,19cyclictest0-21swapper/120:10:291
1756993812,12cyclictest7950irq/56-dwc_otg_21:35:553
1756993812,12cyclictest7950irq/56-dwc_otg_21:35:553
175599386,23cyclictest0-21swapper/219:55:592
175499381,20cyclictest23353-21ssh19:21:031
249372371,32sleep10-21swapper/121:59:241
175699375,20cyclictest7950irq/56-dwc_otg_19:19:053
175599372,4cyclictest0-21swapper/217:41:122
175599372,14cyclictest2033-21ssh18:55:562
175599371,5cyclictest16270-21ssh20:30:572
175499377,17cyclictest0-21swapper/121:05:581
175499372,25cyclictest7950irq/56-dwc_otg_18:20:551
1754993718,5cyclictest8156-21sh21:39:011
1754993718,5cyclictest8156-21sh21:39:011
1754993714,7cyclictest27601-21sh20:44:201
1754993714,7cyclictest27601-21sh20:44:201
175499371,29cyclictest7850irq/56-dwc_otg_19:41:411
1756993613,7cyclictest39-21ksoftirqd/319:21:053
1755993612,11cyclictest0-21swapper/221:56:042
175499363,15cyclictest7750irq/56-dwc_otg20:51:011
1754993627,6cyclictest221ktimersoftd/118:45:561
175499361,7cyclictest2941650ssh21:25:551
175499361,23cyclictest5345-21gpgv16:40:451
175499361,21cyclictest12695-21ls21:05:451
223312350,1sleep30-21swapper/321:56:033
175699357,25cyclictest0-21swapper/317:40:573
175699351,17cyclictest7950irq/56-dwc_otg_19:45:513
175599355,21cyclictest25321-21ssh21:20:572
175599351,3cyclictest7750irq/56-dwc_otg19:05:592
1755993513,4cyclictest908-21runrttasks16:52:102
175599351,16cyclictest0-21swapper/219:04:362
1755993510,12cyclictest0-21swapper/219:25:502
175499358,7cyclictest7750irq/56-dwc_otg21:45:571
175499355,14cyclictest7850irq/56-dwc_otg_17:45:561
175499354,14cyclictest7950irq/56-dwc_otg_20:15:591
175499353,7cyclictest22043-21perl18:41:021
175499353,7cyclictest22043-21perl18:41:021
175499353,29cyclictest18236-21wget17:15:591
175499353,29cyclictest18236-21wget17:15:591
56302340,2sleep30-21swapper/320:17:073
299322340,5sleep2175599cyclictest17:46:102
228502341,4sleep20-21swapper/221:16:542
175699349,12cyclictest0-21swapper/319:34:113
175699346,4cyclictest7950irq/56-dwc_otg_19:28:063
175699342,6cyclictest7950irq/56-dwc_otg_20:36:063
175699342,17cyclictest7850irq/56-dwc_otg_16:46:003
175699341,22cyclictest12478-21latency_hist17:00:453
1756993410,10cyclictest0-21swapper/321:12:263
1756993410,10cyclictest0-21swapper/321:12:263
1755993424,7cyclictest5438-21latency_hist18:10:442
175599341,8cyclictest31497-21fschecks_time19:31:022
1755993418,5cyclictest27958-21/usr/sbin/munin20:06:072
1755993414,10cyclictest5146-21kworker/u8:221:00:452
1755993412,6cyclictest0-21swapper/218:51:012
175599341,24cyclictest11240-21ls18:25:592
1755993410,5cyclictest7850irq/56-dwc_otg_19:21:122
175499346,19cyclictest17294-21ssh18:35:501
175499342,16cyclictest2501-21ssh19:35:441
175499342,15cyclictest0-21swapper/118:51:001
175499341,6cyclictest7750irq/56-dwc_otg19:11:001
175499341,6cyclictest7750irq/56-dwc_otg19:11:001
175499341,5cyclictest7750irq/56-dwc_otg20:15:441
1754993412,4cyclictest0-21swapper/121:41:201
175499341,19cyclictest22434-21unixbench_singl21:16:131
175499341,18cyclictest13027-21fschecks_time17:01:001
1754993411,6cyclictest10664-21latency_hist16:55:451
138992340,2sleep339-21ksoftirqd/321:06:113
175699338,11cyclictest7950irq/56-dwc_otg_18:51:013
175699334,14cyclictest7950irq/56-dwc_otg_20:46:363
175699332,17cyclictest0-21swapper/318:01:093
175699332,12cyclictest0-21swapper/316:41:213
1756993317,9cyclictest0-21swapper/316:36:123
1756993313,4cyclictest0-21swapper/318:56:513
175699331,26cyclictest4029-21ls18:05:593
175599338,15cyclictest0-21swapper/218:35:582
175599335,13cyclictest0-21swapper/220:26:002
175599334,11cyclictest7850irq/56-dwc_otg_20:01:072
175599332,5cyclictest14345-21latency_hist17:05:442
175499332,6cyclictest7950irq/56-dwc_otg_20:31:021
1754993317,5cyclictest20770-21fschecks_time20:36:021
1754993317,5cyclictest1101-21/usr/sbin/munin21:31:021
175499331,5cyclictest6304-21ssh19:00:561
1754993315,4cyclictest27197-21sh20:04:571
175499331,18cyclictest4942-21fschecks_time20:56:031
175499330,24cyclictest0-21swapper/120:26:121
175699328,12cyclictest7950irq/56-dwc_otg_18:31:053
175699327,4cyclictest7950irq/56-dwc_otg_17:16:053
175699327,4cyclictest7950irq/56-dwc_otg_17:16:053
175699327,22cyclictest0-21swapper/321:25:583
175699325,5cyclictest7750irq/56-dwc_otg19:35:593
175699324,7cyclictest37-21rcuc/320:30:493
1756993226,4cyclictest39-21ksoftirqd/318:21:033
175599325,14cyclictest7850irq/56-dwc_otg_16:31:112
175599324,6cyclictest7750irq/56-dwc_otg18:00:562
175599322,7cyclictest25495-21ssh20:41:112
175599322,7cyclictest25495-21ssh20:41:112
175599322,22cyclictest0-21swapper/218:45:482
175599322,14cyclictest7850irq/56-dwc_otg_17:16:092
175599322,14cyclictest7850irq/56-dwc_otg_17:16:092
175599322,11cyclictest10428-21sed18:21:102
175599321,6cyclictest22396-21ntp_kernel_err17:26:072
175599321,28cyclictest7950irq/56-dwc_otg_18:15:572
175499328,13cyclictest0-21swapper/121:53:121
175499324,11cyclictest7750irq/56-dwc_otg19:05:571
175499321,8cyclictest27860-21/usr/sbin/munin17:41:071
175499321,3cyclictest7850irq/56-dwc_otg_17:21:091
175699317,6cyclictest7850irq/56-dwc_otg_20:25:443
175699316,6cyclictest7950irq/56-dwc_otg_17:01:103
175699315,4cyclictest7950irq/56-dwc_otg_20:56:143
175699315,12cyclictest0-21swapper/321:01:423
175699314,14cyclictest7950irq/56-dwc_otg_20:51:013
1756993124,5cyclictest25532-21ssh21:20:593
1756993118,10cyclictest0-21swapper/318:35:503
175699311,21cyclictest9362-21uniq16:51:003
1756993111,4cyclictest0-21swapper/319:56:543
175599317,5cyclictest7850irq/56-dwc_otg_19:11:022
175599317,5cyclictest7850irq/56-dwc_otg_19:11:022
175599317,15cyclictest0-21swapper/218:10:582
175599315,14cyclictest0-21swapper/219:45:572
175599313,6cyclictest7950irq/56-dwc_otg_19:35:552
175599313,14cyclictest0-21swapper/221:49:032
175599313,11cyclictest7850irq/56-dwc_otg_16:55:592
1755993116,8cyclictest17724-21/usr/sbin/munin21:50:552
175599311,4cyclictest0-21swapper/220:51:052
1755993113,5cyclictest908-21runrttasks21:25:582
1754993111,9cyclictest7950irq/56-dwc_otg_19:26:141
175699308,4cyclictest7850irq/56-dwc_otg_18:25:583
175699308,12cyclictest0-21swapper/321:41:093
175699305,5cyclictest7950irq/56-dwc_otg_21:31:043
1756993021,6cyclictest21764-21aten_rbpower_cu17:25:593
175699302,12cyclictest7950irq/56-dwc_otg_17:25:443
175699301,5cyclictest7750irq/56-dwc_otg17:46:013
1756993014,4cyclictest14753-21fschecks_count17:06:003
175699301,19cyclictest0-21swapper/318:12:353
1756993011,4cyclictest0-21swapper/319:11:433
1756993011,4cyclictest0-21swapper/319:11:433
175599306,14cyclictest0-21swapper/217:10:572
175599305,18cyclictest7850irq/56-dwc_otg_20:12:202
175599301,7cyclictest29928-21grep20:46:132
175499308,5cyclictest7950irq/56-dwc_otg_18:30:441
175499307,13cyclictest0-21swapper/116:40:561
175499306,10cyclictest7950irq/56-dwc_otg_18:05:441
1754993017,4cyclictest7950irq/56-dwc_otg_17:35:591
175699294,10cyclictest7950irq/56-dwc_otg_17:11:093
1756992922,5cyclictest1374-21grep17:56:133
175599299,4cyclictest0-21swapper/219:44:322
175599297,9cyclictest7950irq/56-dwc_otg_17:36:012
175599296,14cyclictest0-21swapper/219:15:572
175599295,15cyclictest7850irq/56-dwc_otg_17:21:122
175599293,18cyclictest7750irq/56-dwc_otg16:46:042
175699286,4cyclictest7950irq/56-dwc_otg_18:15:583
175699281,7cyclictest32459-21fschecks_time20:11:023
175599287,3cyclictest0-21swapper/220:39:222
175599282,4cyclictest7750irq/56-dwc_otg17:51:052
175599281,11cyclictest7750irq/56-dwc_otg16:40:442
175499282,6cyclictest5002-21unixbench_singl18:06:121
175699275,11cyclictest7950irq/56-dwc_otg_16:31:043
175699271,18cyclictest0-21swapper/317:50:593
175499277,4cyclictest8283-21sensors_temp18:16:091
13542274,14sleep30-21swapper/316:29:153
175499264,4cyclictest7584-21fschecks_count16:46:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional