You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 18:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu May 07, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2337021650,1sleep30-21swapper/320:56:053
480121340,2sleep14802-21rm21:11:331
20512991027,23cyclictest30216-21ssh19:46:080
176672700,2sleep10-21swapper/120:09:531
2051299698,48cyclictest11961-21fschecks_count21:21:110
2051299693,22cyclictest0-21swapper/018:36:070
2051299693,22cyclictest0-21swapper/018:36:070
2051299684,38cyclictest0-21swapper/019:31:230
20512996835,11cyclictest6136-21awk17:20:550
195872680,1sleep10-21swapper/116:26:211
2051299675,44cyclictest9870-21ssh20:00:550
2051299675,44cyclictest9870-21ssh20:00:550
2051299662,33cyclictest24505-21fschecks_count16:41:120
2051299653,45cyclictest11352-21sort20:41:130
2051299645,32cyclictest21410-21awk18:56:170
2051299642,42cyclictest10606-21unixbench-2d19:21:210
2051299634,37cyclictest26939-21aten_rbpower_en20:21:080
20512996331,17cyclictest0-21swapper/020:06:060
20512996312,47cyclictest101ktimersoftd/019:40:550
96892620,5sleep32051799cyclictest20:00:373
96892620,5sleep32051799cyclictest20:00:373
2051299627,44cyclictest23213-21latency_hist20:55:560
2051299626,45cyclictest3911-21latency_hist21:50:560
2051299622,44cyclictest0-21swapper/021:04:470
142562620,2sleep114238-21missed_timers19:26:171
142562620,2sleep114238-21missed_timers19:26:171
2051299618,26cyclictest13951-21ssh19:26:130
2051299618,26cyclictest13951-21ssh19:26:130
2051299606,43cyclictest0-21swapper/020:50:000
20512996036,13cyclictest101ktimersoftd/016:50:140
20512996036,13cyclictest101ktimersoftd/016:50:140
20512996025,17cyclictest0-21swapper/018:31:170
2051299594,34cyclictest28274-21grep18:21:110
20512995934,10cyclictest7950irq/56-dwc_otg_18:53:340
20512995934,10cyclictest0-21swapper/016:31:060
20512995915,4cyclictest0-21swapper/017:26:070
2051299591,36cyclictest5605-21uniq19:16:120
2051299585,29cyclictest25969-21modprobe18:15:550
2051299582,40cyclictest9-21ksoftirqd/021:16:250
20512995815,3cyclictest0-21swapper/017:00:550
20512995811,44cyclictest101ktimersoftd/017:36:220
2051299574,18cyclictest7750irq/56-dwc_otg17:06:060
2051299571,6cyclictest31706-21rm21:06:070
2051299571,6cyclictest31706-21rm21:06:070
2051299569,27cyclictest12659-21kernelversion18:46:140
2051299569,18cyclictest21241-21grep21:31:240
2051299567,41cyclictest11187-21taskset21:59:270
2051299567,31cyclictest24225-21ssh20:56:200
2051299565,25cyclictest0-21swapper/017:15:560
2051299563,32cyclictest0-21swapper/020:01:160
20512995630,9cyclictest0-21swapper/019:41:180
20512995623,17cyclictest26206-21/usr/sbin/munin18:16:220
2051299561,35cyclictest29616-21idleruntime19:06:120
2051299561,35cyclictest29616-21idleruntime19:06:120
2051299554,47cyclictest101ktimersoftd/020:35:550
20512995536,5cyclictest101ktimersoftd/017:31:140
2051299552,28cyclictest20636-21latency_hist18:00:560
201552552,46sleep00-21swapper/016:29:490
20512995429,13cyclictest0-21swapper/020:29:490
20512995425,4cyclictest3778-21fschecks_time21:11:120
2051299542,36cyclictest0-21swapper/018:10:550
20512995415,17cyclictest25230-21sh20:18:330
2051299541,39cyclictest10792-21latency_hist20:40:560
2051299541,39cyclictest10792-21latency_hist20:40:560
2051299541,35cyclictest3040-21timerwakeupswit19:51:230
2051299541,33cyclictest8899-21grep18:41:210
2051299541,29cyclictest25577-21systemd-journal20:14:110
2051399536,11cyclictest0-21swapper/117:21:061
20512995336,9cyclictest101ktimersoftd/021:28:320
20512995314,36cyclictest101ktimersoftd/018:26:250
2051399524,26cyclictest7850irq/56-dwc_otg_16:30:571
2051699512,34cyclictest7950irq/56-dwc_otg_19:36:082
2051299518,27cyclictest30160-21sh21:42:280
20512995123,7cyclictest24377-21fschecks_count21:36:110
2051299512,31cyclictest2105-21ssh19:11:230
20512995019,5cyclictest7850irq/56-dwc_otg_21:51:120
2051299501,32cyclictest27775-21sh19:04:000
2051799493,8cyclictest7950irq/56-dwc_otg_19:26:073
2051799493,8cyclictest7950irq/56-dwc_otg_19:26:073
2051399493,8cyclictest0-21swapper/118:06:071
20513994925,6cyclictest12588-21if_eth018:46:131
20512994923,4cyclictest28372-21wc16:51:150
2051299492,25cyclictest0-21swapper/017:46:140
20513994823,7cyclictest7750irq/56-dwc_otg20:31:071
2051299482,42cyclictest6678-21grep17:21:110
2051399473,3cyclictest0-21swapper/120:16:081
2051399472,7cyclictest7750irq/56-dwc_otg19:16:061
2051299466,26cyclictest0-21swapper/017:01:090
20512994626,10cyclictest0-21swapper/017:41:580
2051299462,32cyclictest9-21ksoftirqd/016:36:100
20517994516,8cyclictest8231-21/usr/sbin/munin21:56:253
20516994522,5cyclictest13094-21ls18:46:212
20516994521,6cyclictest18342-21ssh20:10:562
2051399453,7cyclictest1969-21rm19:51:091
20513994529,4cyclictest19067-21latency_hist20:50:561
2051299447,24cyclictest0-21swapper/018:01:120
2051299447,24cyclictest0-21swapper/018:01:120
22762430,6sleep22051699cyclictest19:11:252
2051699437,26cyclictest0-21swapper/218:01:082
2051699437,26cyclictest0-21swapper/218:01:082
20516994319,6cyclictest24527-21fschecks_count16:41:132
2051699425,20cyclictest3896-21unixbench_singl17:11:222
2051699421,5cyclictest7950irq/56-dwc_otg_19:31:062
20516994212,11cyclictest7850irq/56-dwc_otg_20:11:232
20513994216,5cyclictest7950irq/56-dwc_otg_21:36:051
135512420,2sleep20-21swapper/220:05:072
326472410,6sleep32051799cyclictest18:31:203
207722410,2sleep10-21swapper/120:51:441
2051799417,24cyclictest0-21swapper/317:11:013
2051699413,4cyclictest7950irq/56-dwc_otg_17:01:082
2051699411,4cyclictest0-21swapper/220:26:222
2051699411,10cyclictest7950irq/56-dwc_otg_21:06:122
2051699411,10cyclictest7950irq/56-dwc_otg_21:06:122
20513994119,5cyclictest7850irq/56-dwc_otg_19:06:051
20513994119,5cyclictest7850irq/56-dwc_otg_19:06:051
20512994115,15cyclictest0-21swapper/017:51:190
2051399401,5cyclictest7750irq/56-dwc_otg17:31:071
288502390,1sleep10-21swapper/118:21:191
2051799391,10cyclictest9429-21ssh19:21:043
20513993921,4cyclictest7862-21fschecks_time21:16:131
20513993917,13cyclictest0-21swapper/117:11:011
2051799384,5cyclictest7850irq/56-dwc_otg_19:36:123
20517993818,5cyclictest5738-21sh20:34:353
20517993813,18cyclictest0-21swapper/320:27:283
2051699384,6cyclictest7950irq/56-dwc_otg_21:16:082
20516993820,4cyclictest26105-21aten_rbpower_en19:41:082
20516993816,5cyclictest23564-21sh21:35:352
2051699381,3cyclictest0-21swapper/221:50:022
2051699381,23cyclictest0-21swapper/216:36:012
2051399387,16cyclictest18566-21ssh20:11:051
20513993818,4cyclictest6945-21/usr/sbin/munin19:56:191
20513993818,4cyclictest6945-21/usr/sbin/munin19:56:191
20513993812,9cyclictest0-21swapper/121:56:211
193992386,22sleep219423-21/usr/sbin/munin16:26:192
2051799378,23cyclictest21690-21ls18:01:213
2051799378,23cyclictest21690-21ls18:01:213
20517993718,6cyclictest15156-21ssh21:25:213
2051699373,11cyclictest0-21swapper/216:30:582
20516993716,4cyclictest2265-21fschecks_time19:51:132
2051699371,25cyclictest29628-21ls19:06:122
2051699371,25cyclictest29628-21ls19:06:122
2051699371,24cyclictest0-21swapper/219:21:062
2051699371,23cyclictest4729-21ls17:16:112
2051699371,22cyclictest25577-21systemd-journal21:55:552
2051399375,6cyclictest7950irq/56-dwc_otg_17:56:061
20513993730,5cyclictest21-21rcuc/121:46:071
2051799364,19cyclictest32389-21ssh21:46:073
20517993629,5cyclictest9225-21ls19:20:563
20517993619,4cyclictest7950irq/56-dwc_otg_20:01:123
2051699368,12cyclictest7850irq/56-dwc_otg_19:15:562
2051699366,4cyclictest7850irq/56-dwc_otg_20:56:102
2051699364,14cyclictest7850irq/56-dwc_otg_19:05:552
2051699362,17cyclictest0-21swapper/220:19:182
20516993617,4cyclictest27974-21latency_hist21:40:552
2051399368,6cyclictest7850irq/56-dwc_otg_20:05:551
2051399368,4cyclictest0-21swapper/121:06:161
2051399368,4cyclictest0-21swapper/121:06:161
2051399366,4cyclictest0-21swapper/118:36:081
2051399366,4cyclictest0-21swapper/118:36:081
2051399365,5cyclictest7950irq/56-dwc_otg_20:56:071
2051399364,4cyclictest7750irq/56-dwc_otg19:01:061
20513993617,4cyclictest22385-21if_wlan019:36:141
20513993615,11cyclictest30488-21ntp_kernel_err18:26:191
20513993614,5cyclictest17961-21cpuspeed19:31:101
2051399361,22cyclictest3241-21latency_hist18:35:561
20513993610,14cyclictest11665-21ssh21:21:051
285542350,1sleep30-21swapper/321:01:203
273952350,6sleep22051699cyclictest18:16:252
2051799356,14cyclictest7950irq/56-dwc_otg_19:31:093
2051799351,22cyclictest0-21swapper/321:53:293
2051699359,4cyclictest7850irq/56-dwc_otg_19:00:552
2051699356,15cyclictest7850irq/56-dwc_otg_21:41:202
2051699355,20cyclictest15839-21ssh21:26:092
2051699355,18cyclictest0-21swapper/220:31:122
2051699353,7cyclictest0-21swapper/221:21:222
2051699352,9cyclictest7950irq/56-dwc_otg_18:36:192
2051699352,9cyclictest7950irq/56-dwc_otg_18:36:192
2051699352,5cyclictest7850irq/56-dwc_otg_20:36:192
2051699352,5cyclictest7850irq/56-dwc_otg_20:36:192
2051699352,22cyclictest0-21swapper/218:55:552
2051699351,4cyclictest7950irq/56-dwc_otg_20:21:132
20516993514,5cyclictest14498-21processes19:26:212
20516993514,5cyclictest14498-21processes19:26:212
2051699351,22cyclictest11758-21/usr/sbin/munin17:36:072
2051399354,5cyclictest16496-21aten_rbpower_po18:51:081
20513993513,7cyclictest25585-21unixbench_singl18:11:221
2051399351,25cyclictest0-21swapper/117:46:081
20513993511,4cyclictest0-21swapper/120:41:391
2051799347,20cyclictest0-21swapper/319:11:093
2051799344,14cyclictest7950irq/56-dwc_otg_17:41:223
2051799343,10cyclictest0-21swapper/316:30:583
20517993428,3cyclictest0-21swapper/318:20:553
20517993415,5cyclictest2441-21kernelversion19:51:153
20517993415,4cyclictest15384-21/usr/sbin/munin17:46:133
20517993410,4cyclictest0-21swapper/321:06:083
20517993410,4cyclictest0-21swapper/321:06:083
2051699346,11cyclictest7950irq/56-dwc_otg_19:46:092
2051699345,6cyclictest9134-21grep18:41:232
2051699341,14cyclictest0-21swapper/218:31:122
2051399349,5cyclictest9292-21sed17:26:241
2051399348,10cyclictest23-21ksoftirqd/121:51:051
2051399343,25cyclictest7950irq/56-dwc_otg_18:56:051
2051399341,3cyclictest7750irq/56-dwc_otg16:36:061
2051799337,16cyclictest0-21swapper/321:26:093
20517993312,12cyclictest0-21swapper/319:03:353
20517993311,4cyclictest0-21swapper/319:10:553
20517993311,4cyclictest0-21swapper/319:10:553
2051699332,15cyclictest0-21swapper/218:11:052
20516993313,12cyclictest15346-21sh20:46:102
2051399339,4cyclictest7950irq/56-dwc_otg_21:41:071
2051399337,5cyclictest26698-21missed_timers19:41:161
2051399337,23cyclictest0-21swapper/116:51:081
2051399331,2cyclictest0-21swapper/117:16:081
2051399331,22cyclictest1735-21missed_timers19:11:181
2051399331,15cyclictest917-21snmpd21:34:581
191962330,2sleep10-21swapper/121:30:091
2051799326,14cyclictest0-21swapper/320:06:013
2051799323,7cyclictest7850irq/56-dwc_otg_17:21:123
2051799322,14cyclictest0-21swapper/321:45:563
20517993218,4cyclictest20422-21if_eth021:31:143
20517993214,4cyclictest8450-21sh18:41:143
2051799321,21cyclictest22706-21cut16:36:113
2051699326,14cyclictest7950irq/56-dwc_otg_17:21:082
2051699321,20cyclictest8161-21/usr/sbin/munin17:26:182
20516993211,4cyclictest0-21swapper/220:51:072
2051399329,5cyclictest0-21swapper/121:04:461
2051399327,22cyclictest0-21swapper/117:51:081
2051399324,14cyclictest7850irq/56-dwc_otg_17:06:211
2051399323,14cyclictest7950irq/56-dwc_otg_17:45:561
2051399321,18cyclictest7750irq/56-dwc_otg19:24:511
20513993211,10cyclictest0-21swapper/120:25:081
2051799318,10cyclictest0-21swapper/319:00:043
2051799315,5cyclictest7202-21fschecks_time20:36:123
2051799315,5cyclictest7202-21fschecks_time20:36:123
2051799314,5cyclictest4846-21unixbench_singl18:36:233
2051799314,5cyclictest4846-21unixbench_singl18:36:233
20517993124,4cyclictest20573-21sh20:51:243
2051799311,9cyclictest12171-21wget18:46:083
2051799311,8cyclictest15390-21grep20:46:113
2051799311,23cyclictest28141-21uniq16:51:123
2051799311,19cyclictest8115-21latency21:16:163
20517993110,6cyclictest18756-21latency_hist17:55:563
2051699313,18cyclictest0-21swapper/220:41:142
20516993123,6cyclictest11097-21grep17:31:222
2051699312,20cyclictest0-21swapper/217:41:202
20516993119,5cyclictest8739-21ssh21:56:162
20516993113,9cyclictest31395-21diskmemload19:58:332
20516993113,9cyclictest31395-21diskmemload19:58:332
2051699311,17cyclictest824-21/usr/sbin/munin17:06:142
2051699311,16cyclictest0-21swapper/221:01:232
2051399319,4cyclictest0-21swapper/120:40:201
2051399319,4cyclictest0-21swapper/120:40:201
2051399319,4cyclictest0-21swapper/118:16:121
2051399317,15cyclictest0-21swapper/117:36:071
2051399316,6cyclictest7950irq/56-dwc_otg_20:26:091
2051399316,6cyclictest7850irq/56-dwc_otg_16:56:101
2051399316,5cyclictest7750irq/56-dwc_otg19:46:171
2051399311,3cyclictest0-21swapper/117:01:141
2051799309,6cyclictest11567-21cat17:35:553
2051799308,5cyclictest22933-21fschecks_time18:06:113
2051799306,15cyclictest0-21swapper/316:41:093
2051799305,11cyclictest0-21swapper/321:12:303
2051799303,4cyclictest39-21ksoftirqd/318:55:553
2051799302,9cyclictest0-21swapper/320:21:053
2051799302,19cyclictest0-21swapper/319:48:433
2051799302,14cyclictest0-21swapper/318:15:553
2051799301,18cyclictest23040-21cut20:16:133
20517993010,6cyclictest7950irq/56-dwc_otg_17:36:103
20517993010,4cyclictest7950irq/56-dwc_otg_20:11:053
2051699304,5cyclictest7850irq/56-dwc_otg_17:58:352
2051699302,21cyclictest0-21swapper/221:14:442
2051699302,11cyclictest0-21swapper/216:47:572
2051699302,11cyclictest0-21swapper/216:47:572
2051699301,20cyclictest29858-21wget18:26:102
2051699301,18cyclictest0-21swapper/216:58:352
2051699301,16cyclictest17189-21/usr/sbin/munin17:51:142
2051699301,13cyclictest0-21swapper/218:06:202
2051399305,6cyclictest0-21swapper/118:41:211
2051399301,13cyclictest7750irq/56-dwc_otg18:01:211
2051399301,13cyclictest7750irq/56-dwc_otg18:01:211
200402302,22sleep30-21swapper/316:28:183
2051799299,11cyclictest0-21swapper/321:39:063
2051799293,16cyclictest12254-21unixbench_singl20:41:243
2051799292,19cyclictest0-21swapper/316:50:313
2051799292,19cyclictest0-21swapper/316:50:313
2051799291,14cyclictest0-21swapper/317:16:133
2051799288,4cyclictest0-21swapper/319:41:203
2051799284,6cyclictest8925-21grep17:26:203
2051799282,6cyclictest30137-21tune2fs18:26:143
2051699282,6cyclictest28634-21processes16:51:192
2051699282,11cyclictest7850irq/56-dwc_otg_18:24:362
2051399283,17cyclictest25318-21unixbench-2d16:41:231
2051399282,4cyclictest0-21swapper/116:46:071
2051399282,4cyclictest0-21swapper/116:46:071
2051799276,7cyclictest7950irq/56-dwc_otg_17:06:183
2051699271,20cyclictest0-21swapper/217:49:392
2051799252,4cyclictest0-21swapper/318:21:153
2051799252,14cyclictest0-21swapper/318:00:303
2051799251,8cyclictest31589-21aten_rbpower_vo17:01:103
2051799232,11cyclictest29689-21taskset16:56:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional