You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 11:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat Mar 21, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204821790,4sleep3278999cyclictest00:10:283
312822840,1sleep30-21swapper/323:27:203
278699839,56cyclictest6330-21latency_hist19:20:060
278699822,6cyclictest7950irq/56-dwc_otg_00:30:210
278699795,64cyclictest21322-21latency_hist23:55:080
278699776,14cyclictest0-21swapper/019:25:200
278699767,54cyclictest0-21swapper/019:35:200
278699767,54cyclictest0-21swapper/019:35:200
278699766,54cyclictest17068-21kernelversion23:10:240
2786997611,47cyclictest20433-21grep21:55:350
2786997514,41cyclictest6123-21grep23:35:360
278699746,61cyclictest0-21swapper/021:05:200
278699736,58cyclictest0-21swapper/020:45:190
278699716,52cyclictest9036-21ssh23:40:200
278699716,52cyclictest9036-21ssh23:40:200
2786997111,40cyclictest10747-21tune2fs21:00:250
2786997110,47cyclictest14681-21sed21:10:260
278699709,46cyclictest14717-21ssh23:07:370
2786997011,44cyclictest28465-21tune2fs22:45:240
278699698,12cyclictest8039-21sort22:20:260
278699696,47cyclictest21430-21fschecks_count20:00:240
278699691,53cyclictest0-21swapper/000:20:070
2786996912,41cyclictest32321-21fschecks_time20:30:240
226892690,1sleep30-21swapper/322:38:163
278799686,25cyclictest12294-21ssh23:05:191
2787996837,8cyclictest10411-21apt-get21:00:191
278699688,49cyclictest30742-21ls00:05:340
2786996812,43cyclictest7301-21ntpq19:20:280
278699679,43cyclictest21889-21ssh23:55:260
278699674,54cyclictest0-21swapper/022:26:190
278699674,50cyclictest0-21swapper/000:26:010
278699673,49cyclictest0-21swapper/021:27:550
278699668,48cyclictest0-21swapper/022:53:410
278699664,48cyclictest20575-21sh21:17:130
278699663,52cyclictest10342-21ssh21:44:450
2786996633,14cyclictest25161-21fschecks_count23:20:240
2786996611,12cyclictest31989-21unixbench_singl21:30:340
2786996611,12cyclictest31989-21unixbench_singl21:30:340
2787996420,14cyclictest21960-21/usr/sbin/munin23:55:261
2786996415,33cyclictest10411-21tr00:20:280
278699641,51cyclictest21038-21rm23:15:240
2786996414,36cyclictest1335-21ssh22:12:070
2786996410,40cyclictest149-21systemd-journal00:00:400
124282640,1sleep30-21swapper/323:44:273
124282640,1sleep30-21swapper/323:44:273
278699636,38cyclictest0-21swapper/021:45:160
278699635,45cyclictest23986-21ssh00:36:360
278699631,51cyclictest0-21swapper/021:35:340
2786996311,41cyclictest20717-21taskset19:57:410
272802630,2sleep0101ktimersoftd/022:05:080
302982620,2sleep20-21swapper/222:47:092
2787996228,12cyclictest32198-21ls20:30:221
2787996220,11cyclictest7093-21/usr/sbin/munin19:20:251
278699626,46cyclictest0-21swapper/019:10:180
278699626,43cyclictest29531-21sh22:06:580
278699624,35cyclictest0-21swapper/022:31:030
2786996217,28cyclictest17004-21latency_hist23:50:070
246882620,1sleep10-21swapper/122:00:421
2787996125,23cyclictest32147-21ssh00:07:451
2787996028,13cyclictest27580-21/usr/sbin/munin21:25:311
2787996027,15cyclictest26533-21latency_hist21:25:081
2787996025,16cyclictest17268-21ntpq23:10:271
2787996024,11cyclictest8956-21wc19:25:261
278699605,41cyclictest0-21swapper/023:25:190
278699601,44cyclictest27777-21kworker/u8:023:00:240
2786996011,46cyclictest5838-21ssh22:56:340
2786996011,46cyclictest5838-21ssh22:56:340
2787995933,10cyclictest0-21swapper/121:50:201
2787995925,19cyclictest28361-21df_inode22:45:221
2787995925,17cyclictest28468-21ssh23:24:461
2787995923,13cyclictest1037-21fschecks_count23:30:251
2787995923,11cyclictest12340-21ssh23:44:171
2787995923,11cyclictest12340-21ssh23:44:171
2787995922,21cyclictest19822-21/usr/sbin/munin19:55:271
2787995918,15cyclictest3518-21ssh00:12:081
278699594,38cyclictest917-21snmpd22:36:040
2786995912,25cyclictest15593-21latency21:50:290
2787995838,7cyclictest0-21swapper/100:20:171
2787995827,20cyclictest2529-21ssh22:53:061
2787995827,12cyclictest8855-21ssh21:42:031
2787995824,12cyclictest10412-21ssh22:23:361
2787995823,17cyclictest1520-21ssh22:12:281
2787995821,23cyclictest7950irq/56-dwc_otg_00:17:281
278699588,32cyclictest3319-21/usr/sbin/munin22:15:200
278699584,42cyclictest0-21swapper/020:08:350
278699583,47cyclictest0-21swapper/022:41:080
2787995725,14cyclictest20080-21ssh21:55:321
2787995724,15cyclictest16081-21ssh21:11:421
2787995723,19cyclictest24151-21ssh00:36:541
2787995723,13cyclictest20369-21ssh21:16:521
2787995722,14cyclictest908-21runrttasks22:15:231
278699577,47cyclictest26465-21sed21:25:070
278699577,38cyclictest29922-21chrt20:24:520
278699571,46cyclictest13-21rcuc/000:10:350
2787995626,22cyclictest28441-21ssh00:03:301
2787995624,14cyclictest21650-21cut20:00:271
2787995614,27cyclictest7850irq/56-dwc_otg_22:40:161
278699566,38cyclictest726-21ssh23:30:210
278699563,41cyclictest0-21swapper/020:50:190
2786995613,23cyclictest4417-21seq20:41:100
2787995529,16cyclictest16605-21ssh23:49:321
2787995528,11cyclictest6163-21/usr/sbin/munin23:35:371
2787995524,11cyclictest17781-21sh22:31:531
2787995524,10cyclictest29556-21ssh22:07:001
2787995523,20cyclictest10521-21ssh23:02:311
278699551,44cyclictest0-21swapper/020:40:070
2787995423,8cyclictest14328-21irqstats00:25:261
278799542,22cyclictest7950irq/56-dwc_otg_00:30:201
2787995419,14cyclictest5789-21ssh22:56:281
2787995419,14cyclictest5789-21ssh22:56:281
2787995419,13cyclictest4524-21apt-config19:15:081
278699545,41cyclictest26013-21taskset20:11:160
2786995413,29cyclictest31546-21resolvconf20:27:370
2786995413,29cyclictest31546-21resolvconf20:27:370
278899532,29cyclictest0-21swapper/220:15:182
2787995324,13cyclictest13057-21rm21:47:121
2787995322,14cyclictest21013-21sh23:55:051
2787995319,16cyclictest14676-21ssh22:28:471
2787995319,13cyclictest7750irq/56-dwc_otg23:30:071
278699535,37cyclictest27884-21seq20:17:030
15272530,3sleep30-21swapper/321:34:033
15272530,3sleep30-21swapper/321:34:033
2787995223,9cyclictest23988-21ssh23:19:101
2787995217,11cyclictest7850irq/56-dwc_otg_20:25:211
2787995217,11cyclictest7850irq/56-dwc_otg_20:25:211
278699525,34cyclictest0-21swapper/019:46:530
278999512,6cyclictest7950irq/56-dwc_otg_00:15:183
2787995127,7cyclictest4380-21apt-get19:15:091
2787995125,18cyclictest4403-21ssh21:36:321
2787995118,18cyclictest31751-21kworker/u8:021:31:101
2787995118,18cyclictest31751-21kworker/u8:021:31:101
278699511,42cyclictest0-21swapper/020:57:590
278999507,18cyclictest7750irq/56-dwc_otg23:05:193
2787995016,15cyclictest6519-21seq20:49:521
2787995016,14cyclictest7950irq/56-dwc_otg_20:55:211
278699501,35cyclictest11448-21unixbench_multi19:30:340
2787994923,18cyclictest18910-21sshd19:52:411
2787994915,20cyclictest24186-21chrt20:05:571
2786994922,16cyclictest7850irq/56-dwc_otg_19:40:290
10092490,4sleep231-21ksoftirqd/200:10:072
2787994815,15cyclictest28108-21chrt20:19:461
2787994810,24cyclictest0-21swapper/122:37:331
278899477,25cyclictest0-21swapper/220:00:202
278899474,10cyclictest7750irq/56-dwc_otg22:25:182
278899472,5cyclictest0-21swapper/220:40:192
278799477,26cyclictest0-21swapper/119:45:201
278699471,36cyclictest0-21swapper/019:52:070
23172473,35sleep00-21swapper/019:07:390
188282470,1sleep20-21swapper/222:33:482
278999467,24cyclictest0-21swapper/319:25:203
278899467,25cyclictest0-21swapper/221:00:212
2787994632,11cyclictest14656-21/usr/sbin/munin19:40:301
2787994629,13cyclictest10004-21apt-get19:30:131
2787994622,11cyclictest13597-21chrt19:38:521
2787994622,11cyclictest13597-21chrt19:38:521
278899455,3cyclictest0-21swapper/221:30:172
278899455,3cyclictest0-21swapper/221:30:172
2787994518,11cyclictest4448-21runrttasks20:41:351
2787994516,10cyclictest0-21swapper/120:54:591
278999447,23cyclictest0-21swapper/320:10:193
2787994410,5cyclictest7950irq/56-dwc_otg_20:10:191
278999438,24cyclictest7850irq/56-dwc_otg_22:20:193
278999437,24cyclictest0-21swapper/320:25:203
278999437,24cyclictest0-21swapper/320:25:203
278999437,24cyclictest0-21swapper/320:05:223
278999437,23cyclictest0-21swapper/300:05:203
278999436,22cyclictest0-21swapper/319:50:203
2789994322,12cyclictest7850irq/56-dwc_otg_21:20:233
2789994227,8cyclictest7850irq/56-dwc_otg_23:10:343
2789994213,6cyclictest21076-21latency_hist23:55:073
2788994216,11cyclictest0-21swapper/221:25:072
2787994218,10cyclictest13425-21chrt21:05:591
2787994215,10cyclictest7850irq/56-dwc_otg_20:38:361
278999418,11cyclictest381ktimersoftd/323:45:283
278999417,24cyclictest0-21swapper/320:55:203
278999417,24cyclictest0-21swapper/319:35:203
278999417,24cyclictest0-21swapper/319:35:203
278999416,25cyclictest0-21swapper/319:15:203
2789994116,12cyclictest381ktimersoftd/300:39:423
278899418,8cyclictest7750irq/56-dwc_otg23:20:182
278899418,6cyclictest25854-21grep20:10:342
2789994027,4cyclictest7750irq/56-dwc_otg19:55:253
278899403,8cyclictest16530-21ssh23:10:162
180352400,1sleep30-21swapper/321:53:443
278999392,18cyclictest0-21swapper/322:05:283
2789993919,4cyclictest7750irq/56-dwc_otg22:50:583
278899397,22cyclictest0-21swapper/219:20:122
2788993912,5cyclictest0-21swapper/223:00:222
2787993926,11cyclictest29090-21ntp_states20:20:311
278999389,9cyclictest0-21swapper/323:30:243
278999388,18cyclictest29745-21taskset20:22:373
278999386,26cyclictest7950irq/56-dwc_otg_21:45:163
278999386,17cyclictest0-21swapper/319:45:203
278999386,10cyclictest0-21swapper/321:28:173
278999385,19cyclictest24876-21ssh23:20:203
278999385,13cyclictest0-21swapper/323:55:133
278999383,11cyclictest0-21swapper/322:00:203
2789993820,9cyclictest381ktimersoftd/322:55:523
2789993820,9cyclictest381ktimersoftd/322:55:523
278999381,24cyclictest18647-21cut21:15:233
2789993812,16cyclictest381ktimersoftd/321:44:543
2789993810,14cyclictest381ktimersoftd/319:20:293
278899386,7cyclictest10633-21/usr/sbin/munin00:20:312
2788993817,4cyclictest3050-21sh22:54:022
278899381,13cyclictest15691-21latency_hist19:45:072
278999378,5cyclictest7950irq/56-dwc_otg_21:00:353
278999377,20cyclictest12788-21unixbench-2d22:25:353
2789993717,6cyclictest7750irq/56-dwc_otg00:03:313
2789993713,10cyclictest381ktimersoftd/300:22:563
2789993710,4cyclictest0-21swapper/323:04:033
278899372,9cyclictest31828-21latency_hist20:30:082
278899372,9cyclictest31828-21latency_hist20:30:082
2788993716,7cyclictest12462-21fschecks_time19:35:242
2788993716,7cyclictest12462-21fschecks_time19:35:242
2788993714,7cyclictest4890-21wget23:35:212
278999364,5cyclictest7750irq/56-dwc_otg22:10:203
2789993623,4cyclictest7750irq/56-dwc_otg22:34:213
278999362,19cyclictest7950irq/56-dwc_otg_20:33:093
2789993617,9cyclictest7950irq/56-dwc_otg_21:55:193
2789993617,4cyclictest15887-21sh21:11:213
2789993613,9cyclictest7850irq/56-dwc_otg_22:45:263
2789993613,13cyclictest381ktimersoftd/320:00:283
278999361,24cyclictest7114-21fschecks_time20:50:233
278999361,19cyclictest0-21swapper/300:28:323
278999361,18cyclictest7950irq/56-dwc_otg_22:17:483
278899367,5cyclictest0-21swapper/200:10:312
278899362,6cyclictest0-21swapper/222:10:182
2788993616,4cyclictest4768-21ssh22:55:302
2788993616,4cyclictest4768-21ssh22:55:302
2788993614,11cyclictest0-21swapper/200:32:052
2788993612,13cyclictest23005-21latency_hist22:00:072
278899361,16cyclictest7750irq/56-dwc_otg20:50:232
278899361,10cyclictest21022-21uniq23:15:242
163252360,2sleep20-21swapper/221:12:082
278999353,18cyclictest39-21ksoftirqd/319:33:293
2789993517,8cyclictest381ktimersoftd/323:35:413
278999351,22cyclictest6398-21latency_hist21:40:073
278899357,15cyclictest0-21swapper/219:30:122
278899351,7cyclictest15353-21fschecks_time21:50:262
2788993513,4cyclictest13792-21/usr/sbin/munin00:25:282
278899351,20cyclictest14680-21latency_hist21:50:072
278899351,11cyclictest13755-21cron21:10:072
41002340,2sleep20-21swapper/223:34:242
2789993415,4cyclictest7750irq/56-dwc_otg19:10:233
278999341,24cyclictest18258-21uniq00:30:253
278899347,13cyclictest7950irq/56-dwc_otg_21:35:252
278899345,6cyclictest7750irq/56-dwc_otg22:00:212
278899345,13cyclictest7850irq/56-dwc_otg_21:29:192
278899344,7cyclictest18255-21switchtime23:50:332
278899342,9cyclictest5301-21cut20:45:242
278899342,9cyclictest0-21swapper/222:20:172
278899342,7cyclictest18226-21ntpq19:50:292
278899342,6cyclictest6324-21ssh00:15:312
278899341,9cyclictest0-21swapper/223:40:252
278899341,9cyclictest0-21swapper/223:40:252
278899341,5cyclictest21480-21ssh23:55:182
272712342,4sleep30-21swapper/320:15:303
278999335,5cyclictest37-21rcuc/322:40:123
2789993319,3cyclictest7850irq/56-dwc_otg_23:16:273
2789993316,10cyclictest917-21snmpd20:40:423
278899335,4cyclictest7950irq/56-dwc_otg_23:45:352
278899332,14cyclictest7850irq/56-dwc_otg_00:35:202
278899332,14cyclictest7750irq/56-dwc_otg19:25:202
2788993317,9cyclictest7950irq/56-dwc_otg_19:10:172
278899331,5cyclictest28979-21/usr/sbin/munin23:25:272
2788993313,6cyclictest0-21swapper/223:05:222
278999327,22cyclictest0-21swapper/320:45:193
278999326,14cyclictest0-21swapper/320:35:213
278899326,23cyclictest5038-21grep19:15:242
278899326,14cyclictest7850irq/56-dwc_otg_22:15:192
278899325,7cyclictest19395-21grep19:55:212
278899325,14cyclictest7850irq/56-dwc_otg_22:05:212
278899325,13cyclictest0-21swapper/222:35:282
278899323,5cyclictest7850irq/56-dwc_otg_21:41:312
23372323,22sleep30-21swapper/319:07:543
2789993124,4cyclictest381ktimersoftd/319:40:183
2789993116,5cyclictest12267-21aten_rbpower_po21:05:213
278899318,4cyclictest25207-21unixbench_multi22:40:332
278899311,7cyclictest2360-21wc20:35:342
2788993114,6cyclictest28408-21/usr/sbin/munin20:20:212
2788993113,5cyclictest32446-21taskset20:30:262
278899309,10cyclictest0-21swapper/221:18:582
278899301,5cyclictest7850irq/56-dwc_otg_00:00:322
13442306,15sleep10-21swapper/119:05:261
278899291,17cyclictest16115-21idleruntime19:45:242
278899291,14cyclictest0-21swapper/220:10:072
19792294,9sleep20-21swapper/219:05:352
278899285,12cyclictest7850irq/56-dwc_otg_20:55:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional