You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 12:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Jul 07, 2026 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28335991299,16cyclictest32690-21proc_pri22:20:380
28335991299,16cyclictest32690-21proc_pri22:20:380
28335991282,18cyclictest8050irq/56-dwc_otg_21:20:410
28335991265,90cyclictest0-21swapper/023:05:380
28337991192,96cyclictest15430-21ssh21:20:391
283359911717,8cyclictest7950irq/56-dwc_otg_23:00:360
283379911616,80cyclictest30102-21/usr/sbin/munin00:15:421
28337991135,75cyclictest0-21swapper/122:20:391
28337991135,75cyclictest0-21swapper/122:20:391
283359911379,15cyclictest7850irq/56-dwc_otg23:35:320
28335991112,45cyclictest15496-21cpuspeed22:00:300
28335991112,45cyclictest15496-21cpuspeed22:00:300
28335991099,78cyclictest9-21ksoftirqd/021:25:420
283359910610,67cyclictest0-21swapper/000:20:390
28337991038,66cyclictest8050irq/56-dwc_otg_00:00:301
28335999640,31cyclictest0-21swapper/022:55:160
2833599944,70cyclictest0-21swapper/000:15:420
2833799935,81cyclictest28402-21runrttasks22:55:151
291372920,2sleep10-21swapper/123:35:141
28337999211,55cyclictest8050irq/56-dwc_otg_23:35:321
2833599925,18cyclictest7950irq/56-dwc_otg_00:00:270
2833799913,8cyclictest0-21swapper/121:40:461
2833599902,36cyclictest32519-21rm21:40:460
2833799895,8cyclictest0-21swapper/123:05:391
2833799894,75cyclictest22880-21/usr/sbin/munin21:30:421
28335998850,10cyclictest7850irq/56-dwc_otg22:10:240
214002880,3sleep021382-21ssh22:45:400
2833599874,45cyclictest0-21swapper/022:05:350
2833599868,21cyclictest0-21swapper/022:55:160
30852840,3sleep13088-21grep19:30:411
2833799834,72cyclictest15416-21aten_rbpower_po22:00:291
2833799834,72cyclictest15416-21aten_rbpower_po22:00:291
2833799833,58cyclictest0-21swapper/122:05:351
2833799833,15cyclictest979-21/usr/sbin/munin23:00:351
2833799825,63cyclictest7950irq/56-dwc_otg_21:25:441
2833599814,59cyclictest465-21fschecks_time19:25:320
28335998129,9cyclictest5713-21diskmemload23:20:270
2833599812,35cyclictest0-21swapper/021:30:420
2833599811,25cyclictest22575-21grep00:05:370
2833999793,52cyclictest0-21swapper/323:35:313
2833799793,9cyclictest23537-21apt-get22:10:241
28335997926,11cyclictest9-21ksoftirqd/023:15:280
2833599791,44cyclictest0-21swapper/023:45:250
38162780,4sleep03815-21aten_rbpower_cu19:35:270
28335997825,14cyclictest0-21swapper/021:05:250
2833799776,64cyclictest0-21swapper/100:20:381
2833599761,24cyclictest0-21swapper/020:20:150
28335997522,4cyclictest3122-21head19:30:410
258742750,2sleep20-21swapper/222:12:332
2833799749,60cyclictest0-21swapper/122:55:161
2833799741,62cyclictest0-21swapper/119:45:251
2833599741,61cyclictest10450-21grep19:50:410
2833899736,30cyclictest0-21swapper/220:00:272
28338997318,24cyclictest8050irq/56-dwc_otg_21:25:422
28338997112,38cyclictest301ktimersoftd/222:20:382
28338997112,38cyclictest301ktimersoftd/222:20:382
2833599712,19cyclictest0-21swapper/000:15:140
28335997117,5cyclictest110750ssh23:40:250
28335997110,19cyclictest5713-21diskmemload22:28:390
2833599702,7cyclictest11073-21ssh23:12:390
2833599702,25cyclictest14837-21unixbench_singl23:55:430
2833899695,16cyclictest0-21swapper/200:15:412
28335996913,16cyclictest0-21swapper/020:45:320
124422690,2sleep20-21swapper/221:17:052
2833599683,32cyclictest0-21swapper/019:45:190
2833599672,31cyclictest0-21swapper/020:28:590
2833599672,31cyclictest0-21swapper/020:28:590
28335996622,5cyclictest7850irq/56-dwc_otg21:50:240
2833599652,48cyclictest11570-21grep21:15:410
2833799647,4cyclictest0-21swapper/123:27:411
28339996311,30cyclictest8050irq/56-dwc_otg_22:20:393
28339996311,30cyclictest8050irq/56-dwc_otg_22:20:393
28338996230,16cyclictest8050irq/56-dwc_otg_21:20:412
2833799625,2cyclictest0-21swapper/123:15:281
2833599621,34cyclictest28923-21latency_hist19:15:150
2833999615,40cyclictest0-21swapper/323:00:353
28338996011,11cyclictest29-21rcuc/223:05:402
2833799606,13cyclictest7950irq/56-dwc_otg_00:36:251
2833799606,11cyclictest21030-21ls22:45:351
2833799605,12cyclictest9864-21ssh21:53:291
28335996023,7cyclictest0-21swapper/019:20:260
28335996023,7cyclictest0-21swapper/019:20:260
2833599601,16cyclictest0-21swapper/000:30:260
2833999593,12cyclictest15386-21cut22:00:283
2833999593,12cyclictest15386-21cut22:00:283
2833899594,39cyclictest15431-21chrt22:00:292
2833899594,39cyclictest15431-21chrt22:00:292
2833799591,8cyclictest22574-21ntpq00:05:371
2833799591,50cyclictest0-21swapper/123:57:351
2833599597,22cyclictest0-21swapper/020:25:150
2833599597,22cyclictest0-21swapper/020:25:150
93502580,7sleep37950irq/56-dwc_otg_22:31:103
2833999589,28cyclictest8050irq/56-dwc_otg_23:05:403
28339995813,18cyclictest0-21swapper/322:55:173
2833899583,23cyclictest28437-21grep22:55:152
2833799587,4cyclictest21208-21unixbench_multi20:20:431
2833799587,4cyclictest21208-21unixbench_multi20:20:431
2833799586,7cyclictest0-21swapper/119:15:231
2833799583,44cyclictest7850irq/56-dwc_otg23:42:171
2833799581,5cyclictest10316-21/usr/sbin/munin21:15:271
28335995815,28cyclictest14973-21rm22:00:140
28339995720,11cyclictest23813-21rt-features21:30:423
28339995713,18cyclictest7950irq/56-dwc_otg_21:20:393
2833899572,12cyclictest20183-21ntpq22:05:372
2833799577,41cyclictest12490-21users21:55:451
2833799574,43cyclictest7342-21apt-get22:30:171
2833799573,4cyclictest31471-21ssh22:20:111
28337995714,40cyclictest7281-21sh00:25:451
2833799571,36cyclictest0-21swapper/121:35:301
2833999565,35cyclictest0-21swapper/300:15:383
2833899566,34cyclictest0-21swapper/223:00:372
2833799565,14cyclictest7950irq/56-dwc_otg_00:15:141
2833799563,15cyclictest7850irq/56-dwc_otg23:50:031
2833799562,9cyclictest7950irq/56-dwc_otg_22:42:221
2833799561,16cyclictest7950irq/56-dwc_otg_20:20:141
28337995611,5cyclictest13713-21expr20:00:391
2833599561,8cyclictest2919-21wget21:45:270
2833599561,34cyclictest11699-21latency_hist22:35:140
2833999557,27cyclictest20181-21ssh22:05:373
2833999553,10cyclictest17922-21df_inode00:00:293
28338995544,6cyclictest0-21swapper/223:35:332
2833799554,8cyclictest31744-21ntp_kernel_pll_20:50:371
2833799551,6cyclictest0-21swapper/120:14:591
2833799551,16cyclictest0-21swapper/121:45:171
2833799551,10cyclictest7950irq/56-dwc_otg_20:45:291
2833599551,30cyclictest28074-21taskset22:15:320
14412550,2sleep30-21swapper/321:43:153
2833999542,10cyclictest8050irq/56-dwc_otg_20:10:253
2833899543,21cyclictest0-21swapper/200:00:292
28338995410,9cyclictest7950irq/56-dwc_otg_00:10:252
2833799543,7cyclictest11948-21aten_rbpower_cu22:35:271
2833799542,4cyclictest9649-21ssh23:50:301
2833799542,4cyclictest9649-21ssh23:50:301
2833799542,12cyclictest0-21swapper/122:26:201
2833799541,6cyclictest10426-21head19:50:411
28335995420,22cyclictest27555-21wget20:40:280
2833599541,8cyclictest27266-21grep20:40:140
2833599541,38cyclictest10036-21latency_hist21:15:140
2833599541,30cyclictest9451-21sh23:50:270
2833599541,30cyclictest9451-21sh23:50:270
278532543,45sleep00-21swapper/019:07:330
158772540,28sleep02833599cyclictest20:05:420
28339995310,8cyclictest7950irq/56-dwc_otg_00:20:393
2833799533,9cyclictest28996-21cron20:45:131
2833799533,43cyclictest0-21swapper/123:22:311
2833799532,42cyclictest0-21swapper/120:37:031
2833799531,6cyclictest8251-21apt-get23:10:171
2833799531,3cyclictest9990-21modprobe21:15:131
2833799531,15cyclictest0-21swapper/100:31:121
2833599532,28cyclictest0-21swapper/020:15:140
28335995322,7cyclictest2353-21wget21:00:290
2833599531,5cyclictest17369-21head22:40:430
2833799523,9cyclictest0-21swapper/119:22:301
2833799523,9cyclictest0-21swapper/119:22:301
2833799523,13cyclictest2730-21latency21:00:341
2833799511,4cyclictest4133-21/usr/sbin/munin21:05:271
2833799511,11cyclictest23234-21rm20:25:491
2833799511,11cyclictest23234-21rm20:25:491
28335995123,11cyclictest7850irq/56-dwc_otg23:30:130
28335995118,6cyclictest0-21swapper/019:40:270
28335995118,5cyclictest7850irq/56-dwc_otg22:40:130
2833599511,47cyclictest26331-21ssh23:30:410
2833599511,29cyclictest27901-21proc_pri21:35:410
2833999504,28cyclictest0-21swapper/321:25:433
2833999503,12cyclictest23537-21apt-get22:10:233
2833899503,9cyclictest0-21swapper/200:20:382
28338995015,17cyclictest20163-21kworker/2:121:30:412
28338995011,21cyclictest32511-21ssh21:40:452
2833799501,9cyclictest727-21runrttasks20:59:071
2833599503,28cyclictest7163-21ssh00:25:440
2833999497,25cyclictest8050irq/56-dwc_otg_22:55:153
28338994914,11cyclictest7950irq/56-dwc_otg_22:55:162
2833799493,15cyclictest0-21swapper/119:42:101
2833999484,8cyclictest8050irq/56-dwc_otg_20:55:283
2833799481,15cyclictest0-21swapper/119:56:281
2833799481,14cyclictest0-21swapper/119:13:531
2833599481,28cyclictest14060-21grep20:00:420
2833599481,27cyclictest31633-21missed_timers20:50:360
2833999477,26cyclictest0-21swapper/323:15:283
2833899476,28cyclictest0-21swapper/220:40:272
2833799472,2cyclictest0-21swapper/120:05:351
2833599474,22cyclictest9-21ksoftirqd/019:55:440
2833999463,10cyclictest0-21swapper/321:45:253
2833799461,5cyclictest3771-21/usr/sbin/munin19:35:401
2833799461,10cyclictest0-21swapper/120:31:261
28335994618,13cyclictest24972-21grep20:30:410
28335994614,20cyclictest0-21swapper/019:19:160
2833999451,7cyclictest8050irq/56-dwc_otg_23:50:253
2833999451,7cyclictest8050irq/56-dwc_otg_23:50:253
2833899452,10cyclictest0-21swapper/220:35:272
28335994524,5cyclictest679-21cut20:55:320
2833899447,24cyclictest0-21swapper/221:05:192
2833899445,12cyclictest7950irq/56-dwc_otg_23:30:272
2833999431,8cyclictest8050irq/56-dwc_otg_23:10:253
2833899437,24cyclictest0-21swapper/220:30:272
2833899433,4cyclictest0-21swapper/220:15:232
2833899423,9cyclictest0-21swapper/222:25:292
2833799421,14cyclictest0-21swapper/119:29:151
2833899417,22cyclictest0-21swapper/221:45:272
2833899415,3cyclictest0-21swapper/222:40:272
2833599412,21cyclictest0-21swapper/000:36:470
166522410,3sleep2301ktimersoftd/223:58:582
2833999405,23cyclictest5978-21ssh00:25:293
218702400,3sleep20-21swapper/222:46:072
136382400,1sleep30-21swapper/300:34:473
2833999399,5cyclictest7907-21sh23:48:093
203462390,6sleep32833999cyclictest22:45:273
2833999386,22cyclictest0-21swapper/319:45:253
2833899387,22cyclictest0-21swapper/220:25:192
2833899387,22cyclictest0-21swapper/220:25:192
2833899387,21cyclictest0-21swapper/200:30:282
28338993823,4cyclictest7950irq/56-dwc_otg_22:35:312
2833999371,6cyclictest7366-21fschecks_time21:50:313
2833999363,4cyclictest7850irq/56-dwc_otg21:15:163
2833899366,16cyclictest0-21swapper/222:15:272
2833899361,3cyclictest7850irq/56-dwc_otg19:30:252
2833899361,18cyclictest11-21rcu_preempt21:15:142
2833999357,24cyclictest1120-21ssh23:40:273
2833999352,5cyclictest8050irq/56-dwc_otg_20:20:283
2833999352,5cyclictest8050irq/56-dwc_otg_20:20:283
2833999352,23cyclictest22811-21switchtime00:05:413
2833899356,20cyclictest27077-21ssh21:35:302
2833899355,7cyclictest7850irq/56-dwc_otg23:55:142
2833899355,7cyclictest7850irq/56-dwc_otg23:55:142
2833899355,13cyclictest7950irq/56-dwc_otg_23:20:292
2833899354,16cyclictest29-21rcuc/221:00:252
2833899353,22cyclictest0-21swapper/222:30:312
2833899351,25cyclictest8872-21ls23:10:282
2833899351,25cyclictest32117-21grep20:50:422
2833899351,24cyclictest427-21cut19:25:322
2833999343,5cyclictest8050irq/56-dwc_otg_19:25:283
2833999341,21cyclictest13834-21cut23:55:323
2833899343,4cyclictest0-21swapper/200:10:142
2833999336,5cyclictest7850irq/56-dwc_otg23:25:303
2833999335,21cyclictest17021-21ssh23:20:283
2833999333,15cyclictest8050irq/56-dwc_otg_19:55:133
2833999333,15cyclictest8050irq/56-dwc_otg_19:35:273
28339993310,4cyclictest0-21swapper/322:42:203
28339993310,10cyclictest0-21swapper/323:35:073
2833899335,5cyclictest7850irq/56-dwc_otg23:30:132
2833899333,27cyclictest8050irq/56-dwc_otg_21:50:242
2833899331,21cyclictest5813-21ls23:45:332
2833899331,20cyclictest11362-21cut19:55:322
2833899331,17cyclictest8050irq/56-dwc_otg_19:25:152
2833899331,17cyclictest8050irq/56-dwc_otg_19:25:152
28338993310,4cyclictest0-21swapper/223:15:382
47682320,22sleep28050irq/56-dwc_otg_19:35:412
2833999327,4cyclictest7950irq/56-dwc_otg_21:55:333
2833999326,5cyclictest7850irq/56-dwc_otg20:30:283
2833999326,22cyclictest0-21swapper/321:05:193
28339993225,5cyclictest2296-21aten_rbpower_en21:00:283
2833999322,13cyclictest8050irq/56-dwc_otg_21:35:383
2833899329,5cyclictest148182chrt20:05:282
2833899325,16cyclictest8050irq/56-dwc_otg_19:40:412
28338993211,14cyclictest0-21swapper/223:44:322
2833999312,8cyclictest0-21swapper/322:20:143
2833999312,19cyclictest0-21swapper/320:36:553
28339993110,3cyclictest0-21swapper/321:14:503
28338993123,6cyclictest301ktimersoftd/200:25:262
2833899311,26cyclictest8050irq/56-dwc_otg_21:55:272
277062313,17sleep10-21swapper/119:05:401
311642300,4sleep32833999cyclictest20:50:293
2833999309,13cyclictest0-21swapper/300:13:053
2833999307,4cyclictest7950irq/56-dwc_otg_22:25:383
2833999306,14cyclictest0-21swapper/320:45:283
2833999301,25cyclictest15225-21ls20:05:343
2833999301,13cyclictest8050irq/56-dwc_otg_20:40:283
2833999301,11cyclictest2321150kworker/u8:322:35:413
2833899306,4cyclictest8050irq/56-dwc_otg_20:20:322
2833899306,4cyclictest8050irq/56-dwc_otg_20:20:322
2833899304,4cyclictest7950irq/56-dwc_otg_19:15:312
2833999293,5cyclictest13978-21unixbench_multi20:00:423
2833999292,3cyclictest7850irq/56-dwc_otg00:39:543
2833999291,16cyclictest8050irq/56-dwc_otg_19:33:403
2833899297,12cyclictest0-21swapper/200:35:292
2833899294,12cyclictest8050irq/56-dwc_otg_19:11:192
28338992913,5cyclictest29566-21cut20:45:322
2833899291,13cyclictest7950irq/56-dwc_otg_20:57:462
2833999282,20cyclictest0-21swapper/319:43:573
2833999282,15cyclictest0-21swapper/320:29:193
2833999282,15cyclictest0-21swapper/320:29:193
2833999281,8cyclictest31015-21/usr/sbin/munin19:20:313
2833999281,8cyclictest31015-21/usr/sbin/munin19:20:313
2833999281,5cyclictest8050irq/56-dwc_otg_19:55:323
2833899287,11cyclictest8050irq/56-dwc_otg_19:49:162
2833899285,12cyclictest8050irq/56-dwc_otg_19:51:212
274262287,10sleep20-21swapper/219:05:372
2833999272,17cyclictest0-21swapper/320:17:563
2833899273,5cyclictest7850irq/56-dwc_otg20:10:282
278012271,19sleep30-21swapper/319:06:553
2833999251,6cyclictest29199-21entropy19:15:293
28339992413,8cyclictest381ktimersoftd/319:15:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional