You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 12:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun May 24, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4072953,12sleep00-21swapper/019:05:340
315912940,5sleep1104699cyclictest00:05:201
268852910,20sleep0104599cyclictest00:00:030
272902860,1sleep00-21swapper/000:39:240
211382790,2sleep30-21swapper/322:35:283
85342750,2sleep30-21swapper/319:30:183
64262740,2sleep30-21swapper/321:40:283
62242730,2sleep00-21swapper/022:55:340
325122720,2sleep10-21swapper/121:33:381
104599721,5cyclictest21023-21apt-config20:05:030
152232700,2sleep30-21swapper/321:13:193
111852660,2sleep20-21swapper/221:45:392
1045996610,24cyclictest0-21swapper/022:35:160
260292650,1sleep20-21swapper/223:58:362
104699654,28cyclictest0-21swapper/123:20:131
288052640,2sleep00-21swapper/023:22:430
1045996420,3cyclictest0-21swapper/021:35:150
1045996415,5cyclictest9550irq/80-mmc023:35:020
40352630,3sleep14038-21unixbench_singl19:15:311
217382630,4sleep321716-21ssh21:20:273
172532630,1sleep00-21swapper/023:09:550
104599637,8cyclictest7950irq/56-dwc_otg_23:25:130
1045996239,11cyclictest0-21swapper/021:40:200
1045996136,5cyclictest6744-21fschecks_time23:35:210
1045996034,5cyclictest7850irq/56-dwc_otg00:15:090
104599601,36cyclictest20844-21acpi21:20:130
1047995946,9cyclictest8050irq/56-dwc_otg_22:00:162
1047995946,9cyclictest8050irq/56-dwc_otg_22:00:162
104599591,44cyclictest22306-21ssh22:00:030
1045995728,21cyclictest31809-21ls20:35:030
104599571,35cyclictest29332-21latency22:45:250
1046995614,24cyclictest0-21swapper/122:15:141
104599566,40cyclictest0-21swapper/021:45:200
104599562,51cyclictest24577-21latency_hist20:15:030
1045995614,23cyclictest13-21rcuc/022:05:180
1045995613,5cyclictest12559-21proc_pri22:25:280
104599561,14cyclictest4467-21unixbench_singl22:15:330
88422550,1sleep30-21swapper/323:37:403
1045995529,13cyclictest8050irq/56-dwc_otg_00:35:030
104599545,26cyclictest0-21swapper/000:05:200
104599544,24cyclictest11948-21latency_hist21:10:030
104599544,24cyclictest11948-21latency_hist21:10:030
1045995433,11cyclictest0-21swapper/023:10:320
1045995425,7cyclictest11440-21ssh00:20:160
104599541,5cyclictest25993-21proc_pri21:25:280
104599541,40cyclictest17213-21grep22:30:320
104599541,38cyclictest26344-21idleruntime-cro20:20:030
104799534,7cyclictest0-21swapper/219:35:152
104699537,26cyclictest0-21swapper/121:50:151
104599539,21cyclictest2910-21cron20:45:020
104599538,42cyclictest101ktimersoftd/020:10:030
104599534,5cyclictest0-21swapper/020:00:030
1045995329,13cyclictest523-21rs:main0
1045995328,5cyclictest4901-21latency_hist20:50:030
1045995325,14cyclictest7950irq/56-dwc_otg_19:15:170
1045995324,5cyclictest0-21swapper/019:40:080
104599532,43cyclictest0-21swapper/022:24:160
104599531,39cyclictest24412-21rm23:17:300
1045995313,3cyclictest0-21swapper/022:10:130
1045995313,3cyclictest0-21swapper/022:10:130
104599531,29cyclictest0-21swapper/000:00:170
104599531,29cyclictest0-21swapper/000:00:170
104599531,22cyclictest14032-21/usr/sbin/munin21:50:280
104599531,20cyclictest17226-21ls19:55:020
104799522,7cyclictest9153-21ssh23:00:152
104599522,47cyclictest101ktimersoftd/022:00:340
104599522,47cyclictest101ktimersoftd/022:00:340
104599522,30cyclictest28138-21sh20:25:020
104599521,30cyclictest4317-21grep00:10:330
104599521,27cyclictest24842-21ssh22:40:200
72102510,2chrt7211-21sh22:57:183
1046995119,15cyclictest8050irq/56-dwc_otg_22:30:321
104599516,42cyclictest0-21swapper/023:00:160
104599511,27cyclictest10557-21wget23:40:170
104699506,29cyclictest28791-21sh22:45:181
104599503,31cyclictest30671-21grep21:30:310
1045995029,3cyclictest0-21swapper/021:20:020
104599502,29cyclictest0-21swapper/020:55:030
104599501,34cyclictest0-21swapper/023:45:220
1047994913,16cyclictest24500-21ls00:35:252
104699491,10cyclictest0-21swapper/100:25:151
104599491,28cyclictest0-21swapper/023:50:200
104599487,7cyclictest0-21swapper/019:25:140
104599486,13cyclictest0-21swapper/000:25:170
104599484,23cyclictest0-21swapper/022:50:310
104599484,23cyclictest0-21swapper/022:50:310
1045994822,6cyclictest13972-21ssh21:11:090
1045994816,16cyclictest1170-21aten_rbpower_vo19:10:160
104599481,33cyclictest0-21swapper/019:35:250
104599481,30cyclictest30078-21wc20:30:030
104599471,29cyclictest8378-21latency_hist21:00:030
104899462,7cyclictest7850irq/56-dwc_otg23:25:133
104799466,25cyclictest0-21swapper/222:15:172
199922451,3sleep20-21swapper/223:50:322
104899457,23cyclictest0-21swapper/319:55:173
104799457,24cyclictest0-21swapper/223:05:162
104799457,24cyclictest0-21swapper/220:05:162
104799456,25cyclictest0-21swapper/220:35:152
104699457,23cyclictest0-21swapper/123:35:161
104699457,23cyclictest0-21swapper/120:30:151
104799445,36cyclictest8050irq/56-dwc_otg_22:35:162
1047994418,14cyclictest0-21swapper/221:00:152
104599441,30cyclictest0-21swapper/019:45:290
104599441,30cyclictest0-21swapper/019:45:290
104699437,6cyclictest6378-21ls19:25:031
104699437,6cyclictest6378-21ls19:25:031
104699437,25cyclictest13432-21ssh23:05:161
1046994325,3cyclictest0-21swapper/119:55:291
104599431,27cyclictest0-21swapper/019:35:020
283782420,2sleep30-21swapper/322:05:533
104899423,7cyclictest8050irq/56-dwc_otg_23:15:173
104799427,24cyclictest0-21swapper/200:15:082
104799422,23cyclictest30484-21ssh23:25:142
104699421,25cyclictest0-21swapper/121:05:141
104699421,25cyclictest0-21swapper/121:05:141
104599421,30cyclictest0-21swapper/021:00:130
104599421,23cyclictest4711-21runrttasks19:20:130
104599421,23cyclictest4711-21runrttasks19:20:130
304512410,2sleep10-21swapper/100:04:071
304512410,2sleep10-21swapper/100:04:071
104899417,24cyclictest0-21swapper/320:55:153
104699418,22cyclictest0-21swapper/121:00:151
104699417,21cyclictest0-21swapper/120:10:151
1046994121,9cyclictest0-21swapper/100:35:191
104699411,7cyclictest8050irq/56-dwc_otg_22:25:141
128642400,2sleep10-21swapper/100:20:451
104899407,12cyclictest3046-21rm00:10:173
1048994010,3cyclictest0-21swapper/321:59:253
104799409,10cyclictest0-21swapper/221:35:152
104799405,9cyclictest7950irq/56-dwc_otg_21:05:162
104799405,9cyclictest7950irq/56-dwc_otg_21:05:162
1047994017,9cyclictest30691-21ssh22:09:552
14532390,2sleep10-21swapper/119:10:201
104899399,13cyclictest39-21ksoftirqd/322:45:163
104899398,4cyclictest0-21swapper/300:07:033
1048993911,16cyclictest0-21swapper/323:40:163
104799396,24cyclictest0-21swapper/219:25:162
104699392,15cyclictest7950irq/56-dwc_otg_23:50:281
104699391,8cyclictest0-21swapper/100:15:171
104899387,18cyclictest11236-21ls21:05:303
104899387,18cyclictest11236-21ls21:05:303
104899381,24cyclictest24697-21ls20:15:043
1047993817,7cyclictest17831-21unixbench_multi21:15:322
104699381,8cyclictest14759-21grep23:45:191
104699381,16cyclictest451950ssh22:55:031
104699381,16cyclictest451950ssh22:55:031
104899375,23cyclictest9166-21ssh23:00:163
104899374,27cyclictest8050irq/56-dwc_otg_22:15:153
104899371,4cyclictest7850irq/56-dwc_otg00:25:143
104799377,21cyclictest7850irq/56-dwc_otg00:03:062
104799377,21cyclictest7850irq/56-dwc_otg00:03:062
104799371,24cyclictest18722-21ls21:55:212
1048993625,7cyclictest381ktimersoftd/300:30:153
104899362,4cyclictest8050irq/56-dwc_otg_23:32:553
104899362,22cyclictest4111-21timerandwakeup20:45:303
1048993618,5cyclictest11015-21ssh22:24:273
104899361,25cyclictest0-21swapper/322:10:443
104899361,25cyclictest0-21swapper/322:10:443
1047993615,4cyclictest15775-21latency_hist22:30:032
1047993614,8cyclictest13481-21ssh00:21:532
104799361,15cyclictest7950irq/56-dwc_otg_22:40:192
104699367,25cyclictest0-21swapper/120:05:161
104699366,6cyclictest7850irq/56-dwc_otg21:55:251
104699363,20cyclictest7950irq/56-dwc_otg_22:10:131
104699363,20cyclictest7950irq/56-dwc_otg_22:10:131
1046993615,10cyclictest11949-21diskmemload22:01:561
1046993615,10cyclictest11949-21diskmemload22:01:561
3132350,2sleep20-21swapper/200:05:492
229732350,1sleep10-21swapper/100:34:061
104899355,11cyclictest7950irq/56-dwc_otg_22:40:253
104799355,20cyclictest29412-21ssh21:30:152
104799354,14cyclictest8050irq/56-dwc_otg_20:05:032
1047993510,4cyclictest0-21swapper/221:10:262
1047993510,4cyclictest0-21swapper/200:10:192
104699358,4cyclictest30552-21aten_rbpower_en23:25:161
1046993512,4cyclictest0-21swapper/122:35:201
1046993512,3cyclictest0-21swapper/123:00:251
104699351,15cyclictest7950irq/56-dwc_otg_00:15:021
1048993412,6cyclictest30654-21grep21:30:303
104899341,23cyclictest25779-21missed_timers21:25:253
104799348,19cyclictest8133-21sh22:20:252
104799345,15cyclictest8050irq/56-dwc_otg_19:50:242
104799342,8cyclictest31233-21ssh22:10:152
104799342,8cyclictest31233-21ssh22:10:152
104799342,8cyclictest0-21swapper/221:40:152
104799341,8cyclictest6197-21latency_hist23:35:032
104699349,14cyclictest0-21swapper/123:12:231
104699349,11cyclictest20837-21/usr/sbin/munin21:20:211
104699348,4cyclictest13590-21idleruntime-cro19:45:031
104699347,15cyclictest0-21swapper/120:50:161
104699346,22cyclictest0-21swapper/119:25:161
104699345,20cyclictest26875-21ssh22:05:171
104699345,14cyclictest8050irq/56-dwc_otg_22:20:171
104699344,16cyclictest8050irq/56-dwc_otg_23:15:331
104699344,15cyclictest8050irq/56-dwc_otg_20:45:301
104699342,15cyclictest0-21swapper/123:30:211
104699341,7cyclictest7850irq/56-dwc_otg20:20:321
104899339,10cyclictest0-21swapper/300:15:083
104899336,17cyclictest0-21swapper/323:08:163
104899333,7cyclictest0-21swapper/321:45:173
104899332,4cyclictest0-21swapper/300:20:323
1048993322,7cyclictest16286-21uniq22:30:203
104899331,3cyclictest8050irq/56-dwc_otg_19:40:163
1048993313,7cyclictest11949-21diskmemload00:38:523
104899331,21cyclictest979-21ls22:50:213
104899331,21cyclictest979-21ls22:50:213
104799337,9cyclictest8050irq/56-dwc_otg_23:10:292
104799337,14cyclictest0-21swapper/220:10:152
104799337,11cyclictest7850irq/56-dwc_otg00:25:262
104799334,9cyclictest8050irq/56-dwc_otg_21:25:082
104799332,5cyclictest7850irq/56-dwc_otg20:50:022
104799332,22cyclictest0-21swapper/200:32:432
1047993321,6cyclictest17217-21grep22:30:322
1047993314,5cyclictest7950irq/56-dwc_otg_20:30:212
104799331,20cyclictest7950irq/56-dwc_otg_21:21:352
1047993310,4cyclictest0-21swapper/223:40:282
104699339,4cyclictest0-21swapper/122:41:481
104699337,11cyclictest7850irq/56-dwc_otg21:10:221
104699335,13cyclictest8050irq/56-dwc_otg_20:40:161
104899326,7cyclictest17971-21ssh23:10:193
1048993224,5cyclictest26334-21cut23:20:163
104899321,4cyclictest8050irq/56-dwc_otg_19:45:183
104899321,4cyclictest8050irq/56-dwc_otg_19:45:183
1048993214,9cyclictest1133-21runrttasks19:22:273
1048993214,9cyclictest1133-21runrttasks19:22:273
1048993214,5cyclictest381ktimersoftd/323:50:573
1048993212,7cyclictest0-21swapper/300:01:243
1048993212,7cyclictest0-21swapper/300:01:243
104799323,13cyclictest8050irq/56-dwc_otg_20:55:032
104799322,16cyclictest0-21swapper/220:25:212
104799321,3cyclictest7950irq/56-dwc_otg_19:30:292
104699327,14cyclictest8050irq/56-dwc_otg_22:55:251
104699325,13cyclictest8050irq/56-dwc_otg_20:35:301
104699325,11cyclictest8050irq/56-dwc_otg_19:40:031
104699324,3cyclictest0-21swapper/121:40:311
1046993219,4cyclictest16552-21/usr/sbin/munin21:15:331
1046993213,7cyclictest15398-21ls19:50:031
1046993213,7cyclictest15398-21ls19:50:031
1046993210,5cyclictest0-21swapper/121:25:281
1046993210,11cyclictest12853-21sh23:42:271
130842310,5sleep2104799cyclictest19:40:322
104899318,10cyclictest0-21swapper/322:00:253
104899318,10cyclictest0-21swapper/322:00:253
104899317,21cyclictest0-21swapper/320:30:153
104899313,16cyclictest0-21swapper/320:52:533
1048993124,5cyclictest8080-21munin-plugin-st19:30:023
104899312,17cyclictest0-21swapper/323:58:593
104899311,5cyclictest26475-21grep20:20:043
104899311,3cyclictest8050irq/56-dwc_otg_20:35:273
1048993113,4cyclictest11949-21diskmemload21:35:063
104799317,6cyclictest7950irq/56-dwc_otg_23:45:192
104799315,11cyclictest7950irq/56-dwc_otg_22:45:262
104799314,11cyclictest7950irq/56-dwc_otg_19:10:162
104799313,21cyclictest7950irq/56-dwc_otg_23:20:302
104799313,19cyclictest7950irq/56-dwc_otg_21:50:472
104799312,5cyclictest15396-21apt19:50:032
104799312,5cyclictest15396-21apt19:50:032
104799311,7cyclictest27563-21cat20:20:312
1047993111,12cyclictest22442-21/usr/sbin/munin23:15:232
1047993110,11cyclictest0-21swapper/219:55:232
104699316,15cyclictest0-21swapper/120:25:161
104699311,7cyclictest16792-21head19:50:311
1046993115,10cyclictest3277-21ssh21:36:461
1046993110,5cyclictest0-21swapper/121:45:171
104899306,10cyclictest8050irq/56-dwc_otg_21:50:273
104899301,16cyclictest8050irq/56-dwc_otg_21:15:173
1048993011,4cyclictest0-21swapper/323:45:593
104799307,20cyclictest0-21swapper/220:15:182
104799304,16cyclictest7950irq/56-dwc_otg_22:55:082
104799304,13cyclictest8050irq/56-dwc_otg_20:42:332
104799303,5cyclictest6991-21/usr/sbin/munin23:35:242
104799302,15cyclictest7950irq/56-dwc_otg_22:54:332
104799302,15cyclictest7950irq/56-dwc_otg_22:54:332
104699305,5cyclictest8050irq/56-dwc_otg_19:35:031
104699305,4cyclictest7850irq/56-dwc_otg23:55:281
104699302,4cyclictest0-21swapper/120:55:131
5932294,16sleep20-21swapper/219:07:512
104899297,13cyclictest0-21swapper/319:50:173
104899295,5cyclictest9726-21grep21:00:303
104899295,10cyclictest7950irq/56-dwc_otg_22:25:243
104899294,11cyclictest7950irq/56-dwc_otg_20:00:303
104899293,12cyclictest8050irq/56-dwc_otg_19:10:243
1048992922,5cyclictest28597-21diskstats20:25:193
104899292,15cyclictest0-21swapper/320:24:383
104799295,15cyclictest1165-21snmpd19:21:152
104799295,15cyclictest1165-21snmpd19:21:152
104699293,10cyclictest8050irq/56-dwc_otg_20:15:181
7602284,16sleep10-21swapper/119:09:521
104899284,10cyclictest8050irq/56-dwc_otg_20:05:313
104899281,12cyclictest8050irq/56-dwc_otg_19:15:263
104799283,12cyclictest7950irq/56-dwc_otg_21:00:032
104699283,5cyclictest8050irq/56-dwc_otg_20:00:241
104899271,4cyclictest8050irq/56-dwc_otg_20:40:273
6352262,14sleep30-21swapper/319:08:163
114042261,4sleep30-21swapper/319:36:433
104799262,15cyclictest0-21swapper/219:15:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional