You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 02:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Aug 07, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2236321970,5sleep10-21swapper/119:05:281
231869911542,46cyclictest22262-21fschecks_count00:15:210
23188991114,78cyclictest0-21swapper/200:15:212
23186991024,15cyclictest29892-21latency_hist21:45:010
23186991024,15cyclictest29892-21latency_hist21:45:010
231869910214,56cyclictest0-21swapper/021:05:140
231869910127,55cyclictest22983-21sed23:35:300
231869910027,60cyclictest9996-21tr22:40:260
23186999926,50cyclictest25491-21sed19:15:210
2318899984,67cyclictest7950irq/56-dwc_otg_23:55:222
23186999823,20cyclictest0-21swapper/023:55:240
23186999731,48cyclictest16307-21fschecks_count20:20:170
23186999531,48cyclictest12047-21ssh21:22:020
23186999424,53cyclictest1355550ssh23:25:020
23186999422,62cyclictest0-21swapper/023:40:460
23186999225,48cyclictest30536-21fschecks_count00:25:190
23186999144,22cyclictest7750irq/56-dwc_otg23:50:160
23186999144,22cyclictest7750irq/56-dwc_otg23:50:160
119242910,2sleep00-21swapper/022:02:210
23186999024,50cyclictest18345-21rm23:30:220
23186999024,48cyclictest3571-21ssh22:32:240
23186999024,48cyclictest3571-21ssh22:32:240
23186999020,59cyclictest0-21swapper/022:10:550
23186999020,59cyclictest0-21swapper/022:10:550
2318699893,54cyclictest29989-21gdbus21:45:030
23186998933,31cyclictest14912-21/usr/sbin/munin22:05:300
23186998926,59cyclictest31713-21unixbench_multi21:00:290
80222880,6sleep12318799cyclictest21:57:311
238962880,2sleep20-21swapper/223:37:042
23186998826,44cyclictest18258-21ssh21:30:170
2318899874,77cyclictest29904-21ls21:45:022
2318899874,77cyclictest29904-21ls21:45:022
23186998724,43cyclictest867-21http22:30:010
23186998723,47cyclictest0-21swapper/021:27:260
23186998526,44cyclictest28479-21ssh23:03:590
23186998524,48cyclictest0-21swapper/019:20:280
23186998518,48cyclictest13266-21http23:25:010
23186998430,33cyclictest7834-21ssh21:16:380
23186998421,54cyclictest0-21swapper/019:30:460
23186998419,35cyclictest0-21swapper/020:05:120
23186998419,35cyclictest0-21swapper/020:05:120
23186998326,9cyclictest0-21swapper/000:00:140
23186998325,44cyclictest0-21swapper/019:28:130
23186998325,44cyclictest0-21swapper/019:28:130
23186998324,42cyclictest20083-21ssh00:12:120
23186998323,43cyclictest0-21swapper/021:50:260
23186998323,43cyclictest0-21swapper/000:35:530
23186998319,44cyclictest0-21swapper/019:10:190
23186998231,33cyclictest7708-21ssh22:37:430
23186998226,36cyclictest2858-21seq19:41:040
23186998219,47cyclictest5042-21sh21:14:000
23186998219,47cyclictest0-21swapper/022:56:270
23186998217,45cyclictest22638-21awk22:15:290
23186998123,42cyclictest28920-21ssh22:24:190
23186998118,49cyclictest15742-21runrttasks22:47:530
23186998020,51cyclictest0-21swapper/021:38:260
23186998020,51cyclictest0-21swapper/021:38:260
23186998014,57cyclictest28625-21runrttasks00:22:440
23186998014,57cyclictest28625-21runrttasks00:22:440
23186997928,48cyclictest15049-21ps20:15:250
23186997915,45cyclictest0-21swapper/000:05:200
23186997721,47cyclictest0-21swapper/021:55:290
23186997721,42cyclictest0-21swapper/020:55:230
23186997718,41cyclictest0-21swapper/020:30:000
2318699766,60cyclictest0-21swapper/019:45:140
23186997621,46cyclictest26660-21modprobe20:50:010
85752750,2sleep20-21swapper/222:39:252
2318899755,65cyclictest0-21swapper/221:45:042
23186997525,37cyclictest2526-21cat00:30:230
23186997520,41cyclictest376-21ssh23:48:360
23186997518,40cyclictest0-21swapper/023:11:540
23186997432,8cyclictest12442-21apt-get20:10:110
23186997421,44cyclictest0-21swapper/022:53:320
23186997415,55cyclictest30438-21ssh23:05:280
23188997323,10cyclictest7750irq/56-dwc_otg22:20:142
23186997324,35cyclictest8562-21taskset19:59:430
2318999722,49cyclictest6219-21ls23:55:253
23186997124,12cyclictest28669-21latency_hist20:55:010
23186997121,39cyclictest0-21swapper/023:19:470
23186997119,39cyclictest0-21swapper/020:36:280
2318899692,10cyclictest32699-21fschecks_time21:05:182
23186996921,35cyclictest0-21swapper/020:02:140
23186996819,29cyclictest0-21swapper/019:54:050
2318799672,34cyclictest9992-21ssh21:20:161
23186996728,11cyclictest28580-21sed20:55:010
23186996715,36cyclictest23311-21aten_rbpower_cu20:40:130
23186996625,37cyclictest1039-21taskset19:36:170
23186996618,26cyclictest0-21swapper/020:30:460
2318899658,5cyclictest0-21swapper/222:15:302
2318799656,41cyclictest7850irq/56-dwc_otg_21:45:011
2318799656,41cyclictest7850irq/56-dwc_otg_21:45:011
127692640,2sleep30-21swapper/322:44:503
2318899631,13cyclictest7750irq/56-dwc_otg22:27:132
48602620,2sleep22318899cyclictest23:14:492
2318899621,6cyclictest9647-21/usr/sbin/munin22:00:202
23187996215,5cyclictest0-21swapper/121:50:311
2318899617,8cyclictest7950irq/56-dwc_otg_23:30:292
2318899612,4cyclictest0-21swapper/221:33:182
2318899612,45cyclictest0-21swapper/200:15:002
2318899612,13cyclictest7950irq/56-dwc_otg_22:30:152
2318899612,13cyclictest7950irq/56-dwc_otg_22:30:152
2318899611,3cyclictest7850irq/56-dwc_otg_21:30:012
2318899607,8cyclictest7950irq/56-dwc_otg_21:22:132
2318899605,5cyclictest7750irq/56-dwc_otg00:05:232
23188996016,41cyclictest12055-21sh22:43:272
2318899599,5cyclictest0-21swapper/221:37:572
2318899599,5cyclictest0-21swapper/221:37:572
2318899595,9cyclictest7950irq/56-dwc_otg_22:11:062
2318899595,9cyclictest7950irq/56-dwc_otg_22:11:062
2318899581,8cyclictest7850irq/56-dwc_otg_20:15:172
2318899581,7cyclictest24319-21grep19:10:282
2318899581,4cyclictest7950irq/56-dwc_otg_22:59:172
2318899581,3cyclictest0-21swapper/223:25:142
23187995810,19cyclictest6154-21wc23:55:221
2318899579,4cyclictest0-21swapper/221:12:282
2318899578,9cyclictest7950irq/56-dwc_otg_23:40:552
2318899576,6cyclictest31462-21grep00:25:312
2318899573,13cyclictest7950irq/56-dwc_otg_23:17:402
23188995717,37cyclictest13634-21/usr/sbin/munin22:05:262
2318799572,8cyclictest7950irq/56-dwc_otg_21:25:121
250262560,2sleep30-21swapper/319:15:153
2318899566,4cyclictest7750irq/56-dwc_otg23:50:142
2318899566,4cyclictest7750irq/56-dwc_otg23:50:142
2318899564,4cyclictest7850irq/56-dwc_otg_21:52:242
2318899562,4cyclictest0-21swapper/223:20:172
23187995614,20cyclictest22361-21fschecks_time00:15:221
2318899556,39cyclictest0-21swapper/221:16:262
2318899551,13cyclictest0-21swapper/222:45:492
2318799555,9cyclictest13720-21ssh00:05:131
2318999547,17cyclictest7950irq/56-dwc_otg_21:45:013
2318999547,17cyclictest7950irq/56-dwc_otg_21:45:013
2318899545,4cyclictest7750irq/56-dwc_otg20:55:142
2318899542,40cyclictest0-21swapper/220:10:182
2318999535,16cyclictest7950irq/56-dwc_otg_21:45:033
2318999535,10cyclictest0-21swapper/300:15:223
2318899531,9cyclictest7750irq/56-dwc_otg00:20:172
2318899531,9cyclictest7750irq/56-dwc_otg00:20:172
23188995316,34cyclictest25324-21sh23:00:102
2318899531,3cyclictest0-21swapper/220:44:442
2318799539,5cyclictest7850irq/56-dwc_otg_23:35:121
23187995314,9cyclictest7850irq/56-dwc_otg_21:45:021
2318899525,4cyclictest7950irq/56-dwc_otg_21:55:172
2318899523,8cyclictest7950irq/56-dwc_otg_00:38:402
2318899523,4cyclictest0-21swapper/200:32:582
2318899522,13cyclictest7850irq/56-dwc_otg_19:40:122
23188995218,31cyclictest11824-21users20:05:322
23188995218,31cyclictest11824-21users20:05:322
2318899521,3cyclictest0-21swapper/220:20:262
2318799528,26cyclictest0-21swapper/122:55:141
23187995228,6cyclictest29650-21rm22:25:141
2318999516,32cyclictest0-21swapper/323:40:023
2318899518,29cyclictest0-21swapper/222:50:072
2318899516,9cyclictest7950irq/56-dwc_otg_00:04:302
23188995112,4cyclictest7585-21date19:55:222
23188995110,4cyclictest0-21swapper/220:45:242
2318799519,29cyclictest0-21swapper/123:05:071
2318799518,28cyclictest0-21swapper/120:40:131
2318999505,12cyclictest7850irq/56-dwc_otg_23:25:143
2318899506,9cyclictest7950irq/56-dwc_otg_20:27:492
2318899502,7cyclictest4125-21ntp_states19:45:252
2318899501,6cyclictest0-21swapper/223:05:172
23187995014,28cyclictest0-21swapper/119:35:141
2318899491,3cyclictest7950irq/56-dwc_otg_21:01:002
2318899491,3cyclictest0-21swapper/223:45:412
2318799498,30cyclictest0-21swapper/123:00:141
2318799498,26cyclictest0-21swapper/121:15:131
2318799497,27cyclictest0-21swapper/121:30:141
2318799497,27cyclictest0-21swapper/119:15:141
2318899482,6cyclictest25716-21ntp_states19:15:252
2318899481,3cyclictest0-21swapper/220:00:242
2318799486,9cyclictest7950irq/56-dwc_otg_23:15:161
267162470,1sleep30-21swapper/323:00:373
2318899472,5cyclictest0-21swapper/220:33:342
23188994712,4cyclictest6574-21taskset19:52:262
2318799478,28cyclictest0-21swapper/122:30:081
2318799478,28cyclictest0-21swapper/122:30:081
2318799476,28cyclictest7750irq/56-dwc_otg23:25:281
2318799471,8cyclictest7750irq/56-dwc_otg22:10:131
2318799471,8cyclictest7750irq/56-dwc_otg22:10:131
2318899462,3cyclictest7950irq/56-dwc_otg_20:50:102
2318899462,3cyclictest7950irq/56-dwc_otg_20:50:102
23187994617,6cyclictest10973-21unixbench_singl22:00:311
227092462,36sleep00-21swapper/019:07:180
2318999457,25cyclictest0-21swapper/319:25:143
2318999457,25cyclictest0-21swapper/319:25:143
2318999451,30cyclictest9508-21ls22:00:023
2318899457,25cyclictest0-21swapper/219:30:142
2318799451,6cyclictest0-21swapper/122:49:251
2318999442,26cyclictest7950irq/56-dwc_otg_22:20:253
2318899443,11cyclictest0-21swapper/220:40:012
2318799443,8cyclictest0-21swapper/119:25:131
2318799443,8cyclictest0-21swapper/119:25:131
2318799441,9cyclictest0-21swapper/120:05:151
2318799441,9cyclictest0-21swapper/120:05:151
2318999437,24cyclictest0-21swapper/319:35:143
2318999435,33cyclictest29415-21ssh23:05:073
2318999435,29cyclictest0-21swapper/322:01:193
2318899431,11cyclictest0-21swapper/219:23:292
2318799436,25cyclictest0-21swapper/100:20:201
2318799436,25cyclictest0-21swapper/100:20:201
281912420,6sleep32318999cyclictest23:42:373
23189994220,4cyclictest30276-21latency_hist21:00:013
2318799429,14cyclictest7950irq/56-dwc_otg_23:50:001
2318799426,6cyclictest9471-21ssh22:40:191
2318799423,7cyclictest0-21swapper/123:50:171
2318799423,7cyclictest0-21swapper/123:50:171
2318799421,19cyclictest9564-21wget23:20:171
106432422,34sleep30-21swapper/320:05:153
106432422,34sleep30-21swapper/320:05:153
244282411,3sleep30-21swapper/322:18:203
2318999417,23cyclictest0-21swapper/321:15:133
2318999416,27cyclictest18047-21ssh21:30:143
23189994129,4cyclictest7950irq/56-dwc_otg_22:30:023
23189994129,4cyclictest7950irq/56-dwc_otg_22:30:023
23189994127,9cyclictest7351-21fschecks_time19:55:193
2318999411,24cyclictest23552-21uniq20:40:173
2318799413,20cyclictest13580-21latency_hist00:05:011
23187994118,3cyclictest0-21swapper/100:10:121
194132410,2sleep30-21swapper/322:52:033
23189994021,5cyclictest1049-21latency_hist22:30:023
23189994014,10cyclictest1544-21diskmemload21:54:203
2318799408,21cyclictest5905-21memory22:35:221
23187994020,5cyclictest1180-21runrttasks23:34:461
23187994012,8cyclictest17211-21sh22:50:081
10242400,1sleep30-21swapper/300:29:033
23189993915,15cyclictest5941-21missed_timers22:35:233
2318799392,5cyclictest7850irq/56-dwc_otg_00:25:131
23187993917,6cyclictest4599-21uptime19:45:301
23187993912,6cyclictest27404-21seq22:21:221
61062380,10sleep339-21ksoftirqd/323:15:273
2318999388,26cyclictest13990-21ssh21:25:073
2318999386,21cyclictest7950irq/56-dwc_otg_23:45:273
2318999381,26cyclictest24162-21ls19:10:263
2318799388,9cyclictest7950irq/56-dwc_otg_22:10:001
2318799388,5cyclictest7750irq/56-dwc_otg22:15:241
2318799382,21cyclictest23-21ksoftirqd/121:35:221
2318799382,21cyclictest23-21ksoftirqd/121:35:221
23187993818,10cyclictest1180-21runrttasks19:32:031
23187993811,4cyclictest0-21swapper/123:10:321
174752380,5sleep317474-21wc22:10:013
2318999375,22cyclictest17849-21ssh00:10:133
23189993720,4cyclictest2592-21sh00:30:243
2318799377,9cyclictest7850irq/56-dwc_otg_19:55:281
2318799376,15cyclictest7950irq/56-dwc_otg_20:55:281
23187993718,4cyclictest2723-21sh00:30:271
23187993713,5cyclictest0-21swapper/121:12:201
2318999368,26cyclictest0-21swapper/322:50:013
2318999367,15cyclictest0-21swapper/321:00:063
2318999363,15cyclictest0-21swapper/321:35:133
2318999363,15cyclictest0-21swapper/321:35:133
2318999361,5cyclictest1191-21snmpd21:12:573
23189993615,15cyclictest7738-21ssh00:36:003
2318999361,22cyclictest13375-21latency_hist00:05:013
2318799361,9cyclictest27453-21ssh23:41:131
23187993611,14cyclictest0-21swapper/100:37:391
23187993610,11cyclictest7950irq/56-dwc_otg_20:10:301
219182360,5sleep20-21swapper/219:05:212
2318999358,12cyclictest7950irq/56-dwc_otg_21:10:013
2318999352,19cyclictest0-21swapper/319:23:513
23189993513,6cyclictest13625-21latency_hist21:25:013
23189993511,13cyclictest18432-21/usr/sbin/munin23:30:243
2318899351,5cyclictest0-21swapper/219:36:002
23187993513,4cyclictest23051-21apt20:40:011
269062341,4sleep326903-21apt-get20:50:113
269062341,4sleep326903-21apt-get20:50:113
2318999347,16cyclictest0-21swapper/322:55:143
23189993412,4cyclictest1180-21runrttasks20:49:433
2318999341,23cyclictest15831-21latency_hist20:20:013
2318999341,23cyclictest14129-21idleruntime00:05:203
23189993411,9cyclictest0-21swapper/322:10:313
23189993411,9cyclictest0-21swapper/322:10:313
2318899344,14cyclictest7950irq/56-dwc_otg_19:25:152
2318899344,14cyclictest7950irq/56-dwc_otg_19:25:152
2318799349,12cyclictest0-21swapper/120:25:221
2318799348,4cyclictest7850irq/56-dwc_otg_20:50:011
23187993415,5cyclictest1180-21runrttasks20:20:231
2318799341,24cyclictest19935-21cut20:30:181
2318799341,20cyclictest0-21swapper/119:20:151
2318999339,11cyclictest0-21swapper/323:20:183
2318999332,4cyclictest0-21swapper/323:50:233
2318999332,4cyclictest0-21swapper/323:50:233
2318999331,22cyclictest26759-21missed_timers00:20:243
2318999331,22cyclictest26759-21missed_timers00:20:243
2318799337,16cyclictest0-21swapper/121:00:061
2318799332,22cyclictest5554-21cut19:50:181
2318799331,18cyclictest7750irq/56-dwc_otg21:10:011
2318799327,7cyclictest24131-21timerandwakeup19:10:261
23187993211,5cyclictest1964-21idleruntime19:40:191
2318999317,18cyclictest0-21swapper/320:25:523
23189993115,6cyclictest3893-21kernelversion19:45:203
2318999302,7cyclictest7950irq/56-dwc_otg_23:13:373
2318999301,6cyclictest9450irq/48-DMA08:56:173
2318999301,21cyclictest0-21swapper/320:10:393
2318999301,16cyclictest0-21swapper/320:35:523
2318999296,4cyclictest0-21swapper/320:03:203
2318799298,5cyclictest14878-21date20:15:221
2318799294,8cyclictest7950irq/56-dwc_otg_20:00:311
2318999286,11cyclictest0-21swapper/319:30:193
2318999281,7cyclictest7850irq/56-dwc_otg_19:50:183
23189992814,4cyclictest1180-21runrttasks20:31:563
2318999281,18cyclictest0-21swapper/320:21:403
2318799286,5cyclictest27405-21latency20:50:181
2318799286,5cyclictest27405-21latency20:50:181
213542273,15sleep30-21swapper/319:05:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional