You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Tue Mar 24, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1478121680,5sleep138799cyclictest21:13:191
385991126,77cyclictest0-21swapper/020:20:230
2869021090,4sleep028695-21sh21:30:090
2869021090,4sleep028695-21sh21:30:090
385991087,67cyclictest0-21swapper/022:20:230
3859910228,54cyclictest0-21swapper/021:55:360
385999829,55cyclictest4824-21ls00:15:380
385999827,59cyclictest15662-21grep23:10:350
385999729,56cyclictest31036-21wget23:30:230
385999623,60cyclictest0-21swapper/019:15:330
385999532,50cyclictest0-21swapper/023:05:200
385999523,57cyclictest0-21swapper/023:50:090
385999427,57cyclictest17824-21/usr/sbin/munin19:55:350
385999229,49cyclictest30515-21ls20:30:360
385999225,50cyclictest20254-21aten_rbpower_cu00:35:250
38599917,48cyclictest0-21swapper/022:05:210
385999121,52cyclictest0-21swapper/022:45:310
385999120,60cyclictest0-21swapper/021:47:220
143522910,2sleep20-21swapper/223:49:172
385998933,45cyclictest22968-21ssh22:01:080
385998925,48cyclictest439-21/usr/sbin/munin19:10:160
7652880,5sleep339799cyclictest23:32:233
385998725,42cyclictest24718-21sh23:21:550
385998631,44cyclictest10242-21ssh23:44:310
385998631,44cyclictest10242-21ssh23:44:310
385998629,36cyclictest18378-21ssh21:17:090
385998530,39cyclictest3942-21ssh21:39:070
385998527,45cyclictest31404-21ssh00:10:000
385998523,53cyclictest30480-21sh21:30:510
385998523,46cyclictest908-21runrttasks22:42:440
385998523,46cyclictest908-21runrttasks22:42:440
385998521,39cyclictest0-21swapper/000:12:470
385998521,39cyclictest0-21swapper/000:12:470
385998432,42cyclictest0-21swapper/020:55:200
385998428,44cyclictest5747-21taskset23:38:520
155002840,2sleep10-21swapper/119:50:271
385998327,45cyclictest20751-21ssh23:55:540
385998324,46cyclictest16165-21switchtime23:50:350
385998322,47cyclictest23560-21aten_rbpower_cu00:00:210
385998231,40cyclictest934-21rm22:53:560
385998222,44cyclictest3849-21ssh22:56:400
385998221,48cyclictest0-21swapper/022:31:350
385998127,45cyclictest881-21ssh22:14:570
385998127,45cyclictest881-21ssh22:14:570
385998125,43cyclictest25482-21sh20:17:080
385998123,46cyclictest20198-21sh23:16:150
385998123,42cyclictest0-21swapper/020:25:340
385998122,46cyclictest18460-21ssh00:32:340
385998122,43cyclictest0-21swapper/019:50:330
385998121,50cyclictest0-21swapper/022:17:450
102672810,1sleep10-21swapper/122:25:291
385998028,39cyclictest5471-21ssh21:40:300
385998026,38cyclictest4251-21apt-get20:50:080
385998021,47cyclictest0-21swapper/023:02:150
385998021,47cyclictest0-21swapper/021:20:130
308082800,2sleep10-21swapper/121:31:271
385997926,36cyclictest11181-21ssh22:26:020
385997925,41cyclictest0-21swapper/021:14:260
385997924,47cyclictest0-21swapper/019:36:530
385997924,41cyclictest0-21swapper/020:09:240
385997923,43cyclictest0-21swapper/020:01:190
385997923,41cyclictest0-21swapper/000:21:150
319422790,2sleep10-21swapper/100:10:211
319422790,2sleep10-21swapper/100:10:211
236212780,5sleep339799cyclictest23:20:353
387997732,25cyclictest1548-21cat22:15:241
385997732,29cyclictest12585-21ls00:25:310
385997728,38cyclictest29371-21ssh23:27:550
385997725,39cyclictest149-21systemd-journal20:41:380
385997723,42cyclictest0-21swapper/019:23:290
385997722,43cyclictest0-21swapper/022:37:070
189902770,2sleep20-21swapper/221:56:242
385997622,37cyclictest13558-21if_err_wlan021:50:270
13262760,5sleep239599cyclictest19:10:312
385997524,36cyclictest4251-21apt-get20:50:090
324092740,2sleep30-21swapper/322:13:163
324092740,2sleep30-21swapper/322:13:163
213402740,5sleep221338-21ssh00:35:382
385997321,37cyclictest0-21swapper/021:09:160
387997237,11cyclictest20396-21ssh23:16:361
385997224,45cyclictest9431-21chrt21:01:060
200272720,3sleep30-21swapper/323:15:563
115922720,2sleep231-21ksoftirqd/219:40:222
387997135,13cyclictest22046-21ssh22:40:201
387997135,13cyclictest22046-21ssh22:40:201
387997133,21cyclictest8710-21ssh23:41:431
387997133,21cyclictest8710-21ssh23:41:431
387997131,15cyclictest17249-21ssh23:13:251
385997114,44cyclictest0-21swapper/019:40:330
221312710,1sleep10-21swapper/100:36:461
387997031,21cyclictest10508-21egrep23:05:231
38599707,44cyclictest0-21swapper/020:10:210
385997021,37cyclictest0-21swapper/019:26:120
385997020,39cyclictest0-21swapper/019:34:100
226612700,2sleep20-21swapper/221:22:102
387996930,19cyclictest4217-21ssh23:36:061
387996831,22cyclictest5587-21ssh22:59:531
387996830,13cyclictest7950irq/56-dwc_otg_21:20:201
387996829,19cyclictest19943-21ssh21:19:521
385996821,37cyclictest0-21swapper/019:45:340
39599671,5cyclictest28112-21sort20:25:272
387996733,12cyclictest27673-21ssh21:28:281
387996733,12cyclictest27673-21ssh21:28:281
387996732,25cyclictest8593-21sh23:02:371
387996731,16cyclictest20083-21ssh22:37:111
316302670,4sleep337-21rcuc/321:32:583
387996631,12cyclictest3208-21/usr/sbin/munin19:15:351
387996630,13cyclictest18593-21ssh21:55:401
387996627,22cyclictest28768-21ls20:25:361
39599653,9cyclictest25407-21rm21:25:302
39599653,9cyclictest25407-21rm21:25:302
387996537,18cyclictest9679-21ssh00:21:391
387996531,15cyclictest24146-21ssh22:03:171
387996531,12cyclictest451-21apt-get19:10:181
387996530,21cyclictest0-21swapper/120:50:141
387996530,12cyclictest31288-21tune2fs23:30:261
387996530,12cyclictest26728-21ssh22:45:311
387996529,21cyclictest30277-21sh23:29:351
387996528,12cyclictest6840-21ssh21:41:521
387996528,11cyclictest15624-21ssh22:31:351
387996434,12cyclictest15878-21ssh21:53:161
387996430,19cyclictest6745-21ssh00:18:511
387996430,13cyclictest6747-21grep22:20:381
387996429,21cyclictest22500-21ssh23:59:061
387996424,24cyclictest18677-21ssh00:32:581
387996424,15cyclictest5557-21seq19:23:491
385996418,34cyclictest0-21swapper/020:38:490
385996418,34cyclictest0-21swapper/020:38:490
39599634,5cyclictest7850irq/56-dwc_otg_22:40:092
387996332,11cyclictest2695-21ssh21:36:461
387996331,13cyclictest27453-21ssh22:06:451
387996228,22cyclictest13333-21ssh23:47:261
387996228,19cyclictest25346-21sh00:01:331
387996227,11cyclictest32099-21ssh22:51:281
39599619,13cyclictest13230-21ssh21:50:232
387996130,12cyclictest24749-21sh23:21:581
387996127,13cyclictest16040-21processes23:50:341
387996126,11cyclictest7850irq/56-dwc_otg_00:10:081
387996124,25cyclictest14207-21ssh00:27:211
39599604,13cyclictest7950irq/56-dwc_otg_23:50:272
387996026,11cyclictest8977-21taskset19:31:311
387996025,15cyclictest31821-21ssh22:12:121
387996025,15cyclictest31821-21ssh22:12:121
39599593,5cyclictest28549-21runrttasks22:08:462
387995937,15cyclictest30798-21sshd20:30:401
387995926,11cyclictest23888-21runrttasks20:14:481
387995924,15cyclictest4235-21cron20:50:081
387995831,12cyclictest17670-21sensors19:55:341
387995828,11cyclictest7802-21taskset20:58:231
387995826,14cyclictest12748-21ssh21:50:061
387995826,10cyclictest0-21swapper/121:00:311
107312580,2sleep30-21swapper/322:25:363
39599577,5cyclictest7950irq/56-dwc_otg_23:30:382
387995730,11cyclictest20110-21taskset20:04:021
387995729,12cyclictest32733-21taskset20:36:061
387995729,12cyclictest32733-21taskset20:36:061
39599566,4cyclictest0-21swapper/200:19:522
39599561,7cyclictest31744-21apt-config00:10:092
395995518,5cyclictest26950-21aten_rbpower_vo23:25:222
39599551,6cyclictest7850irq/56-dwc_otg_23:45:092
39599551,6cyclictest7850irq/56-dwc_otg_23:45:092
387995528,12cyclictest25497-21chrt20:17:121
39599541,20cyclictest0-21swapper/222:21:042
387995324,12cyclictest12742-21taskset19:42:101
39599524,4cyclictest0-21swapper/200:22:182
39599521,5cyclictest31175-21processes22:50:342
387995220,15cyclictest2180-21taskset20:41:551
39599514,12cyclictest0-21swapper/223:20:212
387995122,11cyclictest0-21swapper/121:05:371
387995026,14cyclictest25971-21/usr/sbin/munin20:20:311
387995024,10cyclictest0-21swapper/119:45:171
387995021,13cyclictest7145-21runrttasks19:26:111
387995020,16cyclictest11030-21runrttasks19:39:351
387994835,11cyclictest20542-21munin-node20:05:181
39799473,22cyclictest11933-21ssh21:10:223
39599473,5cyclictest7950irq/56-dwc_otg_23:05:142
39599467,8cyclictest0-21swapper/221:45:092
39599467,24cyclictest0-21swapper/220:35:152
39599467,24cyclictest0-21swapper/220:35:152
39799451,21cyclictest3009-21cut20:45:283
39599452,15cyclictest7750irq/56-dwc_otg21:16:162
323372453,13sleep00-21swapper/019:07:470
39799447,24cyclictest0-21swapper/322:35:223
137582440,19sleep37950irq/56-dwc_otg_23:48:133
39799436,22cyclictest25500-21ssh22:05:183
39599433,9cyclictest7950irq/56-dwc_otg_21:47:552
39599427,26cyclictest0-21swapper/220:50:142
39599427,22cyclictest0-21swapper/221:35:212
128152420,2sleep20-21swapper/222:29:012
92292410,2sleep20-21swapper/223:03:472
39599415,7cyclictest7950irq/56-dwc_otg_23:55:322
304812410,3sleep3379-21cyclictest00:08:183
253122410,2sleep30-21swapper/322:44:123
253122410,2sleep30-21swapper/322:44:123
39599407,24cyclictest0-21swapper/219:50:212
175772400,3sleep30-21swapper/300:30:563
39799391,29cyclictest11723-21ls00:25:093
39799386,23cyclictest0-21swapper/300:10:153
39799386,23cyclictest0-21swapper/300:10:153
39599383,13cyclictest0-21swapper/220:10:212
83232370,5sleep339799cyclictest22:23:223
397993722,4cyclictest0-21swapper/321:25:293
397993722,4cyclictest0-21swapper/321:25:293
39799371,7cyclictest7750irq/56-dwc_otg20:30:243
39799371,21cyclictest12302-21switchtime19:40:333
39599372,3cyclictest0-21swapper/223:10:192
39599371,21cyclictest7850irq/56-dwc_otg_21:00:262
39799365,5cyclictest7750irq/56-dwc_otg21:20:083
39799361,24cyclictest26394-21ls22:45:263
39599361,26cyclictest16410-21uniq00:30:252
40092350,2sleep231-21ksoftirqd/222:56:592
39799353,26cyclictest7950irq/56-dwc_otg_19:10:183
39799351,22cyclictest948-21cpuspeed21:35:233
39799351,22cyclictest7850irq/56-dwc_otg_22:03:393
39599357,24cyclictest0-21swapper/220:05:152
39599355,18cyclictest7950irq/56-dwc_otg_19:20:212
39599353,18cyclictest7950irq/56-dwc_otg_00:05:092
39599352,14cyclictest7850irq/56-dwc_otg_20:20:362
39599351,6cyclictest14397-21unixbench_singl19:45:382
39799341,25cyclictest18515-21ls20:00:093
39599347,7cyclictest7750irq/56-dwc_otg22:00:372
39599347,13cyclictest7950irq/56-dwc_otg_20:55:292
39599346,23cyclictest0-21swapper/219:15:222
39599342,8cyclictest7850irq/56-dwc_otg_20:45:382
395993423,9cyclictest7950irq/56-dwc_otg_00:15:092
395993423,9cyclictest7950irq/56-dwc_otg_00:15:092
39599341,6cyclictest19758-21grep20:00:382
139482340,1sleep30-21swapper/321:50:333
88442330,3sleep38846-21grep21:45:103
39799339,13cyclictest3658-21apt-get00:15:223
39799336,12cyclictest0-21swapper/322:15:213
39799333,7cyclictest7750irq/56-dwc_otg23:40:083
39799333,26cyclictest7950irq/56-dwc_otg_20:50:223
39799331,22cyclictest6686-21cut20:55:263
39599333,7cyclictest7850irq/56-dwc_otg_23:15:222
39599332,6cyclictest7750irq/56-dwc_otg19:25:192
395993310,12cyclictest0-21swapper/222:40:172
395993310,12cyclictest0-21swapper/222:40:172
39799326,4cyclictest0-21swapper/300:00:343
39799322,4cyclictest7750irq/56-dwc_otg20:10:253
397993217,5cyclictest1701-21sensors_temp20:40:313
397993213,17cyclictest3153750wget20:35:243
397993213,17cyclictest3153750wget20:35:243
39799321,20cyclictest9077-21switchtime21:00:363
397993210,6cyclictest37-21rcuc/323:05:323
39599326,14cyclictest0-21swapper/219:55:222
39599325,5cyclictest7850irq/56-dwc_otg_19:35:262
39599321,3cyclictest7950irq/56-dwc_otg_21:10:352
317022326,11sleep20-21swapper/219:05:342
39799319,10cyclictest0-21swapper/322:58:523
39799318,4cyclictest0-21swapper/321:55:573
39799317,4cyclictest7950irq/56-dwc_otg_23:50:243
39799314,7cyclictest6768-21/usr/sbin/munin23:40:233
39799314,7cyclictest6768-21/usr/sbin/munin23:40:233
39799311,17cyclictest23222-21apt-config00:00:093
39799311,13cyclictest7750irq/56-dwc_otg23:25:373
39599319,4cyclictest4710-21seq23:36:592
39599316,15cyclictest0-21swapper/222:30:202
39599312,7cyclictest0-21swapper/221:05:202
395993114,4cyclictest2920750ssh22:48:442
395993111,4cyclictest0-21swapper/221:30:132
39799309,3cyclictest0-21swapper/321:20:263
39799308,5cyclictest7850irq/56-dwc_otg_22:50:303
39799306,6cyclictest0-21swapper/321:50:083
39799302,19cyclictest0-21swapper/320:06:053
39799302,19cyclictest0-21swapper/320:05:003
39799301,6cyclictest6954-21awk23:00:323
39599306,15cyclictest0-21swapper/200:25:242
39599303,6cyclictest7950irq/56-dwc_otg_20:30:382
39599302,5cyclictest0-21swapper/222:10:212
39599302,5cyclictest0-21swapper/222:10:212
39799299,3cyclictest0-21swapper/323:10:143
397992923,4cyclictest25741-21sh20:20:083
39799292,24cyclictest7950irq/56-dwc_otg_20:25:263
39799292,12cyclictest7850irq/56-dwc_otg_19:45:413
39799291,13cyclictest7850irq/56-dwc_otg_22:32:253
397992910,7cyclictest20447-21ssh00:35:273
395992917,5cyclictest3481-21seq22:17:072
39599291,20cyclictest1867-21timerwakeupswit20:40:332
39799289,5cyclictest37-21rcuc/300:25:233
39799283,4cyclictest7750irq/56-dwc_otg21:05:243
39799283,13cyclictest0-21swapper/319:34:453
39799282,17cyclictest0-21swapper/319:36:023
39799281,5cyclictest7850irq/56-dwc_otg_19:50:303
39599287,19cyclictest24526-21ls20:15:282
322702283,17sleep10-21swapper/119:06:551
39799273,12cyclictest7850irq/56-dwc_otg_19:15:383
39799272,5cyclictest7750irq/56-dwc_otg19:20:363
39799271,16cyclictest26292-21fschecks_time20:20:273
39799262,8cyclictest0-21swapper/319:25:233
39599261,5cyclictest8003-21sort19:30:252
317822268,9sleep30-21swapper/319:05:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional