You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 05:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Aug 27, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
766821650,5sleep31225199cyclictest23:27:163
766821650,5sleep31225199cyclictest23:27:163
1844321070,2sleep10-21swapper/121:43:151
122489910510,55cyclictest0-21swapper/021:10:210
1224899973,69cyclictest0-21swapper/021:30:420
12249999122,19cyclictest7750irq/56-dwc_otg23:50:241
12249999122,19cyclictest7750irq/56-dwc_otg23:50:241
73862860,2sleep30-21swapper/321:30:173
302772840,6sleep31225199cyclictest00:34:363
147572840,6sleep31225199cyclictest00:15:273
147572840,6sleep31225199cyclictest00:15:273
228762820,2sleep20-21swapper/221:48:482
228762820,2sleep20-21swapper/221:48:482
1225199803,69cyclictest24017-21memory21:10:313
127232770,2sleep20-21swapper/200:12:402
1225199735,40cyclictest0-21swapper/300:20:253
1224899731,57cyclictest22600-21unixbench_multi21:05:400
116862730,5sleep01224899cyclictest23:31:520
1225199701,49cyclictest13456-21/usr/sbin/munin19:10:393
1224899703,49cyclictest0-21swapper/020:50:220
114682700,4sleep2301ktimersoftd/219:05:382
1225199683,45cyclictest12472-21/usr/sbin/munin20:40:243
1225199671,48cyclictest17757-21ntp_states23:00:343
1225199671,44cyclictest0-21swapper/300:00:343
1224899672,24cyclictest0-21swapper/022:15:240
1225199662,40cyclictest1519-21cut20:10:243
12248996625,22cyclictest0-21swapper/019:15:220
12248996614,12cyclictest0-21swapper/020:10:230
1225199651,48cyclictest19545-21ssh22:24:053
1225199642,51cyclictest11699-21cpu21:35:243
1225199642,44cyclictest28264-21ntpq21:15:333
1225199642,44cyclictest28264-21ntpq21:15:333
1225199642,44cyclictest1709-21ssh00:00:103
1225199641,47cyclictest15199-21ssh22:57:343
1225199641,44cyclictest20631-21ntp_states21:45:353
1225199641,44cyclictest20631-21ntp_states21:45:353
1225099648,30cyclictest301ktimersoftd/221:30:422
1224899641,15cyclictest0-21swapper/020:20:260
1225199631,54cyclictest8033-21ssh22:10:183
1225199631,45cyclictest25266-21perl19:45:293
1225199631,43cyclictest25476-21ssh21:50:593
1225199631,40cyclictest24400-21sh00:26:223
1224899631,39cyclictest0-21swapper/000:15:220
1224899631,39cyclictest0-21swapper/000:15:220
1224899631,25cyclictest25376-21ssh23:10:210
1225199623,42cyclictest7950irq/56-dwc_otg_20:20:273
1225199622,38cyclictest7950irq/56-dwc_otg_19:40:103
1225199621,48cyclictest31176-21ssh22:37:413
1225199621,38cyclictest7950irq/56-dwc_otg_20:45:353
1224899626,53cyclictest0-21swapper/019:35:240
12248996225,4cyclictest13877-21ssh23:35:240
12248996217,6cyclictest0-21swapper/023:40:220
1225199611,4cyclictest21902-21taskset22:25:493
1225199611,48cyclictest12046-21taskset22:54:233
1225199611,47cyclictest0-21swapper/300:14:593
1225199611,45cyclictest1325-21ssh23:20:253
1225199611,44cyclictest18225-21df23:40:263
1225199611,43cyclictest4008-21ntp_states20:15:333
1225199611,43cyclictest4008-21ntp_states20:15:333
1225199611,36cyclictest1724-21ssh21:22:513
12250996125,20cyclictest24003-21sh21:10:302
1224899614,5cyclictest0-21swapper/022:50:240
1224899612,48cyclictest9365-21unixbench_singl22:10:400
12248996114,27cyclictest28728-21/usr/sbin/munin22:35:380
1225199601,49cyclictest23419-21ssh23:07:173
1225199601,47cyclictest3043-21ssh22:42:593
1225199601,47cyclictest3043-21ssh22:42:593
1225199601,43cyclictest28707-21taskset23:14:203
1225199601,43cyclictest17360-21chrt19:24:053
1224899605,26cyclictest16701-21switchtime21:40:370
1224899602,33cyclictest10172-21latency_hist00:10:120
1224899602,29cyclictest30729-21ssh23:15:420
223512590,3sleep022354-21timerwakeupswit23:05:390
1225199591,5cyclictest16202-21sed22:20:113
1225199591,45cyclictest31517-21systemd-journal00:06:233
1225199591,42cyclictest31400-21ssh21:59:153
1225199591,40cyclictest1018-21ssh00:37:523
12249995910,15cyclictest7950irq/56-dwc_otg_21:30:441
1224899594,47cyclictest29898-21kworker/0:220:04:180
1224899593,31cyclictest0-21swapper/019:50:390
12248995928,6cyclictest7127-21munin-plugin-st21:30:110
1224899591,38cyclictest18875-21ssh00:20:300
1225199582,35cyclictest0-21swapper/323:15:373
1225199581,46cyclictest1688-21ssh22:01:403
1225199581,40cyclictest7597-21ssh22:09:553
12248995817,11cyclictest0-21swapper/020:40:210
181532570,1sleep30-21swapper/319:25:303
1225199573,35cyclictest7950irq/56-dwc_otg_22:32:483
1225199572,43cyclictest18408-21ssh21:43:113
1225199571,52cyclictest6941-21apt-get20:25:243
1225199571,48cyclictest16210-21ssh23:37:333
1225199571,46cyclictest7828-21ssh22:49:133
1225199571,44cyclictest6354-21ssh21:28:473
1225199571,43cyclictest23267-21latency_hist21:10:123
1225199571,43cyclictest0-21swapper/320:51:153
1225199571,36cyclictest29560-21kworker/u8:023:33:593
1225199571,2cyclictest0-21swapper/323:46:203
1224899577,40cyclictest20064-21ls21:45:250
1224899577,40cyclictest20064-21ls21:45:250
12248995724,7cyclictest0-21swapper/021:20:240
1225199561,35cyclictest0-21swapper/323:54:493
1225199561,35cyclictest0-21swapper/323:54:493
1224999563,10cyclictest7750irq/56-dwc_otg21:05:231
1224899561,24cyclictest23987-21unixbench_singl00:25:400
1225199552,48cyclictest23131-21apt-get19:40:243
1224899551,29cyclictest25854-21ssh22:30:400
1225199542,40cyclictest7950irq/56-dwc_otg_19:35:023
1225199542,40cyclictest7950irq/56-dwc_otg_19:35:023
1225199541,42cyclictest0-21swapper/320:56:403
1225199541,42cyclictest0-21swapper/320:56:403
1225199541,42cyclictest0-21swapper/320:40:063
1225199541,2cyclictest0-21swapper/320:30:313
1224999547,33cyclictest18020-21ssh23:40:241
12248995412,10cyclictest0-21swapper/022:55:240
1225199533,46cyclictest29646-21unixbench_multi19:55:403
1224899535,18cyclictest26290-21ssh23:50:250
1224899535,18cyclictest26290-21ssh23:50:250
12248995328,5cyclictest2373-21fschecks_count00:00:280
1224899531,8cyclictest17605-21unixbench_multi22:20:410
12248995315,4cyclictest0-21swapper/021:35:200
117762532,44sleep00-21swapper/019:07:390
53582520,2sleep00-21swapper/022:05:490
12249995221,11cyclictest1285-21ntpd23:55:241
12248995215,10cyclictest0-21swapper/000:30:200
1224899521,35cyclictest19768-21fschecks_count19:30:290
1224899521,35cyclictest19768-21fschecks_count19:30:290
1225199511,46cyclictest30374-21/usr/sbin/munin20:00:223
1225199511,40cyclictest0-21swapper/320:07:443
1224899515,23cyclictest0-21swapper/019:25:230
1224899514,22cyclictest0-21swapper/019:55:240
1224899511,27cyclictest27787-21fschecks_count21:15:280
1224899511,27cyclictest27787-21fschecks_count21:15:280
1225199502,44cyclictest19653-21apt-get21:00:183
1225199501,45cyclictest27639-21ls19:50:373
1224999502,10cyclictest7850irq/56-dwc_otg_22:50:241
12248995029,5cyclictest7950irq/56-dwc_otg_22:05:110
12248995022,6cyclictest8043-21unixbench_multi20:25:390
1224899501,29cyclictest22510-21fschecks_time23:45:290
1224899501,29cyclictest0-21swapper/020:05:240
1224899501,27cyclictest21623-21grep22:25:390
1224899501,27cyclictest0-21swapper/019:10:260
316272490,2sleep00-21swapper/021:59:410
1224999495,8cyclictest7750irq/56-dwc_otg22:05:241
12249994916,13cyclictest24060-21ls21:10:321
12248994938,4cyclictest7950irq/56-dwc_otg_23:25:110
12248994923,4cyclictest1252-21kworker/u8:423:55:250
1224899492,30cyclictest10656-21wget20:35:260
12248994920,10cyclictest0-21swapper/000:39:190
1224899482,30cyclictest24825-21latency_hist19:45:110
1224899481,26cyclictest6777-21taskset23:25:400
1224899481,26cyclictest6777-21taskset23:25:400
1225199475,39cyclictest15164-21unixbench_multi19:15:383
1225099475,6cyclictest7850irq/56-dwc_otg_23:50:242
1225099475,6cyclictest7850irq/56-dwc_otg_23:50:242
1224899471,9cyclictest15282-21grep20:45:390
1224899471,27cyclictest9418-21ntp_kernel_pll_20:30:340
1225099467,25cyclictest0-21swapper/220:45:232
1224999467,25cyclictest0-21swapper/119:55:251
12249994615,5cyclictest23-21ksoftirqd/121:20:341
1224899466,22cyclictest0-21swapper/021:00:150
12248994635,4cyclictest4302-21latency_hist22:45:120
12248994635,4cyclictest4302-21latency_hist22:45:120
12248994631,5cyclictest7750irq/56-dwc_otg22:45:370
1224899461,8cyclictest0-21swapper/000:13:370
12250994517,7cyclictest30413-21aten_rbpower_cu23:55:252
12249994519,12cyclictest15920-21aten_rbpower_cu19:20:231
1224899454,5cyclictest0-21swapper/023:00:220
1224899452,24cyclictest0-21swapper/019:20:120
12248994520,5cyclictest17798-21aten_rbpower_cu20:55:260
12248994520,5cyclictest17798-21aten_rbpower_cu20:55:260
1224899451,29cyclictest31517-21systemd-journal19:47:560
1225099447,24cyclictest0-21swapper/223:10:242
12250994412,16cyclictest0-21swapper/220:30:222
1224999447,23cyclictest0-21swapper/120:00:221
1224999447,22cyclictest0-21swapper/120:45:231
12249994424,5cyclictest23-21ksoftirqd/119:45:401
29132430,4sleep1221ktimersoftd/122:42:451
29132430,4sleep1221ktimersoftd/122:42:451
209392430,5sleep11224999cyclictest23:05:121
193342430,2sleep20-21swapper/221:44:532
1225099437,22cyclictest0-21swapper/220:05:262
1225099437,21cyclictest0-21swapper/220:00:222
1224999437,23cyclictest0-21swapper/119:25:241
12249994318,6cyclictest23-21ksoftirqd/100:40:121
1225099426,24cyclictest0-21swapper/223:25:242
1225099426,24cyclictest0-21swapper/223:25:242
1225099424,9cyclictest7950irq/56-dwc_otg_22:35:122
1224999428,5cyclictest7950irq/56-dwc_otg_00:20:111
1224999428,5cyclictest7950irq/56-dwc_otg_00:20:111
1224999426,24cyclictest0-21swapper/121:00:181
1224999424,5cyclictest0-21swapper/120:30:221
12249994220,4cyclictest18160-21if_err_wlan020:55:311
12249994220,4cyclictest18160-21if_err_wlan020:55:311
1224999421,8cyclictest12247-21cyclictest23:25:221
1224999421,8cyclictest12247-21cyclictest23:25:221
12248994212,21cyclictest0-21swapper/020:17:590
12248994212,21cyclictest0-21swapper/020:17:590
1225099409,25cyclictest7950irq/56-dwc_otg_21:55:232
12250994027,5cyclictest27737-21ssh21:15:262
12250994027,5cyclictest27737-21ssh21:15:262
1225099401,19cyclictest18365-21grep20:55:342
1225099401,19cyclictest18365-21grep20:55:342
12249994023,4cyclictest21-21rcuc/100:20:271
1224999401,21cyclictest0-21swapper/119:50:361
12248994021,4cyclictest7850irq/56-dwc_otg_21:50:240
1225099391,26cyclictest18555-21ls19:25:382
1224999397,23cyclictest0-21swapper/123:35:241
1224999396,6cyclictest7750irq/56-dwc_otg23:07:571
12249993915,5cyclictest221ktimersoftd/123:24:391
1224999391,27cyclictest7135-21ls20:25:261
1225099382,20cyclictest0-21swapper/222:50:372
1225099382,19cyclictest11296-21proc_pri20:35:362
1225099381,28cyclictest0-21swapper/200:05:352
1225099381,10cyclictest0-21swapper/222:26:222
1224999387,6cyclictest7750irq/56-dwc_otg20:05:261
1224999386,3cyclictest0-21swapper/100:31:191
1224999381,6cyclictest28844-21awk21:15:411
1224999381,6cyclictest28844-21awk21:15:411
317212370,2sleep10-21swapper/121:59:501
1225099372,9cyclictest29137-21ls19:55:332
12250993729,6cyclictest18324-21latency_hist00:20:122
12250993729,6cyclictest18324-21latency_hist00:20:122
1225099372,12cyclictest0-21swapper/223:40:282
1225099371,6cyclictest0-21swapper/220:15:232
1225099371,6cyclictest0-21swapper/220:15:232
1225099371,18cyclictest25048-21cron23:10:112
1224999375,20cyclictest4561-21ssh22:45:221
12249993713,7cyclictest0-21swapper/100:00:251
12249993710,14cyclictest0-21swapper/100:06:201
322222360,2sleep20-21swapper/222:39:362
231652360,5sleep11224999cyclictest23:45:381
1225099362,15cyclictest0-21swapper/222:47:312
1225099361,18cyclictest0-21swapper/200:20:422
1224999362,3cyclictest0-21swapper/122:15:451
1224999361,4cyclictest11551-21apt-get21:35:221
12249993610,5cyclictest23-21ksoftirqd/121:45:281
12249993610,5cyclictest23-21ksoftirqd/121:45:281
327342350,2sleep112247-21cyclictest23:19:051
277002350,1sleep10-21swapper/123:12:291
1225099352,25cyclictest0-21swapper/223:00:242
1225099352,15cyclictest23127-21sort21:10:112
1225099352,14cyclictest0-21swapper/220:10:282
1225099352,13cyclictest12853-21ssh22:55:242
1225099351,4cyclictest7950irq/56-dwc_otg_22:40:202
1225099351,4cyclictest7950irq/56-dwc_otg_22:40:202
1225099351,24cyclictest0-21swapper/200:30:222
12249993512,4cyclictest0-21swapper/122:34:041
1224999351,22cyclictest3998-21ls21:25:331
113372357,13sleep10-21swapper/119:05:381
1225099347,12cyclictest32199-21grep00:35:392
1225099342,18cyclictest0-21swapper/219:18:432
1225099342,13cyclictest7750irq/56-dwc_otg20:30:112
1225099342,12cyclictest0-21swapper/219:10:372
1225099341,9cyclictest21912-21grep19:35:322
12250993416,4cyclictest31986-21fschecks_count21:20:302
1225099341,22cyclictest16137-21ls22:20:112
1224999349,11cyclictest0-21swapper/100:12:221
1224999342,4cyclictest7750irq/56-dwc_otg22:55:241
1225099338,15cyclictest0-21swapper/223:35:242
1225099333,5cyclictest4292-21uniq22:05:282
1225099333,13cyclictest0-21swapper/219:30:402
1225099333,13cyclictest0-21swapper/219:30:402
1225099332,5cyclictest0-21swapper/223:20:252
1225099332,10cyclictest0-21swapper/220:40:272
1225099331,5cyclictest24621-21cat21:50:322
1225099331,16cyclictest0-21swapper/221:39:032
1225099331,15cyclictest0-21swapper/223:45:402
1225099331,15cyclictest0-21swapper/222:04:432
1224999337,16cyclictest0-21swapper/122:10:251
1224999335,7cyclictest11649-21sed20:35:401
1224999335,19cyclictest20507-21ssh22:25:241
1224999332,6cyclictest21-21rcuc/121:50:301
1224999332,4cyclictest7750irq/56-dwc_otg19:10:121
1224999331,7cyclictest11007-21unixbench_singl23:30:411
12249993317,4cyclictest24746-21latency_hist19:45:111
12249993315,5cyclictest14555-21cut19:15:301
12249993310,10cyclictest0-21swapper/122:04:171
1225099323,12cyclictest0-21swapper/223:31:442
1225099322,14cyclictest0-21swapper/222:21:582
1225099321,19cyclictest21307-21ls21:05:112
12249993225,5cyclictest23-21ksoftirqd/100:25:331
1224999321,9cyclictest3346-21grep20:15:241
1224999321,9cyclictest3346-21grep20:15:241
12249993211,12cyclictest0-21swapper/122:37:321
1225099316,15cyclictest0-21swapper/222:10:252
1225099316,14cyclictest15947-21aten_rbpower_en19:20:232
1225099314,13cyclictest29-21rcuc/219:45:252
1225099312,19cyclictest0-21swapper/219:54:072
1225099311,7cyclictest2874-21ntp_states00:00:372
1225099311,24cyclictest3681-21uniq21:25:292
12249993110,4cyclictest0-21swapper/122:21:031
117222313,20sleep30-21swapper/319:06:583
1225099308,12cyclictest0-21swapper/223:17:422
12250993010,5cyclictest23048-21diskmemload00:28:272
1224999308,8cyclictest23-21ksoftirqd/120:50:421
1224999305,13cyclictest7750irq/56-dwc_otg19:30:291
1224999305,13cyclictest7750irq/56-dwc_otg19:30:291
12249993022,5cyclictest5456-21fschecks_time20:20:301
1224999301,21cyclictest0-21swapper/119:35:351
1224999294,6cyclictest7850irq/56-dwc_otg_20:40:301
1224999292,5cyclictest7750irq/56-dwc_otg20:10:251
1225099272,6cyclictest7950irq/56-dwc_otg_19:40:322
1225099263,10cyclictest7850irq/56-dwc_otg_20:50:302
1225099261,17cyclictest0-21swapper/220:20:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional