You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-03 - 03:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Jul 03, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1677321770,5sleep3157399cyclictest23:09:303
200312770,2sleep10-21swapper/122:34:091
167182720,2sleep00-21swapper/000:26:340
156099721,40cyclictest9216-21apt21:00:111
156099713,47cyclictest7296-21apt-get19:25:221
156099711,44cyclictest593-21ssh00:07:371
122202710,1sleep30-21swapper/321:45:563
156099702,44cyclictest2300-21grep20:40:281
156099701,58cyclictest15095-21ssh21:11:331
156099701,55cyclictest1815-21rm22:12:081
44072690,2sleep00-21swapper/019:15:380
288822690,1sleep10-21swapper/123:23:501
156099691,49cyclictest8425-21wget22:20:271
156099691,49cyclictest8425-21wget22:20:271
156099681,54cyclictest8952-21taskset21:42:401
156099681,45cyclictest24720-21taskset23:18:471
156099681,38cyclictest0-21swapper/123:50:121
156099681,38cyclictest0-21swapper/123:50:121
155399686,49cyclictest0-21swapper/023:25:260
156099671,54cyclictest11528-21ntp_states21:45:351
156099671,53cyclictest0-21swapper/122:00:321
156099671,53cyclictest0-21swapper/122:00:321
11382670,2sleep00-21swapper/022:50:270
11382670,2sleep00-21swapper/022:50:270
157399663,5cyclictest1637450ssh23:47:243
157399663,5cyclictest1637450ssh23:47:243
157399663,4cyclictest17940-21latency_hist19:55:113
156099661,53cyclictest4474-21ssh21:37:181
156099661,51cyclictest17039-21ssh00:27:101
157399659,23cyclictest0-21swapper/323:10:233
156099656,42cyclictest8050irq/56-dwc_otg_22:40:391
156099653,49cyclictest3149-21ntp_states00:10:351
156099653,49cyclictest3149-21ntp_states00:10:351
156099653,46cyclictest18119-21apt-get19:55:241
156099651,53cyclictest12822-21diskmemload21:51:231
156099651,47cyclictest0-21swapper/121:31:141
156099651,47cyclictest0-21swapper/121:31:141
156099651,46cyclictest7850irq/56-dwc_otg00:30:301
156099651,42cyclictest0-21swapper/120:15:221
157399646,13cyclictest8050irq/56-dwc_otg_22:55:293
156099641,54cyclictest15594-21ssh23:07:221
156099641,52cyclictest9452-21ssh00:18:371
156099641,43cyclictest10037-21ntp_states23:00:411
156099641,42cyclictest5019-21ssh22:55:211
156099641,36cyclictest9564-21kernelversion19:30:311
155399643,32cyclictest0-21swapper/022:15:250
155399642,40cyclictest8711-21idleruntime22:20:320
155399642,40cyclictest8711-21idleruntime22:20:320
156099631,48cyclictest1255-21sh23:30:031
1553996336,4cyclictest23370-21aten_rbpower_en00:35:250
1573996210,4cyclictest0-21swapper/322:43:173
156099622,39cyclictest21533-21latency_hist20:05:111
156099621,50cyclictest31038-21runrttasks22:46:361
156099621,44cyclictest23423-21taskset22:37:481
156099621,40cyclictest1667-21/usr/sbin/munin23:30:221
156099621,40cyclictest0-21swapper/122:17:101
1553996239,5cyclictest7850irq/56-dwc_otg20:50:120
1553996239,5cyclictest7850irq/56-dwc_otg20:50:120
1553996229,25cyclictest0-21swapper/019:40:240
1553996220,13cyclictest0-21swapper/000:15:230
156099612,41cyclictest18526-21ssh23:50:291
156099611,44cyclictest29318-21ssh22:05:571
157399602,7cyclictest2343-21grep20:40:283
156099603,33cyclictest0-21swapper/100:00:031
156099602,35cyclictest14509-21ssh00:25:121
156099601,47cyclictest21354-21ssh21:56:581
156099601,39cyclictest27399-21/usr/sbin/munin21:25:401
1553996032,5cyclictest8050irq/56-dwc_otg_00:25:120
155399602,5cyclictest0-21swapper/020:40:220
155399602,27cyclictest0-21swapper/019:35:240
155399601,42cyclictest14989-21/usr/sbin/munin21:50:270
1553996013,15cyclictest1858-21ls19:10:270
156099592,38cyclictest0-21swapper/119:17:051
156099591,49cyclictest26433-21ssh00:38:591
156099591,46cyclictest13348-21rm23:44:261
156099591,38cyclictest18850-21ssh23:10:451
156099591,38cyclictest0-21swapper/123:35:491
157399585,4cyclictest7850irq/56-dwc_otg23:30:303
157399582,4cyclictest7850irq/56-dwc_otg19:30:353
1573995814,4cyclictest15089-21aten_rbpower_en21:50:253
156099582,31cyclictest0-21swapper/122:28:241
156099581,42cyclictest9561-21basename21:00:261
156099581,40cyclictest0-21swapper/100:04:461
1553995825,25cyclictest0-21swapper/021:00:250
156099571,44cyclictest0-21swapper/122:54:061
156099571,44cyclictest0-21swapper/122:54:061
156099571,44cyclictest0-21swapper/121:22:511
1553995727,5cyclictest0-21swapper/023:18:130
157399567,4cyclictest0-21swapper/321:56:253
157399565,8cyclictest0-21swapper/319:55:223
157399564,4cyclictest8050irq/56-dwc_otg_20:32:243
157399564,3cyclictest0-21swapper/322:15:233
157399561,7cyclictest8527-21df22:20:283
157399561,7cyclictest8527-21df22:20:283
1573995617,4cyclictest13756-21/usr/sbin/munin22:25:393
157399561,5cyclictest5495-21taskset00:13:583
157399561,5cyclictest5495-21taskset00:13:583
1573995611,4cyclictest0-21swapper/323:20:363
156099562,42cyclictest19968-21ssh21:17:341
156099561,51cyclictest11295-21wget21:05:261
156099561,39cyclictest28428-21users20:20:411
156099561,35cyclictest0-21swapper/120:09:461
1553995630,14cyclictest17093-21ssh23:10:060
1553995629,13cyclictest7950irq/56-dwc_otg_21:10:350
1553995619,15cyclictest1832250kworker/u9:022:35:340
155399561,42cyclictest0-21swapper/020:55:380
157399559,4cyclictest0-21swapper/323:57:523
157399557,4cyclictest0-21swapper/300:38:543
1573995516,6cyclictest15274-21fschecks_count00:25:333
157399551,4cyclictest0-21swapper/300:05:123
157399551,3cyclictest7950irq/56-dwc_otg_23:44:543
156799552,8cyclictest8050irq/56-dwc_otg_00:00:242
156099552,44cyclictest7950irq/56-dwc_otg_19:11:241
156099552,40cyclictest6919-21chrt19:24:571
156099551,35cyclictest7950irq/56-dwc_otg_19:47:291
156099551,35cyclictest7950irq/56-dwc_otg_19:47:291
155399552,8cyclictest8050irq/56-dwc_otg_21:55:230
155399551,41cyclictest27361-21sh21:25:400
157399542,3cyclictest0-21swapper/300:24:213
1573995418,33cyclictest17133-21fschecks_time22:30:293
157399541,5cyclictest21866-21ssh21:20:263
157399541,4cyclictest0-21swapper/321:17:373
157399541,3cyclictest7950irq/56-dwc_otg_23:00:473
157399541,3cyclictest7950irq/56-dwc_otg_21:31:473
157399541,3cyclictest7950irq/56-dwc_otg_21:31:473
156099542,37cyclictest8050irq/56-dwc_otg_19:41:541
156099541,43cyclictest0-21swapper/120:35:111
1553995421,7cyclictest2571-21fschecks_time21:35:310
1553995421,16cyclictest7950irq/56-dwc_otg_19:50:400
1553995414,5cyclictest7850irq/56-dwc_otg23:30:230
155399541,36cyclictest3605-21ls22:15:120
155399541,32cyclictest13773-21ssh23:45:120
155399541,13cyclictest18439-21switchtime23:10:370
1553995411,25cyclictest727-21runrttasks22:55:330
157399536,3cyclictest0-21swapper/322:50:273
157399536,3cyclictest0-21swapper/322:50:273
157399535,4cyclictest8050irq/56-dwc_otg_21:11:363
157399534,4cyclictest0-21swapper/323:39:373
157399533,9cyclictest8050irq/56-dwc_otg_22:09:183
157399531,5cyclictest7850irq/56-dwc_otg22:00:353
157399531,5cyclictest7850irq/56-dwc_otg22:00:353
1573995312,5cyclictest30174-21ssh22:45:403
157399531,11cyclictest0-21swapper/300:18:323
156799532,3cyclictest0-21swapper/220:55:232
1567995312,17cyclictest8050irq/56-dwc_otg_22:50:262
1567995312,17cyclictest8050irq/56-dwc_otg_22:50:262
156099531,49cyclictest1561-21latency_hist19:10:121
1553995322,7cyclictest7950irq/56-dwc_otg_19:45:390
1553995322,7cyclictest7950irq/56-dwc_otg_19:45:390
155399531,36cyclictest0-21swapper/022:45:390
155399531,28cyclictest0-21swapper/022:33:480
157399527,8cyclictest7850irq/56-dwc_otg23:50:373
157399522,4cyclictest7850irq/56-dwc_otg23:18:383
157399522,3cyclictest7850irq/56-dwc_otg22:37:193
157399521,3cyclictest0-21swapper/321:35:403
156099521,43cyclictest0-21swapper/120:29:301
155399526,43cyclictest0-21swapper/020:30:260
155399523,25cyclictest19750-21ls20:00:110
155399522,36cyclictest0-21swapper/020:05:330
1553995223,3cyclictest0-21swapper/020:50:130
155399522,25cyclictest12015-21unixbench_singl21:45:400
155399521,35cyclictest6252-21latency_hist00:15:120
155399521,35cyclictest6252-21latency_hist00:15:120
157399514,3cyclictest0-21swapper/300:30:533
157399511,4cyclictest0-21swapper/321:25:263
157399511,3cyclictest0-21swapper/320:15:123
156099513,44cyclictest10990-21aten_rbpower_po19:35:251
156099513,39cyclictest7950irq/56-dwc_otg_20:54:331
156099512,36cyclictest0-21swapper/119:53:011
155399516,25cyclictest7205-21latency_hist19:25:120
155399513,18cyclictest23069-21apt-config00:35:120
155399511,37cyclictest0-21swapper/021:20:110
155399511,28cyclictest22604-21ls23:55:270
157399504,4cyclictest0-21swapper/322:13:393
157399501,5cyclictest0-21swapper/300:07:403
155399505,26cyclictest0-21swapper/021:30:250
155399505,26cyclictest0-21swapper/021:30:250
155399502,5cyclictest0-21swapper/023:35:250
1553995023,5cyclictest7950irq/56-dwc_otg_22:00:310
1553995023,5cyclictest7950irq/56-dwc_otg_22:00:310
155399502,34cyclictest18633-21fschecks_time23:50:300
155399501,35cyclictest0-21swapper/021:41:070
155399501,18cyclictest0-21swapper/023:04:430
157399492,3cyclictest7950irq/56-dwc_otg_19:25:273
157399492,13cyclictest0-21swapper/321:40:143
156099492,37cyclictest0-21swapper/120:48:521
156099492,37cyclictest0-21swapper/120:48:521
156099491,38cyclictest0-21swapper/120:37:571
1553994930,3cyclictest0-21swapper/019:30:110
1553994927,5cyclictest7950irq/56-dwc_otg_22:40:230
1553994923,7cyclictest26122-21fschecks_count23:20:300
222642481,4sleep222291-21ssh23:15:342
157399481,4cyclictest31969-21ssh23:26:543
156799487,28cyclictest13196-21ssh21:10:172
155399482,26cyclictest12829-21fschecks_count22:25:280
1553994819,3cyclictest0-21swapper/000:05:390
155399481,27cyclictest0-21swapper/023:48:060
155399481,27cyclictest0-21swapper/023:48:060
155399481,26cyclictest12827-21latency_hist21:10:120
157399473,3cyclictest8050irq/56-dwc_otg_21:05:363
156799477,26cyclictest0-21swapper/220:50:172
155399472,34cyclictest22610-21ntpq21:20:350
1553994712,18cyclictest0-21swapper/022:05:300
1573994615,4cyclictest29032-21/usr/sbin/munin20:25:343
156799462,6cyclictest8050irq/56-dwc_otg_21:55:232
155399462,34cyclictest0-21swapper/020:25:110
1553994620,5cyclictest29331-21fschecks_time20:25:270
1553994615,15cyclictest875-21ntpq20:35:330
155399461,26cyclictest0-21swapper/020:05:120
1553994612,4cyclictest27775-21unixbench_multi00:00:400
157399457,23cyclictest0-21swapper/319:10:163
157399451,4cyclictest518-21rs:main3
1567994520,5cyclictest22187-21grep22:35:412
156099451,5cyclictest23288-21apt-get20:10:171
1553994525,11cyclictest7850irq/56-dwc_otg20:17:010
155399451,4cyclictest9422-21cut19:30:280
11572453,34sleep00-21swapper/019:08:580
156799447,24cyclictest0-21swapper/220:30:262
156799447,23cyclictest0-21swapper/220:35:252
155399442,39cyclictest0-21swapper/020:10:150
156799434,25cyclictest7850irq/56-dwc_otg23:53:042
178292420,2sleep20-21swapper/221:53:162
156799419,12cyclictest8050irq/56-dwc_otg_23:30:262
157399406,24cyclictest0-21swapper/319:20:173
157399406,22cyclictest0-21swapper/320:15:233
157399401,5cyclictest4163-21diskstats20:45:283
157399401,5cyclictest4163-21diskstats20:45:283
156799392,23cyclictest7850irq/56-dwc_otg21:40:392
156799381,24cyclictest23057-21apt-config00:35:122
157399372,12cyclictest0-21swapper/319:36:053
156799372,11cyclictest10746-21ntp_states23:40:352
156799369,4cyclictest0-21swapper/223:20:272
156799366,21cyclictest5070-21ssh22:55:252
156799366,20cyclictest19566-21kworker/u8:221:45:252
156799366,15cyclictest0-21swapper/220:25:222
156799361,24cyclictest12566-21ssh22:25:242
156799356,12cyclictest7950irq/56-dwc_otg_22:20:422
156799356,12cyclictest7950irq/56-dwc_otg_22:20:422
156799355,14cyclictest8050irq/56-dwc_otg_19:25:382
156799353,26cyclictest8050irq/56-dwc_otg_00:20:232
1567993512,7cyclictest0-21swapper/221:33:452
1567993512,7cyclictest0-21swapper/221:33:452
157399341,21cyclictest4027-21missed_timers19:15:333
156799349,4cyclictest0-21swapper/221:15:392
156799347,15cyclictest0-21swapper/219:20:172
156799345,14cyclictest8050irq/56-dwc_otg_21:05:332
156799342,4cyclictest14832-21uniq19:45:292
156799342,4cyclictest14832-21uniq19:45:292
1567993418,5cyclictest301ktimersoftd/220:05:252
1567993417,4cyclictest2658-21grep00:10:282
1567993417,4cyclictest2658-21grep00:10:282
1567993410,10cyclictest0-21swapper/222:41:022
156799340,19cyclictest7850irq/56-dwc_otg00:18:192
157399336,15cyclictest0-21swapper/320:50:183
156799337,14cyclictest0-21swapper/220:45:172
156799337,14cyclictest0-21swapper/220:45:172
156799335,12cyclictest17662-21ssh22:30:362
156799334,6cyclictest0-21swapper/223:55:262
156799334,4cyclictest8050irq/56-dwc_otg_22:15:402
156799333,12cyclictest7950irq/56-dwc_otg_23:45:322
156799333,12cyclictest7950irq/56-dwc_otg_23:45:322
156799331,21cyclictest6135-21cut23:35:292
11002332,24sleep20-21swapper/219:08:152
197512320,1sleep20-21swapper/223:12:232
156799327,6cyclictest0-21swapper/222:00:332
156799327,6cyclictest0-21swapper/222:00:332
156799327,5cyclictest8050irq/56-dwc_otg_19:50:312
156799327,15cyclictest0-21swapper/220:10:242
156799327,12cyclictest0-21swapper/222:15:122
156799326,9cyclictest1983-21grep19:10:302
1567993219,4cyclictest30301-21fschecks_time23:25:302
156799321,7cyclictest31903-21grep00:05:412
1567993214,5cyclictest29314-21sh22:45:272
51612310,2sleep20-21swapper/221:38:302
156799316,14cyclictest0-21swapper/220:20:262
156799314,10cyclictest9090-21/usr/sbin/munin19:30:342
156799313,14cyclictest0-21swapper/223:02:422
156799313,13cyclictest31-21ksoftirqd/222:05:392
1567993123,6cyclictest16323-21ssh00:25:502
1567993117,5cyclictest27892-21sh21:26:302
156799311,6cyclictest0-21swapper/221:23:362
156799311,16cyclictest0-21swapper/219:57:122
157399309,12cyclictest37-21rcuc/321:00:323
157399302,7cyclictest0-21swapper/320:58:583
1567993016,5cyclictest20356-21if_wlan020:00:312
1567993013,5cyclictest1281550wget19:40:262
157399298,6cyclictest27740-21latency20:20:323
157399292,15cyclictest8050irq/56-dwc_otg_19:45:313
157399292,15cyclictest8050irq/56-dwc_otg_19:45:313
156799295,9cyclictest125-21systemd-journal19:36:532
156799293,6cyclictest8050irq/56-dwc_otg_00:35:302
156799291,5cyclictest27012-21latency_hist20:20:122
156799291,5cyclictest13705-21uniq23:05:282
156799291,15cyclictest0-21swapper/220:40:292
157399287,12cyclictest8050irq/56-dwc_otg_20:00:243
157399283,13cyclictest8050irq/56-dwc_otg_20:35:263
157399281,14cyclictest8050irq/56-dwc_otg_19:40:313
156799283,10cyclictest8050irq/56-dwc_otg_21:02:102
156799271,6cyclictest3489-21aten_rbpower_cu19:15:252
11942273,15sleep30-21swapper/319:09:223
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional