You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 12:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Fri Dec 05, 2025 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
53109910132,10cyclictest23502-21wc21:37:261
243799880,4rtkit-daemon2436-21rtkit-daemon18:47:340
273862850,5sleep041ktimersoftd/022:52:280
35622830,5sleep33565-21unixbench_singl22:02:333
5307996925,37cyclictest18330-21grep19:12:230
5307996925,37cyclictest18330-21grep19:12:230
5307996818,33cyclictest0-21swapper/019:07:320
5307996815,46cyclictest0-21swapper/023:32:080
35962680,9sleep1531099cyclictest19:47:371
5307996620,40cyclictest0-21swapper/021:35:560
5307996620,26cyclictest1369-21runrttasks21:01:180
5307996517,28cyclictest186051munin-node20:22:170
5307996417,29cyclictest0-21swapper/021:07:280
5307996416,40cyclictest0-21swapper/000:09:090
5307996416,40cyclictest0-21swapper/000:09:090
5307996414,44cyclictest0-21swapper/019:32:080
5307996323,33cyclictest21434-21unixbench_multi23:47:270
5307996317,26cyclictest0-21swapper/023:03:210
5307996219,26cyclictest22177-21aten_r1power_cu21:37:080
5307996218,37cyclictest0-21swapper/022:07:280
5307996217,26cyclictest0-21swapper/018:52:280
5307996215,40cyclictest0-21swapper/023:37:060
5307996215,40cyclictest0-21swapper/021:27:280
5307996214,32cyclictest0-21swapper/020:17:140
5307996213,30cyclictest0-21swapper/018:57:140
5307996124,12cyclictest28130-21aten2.4-expect22:57:060
5307996118,35cyclictest1243-21snmpd22:05:230
5307996117,38cyclictest0-21swapper/021:47:120
5307996115,30cyclictest0-21swapper/020:47:110
5307996114,40cyclictest0-21swapper/021:42:110
5307996113,40cyclictest0-21swapper/019:52:130
5307996113,32cyclictest0-21swapper/019:42:350
530799609,34cyclictest0-21swapper/019:17:340
530799609,34cyclictest0-21swapper/019:17:340
5307996025,27cyclictest24198-21gltestperf22:47:190
5307996016,36cyclictest5130-21munin-run23:16:530
5307996013,40cyclictest0-21swapper/023:56:530
5307996013,32cyclictest0-21swapper/021:57:210
5307996012,17cyclictest0-21swapper/021:22:280
5307996012,17cyclictest0-21swapper/021:22:280
5307995919,14cyclictest30472-21latency_hist20:47:010
5307995918,25cyclictest12281-21expr20:07:270
5307995916,28cyclictest0-21swapper/023:08:200
5307995913,38cyclictest0-21swapper/023:17:150
5307995818,34cyclictest1243-21snmpd23:59:080
5307995817,25cyclictest4060-21munin-run19:52:020
5307995813,39cyclictest0-21swapper/020:02:360
5307995812,39cyclictest11377-21munin-node22:22:150
5307995812,27cyclictest0-21swapper/019:27:030
5307995741,10cyclictest27739-21grep00:02:160
5307995739,11cyclictest13026-21grep19:02:170
5307995716,26cyclictest13763-21munin-node22:27:210
5307995714,37cyclictest0-21swapper/022:46:090
5307995714,37cyclictest0-21swapper/022:46:090
5307995714,28cyclictest1243-21snmpd21:53:480
5307995712,38cyclictest0-21swapper/000:16:530
5307995712,30cyclictest0-21swapper/021:18:420
5307995711,28cyclictest0-21swapper/020:37:170
5307995711,28cyclictest0-21swapper/020:37:170
243799570,5rtkit-daemon2436-21rtkit-daemon23:11:542
240222573,46sleep124123-21fschecks_time22:47:181
5310995628,10cyclictest30482-21needreboot00:07:191
5310995628,10cyclictest30482-21needreboot00:07:191
530799568,32cyclictest0-21swapper/019:27:150
5307995636,14cyclictest0-21swapper/021:12:120
5307995618,12cyclictest8708-21chrt20:00:300
5307995618,12cyclictest8708-21chrt20:00:300
5307995615,34cyclictest2836-21idleruntime-cro20:56:580
5307995613,30cyclictest0-21swapper/022:33:540
531599553,45cyclictest10202-21telnet23:27:053
531599553,45cyclictest10202-21telnet23:27:053
531599553,43cyclictest1243-21snmpd21:58:483
5310995516,23cyclictest14094-21rt-features19:02:321
530799559,39cyclictest0-21swapper/021:02:160
5307995535,14cyclictest0-21swapper/020:32:230
5307995533,17cyclictest1369-21runrttasks23:42:080
5307995524,15cyclictest0-21swapper/023:23:320
5307995522,27cyclictest6934-21grep22:12:110
5307995519,21cyclictest1243-21snmpd22:18:510
5307995510,37cyclictest0-21swapper/019:37:150
243799550,5rtkit-daemon2436-21rtkit-daemon23:51:523
243799550,5rtkit-daemon2436-21rtkit-daemon23:42:303
531099544,27cyclictest0-21swapper/119:33:281
5310995426,13cyclictest0-21swapper/122:02:161
5310995425,14cyclictest0-21swapper/120:32:201
5307995427,21cyclictest0-21swapper/018:47:050
5307995412,35cyclictest0-21swapper/022:37:250
243799540,6rtkit-daemon2436-21rtkit-daemon00:12:143
222992543,11sleep10-21swapper/120:27:321
222482542,45sleep122315-21processes22:42:271
222482542,45sleep122315-21processes22:42:271
5315995335,10cyclictest7030-21grep23:17:273
5315995334,12cyclictest15759-21fschecks_time19:07:213
531599533,42cyclictest21024-21stty20:27:153
5314995338,9cyclictest26218-21grep23:57:272
5314995337,10cyclictest19396-21grep20:22:212
531099537,29cyclictest0-21swapper/122:27:301
531099535,30cyclictest0-21swapper/121:52:231
531099534,30cyclictest0-21swapper/100:12:251
5310995324,13cyclictest0-21swapper/122:32:291
5310995323,13cyclictest0-21swapper/121:57:261
5310995322,13cyclictest0-21swapper/121:27:221
5307995317,29cyclictest0-21swapper/020:15:460
293332533,43sleep329341-21cut21:52:073
243799530,5rtkit-daemon2436-21rtkit-daemon23:52:223
243799530,5rtkit-daemon2436-21rtkit-daemon23:02:242
130232533,8sleep313131-21munin-node20:07:513
117042533,42sleep111744-21sensors_temp18:57:331
86732522,43sleep10-21swapper/122:15:591
5315995237,9cyclictest5161-21tail19:52:213
531599523,42cyclictest7729-21sshd19:57:263
531599523,42cyclictest7729-21sshd19:57:263
531599523,41cyclictest14740-21cat23:36:543
531599523,41cyclictest1243-21snmpd20:18:153
531599523,40cyclictest2354-21df_abs19:47:193
531599523,39cyclictest16683-21munin-node21:22:353
531599523,39cyclictest16683-21munin-node21:22:353
531099525,29cyclictest0-21swapper/123:57:181
531099525,29cyclictest0-21swapper/118:47:361
531099525,28cyclictest0-21swapper/121:47:181
5310995221,13cyclictest0-21swapper/123:22:131
5310995221,13cyclictest0-21swapper/120:17:331
530799524,41cyclictest0-21swapper/020:27:210
282742523,12sleep20-21swapper/219:32:322
243799520,5rtkit-daemon2436-21rtkit-daemon23:22:232
243799520,5rtkit-daemon2436-21rtkit-daemon00:11:363
243799520,5rtkit-daemon2436-21rtkit-daemon00:11:363
220142523,12sleep20-21swapper/219:22:042
220142523,12sleep20-21swapper/219:22:042
5315995138,8cyclictest22561-21cpuspeed_turbos19:22:183
5315995132,12cyclictest8608-21gltestperf23:22:153
531599513,41cyclictest28033-21cat20:42:003
531599513,41cyclictest28033-21cat20:42:003
531599513,41cyclictest0-21swapper/323:07:083
5314995135,10cyclictest13223-21df19:02:192
5314995133,11cyclictest27726-21fschecks_time21:47:182
5314995125,20cyclictest0-21swapper/223:37:112
5314995112,32cyclictest281ktimersoftd/220:42:352
531099516,28cyclictest0-21swapper/119:17:271
531099516,28cyclictest0-21swapper/119:17:271
531099514,30cyclictest0-21swapper/119:27:301
531099514,29cyclictest0-21swapper/123:47:141
531099514,29cyclictest0-21swapper/120:47:101
531099514,28cyclictest0-21swapper/120:17:001
531099514,28cyclictest0-21swapper/119:52:361
5310995122,13cyclictest0-21swapper/123:19:011
5310995121,13cyclictest0-21swapper/123:56:531
5307995112,23cyclictest0-21swapper/023:27:240
5307995112,23cyclictest0-21swapper/023:27:240
43432513,40sleep20-21swapper/220:57:282
243799510,5rtkit-daemon2436-21rtkit-daemon23:27:082
243799510,5rtkit-daemon2436-21rtkit-daemon23:27:082
217662512,43sleep10-21swapper/121:35:041
216272512,9sleep30-21swapper/321:33:163
204672512,9sleep30-21swapper/322:38:583
531599503,40cyclictest29978-21sshd22:58:223
531599503,39cyclictest13687-21munin-node19:02:253
5315995014,29cyclictest361ktimersoftd/322:07:303
5314995030,14cyclictest2833-21grep19:47:252
531499502,41cyclictest28200-21telnet22:57:072
531499502,41cyclictest25968-21ps19:27:312
531499502,41cyclictest25871-21telnet20:37:172
531499502,41cyclictest25871-21telnet20:37:172
531499502,40cyclictest21638-21sshd21:33:242
531499502,13cyclictest0-21swapper/219:07:322
5314995011,32cyclictest0-21swapper/219:17:042
5314995011,32cyclictest0-21swapper/219:17:042
531099505,29cyclictest0-21swapper/123:37:231
531099505,28cyclictest0-21swapper/123:42:301
531099505,28cyclictest0-21swapper/120:22:261
531099505,28cyclictest0-21swapper/118:52:151
531099505,27cyclictest0-21swapper/119:22:371
531099504,28cyclictest0-21swapper/123:02:271
531099504,28cyclictest0-21swapper/121:12:111
531099504,28cyclictest0-21swapper/120:07:381
531099504,26cyclictest31536-21yum-updatesd-he19:42:021
243799500,5rtkit-daemon2436-21rtkit-daemon23:37:303
243799500,5rtkit-daemon2436-21rtkit-daemon23:21:532
132252502,41sleep10-21swapper/123:32:161
80942493,39sleep18236-21ls21:07:191
55282492,9sleep30-21swapper/318:47:153
5315994933,11cyclictest26585-21tail22:52:153
5315994911,32cyclictest361ktimersoftd/320:57:303
5314994934,10cyclictest12224-21sed22:22:182
5314994932,10cyclictest32623-21grep19:42:232
531499492,39cyclictest24631-21telnet21:42:112
531499492,38cyclictest558-21telnet20:52:132
531499492,38cyclictest29493-21telnet21:52:082
5314994911,21cyclictest0-21swapper/222:52:192
531099498,26cyclictest0-21swapper/119:07:341
531099495,28cyclictest0-21swapper/120:42:201
531099495,26cyclictest0-21swapper/121:44:241
531099494,28cyclictest0-21swapper/123:27:241
531099494,28cyclictest0-21swapper/123:27:241
531099494,28cyclictest0-21swapper/122:57:061
531099494,28cyclictest0-21swapper/100:02:181
531099494,27cyclictest0-21swapper/123:12:161
531099494,27cyclictest0-21swapper/121:17:101
49272492,41sleep20-21swapper/223:14:242
326832494,38sleep20-21swapper/200:12:202
245702492,9sleep30-21swapper/321:42:113
229522492,9sleep30-21swapper/322:45:113
229522492,9sleep30-21swapper/322:45:113
121812492,9sleep20-21swapper/218:58:342
531599484,37cyclictest0-21swapper/322:47:333
531599482,39cyclictest1243-21snmpd20:33:173
531099486,26cyclictest0-21swapper/122:53:571
531099485,26cyclictest0-21swapper/122:37:091
531099483,27cyclictest0-21swapper/121:06:591
5310994819,13cyclictest0-21swapper/122:17:101
49372484,6sleep20-21swapper/218:42:402
312992482,9sleep30-21swapper/319:38:413
243799480,5rtkit-daemon2436-21rtkit-daemon00:01:373
241062482,9sleep30-21swapper/321:39:093
78372472,38sleep30-21swapper/321:07:093
531599474,38cyclictest0-21swapper/323:16:533
531599474,37cyclictest0-21swapper/321:21:593
5315994730,11cyclictest2396-21sed20:52:343
5314994732,9cyclictest22201-21aten_r1power_fr23:52:052
5314994732,9cyclictest14638-21gltestperf20:12:222
5314994732,10cyclictest6877-21fgrep21:02:312
5314994731,11cyclictest32528-21grep21:57:172
5314994731,10cyclictest3562-21taskset22:02:332
5314994730,11cyclictest27721-21grep00:02:162
531499473,34cyclictest0-21swapper/218:47:162
5314994729,11cyclictest11568-21grep21:12:262
5314994728,13cyclictest21475-21grep23:47:282
531499472,38cyclictest17457-21telnet23:42:052
531099478,17cyclictest0-21swapper/119:58:101
531099478,17cyclictest0-21swapper/119:58:101
5310994719,12cyclictest0-21swapper/123:09:001
326892472,9sleep30-21swapper/320:51:373
318002472,38sleep10-21swapper/119:42:141
243442472,39sleep20-21swapper/219:26:352
209812472,38sleep20-21swapper/220:27:052
110482472,9sleep30-21swapper/322:20:483
531599465,27cyclictest0-21swapper/319:12:163
531599465,27cyclictest0-21swapper/319:12:163
531599465,26cyclictest0-21swapper/322:22:093
531599464,37cyclictest0-21swapper/321:47:303
531599463,23cyclictest12494-21latency_hist19:02:043
5315994616,25cyclictest0-21swapper/320:12:243
531499464,28cyclictest0-21swapper/220:02:262
531499464,27cyclictest0-21swapper/219:42:022
531499462,36cyclictest12641-21telnet21:17:092
5314994618,7cyclictest10069-21latency_hist18:57:042
531499461,40cyclictest0-21swapper/222:27:122
531099465,25cyclictest0-21swapper/120:58:381
531099465,15cyclictest0-21swapper/122:08:501
5310994625,14cyclictest9381-21cpuspeed_turbos20:02:161
5310994616,13cyclictest0-21swapper/120:52:231
240752462,9sleep20-21swapper/221:38:432
135142462,9sleep20-21swapper/220:10:572
531599455,33cyclictest7572-21if_err_eth022:12:183
531599455,11cyclictest26357-21unixbench_singl19:27:363
531599454,36cyclictest32650-21munin-run23:06:543
5315994530,9cyclictest11146-21hwlatdetect21:12:183
531599453,34cyclictest0-21swapper/320:42:183
5314994532,8cyclictest29417-21basename00:07:062
5314994532,8cyclictest29417-21basename00:07:062
531499453,35cyclictest20971-21telnet22:42:092
531499453,35cyclictest20971-21telnet22:42:092
531499453,35cyclictest0-21swapper/222:47:242
5314994528,11cyclictest9859-21sed22:17:182
5314994523,16cyclictest0-21swapper/221:22:202
5314994523,16cyclictest0-21swapper/221:22:202
531499452,38cyclictest0-21swapper/220:51:112
531499452,37cyclictest0-21swapper/220:33:062
531499452,37cyclictest0-21swapper/220:19:312
5314994517,14cyclictest0-21swapper/219:58:372
5314994517,14cyclictest0-21swapper/219:58:372
5314994515,8cyclictest19531-21latency_hist21:31:592
531099455,15cyclictest0-21swapper/120:41:271
531099455,15cyclictest0-21swapper/120:41:271
531099453,26cyclictest0-21swapper/118:47:061
243799450,4rtkit-daemon2436-21rtkit-daemon00:02:223
170812452,37sleep10-21swapper/121:26:131
170812452,37sleep10-21swapper/121:26:131
531599444,35cyclictest0-21swapper/322:36:223
531599443,26cyclictest0-21swapper/319:32:153
531599442,36cyclictest0-21swapper/322:27:083
5314994421,17cyclictest0-21swapper/222:32:192
531499442,36cyclictest0-21swapper/222:12:302
531499442,36cyclictest0-21swapper/222:07:212
205582442,9sleep30-21swapper/320:24:323
192312442,9sleep30-21swapper/321:28:083
89112432,9sleep30-21swapper/320:02:033
5315994322,15cyclictest0-21swapper/321:03:593
5315994320,10cyclictest0-21swapper/318:52:163
531499434,32cyclictest12572-21taskset23:32:082
531499433,30cyclictest0-21swapper/219:52:252
5314994325,12cyclictest0-21swapper/221:07:342
531099435,23cyclictest0-21swapper/119:12:371
531099435,23cyclictest0-21swapper/119:12:371
531099434,21cyclictest0-21swapper/122:22:101
14822432,35sleep30-21swapper/319:46:273
531599423,24cyclictest0-21swapper/319:17:353
531599423,24cyclictest0-21swapper/319:17:353
531499423,33cyclictest0-21swapper/222:37:082
531599393,30cyclictest5387-21cat18:47:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional