You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 06:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Mon Sep 15, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
223099820,4rtkit-daemon2229-21rtkit-daemon19:34:403
223099810,4rtkit-daemon2229-21rtkit-daemon18:29:213
223099770,7rtkit-daemon0-21swapper/022:41:180
11308997130,26cyclictest23949-21aten_r1power_cu22:26:040
11308997130,26cyclictest23949-21aten_r1power_cu22:26:040
11308997020,42cyclictest0-21swapper/020:42:370
223099690,6rtkit-daemon9920-21tr21:56:050
1130899696,56cyclictest0-21swapper/020:51:110
11308996917,45cyclictest23033-21chrt17:48:340
11308996822,39cyclictest1265-21runrttasks20:26:100
11308996818,18cyclictest0-21swapper/018:11:170
11308996818,18cyclictest0-21swapper/018:11:170
1130899679,52cyclictest0-21swapper/020:46:140
1130899674,54cyclictest0-21swapper/017:56:000
11308996720,25cyclictest1160-21snmpd19:22:550
11308996719,42cyclictest30872-21munin-run22:40:500
11314996622,38cyclictest10384-21taskset20:48:173
11308996623,36cyclictest13346-21seq20:56:070
11308996517,32cyclictest0-21swapper/022:31:180
11308996516,31cyclictest0-21swapper/020:15:550
11308996511,47cyclictest0-21swapper/020:21:180
11308996511,47cyclictest0-21swapper/020:21:180
1130899649,48cyclictest0-21swapper/021:13:220
1130899644,52cyclictest0-21swapper/020:36:080
11308996421,36cyclictest2366-21sendmail19:25:590
11308996419,38cyclictest0-21swapper/020:06:190
11308996417,42cyclictest7685-21awk18:26:000
11308996416,42cyclictest0-21swapper/018:57:430
11308996416,42cyclictest0-21swapper/018:57:430
11308996416,41cyclictest0-21swapper/022:06:080
1131499638,46cyclictest0-21swapper/319:20:583
11314996313,43cyclictest21073-21munin-node17:46:333
11314996312,43cyclictest0-21swapper/319:22:453
11308996319,26cyclictest0-21swapper/021:01:100
11308996319,26cyclictest0-21swapper/021:01:100
11308996318,38cyclictest1160-21snmpd19:58:020
11308996317,39cyclictest0-21swapper/018:51:110
11308996317,39cyclictest0-21swapper/018:51:110
11308996310,34cyclictest0-21swapper/019:41:100
223099620,7rtkit-daemon0-21swapper/220:24:313
223099620,7rtkit-daemon0-21swapper/220:24:313
11308996223,32cyclictest12915-21sed19:46:180
11308996217,39cyclictest25313-21aten2.4-expect19:06:100
11308996217,36cyclictest1160-21snmpd22:12:070
11308996217,28cyclictest0-21swapper/019:11:100
11308996216,40cyclictest0-21swapper/019:36:190
11308996216,40cyclictest0-21swapper/019:36:190
11308996215,40cyclictest0-21swapper/022:20:510
11308996210,44cyclictest0-21swapper/017:56:160
223099610,7rtkit-daemon0-21swapper/219:11:263
11314996143,11cyclictest3905-21gltestperf18:16:193
11314996125,30cyclictest23565-21gltestperf21:16:133
11314996115,39cyclictest17226-21munin-run18:45:583
11314996111,43cyclictest0-21swapper/317:46:013
1130899614,49cyclictest0-21swapper/019:51:260
11308996118,30cyclictest0-21swapper/019:17:550
11308996115,17cyclictest1160-21snmpd21:53:210
1131499607,41cyclictest0-21swapper/318:23:313
1131399602,52cyclictest0-21swapper/219:32:572
1131299604,49cyclictest0-21swapper/122:36:211
1131299604,48cyclictest0-21swapper/121:46:201
11308996033,20cyclictest0-21swapper/017:26:020
11308996016,36cyclictest4902-21munin-node19:31:150
11308996015,30cyclictest0-21swapper/022:46:030
223099590,4rtkit-daemon2229-21rtkit-daemon20:43:302
1131499598,42cyclictest0-21swapper/319:30:573
11314995918,33cyclictest1260-21aten_r1power_po20:31:053
11314995913,39cyclictest0-21swapper/318:36:153
11314995911,41cyclictest0-21swapper/318:11:013
1131299594,48cyclictest0-21swapper/118:46:281
1130899596,46cyclictest0-21swapper/021:48:070
11308995924,17cyclictest0-21swapper/021:26:090
11308995923,29cyclictest655-21fschecks_time21:36:130
11308995918,34cyclictest1265-21runrttasks22:51:030
11308995917,35cyclictest1160-21snmpd20:33:080
11308995914,38cyclictest0-21swapper/020:03:370
11308995912,40cyclictest0-21swapper/021:21:100
11308995911,32cyclictest0-21swapper/021:06:250
223099580,6rtkit-daemon0-21swapper/220:13:013
223099580,5rtkit-daemon2229-21rtkit-daemon20:15:012
11314995816,34cyclictest18909-21crond22:15:503
1131299584,28cyclictest0-21swapper/122:21:041
1131299583,47cyclictest0-21swapper/118:11:261
1131299583,47cyclictest0-21swapper/118:11:261
11308995841,12cyclictest8793-21tail18:26:230
11308995823,29cyclictest1160-21snmpd17:32:380
11308995818,16cyclictest0-21swapper/022:21:220
11308995816,36cyclictest15099-21chrt18:41:110
11308995814,37cyclictest0-21swapper/021:41:250
11308995814,37cyclictest0-21swapper/021:41:250
11308995811,40cyclictest0-21swapper/018:31:150
223099570,7rtkit-daemon0-21swapper/221:33:133
223099570,5rtkit-daemon2229-21rtkit-daemon18:22:282
11314995746,6cyclictest91662sleep321:53:273
11314995720,18cyclictest30201-21switchtime22:36:233
11314995716,35cyclictest0-21swapper/321:15:303
11314995715,33cyclictest0-21swapper/322:06:053
1131299574,45cyclictest16276-21needreboot18:41:251
1131299573,28cyclictest0-21swapper/120:35:231
1130899579,31cyclictest0-21swapper/018:49:310
1130899578,41cyclictest0-21swapper/022:02:040
1130899574,28cyclictest0-21swapper/017:41:270
11308995718,25cyclictest1160-21snmpd21:18:100
11308995712,39cyclictest0-21swapper/018:37:480
11308995712,39cyclictest0-21swapper/018:16:310
223099560,5rtkit-daemon2229-21rtkit-daemon18:12:362
223099560,5rtkit-daemon2229-21rtkit-daemon18:12:362
223099560,4rtkit-daemon2229-21rtkit-daemon18:29:502
1131499564,38cyclictest0-21swapper/320:16:173
11314995613,36cyclictest29706-21latency_hist21:30:523
11314995610,38cyclictest11579-21perl20:51:123
1131299565,44cyclictest0-21swapper/118:51:101
1131299565,44cyclictest0-21swapper/118:51:101
1131299563,46cyclictest0-21swapper/119:46:191
1130899567,43cyclictest0-21swapper/018:07:430
11308995618,32cyclictest0-21swapper/020:16:400
11308995614,36cyclictest0-21swapper/017:36:280
223099550,6rtkit-daemon0-21swapper/220:40:003
223099550,5rtkit-daemon2229-21rtkit-daemon18:59:342
223099550,5rtkit-daemon2229-21rtkit-daemon18:59:342
223099550,5rtkit-daemon2229-21rtkit-daemon17:38:082
223099550,4rtkit-daemon2229-21rtkit-daemon20:23:392
223099550,4rtkit-daemon2229-21rtkit-daemon20:23:392
223099550,4rtkit-daemon2229-21rtkit-daemon18:47:052
223099550,4rtkit-daemon2229-21rtkit-daemon17:34:082
20802552,45sleep10-21swapper/121:40:071
1131499558,39cyclictest0-21swapper/317:51:203
1131499556,40cyclictest19224-21munin-node20:01:183
1131499554,43cyclictest0-21swapper/321:06:273
11314995516,24cyclictest19839-21munin-node18:51:273
11314995516,24cyclictest19839-21munin-node18:51:273
11314995514,26cyclictest16841-21munin-node19:56:173
11314995511,37cyclictest22920-21aten2.4-expect19:01:063
11314995510,38cyclictest0-21swapper/319:55:563
11314995510,24cyclictest11829-21crond19:45:563
1131399552,47cyclictest0-21swapper/220:31:062
1131299554,28cyclictest0-21swapper/121:01:061
1131299554,28cyclictest0-21swapper/121:01:061
11312995536,11cyclictest14045-21gltestperf20:56:141
11308995525,24cyclictest31140-21grep21:31:200
11308995513,28cyclictest0-21swapper/019:01:190
11308995512,37cyclictest0-21swapper/017:30:350
11308995512,37cyclictest0-21swapper/017:30:350
223099540,6rtkit-daemon2229-21rtkit-daemon21:23:293
1131499545,31cyclictest0-21swapper/322:46:013
11314995436,12cyclictest26925-21unixbench_multi19:06:313
11314995433,15cyclictest0-21swapper/317:35:243
11314995424,24cyclictest6973-21hwlatdetect20:41:163
11314995412,35cyclictest14129-21hostname20:56:153
11314995412,26cyclictest9593-21munin-node21:56:123
11314995411,35cyclictest759-21munin-node21:36:143
11314995410,38cyclictest17582-21ntpq17:36:293
11313995436,11cyclictest21579-21head18:51:322
11313995436,11cyclictest21579-21head18:51:322
1131399543,43cyclictest0-21swapper/219:51:282
1131399543,29cyclictest0-21swapper/221:56:172
1131299543,45cyclictest0-21swapper/119:56:221
1131299543,44cyclictest8512-21uniq18:26:201
1131299543,44cyclictest3357-21munin-run22:50:481
11308995434,15cyclictest0-21swapper/018:01:160
295162532,9sleep20-21swapper/219:12:542
254852532,9sleep20-21swapper/217:54:302
223099530,7rtkit-daemon2229-21rtkit-daemon18:47:503
11314995316,30cyclictest0-21swapper/322:01:373
11314995315,32cyclictest5696-21sshd22:55:183
11314995314,33cyclictest2423-21munin-node21:41:053
11314995314,33cyclictest2423-21munin-node21:41:053
11314995313,32cyclictest22819-21munin-node22:21:193
11314995310,29cyclictest24960-21iostat22:26:153
11314995310,29cyclictest24960-21iostat22:26:153
1131399534,42cyclictest0-21swapper/218:06:342
11312995336,10cyclictest26207-21tr17:56:151
1131299533,45cyclictest0-21swapper/117:31:371
1131299533,45cyclictest0-21swapper/117:26:251
1131299533,45cyclictest0-21swapper/117:26:251
1131299533,44cyclictest0-21swapper/119:25:571
1131299533,44cyclictest0-21swapper/118:16:281
1131299533,44cyclictest0-21swapper/118:06:311
11312995322,13cyclictest0-21swapper/122:51:201
115872522,43sleep10-21swapper/121:58:491
1131499524,31cyclictest0-21swapper/317:31:013
1131499524,31cyclictest0-21swapper/317:31:013
11314995222,23cyclictest13271-21needreboot19:46:233
11314995221,25cyclictest1265-21runrttasks18:00:153
11314995215,29cyclictest1265-21runrttasks21:01:103
11314995215,29cyclictest1265-21runrttasks21:01:103
11314995211,27cyclictest11558-21ntp_kernel_pll_18:31:263
1131399524,29cyclictest0-21swapper/220:36:232
1131399524,12cyclictest0-21swapper/222:31:182
1131399523,38cyclictest30382-21crond18:06:002
1131399522,44cyclictest0-21swapper/218:41:302
1131299524,40cyclictest0-21swapper/120:01:281
1131299524,30cyclictest0-21swapper/121:06:141
1131299524,29cyclictest0-21swapper/120:36:201
1131299523,44cyclictest0-21swapper/120:16:201
1131299523,43cyclictest0-21swapper/121:41:181
1131299523,43cyclictest0-21swapper/121:41:181
1131299523,43cyclictest0-21swapper/117:36:131
1131299523,42cyclictest0-21swapper/121:26:051
11312995220,13cyclictest0-21swapper/119:36:131
11312995220,13cyclictest0-21swapper/119:36:131
230192512,41sleep10-21swapper/121:16:081
223099510,5rtkit-daemon2229-21rtkit-daemon17:21:232
11314995132,13cyclictest31713-21grep20:26:133
11314995123,22cyclictest28504-21cpuspeed_turbos18:01:153
11314995122,23cyclictest19508-21cpuspeed_turbos22:16:063
11314995120,25cyclictest7934-21fschecks_count19:36:143
11314995120,25cyclictest7934-21fschecks_count19:36:143
11314995111,33cyclictest21707-21telnet20:06:093
1131399514,39cyclictest0-21swapper/222:11:162
1131399514,29cyclictest0-21swapper/221:46:162
1131399514,29cyclictest0-21swapper/221:36:172
1131399514,28cyclictest0-21swapper/222:40:502
1131399514,28cyclictest0-21swapper/217:46:232
1131399514,27cyclictest0-21swapper/221:21:042
1131399512,43cyclictest0-21swapper/220:16:292
1131399512,43cyclictest0-21swapper/219:46:212
1131399512,43cyclictest0-21swapper/219:30:582
1131399512,43cyclictest0-21swapper/219:16:212
11313995118,13cyclictest0-21swapper/218:16:202
1131299518,13cyclictest0-21swapper/119:26:281
1131299515,29cyclictest0-21swapper/122:41:131
1131299514,40cyclictest0-21swapper/120:50:531
1131299514,39cyclictest0-21swapper/122:16:151
1131299514,29cyclictest0-21swapper/122:01:111
1131299514,29cyclictest0-21swapper/119:06:191
11312995134,11cyclictest14450-21unixbench_singl18:36:331
1131299513,9cyclictest0-21swapper/117:52:421
1131299513,43cyclictest0-21swapper/120:26:291
1131299513,43cyclictest0-21swapper/119:11:221
1131299513,43cyclictest0-21swapper/118:21:241
258772502,9sleep10-21swapper/122:28:071
258772502,9sleep10-21swapper/122:28:071
246762502,9sleep20-21swapper/221:17:422
223099500,4rtkit-daemon2229-21rtkit-daemon18:58:193
223099500,4rtkit-daemon2229-21rtkit-daemon18:58:193
163692502,41sleep10-21swapper/119:52:561
11314995018,13cyclictest0-21swapper/322:45:453
11314995010,34cyclictest1437-21sed18:11:193
11314995010,34cyclictest1437-21sed18:11:193
1131399504,38cyclictest0-21swapper/219:36:302
1131399504,38cyclictest0-21swapper/219:36:302
1131399504,28cyclictest0-21swapper/222:46:182
1131399504,28cyclictest0-21swapper/221:06:312
1131399504,28cyclictest0-21swapper/218:36:322
1131399504,28cyclictest0-21swapper/218:31:262
1131399504,28cyclictest0-21swapper/217:41:392
1131399503,39cyclictest0-21swapper/221:11:112
1131399503,38cyclictest0-21swapper/221:01:052
1131399503,38cyclictest0-21swapper/221:01:052
11313995023,7cyclictest13680-21latency_hist17:31:022
11313995023,7cyclictest13680-21latency_hist17:31:022
1131299504,28cyclictest0-21swapper/121:22:101
1131299504,28cyclictest0-21swapper/120:51:161
1131299504,28cyclictest0-21swapper/119:41:311
1131299504,28cyclictest0-21swapper/119:31:181
11312995032,11cyclictest26623-21cpuspeed_turbos22:31:051
11312995024,13cyclictest0-21swapper/122:06:141
102302504,6sleep10-21swapper/117:21:251
317502492,40sleep10-21swapper/121:31:321
11314994934,9cyclictest5169-21taskset21:46:063
1131399494,39cyclictest0-21swapper/220:26:122
1131399494,38cyclictest0-21swapper/219:21:212
1131399494,37cyclictest0-21swapper/221:41:092
1131399494,37cyclictest0-21swapper/221:41:092
1131399494,28cyclictest0-21swapper/222:27:092
1131399494,28cyclictest0-21swapper/222:27:092
1131399494,27cyclictest0-21swapper/221:51:222
1131399494,27cyclictest0-21swapper/221:31:072
1131399494,27cyclictest0-21swapper/220:56:152
11313994931,12cyclictest0-21swapper/219:56:252
1131399493,38cyclictest21650-21aten2.4-expect22:21:052
11313994916,26cyclictest0-21swapper/221:26:192
1131299494,29cyclictest0-21swapper/120:21:151
1131299494,29cyclictest0-21swapper/120:21:151
206652482,38sleep10-21swapper/117:43:531
1131399484,27cyclictest0-21swapper/222:07:062
1131399484,27cyclictest0-21swapper/220:03:022
1131399484,26cyclictest0-21swapper/220:06:082
1131399484,24cyclictest0-21swapper/217:56:132
11313994831,11cyclictest32265-21wc22:41:152
1131399482,41cyclictest0-21swapper/222:01:242
1131299484,37cyclictest0-21swapper/121:51:131
1131299483,39cyclictest17253-21df_abs22:11:071
1131299483,39cyclictest0-21swapper/118:56:001
1131299483,39cyclictest0-21swapper/118:56:001
11312994818,14cyclictest0-21swapper/121:11:151
11312994818,14cyclictest0-21swapper/120:41:221
260002472,37sleep10-21swapper/120:14:081
115602472,9sleep20-21swapper/219:42:292
11314994714,26cyclictest0-21swapper/322:31:203
1131399475,35cyclictest0-21swapper/220:46:132
1131399474,26cyclictest0-21swapper/219:07:532
1131399473,38cyclictest0-21swapper/222:52:132
1131399473,37cyclictest0-21swapper/222:16:032
1131399473,27cyclictest0-21swapper/219:05:582
11313994723,10cyclictest0-21swapper/220:51:102
1131299474,28cyclictest0-21swapper/117:46:191
1131299474,27cyclictest0-21swapper/119:02:521
1131299474,24cyclictest0-21swapper/119:16:111
1131299453,9cyclictest0-21swapper/118:01:361
1131299453,10cyclictest1586-21nscd20:06:091
1131299443,10cyclictest0-21swapper/118:31:341
11314994323,14cyclictest0-21swapper/317:26:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional