You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 04:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Thu Dec 04, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
297599910940,13cyclictest0-21swapper/221:53:082
297599910040,10cyclictest18985-21crond21:47:532
297599910040,10cyclictest18985-21crond21:47:532
29759999832,13cyclictest0-21swapper/219:38:292
2975199973,46cyclictest0-21swapper/022:46:080
29751999535,10cyclictest17606-21tune2fs19:28:170
29751999136,10cyclictest18346-21tr20:38:110
29759998613,16cyclictest0-21swapper/220:43:302
29751998632,10cyclictest2453-21head22:18:160
2975199853,42cyclictest0-21swapper/020:47:560
29751998524,12cyclictest0-21swapper/023:07:510
29751998238,10cyclictest0-21swapper/021:08:030
29759998035,17cyclictest0-21swapper/223:16:362
29759996738,11cyclictest14686-21find20:28:272
2975599654,53cyclictest0-21swapper/123:02:511
2976099644,41cyclictest0-21swapper/320:58:193
2976099644,41cyclictest0-21swapper/320:58:193
29759996432,14cyclictest0-21swapper/223:18:042
2975599644,11cyclictest5044-21tail21:18:081
29760996226,29cyclictest1369-21runrttasks21:53:263
29759996137,11cyclictest0-21swapper/218:58:362
29759996133,11cyclictest0-21swapper/222:14:532
29759996132,13cyclictest0-21swapper/221:33:062
29759996131,13cyclictest0-21swapper/222:53:142
2975599614,11cyclictest14397-21aten2.4-expect23:53:031
2975599614,11cyclictest14397-21aten2.4-expect23:53:031
2975599604,49cyclictest0-21swapper/119:48:111
2975599604,12cyclictest0-21swapper/120:18:431
2975599604,12cyclictest0-21swapper/120:18:431
2975599595,12cyclictest0-21swapper/123:10:221
250332597,44sleep00-21swapper/019:43:230
29759995831,8cyclictest22962-21mailstats00:08:182
2975599584,47cyclictest20143-21awk22:57:501
29751995839,12cyclictest0-21swapper/020:37:570
29751995836,17cyclictest0-21swapper/000:16:230
117692582,48sleep00-21swapper/023:47:500
86362573,9sleep38640-21swap23:38:223
29755995722,12cyclictest0-21swapper/121:08:171
2975199573,46cyclictest0-21swapper/023:38:060
205772573,11sleep30-21swapper/300:03:213
2975599564,29cyclictest0-21swapper/121:23:201
2975599564,28cyclictest0-21swapper/119:33:001
29755995622,12cyclictest0-21swapper/122:07:531
29751995625,13cyclictest0-21swapper/022:53:190
2976099555,34cyclictest0-21swapper/323:29:093
2976099554,29cyclictest0-21swapper/320:23:273
29759995531,8cyclictest10963-21open_inodes21:28:242
29759995510,28cyclictest30433-21grep22:08:262
29759995510,20cyclictest0-21swapper/221:42:352
2975599554,29cyclictest0-21swapper/119:54:491
2975599554,28cyclictest0-21swapper/123:38:171
2975599554,28cyclictest0-21swapper/123:13:161
29755995520,13cyclictest0-21swapper/100:09:161
2976099544,31cyclictest0-21swapper/321:43:263
2976099544,31cyclictest0-21swapper/321:43:263
2975999544,31cyclictest0-21swapper/219:53:182
2975999544,30cyclictest0-21swapper/220:58:112
2975999544,30cyclictest0-21swapper/220:58:112
29759995431,11cyclictest0-21swapper/219:04:582
2975599545,27cyclictest0-21swapper/120:48:071
29755995440,8cyclictest2753-21cpuspeed_turbos23:28:041
2975599544,28cyclictest0-21swapper/120:33:331
2975599544,13cyclictest0-21swapper/121:16:321
2975599543,9cyclictest0-21swapper/122:18:131
2975599543,10cyclictest0-21swapper/122:23:121
2975599543,10cyclictest0-21swapper/122:23:121
2975199544,43cyclictest0-21swapper/023:37:490
2975199543,45cyclictest0-21swapper/023:58:000
2975199543,45cyclictest0-21swapper/020:08:100
29751995424,13cyclictest0-21swapper/019:27:590
243799540,5rtkit-daemon2436-21rtkit-daemon00:09:590
203342543,9sleep10-21swapper/119:33:241
203342543,9sleep10-21swapper/119:33:241
29760995324,10cyclictest14206-21munin-run21:37:543
2975999535,25cyclictest0-21swapper/220:23:302
2975999534,28cyclictest0-21swapper/219:28:202
29759995330,10cyclictest0-21swapper/219:26:582
29759995320,13cyclictest0-21swapper/222:48:082
29759995320,13cyclictest0-21swapper/200:14:172
2975999532,36cyclictest0-21swapper/219:18:112
29759995313,23cyclictest26379-21munin-node22:03:062
2975599535,26cyclictest0-21swapper/120:55:001
2975599535,26cyclictest0-21swapper/120:13:321
2975599534,27cyclictest0-21swapper/122:28:281
2975599534,27cyclictest0-21swapper/122:28:281
2975599534,26cyclictest0-21swapper/121:07:561
2975599533,9cyclictest0-21swapper/100:13:061
2975199533,43cyclictest0-21swapper/021:23:250
5432522,43sleep00-21swapper/022:13:580
2976099523,27cyclictest0-21swapper/319:39:473
29760995221,13cyclictest0-21swapper/320:08:253
2976099522,44cyclictest0-21swapper/323:58:023
2975999524,29cyclictest0-21swapper/223:43:252
2975999524,29cyclictest0-21swapper/220:33:192
29759995229,9cyclictest22840-21munin-node20:48:162
29759995229,11cyclictest0-21swapper/221:09:192
29759995222,14cyclictest0-21swapper/223:53:072
29759995222,14cyclictest0-21swapper/223:53:072
29759995219,13cyclictest0-21swapper/223:08:082
2975599524,26cyclictest0-21swapper/123:03:251
2975599523,9cyclictest0-21swapper/119:23:001
2975599523,8cyclictest25004-21idleruntime19:43:221
2975599523,12cyclictest0-21swapper/119:08:221
29755995228,15cyclictest0-21swapper/122:43:201
29755995226,9cyclictest5141-21sendmail_mailqu20:08:271
29755995222,12cyclictest0-21swapper/122:13:261
29751995234,11cyclictest13881-21unixbench_singl19:18:320
2975199523,42cyclictest0-21swapper/019:57:580
29751995223,12cyclictest0-21swapper/019:17:590
264142524,40sleep00-21swapper/022:03:060
212422523,9sleep00-21swapper/000:07:170
79482516,10sleep00-21swapper/022:29:500
79482516,10sleep00-21swapper/022:29:500
69672513,9sleep30-21swapper/321:22:063
305222512,42sleep00-21swapper/023:18:110
2976099514,29cyclictest0-21swapper/319:08:333
2976099514,28cyclictest0-21swapper/322:18:103
2976099514,27cyclictest0-21swapper/322:12:533
2976099514,21cyclictest0-21swapper/323:43:153
29760995134,10cyclictest30325-21tr18:48:143
29760995134,10cyclictest30325-21tr18:48:143
2975999514,31cyclictest0-21swapper/220:38:102
2975999514,28cyclictest0-21swapper/219:58:372
2975999514,28cyclictest0-21swapper/219:08:122
2975999514,27cyclictest0-21swapper/220:13:102
2975999513,28cyclictest0-21swapper/223:23:172
2975999513,28cyclictest0-21swapper/222:18:232
2975999513,28cyclictest0-21swapper/219:48:292
29759995129,10cyclictest0-21swapper/222:41:392
29759995129,10cyclictest0-21swapper/218:53:132
29759995128,11cyclictest0-21swapper/220:07:192
29759995128,11cyclictest0-21swapper/219:33:112
29759995128,11cyclictest0-21swapper/219:33:112
29759995119,12cyclictest0-21swapper/222:23:212
29759995119,12cyclictest0-21swapper/222:23:212
2975599515,31cyclictest0-21swapper/100:03:011
2975599515,27cyclictest0-21swapper/122:02:531
2975599515,27cyclictest0-21swapper/119:04:411
29755995132,10cyclictest12042-21grep20:23:211
2975599513,9cyclictest0-21swapper/122:38:171
2975599513,8cyclictest20486-21ntp_kernel_pll_21:48:231
29755995121,14cyclictest0-21swapper/122:33:241
29755995118,12cyclictest31237-21munin-node19:58:221
2975199514,41cyclictest0-21swapper/023:28:030
29751995136,10cyclictest0-21swapper/021:57:540
2975199513,43cyclictest0-21swapper/020:13:300
2975199513,40cyclictest1369-21runrttasks18:48:140
2975199513,40cyclictest1369-21runrttasks18:48:140
294492514,6sleep00-21swapper/018:44:270
201942514,12sleep10-21swapper/120:42:501
163152512,9sleep30-21swapper/323:55:143
163152512,9sleep30-21swapper/323:55:143
43032504,9sleep00-21swapper/021:13:320
2976099505,28cyclictest0-21swapper/321:28:293
2976099505,26cyclictest0-21swapper/319:29:453
2976099505,25cyclictest0-21swapper/322:03:063
2976099504,29cyclictest0-21swapper/322:53:273
2976099504,29cyclictest0-21swapper/320:28:153
2976099504,29cyclictest0-21swapper/320:18:113
2976099504,29cyclictest0-21swapper/320:18:113
2976099504,28cyclictest0-21swapper/321:13:183
2976099504,28cyclictest0-21swapper/320:43:163
2976099504,21cyclictest0-21swapper/319:04:413
29760995020,13cyclictest0-21swapper/323:48:213
29760995019,13cyclictest0-21swapper/320:53:153
2975999505,25cyclictest0-21swapper/222:43:252
2975999504,38cyclictest0-21swapper/223:47:532
2975999504,30cyclictest0-21swapper/222:33:062
2975999504,28cyclictest0-21swapper/223:00:212
2975999504,28cyclictest0-21swapper/221:03:122
2975999504,27cyclictest0-21swapper/222:32:532
2975999504,27cyclictest0-21swapper/222:32:532
2975999504,27cyclictest0-21swapper/219:13:212
2975999503,28cyclictest0-21swapper/219:43:232
29759995029,10cyclictest0-21swapper/221:13:162
29759995021,14cyclictest0-21swapper/221:58:092
29759995017,13cyclictest0-21swapper/223:59:482
2975599505,27cyclictest0-21swapper/118:59:401
2975599505,26cyclictest0-21swapper/119:39:471
2975599504,9cyclictest0-21swapper/118:51:301
2975599504,9cyclictest0-21swapper/118:51:301
2975599504,27cyclictest0-21swapper/123:48:121
2975599504,27cyclictest0-21swapper/122:08:161
29755995023,10cyclictest10317-21fschecks_time23:43:101
29755995016,12cyclictest0-21swapper/123:19:301
29755995015,12cyclictest0-21swapper/122:48:161
2975199504,39cyclictest0-21swapper/022:58:010
2975199503,42cyclictest0-21swapper/019:03:090
2975199503,41cyclictest0-21swapper/023:54:140
2975199503,41cyclictest0-21swapper/023:54:140
2975199503,41cyclictest0-21swapper/021:33:050
2975199503,28cyclictest113582grep21:28:300
29751995024,12cyclictest0-21swapper/022:23:080
29751995024,12cyclictest0-21swapper/022:23:080
29751995024,12cyclictest0-21swapper/020:57:500
2975199502,43cyclictest0-21swapper/020:58:090
2975199502,43cyclictest0-21swapper/020:58:090
284712502,43sleep00-21swapper/019:51:170
323022493,41sleep10-21swapper/123:22:561
2976099495,27cyclictest0-21swapper/322:13:243
2976099495,27cyclictest0-21swapper/320:33:123
2976099494,28cyclictest0-21swapper/322:33:223
2976099494,28cyclictest0-21swapper/321:48:163
2976099494,28cyclictest0-21swapper/318:58:123
2976099494,28cyclictest0-21swapper/318:53:283
2976099494,27cyclictest0-21swapper/319:43:163
29760994932,10cyclictest15872-21cpuspeed_turbos22:48:053
29760994931,12cyclictest23832-21nscd20:07:573
2976099493,27cyclictest0-21swapper/322:27:533
2976099493,27cyclictest0-21swapper/322:27:533
2975999495,26cyclictest0-21swapper/223:03:012
2975999494,30cyclictest0-21swapper/221:23:102
2975999494,27cyclictest0-21swapper/220:54:282
2975999494,26cyclictest0-21swapper/218:48:112
2975999494,26cyclictest0-21swapper/218:48:112
2975999494,25cyclictest0-21swapper/221:18:242
29759994926,11cyclictest0-21swapper/221:48:392
29759994921,13cyclictest0-21swapper/223:28:232
29759994917,13cyclictest0-21swapper/200:03:022
2975599494,26cyclictest0-21swapper/123:58:001
2975599494,26cyclictest0-21swapper/120:04:511
29755994922,12cyclictest0-21swapper/121:33:171
29755994916,10cyclictest0-21swapper/123:34:441
2975199495,37cyclictest0-21swapper/021:18:060
29751994937,8cyclictest12674-21grep23:48:090
29751994932,11cyclictest20422-21grep21:48:210
2975199493,40cyclictest0-21swapper/021:58:050
2975199493,40cyclictest0-21swapper/020:48:070
2975199493,40cyclictest0-21swapper/019:33:130
2975199493,40cyclictest0-21swapper/019:33:130
2975199493,39cyclictest0-21swapper/023:08:010
2975199493,12cyclictest0-21swapper/018:58:330
29751994922,12cyclictest0-21swapper/020:18:210
29751994922,12cyclictest0-21swapper/020:18:210
2976099485,27cyclictest0-21swapper/323:02:303
2976099484,27cyclictest0-21swapper/323:13:143
2976099484,27cyclictest0-21swapper/319:23:253
2976099484,26cyclictest0-21swapper/322:02:543
2976099484,26cyclictest0-21swapper/320:02:583
29760994817,14cyclictest0-21swapper/323:34:083
29760994817,11cyclictest0-21swapper/320:13:193
2975999484,26cyclictest0-21swapper/223:34:102
2975999484,25cyclictest0-21swapper/223:38:032
2975599486,34cyclictest0-21swapper/120:47:551
2975599484,26cyclictest0-21swapper/121:00:011
2975599484,26cyclictest0-21swapper/121:00:011
2975599484,26cyclictest0-21swapper/119:23:141
29755994828,14cyclictest0-21swapper/121:38:151
29755994818,12cyclictest0-21swapper/118:54:391
2975199483,40cyclictest0-21swapper/022:36:300
29751994821,12cyclictest0-21swapper/019:58:130
236572482,39sleep00-21swapper/019:40:490
18922482,40sleep00-21swapper/018:54:210
187042482,40sleep00-21swapper/021:44:200
187042482,40sleep00-21swapper/021:44:200
120802482,9sleep00-21swapper/022:38:210
93192472,9sleep00-21swapper/019:10:420
2976099475,25cyclictest0-21swapper/323:20:243
2976099474,26cyclictest0-21swapper/321:42:423
2976099474,26cyclictest0-21swapper/300:13:033
2976099473,24cyclictest28614-21munin-run19:52:583
29760994719,12cyclictest0-21swapper/319:53:103
29760994718,13cyclictest0-21swapper/321:25:043
2975999474,28cyclictest0-21swapper/220:08:112
2975999473,26cyclictest0-21swapper/220:18:152
2975999473,26cyclictest0-21swapper/220:18:152
2975599474,25cyclictest0-21swapper/121:28:071
29755994721,11cyclictest0-21swapper/121:43:281
29755994721,11cyclictest0-21swapper/121:43:281
2975199473,37cyclictest0-21swapper/020:23:100
29751994720,10cyclictest0-21swapper/023:23:000
29751994718,12cyclictest0-21swapper/020:28:100
2976099464,26cyclictest0-21swapper/322:30:163
2976099464,26cyclictest0-21swapper/322:30:163
2976099464,26cyclictest0-21swapper/320:49:593
29755994616,13cyclictest0-21swapper/119:13:311
2975199463,38cyclictest0-21swapper/022:48:310
2975199463,37cyclictest0-21swapper/022:08:200
2975199463,36cyclictest0-21swapper/021:03:120
2975199463,11cyclictest0-21swapper/023:13:320
2975199462,38cyclictest0-21swapper/021:38:050
295892464,6sleep10-21swapper/118:46:171
216002463,37sleep10-21swapper/121:53:031
2976099454,24cyclictest0-21swapper/323:08:133
2976099454,13cyclictest0-21swapper/321:08:253
29760994527,11cyclictest21878-21cpuspeed_turbos00:08:023
29760994527,11cyclictest20474-21head19:33:253
29760994527,11cyclictest20474-21head19:33:253
29760994517,12cyclictest0-21swapper/323:05:283
29760994515,13cyclictest0-21swapper/320:38:253
2975999454,35cyclictest0-21swapper/218:48:012
2976099444,33cyclictest0-21swapper/322:38:133
2976099444,33cyclictest0-21swapper/321:03:293
2976099444,24cyclictest0-21swapper/322:43:043
29760994415,12cyclictest0-21swapper/319:18:103
2975599444,21cyclictest15473-21cat20:32:571
115552442,35sleep30-21swapper/319:13:473
2976099434,33cyclictest0-21swapper/323:23:063
2975199439,27cyclictest1293-21telnet20:03:080
2976099313,22cyclictest29953-21cat18:48:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional