You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 12:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Tue Dec 30, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
248599750,4rtkit-daemon2484-21rtkit-daemon21:39:300
248599700,5rtkit-daemon2484-21rtkit-daemon23:40:020
248599690,5rtkit-daemon2484-21rtkit-daemon21:58:210
248599690,4rtkit-daemon2484-21rtkit-daemon21:34:450
248599690,4rtkit-daemon2484-21rtkit-daemon20:59:310
248599690,4rtkit-daemon2484-21rtkit-daemon20:59:310
248599680,5rtkit-daemon2484-21rtkit-daemon19:14:430
248599670,4rtkit-daemon2484-21rtkit-daemon20:00:040
2465299674,55cyclictest0-21swapper/119:24:271
248599660,5rtkit-daemon2484-21rtkit-daemon20:24:550
248599660,4rtkit-daemon2484-21rtkit-daemon21:44:440
248599660,4rtkit-daemon2484-21rtkit-daemon21:44:440
248599660,4rtkit-daemon2484-21rtkit-daemon19:54:490
248599650,4rtkit-daemon2484-21rtkit-daemon23:19:410
248599650,4rtkit-daemon2484-21rtkit-daemon23:14:110
248599650,4rtkit-daemon2484-21rtkit-daemon23:14:110
248599650,4rtkit-daemon2484-21rtkit-daemon21:51:290
248599650,4rtkit-daemon2484-21rtkit-daemon19:44:490
248599650,4rtkit-daemon2484-21rtkit-daemon19:19:580
248599640,4rtkit-daemon2484-21rtkit-daemon20:14:330
248599640,4rtkit-daemon2484-21rtkit-daemon18:56:590
248599640,4rtkit-daemon2484-21rtkit-daemon18:56:590
248599640,4rtkit-daemon15-21rcuc/022:24:200
24653996444,13cyclictest0-21swapper/223:19:192
248599630,8rtkit-daemon0-21swapper/021:30:520
248599630,7rtkit-daemon0-21swapper/022:20:430
248599630,4rtkit-daemon3-21ksoftirqd/022:57:040
248599630,4rtkit-daemon3-21ksoftirqd/020:48:240
248599630,3rtkit-daemon2484-21rtkit-daemon22:09:280
248599630,12rtkit-daemon2484-21rtkit-daemon18:56:441
248599630,12rtkit-daemon2484-21rtkit-daemon18:56:441
24653996341,15cyclictest0-21swapper/220:19:362
2465299634,51cyclictest0-21swapper/122:49:311
248599620,9rtkit-daemon0-21swapper/022:42:420
248599620,9rtkit-daemon0-21swapper/019:41:340
248599620,9rtkit-daemon0-21swapper/000:19:010
248599620,8rtkit-daemon0-21swapper/023:49:400
248599620,8rtkit-daemon0-21swapper/021:12:310
248599620,8rtkit-daemon0-21swapper/019:10:200
248599620,7rtkit-daemon0-21swapper/023:31:250
248599620,7rtkit-daemon0-21swapper/021:21:230
248599620,7rtkit-daemon0-21swapper/020:05:330
248599620,7rtkit-daemon0-21swapper/019:50:490
248599620,7rtkit-daemon0-21swapper/019:29:420
248599620,7rtkit-daemon0-21swapper/000:12:010
248599620,6rtkit-daemon0-21swapper/023:47:250
248599620,6rtkit-daemon0-21swapper/023:28:400
248599620,6rtkit-daemon0-21swapper/023:04:560
248599620,6rtkit-daemon0-21swapper/000:07:090
248599620,5rtkit-daemon3-21ksoftirqd/023:56:390
248599620,4rtkit-daemon3-21ksoftirqd/021:16:310
248599620,4rtkit-daemon0-21swapper/019:00:430
248599620,3rtkit-daemon15-21rcuc/020:19:250
248599620,3rtkit-daemon0-21swapper/023:00:040
248599620,3rtkit-daemon0-21swapper/021:04:090
24654996244,12cyclictest6441-21grep22:44:233
24653996239,15cyclictest0-21swapper/219:04:442
2465299624,49cyclictest0-21swapper/120:24:091
248599610,8rtkit-daemon0-21swapper/020:29:100
248599610,8rtkit-daemon0-21swapper/019:07:360
248599610,7rtkit-daemon0-21swapper/023:37:100
248599610,7rtkit-daemon0-21swapper/022:35:120
248599610,7rtkit-daemon0-21swapper/022:14:580
248599610,7rtkit-daemon0-21swapper/022:05:510
248599610,7rtkit-daemon0-21swapper/021:59:360
248599610,7rtkit-daemon0-21swapper/020:49:470
248599610,7rtkit-daemon0-21swapper/019:36:420
248599610,7rtkit-daemon0-21swapper/019:36:420
248599610,7rtkit-daemon0-21swapper/000:15:010
248599610,7rtkit-daemon0-21swapper/000:02:540
248599610,7rtkit-daemon0-21swapper/000:02:540
248599610,6rtkit-daemon0-21swapper/022:30:200
248599610,6rtkit-daemon0-21swapper/022:30:200
248599610,6rtkit-daemon0-21swapper/020:40:390
248599610,6rtkit-daemon0-21swapper/020:36:250
248599610,5rtkit-daemon3-21ksoftirqd/022:51:040
248599610,3rtkit-daemon0-21swapper/019:27:200
248599610,2rtkit-daemon0-21swapper/023:09:110
248599610,2rtkit-daemon0-21swapper/023:09:110
2465499619,45cyclictest0-21swapper/319:34:213
2465499619,45cyclictest0-21swapper/319:34:213
24654996143,12cyclictest10890-21sed23:59:343
24654996143,12cyclictest10890-21sed23:59:343
24654996143,11cyclictest3338-21cut22:39:163
2465499614,50cyclictest0-21swapper/323:34:213
2465499614,49cyclictest0-21swapper/322:34:283
24653996140,14cyclictest0-21swapper/223:54:232
2465299613,51cyclictest11090-21fschecks_count20:39:261
293532600,3sleep229496-21latency_hist21:19:062
248599600,6rtkit-daemon0-21swapper/022:45:420
248599600,6rtkit-daemon0-21swapper/020:54:460
248599600,3rtkit-daemon0-21swapper/021:24:080
24654996040,14cyclictest5278-21gltestperf23:49:203
24653996040,13cyclictest0-21swapper/221:09:072
24653996039,13cyclictest0-21swapper/221:46:562
24653996039,13cyclictest0-21swapper/221:46:562
2465299603,50cyclictest0-21swapper/122:39:031
2465299603,49cyclictest0-21swapper/119:49:101
248599590,6rtkit-daemon0-21swapper/020:10:180
24654995941,12cyclictest3224-21ls19:14:353
24654995940,12cyclictest28467-21ntp_offset21:14:333
24654995939,13cyclictest4850-21cpuspeed_turbos19:19:263
24653995932,20cyclictest20798-21cut23:14:232
24653995932,20cyclictest20798-21cut23:14:232
24653995841,10cyclictest32275-21tr21:24:202
2465299585,46cyclictest0-21swapper/119:19:281
248599570,5rtkit-daemon2484-21rtkit-daemon23:19:181
24654995739,11cyclictest12256-21grep00:04:193
24654995738,13cyclictest16988-21fschecks_count00:14:183
21102575,11sleep20-21swapper/221:29:182
128222573,11sleep20-21swapper/221:49:332
2465499564,44cyclictest0-21swapper/321:59:153
24654995639,11cyclictest8404-21cat22:49:163
24654995638,11cyclictest18648-21ls23:09:233
24654995638,11cyclictest18648-21ls23:09:233
24654995637,12cyclictest10613-21cpu20:39:213
24654995634,16cyclictest31156-21munin-node22:29:183
24654995634,16cyclictest31156-21munin-node22:29:183
2465399565,30cyclictest0-21swapper/220:14:082
248599550,6rtkit-daemon0-21swapper/023:43:401
2465499554,34cyclictest0-21swapper/320:50:253
24654995537,12cyclictest23930-21cpuspeed_turbos19:59:243
24654995537,12cyclictest21551-21timerwakeupswit23:14:363
24654995537,12cyclictest21551-21timerwakeupswit23:14:363
24654995536,12cyclictest28705-21cpuspeed_turbos22:24:183
24654995532,17cyclictest0-21swapper/323:59:003
2465399555,27cyclictest0-21swapper/223:04:142
2465299555,32cyclictest0-21swapper/120:49:351
2465499544,42cyclictest0-21swapper/320:19:383
2465499544,41cyclictest0-21swapper/321:29:213
24654995437,11cyclictest5680-21netstat21:34:313
24654995436,11cyclictest14135-21processes22:59:323
24654995433,15cyclictest0-21swapper/321:19:223
2465399546,41cyclictest0-21swapper/223:24:132
2465399544,42cyclictest0-21swapper/219:59:432
2465299543,45cyclictest0-21swapper/120:12:411
2465499534,41cyclictest0-21swapper/319:29:213
2465499534,40cyclictest0-21swapper/319:24:303
24654995335,12cyclictest1419-21runrttasks19:47:043
24654995332,15cyclictest0-21swapper/318:59:463
2465499533,33cyclictest0-21swapper/318:54:403
2465499533,33cyclictest0-21swapper/318:54:403
2465499532,44cyclictest0-21swapper/319:09:383
2465499532,42cyclictest21621-21cstates19:54:253
24654995316,31cyclictest14381-21sed00:09:143
2465399535,39cyclictest0-21swapper/222:24:152
2465399534,40cyclictest0-21swapper/222:29:212
2465399534,40cyclictest0-21swapper/222:29:212
2465399534,27cyclictest0-21swapper/200:02:302
2465399534,27cyclictest0-21swapper/200:02:302
24653995334,11cyclictest1419-21runrttasks19:34:272
24653995334,11cyclictest1419-21runrttasks19:34:272
24653995322,13cyclictest0-21swapper/223:09:242
24653995322,13cyclictest0-21swapper/223:09:242
2465499524,42cyclictest19430-21munin-node19:49:263
2465499524,41cyclictest0-21swapper/320:34:393
2465499522,42cyclictest0-21swapper/321:29:043
2465399525,40cyclictest0-21swapper/220:39:202
2465399525,39cyclictest0-21swapper/221:54:242
2465399524,40cyclictest0-21swapper/220:14:192
2465399524,28cyclictest0-21swapper/200:23:592
2465399524,11cyclictest1128-21crond23:44:012
24653995225,9cyclictest1011-21mailstats19:09:422
24653995221,13cyclictest0-21swapper/223:29:112
24653995220,13cyclictest0-21swapper/221:39:372
24652995236,10cyclictest25239-21unixbench-2d22:14:381
2465299523,9cyclictest30786-21needreboot21:19:331
2465299523,8cyclictest24903-21ntpdc19:59:371
59642513,11sleep20-21swapper/219:19:442
248599510,5rtkit-daemon2484-21rtkit-daemon19:15:051
2465499514,31cyclictest0-21swapper/320:04:423
2465499514,30cyclictest0-21swapper/320:44:373
24654995130,15cyclictest0-21swapper/323:24:233
2465499513,40cyclictest0-21swapper/323:43:323
24654995129,16cyclictest0-21swapper/321:04:073
24654995129,16cyclictest0-21swapper/321:04:073
2465499512,41cyclictest15245-21netstat21:54:323
2465399515,28cyclictest0-21swapper/222:54:202
2465399514,39cyclictest0-21swapper/221:34:292
2465399514,39cyclictest0-21swapper/220:49:202
2465399514,39cyclictest0-21swapper/219:54:222
2465399514,39cyclictest0-21swapper/219:49:382
2465399514,28cyclictest0-21swapper/200:14:292
2465399512,44cyclictest0-21swapper/223:37:412
2465299514,40cyclictest0-21swapper/123:04:141
2465299514,39cyclictest0-21swapper/121:39:061
2465299514,24cyclictest0-21swapper/121:09:201
24652995133,11cyclictest20833-21gltestperf20:59:271
24652995133,11cyclictest20833-21gltestperf20:59:271
2465299513,9cyclictest0-21swapper/121:29:321
24652995120,12cyclictest0-21swapper/119:34:361
24652995120,12cyclictest0-21swapper/119:34:361
24654995033,10cyclictest11654-21head21:49:183
24654995029,15cyclictest0-21swapper/322:14:303
24654995021,13cyclictest22098-21crond23:23:563
24654995017,18cyclictest0-21swapper/320:54:273
2465399504,39cyclictest0-21swapper/221:09:272
2465399504,28cyclictest0-21swapper/222:19:312
2465399504,28cyclictest0-21swapper/222:14:252
2465399504,28cyclictest0-21swapper/219:14:392
24653995034,10cyclictest7267-21grep19:24:242
2465299504,39cyclictest0-21swapper/122:54:241
2465299504,27cyclictest0-21swapper/120:29:301
2465299503,28cyclictest0-21swapper/123:49:131
2465299503,28cyclictest0-21swapper/100:04:291
2465299503,25cyclictest11440-21sadc19:34:101
244842504,6sleep00-21swapper/018:52:300
137382502,9sleep20-21swapper/220:44:322
282772496,36sleep10-21swapper/118:59:361
2465499495,27cyclictest0-21swapper/321:09:303
2465499494,37cyclictest0-21swapper/322:24:043
2465499493,31cyclictest0-21swapper/319:40:123
24654994928,16cyclictest0-21swapper/300:19:193
2465399494,37cyclictest0-21swapper/222:34:202
2465399494,37cyclictest0-21swapper/220:54:082
2465399494,28cyclictest0-21swapper/221:59:252
2465399494,28cyclictest0-21swapper/218:54:232
2465399494,28cyclictest0-21swapper/218:54:232
2465399494,27cyclictest0-21swapper/220:29:312
2465399494,27cyclictest0-21swapper/219:44:302
2465399494,25cyclictest1419-21runrttasks18:59:242
2465399494,25cyclictest0-21swapper/200:04:112
24653994919,12cyclictest0-21swapper/222:05:212
2465299494,38cyclictest0-21swapper/121:24:301
2465299494,28cyclictest0-21swapper/120:54:201
24652994933,9cyclictest14004-21head00:09:111
24652994931,11cyclictest17198-21gltestperf21:59:221
2465299493,41cyclictest0-21swapper/120:29:021
160042492,40sleep10-21swapper/119:41:271
2465499484,37cyclictest0-21swapper/322:54:193
24654994831,10cyclictest9971-21grep21:44:233
24654994831,10cyclictest9971-21grep21:44:233
24654994830,12cyclictest29478-21ls20:09:333
24654994825,16cyclictest0-21swapper/319:04:293
24654994823,19cyclictest0-21swapper/323:29:333
2465499482,38cyclictest151701munin-node23:04:303
2465499482,37cyclictest3448-21smartctl23:44:333
2465499482,32cyclictest0-21swapper/322:10:253
2465399487,34cyclictest0-21swapper/223:01:112
2465399484,37cyclictest0-21swapper/221:19:202
2465399484,36cyclictest0-21swapper/222:09:362
2465399484,28cyclictest0-21swapper/223:40:552
2465399484,27cyclictest0-21swapper/223:49:132
2465399484,27cyclictest0-21swapper/222:49:302
2465399484,27cyclictest0-21swapper/222:44:352
2465399484,27cyclictest0-21swapper/220:04:292
2465299484,38cyclictest0-21swapper/121:59:051
2465299484,28cyclictest0-21swapper/122:44:031
2465299484,28cyclictest0-21swapper/122:24:251
2465299484,28cyclictest0-21swapper/121:49:271
2465299484,28cyclictest0-21swapper/120:34:291
2465299484,26cyclictest0-21swapper/123:55:571
2465299484,26cyclictest0-21swapper/123:48:361
2465299484,26cyclictest0-21swapper/122:30:441
2465299484,26cyclictest0-21swapper/122:30:441
2465299484,25cyclictest0-21swapper/122:05:401
2465299484,13cyclictest0-21swapper/119:50:141
282192472,9sleep10-21swapper/121:14:301
2465399475,26cyclictest0-21swapper/220:35:222
2465399474,34cyclictest0-21swapper/222:39:142
2465399474,26cyclictest0-21swapper/221:00:262
2465399474,26cyclictest0-21swapper/221:00:262
24653994724,16cyclictest0-21swapper/200:09:182
2465299474,36cyclictest1287-21snmpd21:39:181
2465299474,29cyclictest0-21swapper/122:44:311
2465299474,28cyclictest0-21swapper/123:09:231
2465299474,28cyclictest0-21swapper/123:09:231
2465299474,28cyclictest0-21swapper/122:59:351
2465299474,28cyclictest0-21swapper/100:19:211
2465299474,27cyclictest0-21swapper/119:09:451
2465299474,27cyclictest0-21swapper/119:09:121
2465299474,25cyclictest0-21swapper/123:37:001
2465299474,25cyclictest0-21swapper/121:45:361
2465299474,25cyclictest0-21swapper/121:45:361
24652994731,10cyclictest27716-21unixbench_singl22:19:381
24652994730,11cyclictest22294-21kernelversion19:54:341
24652994716,11cyclictest0-21swapper/120:08:571
24652994715,13cyclictest0-21swapper/122:09:281
2465499464,35cyclictest0-21swapper/320:24:313
2465499464,34cyclictest6839-21munin-node21:39:243
24654994626,11cyclictest6052-21munin-node20:29:243
2465399464,25cyclictest0-21swapper/219:39:312
24652994620,12cyclictest0-21swapper/120:15:181
24652994615,13cyclictest0-21swapper/123:24:341
2465499454,33cyclictest30891-21tail20:14:193
24654994532,7cyclictest23282-21gltestperf21:04:253
24654994530,9cyclictest20941-21crond22:09:033
2465399454,35cyclictest0-21swapper/220:24:242
24653994529,10cyclictest1419-21runrttasks19:29:322
2465299453,25cyclictest0-21swapper/123:59:261
2465299453,25cyclictest0-21swapper/123:59:261
24652994524,14cyclictest0-21swapper/120:44:401
24652994515,12cyclictest0-21swapper/100:14:121
24652994418,12cyclictest0-21swapper/123:14:151
24652994418,12cyclictest0-21swapper/123:14:151
24652994415,12cyclictest0-21swapper/123:29:131
2465399424,13cyclictest0-21swapper/218:54:122
2465299414,13cyclictest0-21swapper/121:04:171
24654993813,13cyclictest0-21swapper/318:54:123
2465299345,19cyclictest0-21swapper/118:54:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional