You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 15:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Tue Dec 02, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
856127810,57sleep00-21swapper/006:44:540
139852710,7sleep0918199cyclictest09:14:200
243799670,6rtkit-daemon2436-21rtkit-daemon08:14:410
9188996537,11cyclictest27743-21sshd10:49:342
243799640,5rtkit-daemon2436-21rtkit-daemon09:59:300
243799630,5rtkit-daemon2436-21rtkit-daemon06:49:370
243799630,5rtkit-daemon2436-21rtkit-daemon06:49:370
275842624,13sleep10-21swapper/110:49:341
243799620,7rtkit-daemon2436-21rtkit-daemon09:39:310
194512624,9sleep019534-21tune2fs09:24:370
9188996034,9cyclictest16195-21grep08:09:412
243799580,4rtkit-daemon2436-21rtkit-daemon10:09:440
9188995711,27cyclictest0-21swapper/209:19:202
918999564,47cyclictest0-21swapper/311:24:393
9188995636,13cyclictest26434-21cpuspeed_turbos07:24:382
918899563,45cyclictest11423-21cut07:59:412
918699565,43cyclictest0-21swapper/108:54:501
194742565,9sleep00-21swapper/010:30:140
194742565,9sleep00-21swapper/010:30:140
9188995538,10cyclictest21553-21tr11:44:302
918699554,13cyclictest0-21swapper/111:29:271
283982550,6sleep228399-21aten_r1power_en11:59:262
243799550,5rtkit-daemon2436-21rtkit-daemon11:59:330
243799550,5rtkit-daemon2436-21rtkit-daemon11:06:120
243799550,5rtkit-daemon2436-21rtkit-daemon10:34:360
243799550,5rtkit-daemon2436-21rtkit-daemon10:24:360
243799550,5rtkit-daemon2436-21rtkit-daemon08:54:400
243799550,5rtkit-daemon2436-21rtkit-daemon07:39:500
243799550,5rtkit-daemon2436-21rtkit-daemon07:14:440
9189995438,10cyclictest4812-21cpuspeed_turbos11:09:293
9189995438,10cyclictest4812-21cpuspeed_turbos11:09:293
9189995423,24cyclictest0-21swapper/310:59:153
9188995433,9cyclictest30543-21runrttasks08:39:422
9188995427,12cyclictest0-21swapper/207:44:502
9188995424,13cyclictest0-21swapper/211:09:462
9188995424,13cyclictest0-21swapper/211:09:462
918699545,12cyclictest0-21swapper/109:39:391
243799540,6rtkit-daemon2436-21rtkit-daemon06:55:000
243799540,5rtkit-daemon2436-21rtkit-daemon10:44:360
243799540,5rtkit-daemon2436-21rtkit-daemon10:43:580
243799540,5rtkit-daemon2436-21rtkit-daemon09:49:380
243799540,5rtkit-daemon2436-21rtkit-daemon09:04:470
243799540,5rtkit-daemon2436-21rtkit-daemon07:44:430
243799540,5rtkit-daemon2436-21rtkit-daemon07:29:430
223572544,42sleep20-21swapper/207:14:502
9189995337,10cyclictest9376-21strings07:54:463
9189995321,11cyclictest6729-21iostat_ios10:04:423
9189995311,34cyclictest0-21swapper/310:39:163
9188995336,9cyclictest31672-21wc12:04:342
9188995324,14cyclictest0-21swapper/209:39:202
918699535,12cyclictest0-21swapper/107:04:361
918699534,33cyclictest0-21swapper/110:39:081
918699534,14cyclictest0-21swapper/111:04:251
918699534,12cyclictest0-21swapper/108:18:471
9186995322,13cyclictest0-21swapper/106:59:371
81592533,9sleep30-21swapper/310:09:303
267352533,43sleep10-21swapper/108:30:381
243799530,5rtkit-daemon2436-21rtkit-daemon12:13:020
243799530,5rtkit-daemon2436-21rtkit-daemon12:04:320
243799530,5rtkit-daemon2436-21rtkit-daemon11:34:340
243799530,5rtkit-daemon2436-21rtkit-daemon10:49:350
243799530,5rtkit-daemon2436-21rtkit-daemon10:04:520
243799530,5rtkit-daemon2436-21rtkit-daemon08:24:410
243799530,5rtkit-daemon2436-21rtkit-daemon08:19:410
243799530,5rtkit-daemon2436-21rtkit-daemon08:19:410
243799530,5rtkit-daemon2436-21rtkit-daemon07:54:500
9189995221,13cyclictest0-21swapper/308:19:543
9189995221,13cyclictest0-21swapper/308:19:543
9188995236,10cyclictest21731-21unixbench_multi10:34:512
918899522,38cyclictest0-21swapper/207:05:442
918699524,29cyclictest0-21swapper/107:49:471
918699523,11cyclictest0-21swapper/108:09:411
9181995222,11cyclictest0-21swapper/010:54:300
243799520,5rtkit-daemon2436-21rtkit-daemon11:54:330
243799520,5rtkit-daemon2436-21rtkit-daemon11:54:330
243799520,5rtkit-daemon2436-21rtkit-daemon10:59:270
243799520,5rtkit-daemon2436-21rtkit-daemon10:19:360
243799520,5rtkit-daemon2436-21rtkit-daemon09:44:530
243799520,5rtkit-daemon2436-21rtkit-daemon09:44:530
243799520,5rtkit-daemon2436-21rtkit-daemon09:20:460
243799520,5rtkit-daemon2436-21rtkit-daemon08:34:410
243799520,5rtkit-daemon2436-21rtkit-daemon08:33:110
243799520,5rtkit-daemon2436-21rtkit-daemon07:09:440
232422522,43sleep00-21swapper/009:32:110
210142523,9sleep30-21swapper/307:13:263
207542524,12sleep20-21swapper/207:10:022
149042523,42sleep10-21swapper/108:07:001
101822523,42sleep10-21swapper/110:13:311
9189995137,9cyclictest31339-21unixbench_multi08:39:543
9189995136,9cyclictest15057-21gltestperf06:59:453
9189995122,12cyclictest0-21swapper/311:14:323
9189995118,13cyclictest28172-21yum-updatesd-he09:43:593
9188995131,7cyclictest23411-21latency_hist09:34:202
9188995126,10cyclictest367-21sshd08:44:402
9188995124,12cyclictest0-21swapper/211:49:342
9188995122,12cyclictest0-21swapper/211:39:342
918699515,9cyclictest9013-21idleruntime-cro11:19:151
918699515,26cyclictest0-21swapper/111:24:161
918699515,13cyclictest0-21swapper/107:46:421
918699514,12cyclictest0-21swapper/111:09:351
918699514,12cyclictest0-21swapper/111:09:351
918699514,10cyclictest24055-21cpuspeed_turbos07:19:381
9186995124,8cyclictest27032-21cat10:49:161
257272512,9sleep30-21swapper/311:54:083
243799510,5rtkit-daemon2436-21rtkit-daemon12:14:320
243799510,5rtkit-daemon2436-21rtkit-daemon09:02:250
243799510,5rtkit-daemon2436-21rtkit-daemon09:02:250
243799510,5rtkit-daemon2436-21rtkit-daemon08:09:420
243799510,5rtkit-daemon2436-21rtkit-daemon08:04:420
22352515,40sleep00-21swapper/009:54:480
138342513,42sleep10-21swapper/109:13:581
116222514,9sleep10-21swapper/111:24:201
918999505,27cyclictest0-21swapper/306:54:353
918999504,28cyclictest0-21swapper/309:54:503
9189995035,10cyclictest19432-21uniq11:39:303
9189995021,12cyclictest0-21swapper/311:09:123
9189995021,12cyclictest0-21swapper/308:59:393
9189995021,12cyclictest0-21swapper/308:59:393
9188995033,11cyclictest23202-21tune2fs08:24:422
9188995033,11cyclictest12154-21grep06:54:382
9188995021,12cyclictest0-21swapper/210:29:392
9188995021,12cyclictest0-21swapper/210:29:392
918699506,26cyclictest0-21swapper/109:38:581
918699505,27cyclictest0-21swapper/107:59:341
918699504,28cyclictest0-21swapper/107:24:531
918699504,28cyclictest0-21swapper/107:18:371
918699504,13cyclictest0-21swapper/109:44:401
918699504,13cyclictest0-21swapper/109:44:401
918699503,13cyclictest0-21swapper/109:19:201
9186995021,13cyclictest0-21swapper/108:59:331
9186995021,13cyclictest0-21swapper/108:59:331
9186995021,12cyclictest0-21swapper/109:59:021
9186995020,13cyclictest0-21swapper/109:25:381
9186995020,13cyclictest0-21swapper/108:49:501
9186995019,13cyclictest0-21swapper/108:34:531
9186995011,21cyclictest0-21swapper/112:04:251
918199503,17cyclictest0-21swapper/011:09:340
918199503,17cyclictest0-21swapper/011:09:340
302312502,9sleep30-21swapper/312:00:373
256332503,9sleep20-21swapper/207:21:192
243799500,5rtkit-daemon2436-21rtkit-daemon08:49:550
243799500,5rtkit-daemon2436-21rtkit-daemon08:44:330
243799500,5rtkit-daemon2436-21rtkit-daemon07:51:270
243799500,5rtkit-daemon2436-21rtkit-daemon07:19:360
243799500,4rtkit-daemon2436-21rtkit-daemon09:34:460
201912502,42sleep00-21swapper/011:39:420
17172503,9sleep30-21swapper/308:48:253
161412502,42sleep00-21swapper/009:17:560
147032502,9sleep20-21swapper/210:20:132
136602503,9sleep10-21swapper/106:55:501
111902502,9sleep30-21swapper/309:05:383
918999496,25cyclictest0-21swapper/310:29:063
918999495,27cyclictest0-21swapper/310:04:033
918999495,27cyclictest0-21swapper/309:53:373
918999495,26cyclictest0-21swapper/311:04:123
918999495,26cyclictest0-21swapper/308:53:523
918999495,26cyclictest0-21swapper/308:13:473
9189994934,10cyclictest17316-21fschecks_time07:04:433
9189994920,12cyclictest0-21swapper/308:14:353
9189994918,13cyclictest0-21swapper/307:59:373
918899494,29cyclictest0-21swapper/211:19:332
9188994934,9cyclictest15044-21grep06:59:452
9188994933,9cyclictest28683-21find08:34:532
918899493,27cyclictest0-21swapper/208:49:352
918699496,26cyclictest0-21swapper/111:54:201
918699496,26cyclictest0-21swapper/111:54:201
918699496,25cyclictest0-21swapper/107:55:421
918699495,27cyclictest0-21swapper/110:29:061
918699495,27cyclictest0-21swapper/110:14:521
918699495,26cyclictest0-21swapper/107:38:401
918699495,26cyclictest0-21swapper/107:38:401
918699495,24cyclictest20124-21aten2.4-expect08:19:351
918699495,24cyclictest20124-21aten2.4-expect08:19:351
918699494,39cyclictest0-21swapper/109:29:441
918699494,26cyclictest7922-21latency_hist10:09:171
918699493,27cyclictest29094-21fschecks_count07:29:391
9186994920,13cyclictest0-21swapper/110:39:471
9186994920,12cyclictest0-21swapper/107:39:441
9186994918,14cyclictest0-21swapper/109:59:321
918199493,41cyclictest0-21swapper/010:17:550
9181994923,12cyclictest0-21swapper/011:19:340
315932495,9sleep00-21swapper/008:41:330
250192493,39sleep00-21swapper/011:49:460
221302492,9sleep20-21swapper/208:22:532
221302492,9sleep20-21swapper/208:22:532
207232492,9sleep30-21swapper/309:25:293
186342494,9sleep00-21swapper/007:08:310
147912492,9sleep20-21swapper/208:05:322
112882492,9sleep20-21swapper/209:06:542
918999485,26cyclictest0-21swapper/311:54:203
918999485,26cyclictest0-21swapper/311:54:203
918999485,26cyclictest0-21swapper/310:54:103
918999485,26cyclictest0-21swapper/309:44:493
918999485,26cyclictest0-21swapper/309:44:493
918999485,26cyclictest0-21swapper/309:18:563
918999485,26cyclictest0-21swapper/308:28:493
918999485,26cyclictest0-21swapper/307:34:473
918999485,26cyclictest0-21swapper/307:34:473
918999484,27cyclictest0-21swapper/306:49:473
918999484,27cyclictest0-21swapper/306:49:473
918999484,26cyclictest0-21swapper/311:29:353
918999484,26cyclictest0-21swapper/307:44:373
9189994834,8cyclictest6299-21unixbench_singl08:54:543
918999483,8cyclictest34-21rcuc/311:24:153
918999482,31cyclictest0-21swapper/306:49:263
918899484,38cyclictest0-21swapper/207:49:522
918899484,26cyclictest0-21swapper/209:49:452
918899483,27cyclictest0-21swapper/210:59:512
918899483,27cyclictest0-21swapper/209:19:532
9188994821,13cyclictest0-21swapper/210:09:362
918899482,34cyclictest0-21swapper/211:28:512
9188994819,13cyclictest0-21swapper/208:29:402
918699485,27cyclictest0-21swapper/108:44:421
918699485,26cyclictest0-21swapper/111:59:431
918699485,23cyclictest9174-21munin-run06:49:251
918699484,27cyclictest0-21swapper/112:14:231
918699484,26cyclictest0-21swapper/111:44:141
918699484,26cyclictest0-21swapper/111:34:361
918699484,26cyclictest0-21swapper/110:19:411
918699484,12cyclictest0-21swapper/111:49:191
9186994821,11cyclictest0-21swapper/110:54:371
9186994817,14cyclictest0-21swapper/107:09:401
9186994817,12cyclictest0-21swapper/108:28:481
3822482,41sleep00-21swapper/007:37:050
3822482,41sleep00-21swapper/007:37:050
305552483,9sleep30-21swapper/307:33:173
170682483,9sleep20-21swapper/210:25:002
102272482,9sleep20-21swapper/207:58:152
918999476,25cyclictest0-21swapper/309:33:583
918999475,26cyclictest0-21swapper/310:44:093
918999475,26cyclictest0-21swapper/309:38:593
918999475,25cyclictest0-21swapper/311:34:313
918999475,25cyclictest0-21swapper/307:53:433
918999475,25cyclictest0-21swapper/307:19:473
918999474,26cyclictest0-21swapper/312:09:423
918999474,26cyclictest0-21swapper/310:44:303
918999474,26cyclictest0-21swapper/307:44:243
918999474,26cyclictest0-21swapper/307:24:453
918999474,25cyclictest0-21swapper/311:44:273
918999474,12cyclictest0-21swapper/310:29:313
918999474,12cyclictest0-21swapper/310:29:313
918999473,24cyclictest13687-21perl08:04:403
9188994732,10cyclictest30757-21cat10:54:412
918899473,38cyclictest0-21swapper/207:29:372
918899473,26cyclictest0-21swapper/208:59:392
918899473,26cyclictest0-21swapper/208:59:392
9188994721,11cyclictest0-21swapper/211:29:352
9188994718,12cyclictest0-21swapper/206:49:562
9188994718,12cyclictest0-21swapper/206:49:562
918699475,26cyclictest0-21swapper/106:53:441
918699475,26cyclictest0-21swapper/106:53:441
918699474,26cyclictest0-21swapper/109:49:411
918699473,25cyclictest956-21sendmail_mailqu10:59:441
9186994720,11cyclictest0-21swapper/110:29:481
9186994720,11cyclictest0-21swapper/110:29:481
9186994719,10cyclictest29645-21munin-node08:39:391
918199474,26cyclictest0-21swapper/007:24:450
51672472,9sleep30-21swapper/312:16:293
244322472,9sleep20-21swapper/210:42:212
231652472,39sleep00-21swapper/011:46:560
183232473,38sleep10-21swapper/109:20:151
137642472,39sleep00-21swapper/011:29:060
105462472,39sleep00-21swapper/007:59:260
918999463,24cyclictest1369-21runrttasks08:34:343
9189994616,12cyclictest0-21swapper/309:23:563
918899464,27cyclictest0-21swapper/208:14:352
918899464,26cyclictest0-21swapper/208:54:322
9188994633,8cyclictest7657-21gltestperf11:14:352
9188994632,9cyclictest4051-21gltestperf12:14:342
9188994632,9cyclictest26034-21head10:44:432
9188994630,10cyclictest27241-21needreboot09:39:432
9188994630,10cyclictest26301-21grep11:54:282
9188994630,10cyclictest26301-21grep11:54:282
918899462,39cyclictest0-21swapper/210:18:362
9188994618,12cyclictest0-21swapper/207:34:492
9188994618,12cyclictest0-21swapper/207:34:492
918699464,26cyclictest0-21swapper/109:08:541
9186994626,14cyclictest0-21swapper/111:44:311
918199463,39cyclictest0-21swapper/011:14:490
9181994619,12cyclictest0-21swapper/011:29:330
9181994617,12cyclictest0-21swapper/006:59:450
28852462,9sleep20-21swapper/207:40:582
147192462,9sleep30-21swapper/310:20:263
125672462,39sleep30-21swapper/310:18:323
9189994518,12cyclictest0-21swapper/312:04:363
9188994530,10cyclictest5038-21unixbench_multi09:59:512
918899453,36cyclictest0-21swapper/212:09:442
918899453,26cyclictest0-21swapper/211:09:152
918899453,26cyclictest0-21swapper/209:09:532
9188994516,11cyclictest0-21swapper/209:57:372
75842453,9sleep20-21swapper/210:05:502
302832453,9sleep20-21swapper/209:45:192
302832453,9sleep20-21swapper/209:45:192
27502453,36sleep10-21swapper/112:11:031
163882452,9sleep20-21swapper/211:34:162
135992452,9sleep30-21swapper/309:10:543
234022432,35sleep30-21swapper/307:18:303
9189994228,9cyclictest25941-21ls08:29:453
918899414,31cyclictest0-21swapper/209:24:382
89382394,29sleep20-21swapper/206:46:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional