You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 18:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Sun Nov 16, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2734599915,26cyclictest0-21swapper/307:22:593
2734599915,26cyclictest0-21swapper/307:22:593
2734599903,25cyclictest0-21swapper/308:14:463
2734599884,26cyclictest0-21swapper/312:13:443
2734599874,29cyclictest0-21swapper/308:54:543
2734599874,28cyclictest0-21swapper/310:54:363
2734599874,28cyclictest0-21swapper/310:54:363
27345998710,59cyclictest31040-21latency07:09:533
2734599864,27cyclictest0-21swapper/309:54:413
2734599843,22cyclictest6691-21unixbench_multi10:49:533
2734599834,27cyclictest0-21swapper/311:19:533
2734599834,27cyclictest0-21swapper/307:54:473
27345998326,50cyclictest10269-21unixbench_multi12:04:503
27345998316,13cyclictest0-21swapper/308:20:023
2734599824,22cyclictest0-21swapper/307:44:383
2734599823,72cyclictest0-21swapper/311:14:463
2734599823,27cyclictest0-21swapper/311:24:353
27345998225,50cyclictest13806-21fschecks_time08:49:423
27345998225,50cyclictest13806-21fschecks_time08:49:423
27345998225,49cyclictest19769-21unixbench-2d12:24:493
27345998225,49cyclictest19769-21unixbench-2d12:24:493
27345998224,51cyclictest13811-21timerwakeupswit11:04:513
2734599814,27cyclictest0-21swapper/308:59:573
2734599814,23cyclictest0-21swapper/308:39:363
27345998124,50cyclictest14208-21fschecks_time09:59:413
2734599804,26cyclictest0-21swapper/310:04:373
2734599804,24cyclictest0-21swapper/310:23:273
2734599804,24cyclictest0-21swapper/307:17:583
2734599794,24cyclictest0-21swapper/311:59:453
27345997924,48cyclictest30712-21unixbench_singl09:24:553
2734399794,69cyclictest0-21swapper/207:04:492
2734599784,68cyclictest0-21swapper/310:33:293
2734599784,54cyclictest0-21swapper/312:19:553
2734599784,54cyclictest0-21swapper/312:19:553
2734599784,22cyclictest0-21swapper/309:53:223
2734599783,9cyclictest2784-21munin-node10:44:333
27345997823,48cyclictest21898-21cpuspeed_turbos07:59:433
27345997821,10cyclictest16293-21unixbench_multi11:09:523
2734599776,26cyclictest0-21swapper/310:43:303
2734599776,19cyclictest0-21swapper/309:44:213
2734599774,66cyclictest0-21swapper/311:44:413
2734599774,21cyclictest0-21swapper/307:39:273
2734599773,21cyclictest0-21swapper/308:44:363
27345997723,47cyclictest6160-21proc_pri07:24:573
27345997720,10cyclictest9795-21munin-node10:59:473
27345997720,10cyclictest9795-21munin-node10:59:473
2734599766,16cyclictest0-21swapper/310:39:193
2734599765,63cyclictest0-21swapper/311:49:403
2734599764,26cyclictest0-21swapper/311:54:413
2734599764,22cyclictest0-21swapper/308:09:363
2734599764,20cyclictest0-21swapper/310:24:563
2734599764,13cyclictest0-21swapper/308:29:243
2734599763,33cyclictest1369-21runrttasks07:52:393
27345997623,47cyclictest1369-21runrttasks09:34:383
27345997623,47cyclictest1369-21runrttasks09:34:383
27345997622,47cyclictest6551-21cpuspeed_turbos09:44:353
27345997621,9cyclictest27598-21latency11:34:423
27345997621,49cyclictest0-21swapper/311:39:283
27345997621,49cyclictest0-21swapper/311:39:283
27345997619,9cyclictest24464-21df_abs08:04:423
27345997619,9cyclictest24464-21df_abs08:04:423
2734599754,14cyclictest0-21swapper/308:33:093
2734599754,12cyclictest0-21swapper/311:29:463
27345997518,10cyclictest25957-21gltestperf09:14:443
27345997517,9cyclictest7556-21cpuspeed_turbos07:29:443
27345997515,8cyclictest0-21swapper/312:14:313
27345997511,12cyclictest0-21swapper/312:29:433
2734599744,19cyclictest0-21swapper/310:12:313
2734599743,12cyclictest0-21swapper/309:04:243
27345997416,9cyclictest28830-21ntp_offset07:04:533
27345997414,13cyclictest0-21swapper/309:14:213
2734599733,24cyclictest0-21swapper/307:39:483
27345997317,9cyclictest28128-21fschecks_count09:19:433
27345997220,9cyclictest7169-21needreboot08:34:513
27345997220,11cyclictest0-21swapper/310:16:583
243799720,2rtkit-daemon0-21swapper/210:58:212
243799720,2rtkit-daemon0-21swapper/210:58:212
243799710,21rtkit-daemon2436-21rtkit-daemon11:49:492
27345997018,8cyclictest32493-21missed_timers09:29:463
243799670,2rtkit-daemon0-21swapper/211:39:572
243799670,2rtkit-daemon0-21swapper/211:39:572
2734399664,55cyclictest0-21swapper/209:39:542
2734099654,53cyclictest0-21swapper/011:34:380
2734099654,53cyclictest0-21swapper/008:24:380
2734099644,53cyclictest0-21swapper/009:39:220
2734099644,53cyclictest0-21swapper/009:39:220
243799640,4rtkit-daemon2436-21rtkit-daemon07:29:142
2734099633,53cyclictest0-21swapper/008:29:460
2734099623,53cyclictest0-21swapper/008:04:370
2734099623,53cyclictest0-21swapper/008:04:370
2734099622,54cyclictest0-21swapper/011:44:290
243799620,9rtkit-daemon0-21swapper/210:39:442
2734099615,49cyclictest28460-21runrttasks07:04:470
2734099615,49cyclictest0-21swapper/010:04:500
2734099613,50cyclictest0-21swapper/012:29:460
243799610,9rtkit-daemon0-21swapper/209:52:082
243799610,8rtkit-daemon0-21swapper/210:51:142
243799610,8rtkit-daemon0-21swapper/210:29:442
243799610,5rtkit-daemon1165-21lldpd10:15:222
243799610,4rtkit-daemon0-21swapper/211:04:432
243799610,3rtkit-daemon0-21swapper/212:04:112
243799610,3rtkit-daemon0-21swapper/211:17:132
243799610,3rtkit-daemon0-21swapper/209:54:462
243799610,3rtkit-daemon0-21swapper/209:48:162
243799610,3rtkit-daemon0-21swapper/208:56:112
243799610,10rtkit-daemon0-21swapper/212:24:402
243799610,10rtkit-daemon0-21swapper/212:24:402
311992603,8sleep031225-21grep07:09:570
27343996041,11cyclictest1102-21tail07:14:522
2734099604,50cyclictest25310-21munin-node11:29:440
2734099604,49cyclictest21818-21tail11:24:300
2734099603,52cyclictest0-21swapper/009:14:500
2734099603,51cyclictest0-21swapper/007:14:380
243799600,9rtkit-daemon0-21swapper/212:31:172
243799600,9rtkit-daemon0-21swapper/211:45:262
243799600,9rtkit-daemon0-21swapper/210:47:592
243799600,9rtkit-daemon0-21swapper/210:28:142
243799600,8rtkit-daemon0-21swapper/212:04:262
243799600,8rtkit-daemon0-21swapper/211:10:432
243799600,8rtkit-daemon0-21swapper/210:36:222
243799600,8rtkit-daemon0-21swapper/210:09:452
243799600,8rtkit-daemon0-21swapper/210:05:232
243799600,8rtkit-daemon0-21swapper/208:26:042
243799600,8rtkit-daemon0-21swapper/208:13:052
243799600,8rtkit-daemon0-21swapper/208:01:282
243799600,7rtkit-daemon0-21swapper/212:20:332
243799600,7rtkit-daemon0-21swapper/212:20:332
243799600,7rtkit-daemon0-21swapper/208:20:192
243799600,6rtkit-daemon0-21swapper/208:35:342
243799600,4rtkit-daemon0-21swapper/209:35:092
243799600,4rtkit-daemon0-21swapper/209:35:092
243799600,3rtkit-daemon0-21swapper/211:00:132
243799600,3rtkit-daemon0-21swapper/211:00:132
243799600,2rtkit-daemon0-21swapper/211:56:112
243799600,10rtkit-daemon0-21swapper/211:30:052
243799600,10rtkit-daemon0-21swapper/210:00:082
243799600,10rtkit-daemon0-21swapper/208:32:192
89272593,48sleep00-21swapper/008:39:410
2734599598,13cyclictest0-21swapper/307:04:283
27343995940,11cyclictest2749-21grep07:19:412
27343995940,11cyclictest2749-21grep07:19:412
2734399594,48cyclictest0-21swapper/208:44:532
27342995933,13cyclictest0-21swapper/111:34:311
2734099594,46cyclictest0-21swapper/012:04:460
2734099593,51cyclictest0-21swapper/010:49:520
243799590,8rtkit-daemon0-21swapper/208:14:352
243799590,4rtkit-daemon0-21swapper/208:50:262
243799590,4rtkit-daemon0-21swapper/208:50:262
243799590,3rtkit-daemon0-21swapper/211:36:042
243799590,3rtkit-daemon0-21swapper/211:25:202
243799590,3rtkit-daemon0-21swapper/208:05:052
243799590,3rtkit-daemon0-21swapper/208:05:052
243799590,3rtkit-daemon0-21swapper/207:56:202
243799590,3rtkit-daemon0-21swapper/207:29:292
243799590,2rtkit-daemon0-21swapper/207:40:212
243799590,10rtkit-daemon0-21swapper/207:34:362
49062580,8sleep027333-21cyclictest10:49:190
27342995832,12cyclictest0-21swapper/107:14:271
2734099583,50cyclictest0-21swapper/012:04:150
2734099583,49cyclictest0-21swapper/008:59:240
2734099583,49cyclictest0-21swapper/008:34:520
2734099583,49cyclictest0-21swapper/007:49:560
2734099583,48cyclictest0-21swapper/011:09:420
27340995812,39cyclictest1374-21tail11:49:290
243799580,7rtkit-daemon0-21swapper/210:19:452
2734299573,48cyclictest0-21swapper/111:04:401
2734299573,47cyclictest0-21swapper/108:41:341
27342995712,38cyclictest30289-21tail10:34:331
2734099574,48cyclictest0-21swapper/009:53:220
2734099573,48cyclictest0-21swapper/007:48:020
243799570,2rtkit-daemon0-21swapper/211:19:272
27343995638,12cyclictest16133-21unixbench_singl07:44:592
2734299562,49cyclictest0-21swapper/110:59:311
2734299562,49cyclictest0-21swapper/110:59:311
2734099563,48cyclictest0-21swapper/010:39:470
2734099563,47cyclictest0-21swapper/011:39:230
2734099563,47cyclictest0-21swapper/011:39:230
2734099563,47cyclictest0-21swapper/009:24:430
2734099562,48cyclictest0-21swapper/009:19:450
2734399554,42cyclictest30292-21grep07:09:442
2734299553,46cyclictest0-21swapper/112:23:451
2734299553,46cyclictest0-21swapper/112:23:451
2734099554,44cyclictest0-21swapper/007:39:260
2734099554,43cyclictest0-21swapper/010:09:420
2734099553,47cyclictest0-21swapper/011:54:290
2734099553,47cyclictest0-21swapper/009:54:340
2734099553,47cyclictest0-21swapper/009:00:220
2734099553,47cyclictest0-21swapper/008:24:240
2734099553,46cyclictest0-21swapper/009:44:490
2734099553,46cyclictest0-21swapper/009:09:370
2734099553,45cyclictest0-21swapper/008:49:540
2734099553,45cyclictest0-21swapper/008:49:540
78342542,45sleep10-21swapper/108:35:471
297272543,12sleep20-21swapper/209:24:412
27342995436,12cyclictest0-21swapper/111:24:311
2734299543,44cyclictest32729-21munin-node11:44:481
2734099543,46cyclictest0-21swapper/007:59:570
2734099543,45cyclictest0-21swapper/008:14:510
2734099543,45cyclictest0-21swapper/007:19:550
2734099543,45cyclictest0-21swapper/007:19:550
2734099542,47cyclictest0-21swapper/007:29:510
2734099542,46cyclictest0-21swapper/010:14:450
2734099542,45cyclictest0-21swapper/010:24:480
27340995415,34cyclictest14231-21http_loadtime12:14:350
193892544,43sleep219516-21timerwakeupswit08:59:592
177272543,44sleep00-21swapper/012:22:300
177272543,44sleep00-21swapper/012:22:300
2734399535,29cyclictest0-21swapper/209:18:162
2734299533,44cyclictest0-21swapper/107:29:591
27342995326,14cyclictest0-21swapper/111:49:381
2734099534,12cyclictest0-21swapper/009:39:380
2734099533,46cyclictest0-21swapper/012:14:020
2734099533,45cyclictest0-21swapper/008:09:530
2734099533,45cyclictest0-21swapper/007:54:410
2734099533,42cyclictest0-21swapper/010:59:400
2734099533,42cyclictest0-21swapper/010:59:400
133692533,10sleep00-21swapper/009:59:220
27343995235,11cyclictest12652-21unixbench_multi12:09:502
27343995233,12cyclictest8951-21cut08:39:412
2734299524,41cyclictest0-21swapper/107:59:251
27342995238,9cyclictest28978-21munin-node09:24:431
27342995238,9cyclictest22696-21munin-node09:09:581
27342995238,9cyclictest0-21swapper/108:00:301
27342995237,10cyclictest0-21swapper/112:00:211
27342995235,12cyclictest0-21swapper/108:30:221
2734299523,42cyclictest10897-21telnet08:44:361
2734299523,25cyclictest0-21swapper/108:24:531
27343995131,12cyclictest23400-21tune2fs09:09:412
27342995137,9cyclictest5429-21munin-node10:49:341
27342995136,11cyclictest0-21swapper/108:57:521
27342995135,11cyclictest0-21swapper/112:04:521
2734299513,41cyclictest0-21swapper/109:14:361
2734299512,42cyclictest0-21swapper/107:34:411
2734099515,40cyclictest0-21swapper/008:44:500
2734099514,42cyclictest0-21swapper/012:24:280
2734099514,42cyclictest0-21swapper/012:24:280
2734099513,42cyclictest0-21swapper/009:04:570
151872512,42sleep10-21swapper/108:53:251
151872512,42sleep10-21swapper/108:53:251
83302502,10sleep10-21swapper/109:48:431
285662502,9sleep10-21swapper/109:22:101
2734399505,38cyclictest0-21swapper/207:49:432
27343995021,15cyclictest0-21swapper/209:19:392
2734299504,40cyclictest0-21swapper/110:39:301
27342995037,8cyclictest24508-21expr11:29:331
27342995036,10cyclictest0-21swapper/108:18:361
27342995035,10cyclictest21186-21gltestperf09:04:431
2734299503,42cyclictest0-21swapper/111:18:361
2734299503,41cyclictest0-21swapper/109:38:191
2734299503,41cyclictest0-21swapper/109:38:191
27342995017,27cyclictest24234-21ntpdc10:19:481
27342995013,31cyclictest15700-21latency_hist10:04:201
2734099506,27cyclictest0-21swapper/011:19:330
243799500,4rtkit-daemon2436-21rtkit-daemon09:34:172
151382502,10sleep10-21swapper/111:09:361
140572502,10sleep10-21swapper/107:43:391
58412492,39sleep10-21swapper/111:58:051
307972492,10sleep10-21swapper/111:41:081
307972492,10sleep10-21swapper/111:41:081
27343994914,13cyclictest0-21swapper/209:04:452
27342994936,9cyclictest0-21swapper/112:29:311
27342994933,11cyclictest0-21swapper/107:52:271
2734299493,41cyclictest0-21swapper/109:51:271
2734299493,40cyclictest0-21swapper/107:19:511
2734299493,40cyclictest0-21swapper/107:19:511
2734299493,38cyclictest0-21swapper/110:29:401
2734099495,37cyclictest0-21swapper/010:29:360
2734099493,39cyclictest0-21swapper/007:24:570
265372493,39sleep126688-21cstates08:09:391
122622493,10sleep00-21swapper/011:04:310
8592482,40sleep10-21swapper/109:31:541
27342994836,7cyclictest4410-21df_abs09:39:371
27342994834,10cyclictest0-21swapper/110:16:121
2734299483,39cyclictest0-21swapper/108:23:081
2734099484,37cyclictest0-21swapper/007:44:260
27340994833,10cyclictest7736-21cpuspeed_turbos10:54:330
27340994833,10cyclictest7736-21cpuspeed_turbos10:54:330
2734099483,39cyclictest0-21swapper/010:19:320
2734099483,38cyclictest0-21swapper/009:31:070
2734099482,39cyclictest0-21swapper/011:14:370
2734099482,10cyclictest0-21swapper/010:34:360
202332482,39sleep10-21swapper/110:13:051
2734399474,24cyclictest0-21swapper/212:14:392
2734299474,36cyclictest0-21swapper/110:24:341
2734299473,40cyclictest0-21swapper/112:27:211
2734299473,40cyclictest0-21swapper/112:27:211
2734299473,38cyclictest0-21swapper/107:04:421
2734299466,35cyclictest0-21swapper/110:58:321
2734299466,35cyclictest0-21swapper/110:58:321
2734299464,37cyclictest0-21swapper/112:09:381
2734299464,36cyclictest0-21swapper/112:19:151
2734299463,39cyclictest0-21swapper/111:22:241
270842464,6sleep00-21swapper/007:01:350
260442463,37sleep10-21swapper/108:09:031
260442463,37sleep10-21swapper/108:09:031
176872462,38sleep10-21swapper/110:05:561
42752453,36sleep14395-21uptime10:44:531
27342994527,13cyclictest14361-21munin-node07:44:471
271622454,6sleep10-21swapper/107:02:371
131762452,37sleep10-21swapper/109:59:191
67722442,36sleep10-21swapper/107:26:591
2734299443,35cyclictest0-21swapper/108:59:381
2734299442,37cyclictest0-21swapper/107:14:391
2734399434,31cyclictest0-21swapper/207:04:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional