You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 21:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Thu Dec 04, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2437991000,3rtkit-daemon0-21swapper/310:02:382
243799990,6rtkit-daemon0-21swapper/311:36:492
3205599914,52cyclictest0-21swapper/310:27:473
32053999028,10cyclictest28217-21gltestperf11:07:441
194522900,5sleep041ktimersoftd/009:42:410
243799890,5rtkit-daemon2436-21rtkit-daemon10:36:291
243799890,5rtkit-daemon2436-21rtkit-daemon10:36:291
3205599883,11cyclictest0-21swapper/311:37:383
3205599865,63cyclictest0-21swapper/311:27:533
3205599864,75cyclictest0-21swapper/312:12:383
3205599864,59cyclictest0-21swapper/309:13:033
3205599854,57cyclictest0-21swapper/310:57:433
3205599853,12cyclictest0-21swapper/311:42:513
32053998529,10cyclictest5216-21unixbench_multi08:03:011
32053998524,55cyclictest1369-21runrttasks08:55:031
32053998433,11cyclictest16808-21aten2.4-expect07:22:421
32053998430,9cyclictest27938-21unixbench_singl09:58:001
3205599834,61cyclictest0-21swapper/309:57:403
32055998315,44cyclictest30955-21munin-node10:07:443
32053998332,9cyclictest23495-21sed10:57:461
3205399833,20cyclictest20187-21hostname10:52:371
32053998328,9cyclictest1409-21unixbench_multi11:17:561
32053998328,9cyclictest1409-21unixbench_multi11:17:561
32053998327,9cyclictest15728-21cpuspeed_turbos10:42:381
32053998327,10cyclictest31774-21gltestperf10:07:441
32053998324,9cyclictest1346-21cpuspeed_turbos07:57:441
32053998319,20cyclictest1369-21runrttasks11:16:001
3205599824,55cyclictest0-21swapper/307:02:503
32053998228,10cyclictest19362-21cpuspeed_turbos09:42:411
3205599817,59cyclictest30997-21sshd08:57:523
3205599815,54cyclictest0-21swapper/308:12:493
3205599814,61cyclictest0-21swapper/309:27:303
3205599814,61cyclictest0-21swapper/308:00:353
3205599814,54cyclictest0-21swapper/307:22:573
32053998128,9cyclictest16207-21fschecks_time08:27:481
32053998128,10cyclictest21797-21ls08:38:001
32053998127,9cyclictest15208-21kernelversion11:47:451
243799810,4rtkit-daemon2436-21rtkit-daemon06:48:233
3205599804,54cyclictest0-21swapper/311:07:573
3205599804,53cyclictest0-21swapper/309:32:573
3205599804,12cyclictest0-21swapper/311:26:073
3205599803,8cyclictest6856-21iostat_ios10:22:463
3205599803,13cyclictest0-21swapper/308:02:473
32053998027,10cyclictest4048-21fschecks_count10:17:411
3205599794,61cyclictest0-21swapper/310:52:453
3205599794,59cyclictest0-21swapper/311:47:543
3205599794,59cyclictest0-21swapper/309:52:383
3205599794,53cyclictest0-21swapper/312:01:133
32055997910,53cyclictest1618-21nscd11:37:203
3205399794,34cyclictest0-21swapper/111:37:341
32053997933,11cyclictest3077-21gltestperf11:22:421
32053997927,10cyclictest1369-21runrttasks08:47:281
243799790,6rtkit-daemon2436-21rtkit-daemon07:44:351
243799790,6rtkit-daemon2436-21rtkit-daemon07:44:351
243799790,4rtkit-daemon2436-21rtkit-daemon08:23:193
243799790,4rtkit-daemon2436-21rtkit-daemon08:23:193
3205599784,66cyclictest0-21swapper/308:12:293
3205599784,58cyclictest0-21swapper/311:12:353
3205599784,55cyclictest0-21swapper/308:32:513
243799780,4rtkit-daemon2436-21rtkit-daemon08:49:033
243799780,4rtkit-daemon2436-21rtkit-daemon07:33:283
3205599776,54cyclictest2196-21nscd10:17:243
3205599774,56cyclictest0-21swapper/311:53:533
32055997718,37cyclictest0-21swapper/308:42:413
243799770,4rtkit-daemon2436-21rtkit-daemon12:08:553
243799770,4rtkit-daemon2436-21rtkit-daemon07:45:583
243799770,4rtkit-daemon2436-21rtkit-daemon07:45:583
3205599764,65cyclictest0-21swapper/307:12:543
3205599764,57cyclictest0-21swapper/307:17:443
3205599764,56cyclictest0-21swapper/309:17:433
3205599764,55cyclictest0-21swapper/309:07:413
3205599764,55cyclictest0-21swapper/309:07:413
3205599763,8cyclictest22345-21iostat_ios12:02:443
32055997612,41cyclictest0-21swapper/308:37:573
3205399764,33cyclictest0-21swapper/110:42:141
3205399763,39cyclictest0-21swapper/109:39:561
32053997622,9cyclictest0-21swapper/108:32:441
243799760,4rtkit-daemon2436-21rtkit-daemon11:18:203
243799760,4rtkit-daemon2436-21rtkit-daemon11:18:203
3205599755,62cyclictest0-21swapper/309:02:463
3205599755,12cyclictest16640-21aten2.4-expect09:37:353
3205599754,64cyclictest0-21swapper/310:33:003
3205599754,64cyclictest0-21swapper/310:33:003
3205599754,58cyclictest0-21swapper/308:27:593
3205599754,47cyclictest0-21swapper/306:47:333
3205599753,8cyclictest21987-21diskstats09:47:423
3205599753,8cyclictest21220-21crond07:32:303
32055997511,40cyclictest114550irq/46-eth006:58:043
32055997511,40cyclictest114550irq/46-eth006:58:043
3205599744,63cyclictest0-21swapper/310:42:503
3205599744,56cyclictest0-21swapper/308:22:283
3205599744,50cyclictest0-21swapper/310:41:003
243799740,5rtkit-daemon2436-21rtkit-daemon07:48:582
3205599735,50cyclictest0-21swapper/309:45:503
3205599734,9cyclictest114550irq/46-eth007:50:333
3205599734,54cyclictest0-21swapper/307:52:593
3205599734,49cyclictest0-21swapper/307:37:413
32053997331,10cyclictest0-21swapper/107:30:531
3205599724,47cyclictest0-21swapper/310:17:543
3205599724,11cyclictest0-21swapper/306:55:253
3205599723,49cyclictest28571-21munin-node10:02:503
32055997217,49cyclictest19208-21sendmail_mailqu10:47:533
3205399723,36cyclictest0-21swapper/111:54:541
3205399723,32cyclictest0-21swapper/111:47:211
32053997222,9cyclictest0-21swapper/107:51:361
3205599713,52cyclictest27449-21taskset08:52:393
3205599713,47cyclictest0-21swapper/307:07:443
3205599703,10cyclictest0-21swapper/309:24:513
32053997012,26cyclictest0-21swapper/110:31:541
3205599693,8cyclictest1243-21snmpd11:07:143
3052226912,46sleep20-21swapper/206:42:502
243799690,6rtkit-daemon2436-21rtkit-daemon08:25:492
243799690,6rtkit-daemon2436-21rtkit-daemon08:25:492
243799690,6rtkit-daemon2436-21rtkit-daemon06:50:152
32053996821,10cyclictest0-21swapper/108:58:351
243799680,6rtkit-daemon2436-21rtkit-daemon10:55:582
243799680,4rtkit-daemon2436-21rtkit-daemon08:49:102
32053996721,10cyclictest0-21swapper/107:15:531
243799670,6rtkit-daemon2436-21rtkit-daemon12:12:332
243799670,5rtkit-daemon2436-21rtkit-daemon12:08:182
243799670,5rtkit-daemon2436-21rtkit-daemon10:38:142
243799670,5rtkit-daemon2436-21rtkit-daemon10:22:522
243799670,5rtkit-daemon2436-21rtkit-daemon07:33:212
243799670,4rtkit-daemon2436-21rtkit-daemon09:37:462
243799670,4rtkit-daemon2436-21rtkit-daemon07:44:062
243799670,4rtkit-daemon2436-21rtkit-daemon07:44:062
243799660,5rtkit-daemon2436-21rtkit-daemon11:43:042
243799660,5rtkit-daemon2436-21rtkit-daemon11:38:262
243799660,5rtkit-daemon2436-21rtkit-daemon11:23:042
243799660,5rtkit-daemon2436-21rtkit-daemon08:18:272
243799660,5rtkit-daemon2436-21rtkit-daemon07:11:522
243799660,4rtkit-daemon2436-21rtkit-daemon11:18:122
243799660,4rtkit-daemon2436-21rtkit-daemon11:18:122
243799660,4rtkit-daemon2436-21rtkit-daemon09:06:552
243799660,4rtkit-daemon2436-21rtkit-daemon08:58:182
243799660,4rtkit-daemon2436-21rtkit-daemon08:27:342
32053996521,9cyclictest0-21swapper/107:11:401
243799650,4rtkit-daemon2436-21rtkit-daemon12:02:552
243799650,4rtkit-daemon2436-21rtkit-daemon08:16:272
243799650,4rtkit-daemon2436-21rtkit-daemon07:38:132
243799650,4rtkit-daemon2436-21rtkit-daemon07:12:372
315322647,50sleep00-21swapper/008:58:010
243799640,4rtkit-daemon2436-21rtkit-daemon07:03:522
3205299624,50cyclictest0-21swapper/011:22:560
72422617,9sleep07381-21tr09:17:400
150692618,45sleep00-21swapper/009:32:470
3205399603,16cyclictest0-21swapper/107:37:471
3205499594,37cyclictest0-21swapper/209:12:542
3205499594,13cyclictest0-21swapper/211:56:112
243799590,4rtkit-daemon2436-21rtkit-daemon11:27:492
3205499583,12cyclictest0-21swapper/210:02:262
243799580,4rtkit-daemon2436-21rtkit-daemon11:05:202
62972573,47sleep20-21swapper/208:07:452
3205499574,31cyclictest0-21swapper/211:11:472
32054995737,11cyclictest7442-21tr09:17:402
27312573,47sleep00-21swapper/009:07:440
27312573,47sleep00-21swapper/009:07:440
32054995638,10cyclictest20710-21unixbench_singl11:57:542
246072554,11sleep00-21swapper/009:52:460
79192542,12sleep00-21swapper/008:10:450
3205499544,31cyclictest0-21swapper/210:07:582
3205499544,30cyclictest0-21swapper/210:17:402
243799540,4rtkit-daemon2436-21rtkit-daemon06:52:532
185062544,13sleep00-21swapper/011:54:490
325942530,7sleep032592-21tr06:47:460
3205499535,13cyclictest0-21swapper/211:16:062
3205499534,30cyclictest0-21swapper/210:57:462
3205499534,30cyclictest0-21swapper/209:07:552
3205499534,30cyclictest0-21swapper/209:07:552
3205499534,30cyclictest0-21swapper/208:42:572
3205499534,30cyclictest0-21swapper/207:17:482
3205499534,29cyclictest0-21swapper/208:33:012
32054995321,13cyclictest0-21swapper/206:57:552
32054995321,13cyclictest0-21swapper/206:57:552
32052995324,10cyclictest1954-21latency_hist09:07:280
270172535,41sleep00-21swapper/011:06:210
243799530,4rtkit-daemon2436-21rtkit-daemon07:52:432
76532523,43sleep00-21swapper/010:23:090
3205499525,29cyclictest0-21swapper/208:00:352
3205499524,29cyclictest0-21swapper/209:52:462
3205499524,29cyclictest0-21swapper/208:02:492
32054995236,10cyclictest22259-21head09:47:452
32054995223,11cyclictest127602sadc10:37:232
32054995223,11cyclictest127602sadc10:37:232
32054995220,13cyclictest0-21swapper/207:32:312
3205499522,36cyclictest0-21swapper/210:44:172
3205299524,42cyclictest0-21swapper/008:27:410
3205299524,42cyclictest0-21swapper/007:28:040
32052995236,11cyclictest0-21swapper/008:05:360
32052995235,9cyclictest28172-21grep08:52:460
32052995233,14cyclictest0-21swapper/007:52:560
32052995231,15cyclictest0-21swapper/011:47:390
281042523,42sleep00-21swapper/009:59:270
250662527,37sleep00-21swapper/008:47:410
213102524,41sleep00-21swapper/012:02:230
64282514,41sleep00-21swapper/011:28:140
3205499514,28cyclictest0-21swapper/211:52:212
32054995119,14cyclictest0-21swapper/209:22:492
32054995119,14cyclictest0-21swapper/208:52:452
32054995115,10cyclictest0-21swapper/210:47:352
3205299515,27cyclictest0-21swapper/009:27:530
32052995133,10cyclictest7169-21head07:02:440
3205299513,43cyclictest0-21swapper/007:12:020
32052995122,12cyclictest0-21swapper/011:57:400
236992513,12sleep00-21swapper/012:07:230
195982513,41sleep00-21swapper/010:47:590
126592514,11sleep00-21swapper/010:36:110
126592514,11sleep00-21swapper/010:36:110
52842503,12sleep00-21swapper/010:18:210
3205499505,27cyclictest0-21swapper/207:25:302
3205499503,28cyclictest0-21swapper/209:42:482
3205299504,27cyclictest0-21swapper/008:37:490
32052995032,13cyclictest0-21swapper/007:12:440
3205299503,40cyclictest0-21swapper/011:12:350
32052995025,11cyclictest0-21swapper/011:42:420
32052995025,10cyclictest0-21swapper/009:22:390
32052995023,9cyclictest4526-21latency_hist06:57:320
32052995023,13cyclictest0-21swapper/009:13:010
32052995021,13cyclictest0-21swapper/009:37:540
3205499495,36cyclictest0-21swapper/210:12:482
3205499494,24cyclictest0-21swapper/209:27:392
32054994921,10cyclictest21318-21latency08:37:522
3205299495,37cyclictest0-21swapper/010:27:550
3205299494,38cyclictest0-21swapper/008:17:300
3205299494,27cyclictest0-21swapper/007:58:010
3205299494,27cyclictest0-21swapper/006:57:540
3205299494,27cyclictest0-21swapper/006:57:540
32052994925,10cyclictest0-21swapper/008:43:010
29922494,11sleep00-21swapper/010:14:340
198352493,39sleep00-21swapper/008:35:320
174132494,10sleep00-21swapper/010:45:380
149552492,13sleep00-21swapper/010:39:450
3205499484,27cyclictest0-21swapper/210:27:382
3205299484,26cyclictest0-21swapper/007:42:520
3205299484,26cyclictest0-21swapper/007:42:520
3205299483,39cyclictest0-21swapper/011:37:330
3205299483,39cyclictest0-21swapper/011:07:450
3205299483,39cyclictest0-21swapper/007:32:420
304762482,11sleep00-21swapper/010:04:190
161802483,39sleep00-21swapper/007:18:580
154002483,39sleep115444-21needreboot09:32:521
3205499474,24cyclictest0-21swapper/209:32:382
3205399474,38cyclictest5881-21munin-node09:12:541
3205399473,27cyclictest0-21swapper/107:37:321
32053994719,22cyclictest0-21swapper/107:57:051
32052994731,11cyclictest0-21swapper/010:57:500
32052994730,11cyclictest0-21swapper/009:47:550
32052994728,13cyclictest0-21swapper/010:52:370
260922474,36sleep10-21swapper/112:12:241
199272475,36sleep10-21swapper/110:52:161
161692477,34sleep10-21swapper/107:18:511
3205399463,36cyclictest0-21swapper/106:47:501
32053994613,8cyclictest114550irq/46-eth007:03:021
32052994629,12cyclictest0-21swapper/008:18:000
32052994628,13cyclictest0-21swapper/012:17:210
32052994618,13cyclictest0-21swapper/011:34:450
32052994615,9cyclictest26022-21latency_hist07:42:310
77232457,33sleep10-21swapper/110:24:021
5782454,35sleep00-21swapper/010:11:320
3205399453,25cyclictest0-21swapper/111:02:361
32053994519,12cyclictest0-21swapper/110:02:461
3205299454,25cyclictest0-21swapper/007:47:420
32053994429,10cyclictest1369-21runrttasks09:02:441
32053994417,12cyclictest0-21swapper/108:17:301
3205299444,35cyclictest0-21swapper/008:26:290
3205299444,35cyclictest0-21swapper/008:26:290
32052994415,13cyclictest0-21swapper/011:17:410
32052994415,13cyclictest0-21swapper/011:17:410
317732443,35sleep00-21swapper/006:44:210
186602443,35sleep00-21swapper/007:25:150
3205399434,32cyclictest0-21swapper/108:47:481
3205399433,32cyclictest0-21swapper/108:22:291
32053994327,11cyclictest0-21swapper/109:47:491
32053994321,9cyclictest0-21swapper/106:53:031
32053994321,10cyclictest0-21swapper/111:59:151
32053994311,25cyclictest0-21swapper/108:07:421
3205399423,12cyclictest0-21swapper/109:56:321
32053994228,10cyclictest0-21swapper/108:22:561
32053994228,10cyclictest0-21swapper/108:22:561
32053994221,9cyclictest0-21swapper/110:13:321
32053994220,10cyclictest0-21swapper/109:29:241
3205399413,34cyclictest0-21swapper/106:57:501
3205399413,34cyclictest0-21swapper/106:57:501
3205399413,33cyclictest0-21swapper/111:27:331
3205399413,31cyclictest0-21swapper/109:22:271
3205399413,11cyclictest0-21swapper/112:02:361
3205399403,32cyclictest0-21swapper/106:47:331
32053994026,9cyclictest7580-21munin-node11:32:381
3205399393,32cyclictest0-21swapper/109:09:101
3205399393,32cyclictest0-21swapper/109:09:101
32053993927,8cyclictest24087-21tr12:07:361
32053993921,11cyclictest9459-21munin-node09:22:381
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional