You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-05 - 16:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Wed Nov 05, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3221599914,68cyclictest0-21swapper/009:02:440
3221599914,68cyclictest0-21swapper/009:02:440
3221599905,64cyclictest0-21swapper/005:42:470
3221599904,67cyclictest0-21swapper/008:52:090
3221599893,68cyclictest9475-21ls04:57:350
3221599884,68cyclictest0-21swapper/009:52:530
3221599884,63cyclictest0-21swapper/004:57:560
3221599884,62cyclictest0-21swapper/004:38:080
3221599874,68cyclictest0-21swapper/008:07:510
3221599864,66cyclictest0-21swapper/007:37:440
3221599864,66cyclictest0-21swapper/007:37:440
3221599864,65cyclictest0-21swapper/005:41:380
3221599864,63cyclictest0-21swapper/008:27:530
3221599864,60cyclictest0-21swapper/009:36:520
3221599864,59cyclictest0-21swapper/005:22:510
3221599854,60cyclictest0-21swapper/005:52:570
3221599853,21cyclictest0-21swapper/008:13:060
32215998523,46cyclictest0-21swapper/006:07:320
3221599844,14cyclictest0-21swapper/006:37:530
32215998428,49cyclictest696-21gltestperf09:07:470
32215998415,12cyclictest0-21swapper/005:57:470
32215998413,65cyclictest9549-21munin-node09:28:010
3221599834,65cyclictest0-21swapper/009:37:550
3221599834,64cyclictest0-21swapper/006:12:440
32215998321,48cyclictest0-21swapper/006:52:590
3221599824,70cyclictest0-21swapper/007:42:410
3221599824,63cyclictest0-21swapper/006:58:020
3221599824,22cyclictest0-21swapper/008:32:560
3221599823,62cyclictest0-21swapper/007:52:570
32215998217,48cyclictest0-21swapper/007:47:570
3221599814,63cyclictest0-21swapper/009:12:490
3221599813,23cyclictest0-21swapper/008:37:470
32215998129,46cyclictest17050-21grep09:42:390
32215998129,46cyclictest17050-21grep09:42:390
3221599804,69cyclictest0-21swapper/008:22:040
3221599804,69cyclictest0-21swapper/008:22:040
3221599804,62cyclictest0-21swapper/005:48:060
3221599804,12cyclictest0-21swapper/009:02:110
3221599803,70cyclictest0-21swapper/009:49:360
3221599803,70cyclictest0-21swapper/005:33:030
3221599803,70cyclictest0-21swapper/005:11:170
3221599803,62cyclictest0-21swapper/007:17:390
32215998014,12cyclictest0-21swapper/007:08:120
3221599794,68cyclictest0-21swapper/007:02:520
3221599794,67cyclictest0-21swapper/007:22:500
3221599793,69cyclictest17428-21grep07:27:470
32215997920,53cyclictest0-21swapper/008:52:370
3221599785,58cyclictest0-21swapper/006:08:080
3221599785,58cyclictest0-21swapper/006:08:080
3221599784,66cyclictest0-21swapper/006:42:550
32215997815,56cyclictest0-21swapper/008:02:490
3221599774,12cyclictest0-21swapper/008:27:050
3221599773,21cyclictest0-21swapper/008:02:010
3221599773,20cyclictest0-21swapper/005:06:320
32215997727,45cyclictest19468-21sort05:17:480
32215997722,49cyclictest3445-21time04:42:580
32215997722,49cyclictest3445-21time04:42:580
3221599765,14cyclictest0-21swapper/006:23:210
3221599764,65cyclictest0-21swapper/010:02:450
3221599764,20cyclictest1265-21runrttasks05:29:110
3221599764,20cyclictest1265-21runrttasks05:29:110
32215997618,8cyclictest1265-21runrttasks07:13:360
32215997616,8cyclictest0-21swapper/006:47:500
32215997614,8cyclictest0-21swapper/005:13:030
3221599755,13cyclictest0-21swapper/006:28:330
3221599754,24cyclictest0-21swapper/009:17:550
3221599753,13cyclictest0-21swapper/007:33:530
3221599753,13cyclictest0-21swapper/007:33:530
32215997528,40cyclictest16501-21grep06:17:510
32215997521,46cyclictest24559-21unixbench_singl06:33:040
3221599744,62cyclictest0-21swapper/009:22:480
3221599744,21cyclictest0-21swapper/008:44:240
3221599743,64cyclictest0-21swapper/009:58:010
3221599743,22cyclictest0-21swapper/004:47:530
32218996815,45cyclictest9051-21entropy06:02:503
3221699674,12cyclictest0-21swapper/109:57:381
3221599644,19cyclictest32188-21latency_hist04:37:360
32218996021,12cyclictest0-21swapper/309:57:263
223099600,4rtkit-daemon2229-21rtkit-daemon05:59:203
156612600,4sleep115745-21rt-features09:37:531
3221899594,13cyclictest0-21swapper/310:02:423
3221899594,13cyclictest0-21swapper/305:32:353
3221899594,13cyclictest0-21swapper/305:32:353
223099590,4rtkit-daemon2229-21rtkit-daemon06:34:193
127392594,48sleep112825-21gltestperf07:17:471
3221899585,29cyclictest0-21swapper/308:47:593
3221899585,29cyclictest0-21swapper/307:52:453
3221899585,29cyclictest0-21swapper/307:32:293
3221899584,28cyclictest0-21swapper/307:42:303
3221899584,28cyclictest0-21swapper/307:42:303
3221899584,28cyclictest0-21swapper/306:08:023
3221899584,28cyclictest0-21swapper/306:08:023
3221899584,13cyclictest0-21swapper/305:47:453
32218995838,12cyclictest12922-21crond08:27:263
32218995819,13cyclictest0-21swapper/306:42:413
223099580,4rtkit-daemon2229-21rtkit-daemon06:38:193
223099580,4rtkit-daemon2229-21rtkit-daemon05:35:593
3221899575,30cyclictest0-21swapper/309:14:083
3221899575,29cyclictest0-21swapper/308:53:013
3221899575,29cyclictest0-21swapper/307:47:513
3221899574,29cyclictest0-21swapper/308:12:273
3221899574,29cyclictest0-21swapper/307:02:463
3221899574,29cyclictest0-21swapper/305:08:083
3221899574,28cyclictest0-21swapper/309:07:263
3221899574,28cyclictest0-21swapper/309:07:263
3221899574,13cyclictest0-21swapper/308:07:283
3221899574,12cyclictest0-21swapper/306:18:073
3221699575,11cyclictest0-21swapper/105:57:331
3221899565,27cyclictest0-21swapper/306:12:533
3221899564,30cyclictest0-21swapper/306:23:013
3221899564,29cyclictest0-21swapper/309:52:503
3221899564,29cyclictest0-21swapper/309:32:453
3221899564,29cyclictest0-21swapper/307:42:473
3221899564,28cyclictest0-21swapper/309:22:553
3221899564,28cyclictest0-21swapper/307:17:473
3221899564,27cyclictest0-21swapper/304:52:363
3221899564,27cyclictest0-21swapper/304:42:483
3221899564,27cyclictest0-21swapper/304:42:483
32218995638,10cyclictest29639-21grep05:37:553
3221799563,45cyclictest0-21swapper/205:58:002
3221699564,34cyclictest0-21swapper/107:49:111
3221699564,28cyclictest0-21swapper/108:57:081
32216995637,10cyclictest1118-21ntpdc09:07:531
32216995636,12cyclictest29631-21gltestperf05:37:551
3221899555,29cyclictest0-21swapper/309:07:463
3221899554,32cyclictest0-21swapper/308:12:513
3221899554,29cyclictest0-21swapper/305:03:073
3221899554,28cyclictest0-21swapper/308:27:443
3221899554,10cyclictest5847-21munin-run05:57:323
3221699553,27cyclictest0-21swapper/107:42:111
3221699553,27cyclictest0-21swapper/107:42:111
169862553,44sleep10-21swapper/105:12:481
3221699545,29cyclictest0-21swapper/109:27:531
3221699545,29cyclictest0-21swapper/106:52:301
3221699544,13cyclictest0-21swapper/108:32:061
32216995421,14cyclictest0-21swapper/105:23:101
223099540,5rtkit-daemon2229-21rtkit-daemon07:07:552
47032532,45sleep14782-21missed_timers08:07:531
32217995334,12cyclictest0-21swapper/206:13:052
32217995322,24cyclictest0-21swapper/206:07:322
3221699536,29cyclictest0-21swapper/107:22:431
3221699536,28cyclictest0-21swapper/107:33:061
3221699536,28cyclictest0-21swapper/107:33:061
3221699535,29cyclictest0-21swapper/107:57:271
3221699534,29cyclictest0-21swapper/106:56:501
3221699534,29cyclictest0-21swapper/105:58:011
3221699534,29cyclictest0-21swapper/105:22:341
32216995334,11cyclictest24485-21taskset08:47:581
32216995320,13cyclictest0-21swapper/108:42:261
32216995319,14cyclictest0-21swapper/109:42:561
32216995319,14cyclictest0-21swapper/109:42:561
223099530,5rtkit-daemon2229-21rtkit-daemon09:12:582
32218995234,10cyclictest15189-21cut09:37:443
32217995221,13cyclictest0-21swapper/204:57:532
3221799522,44cyclictest0-21swapper/209:06:592
3221799522,44cyclictest0-21swapper/209:06:592
32217995213,32cyclictest0-21swapper/206:27:522
3221699526,27cyclictest0-21swapper/108:02:411
3221699525,28cyclictest0-21swapper/104:42:521
3221699525,28cyclictest0-21swapper/104:42:521
3221699525,27cyclictest0-21swapper/106:38:051
3221699524,29cyclictest0-21swapper/108:17:031
3221699524,29cyclictest0-21swapper/107:42:571
3221699524,29cyclictest0-21swapper/107:02:431
3221699524,28cyclictest0-21swapper/108:17:481
3221699524,28cyclictest0-21swapper/108:17:481
3221699524,28cyclictest0-21swapper/106:17:331
3221699524,25cyclictest8591-21aten2.4-expect06:02:461
32216995235,11cyclictest4068-21unixbench_multi09:13:001
32216995234,11cyclictest19713-21unixbench_multi06:23:051
32216995221,13cyclictest0-21swapper/109:22:551
32216995221,13cyclictest0-21swapper/106:43:021
204002524,41sleep320449-21munin-node07:32:563
204002524,41sleep320449-21munin-node07:32:563
196882523,42sleep10-21swapper/109:47:411
176582522,9sleep20-21swapper/206:21:092
3221899515,38cyclictest0-21swapper/305:27:343
3221799514,39cyclictest0-21swapper/209:32:342
3221799513,41cyclictest0-21swapper/209:52:532
32217995124,20cyclictest1773-21Xorg05:28:322
32217995124,20cyclictest1773-21Xorg05:28:322
3221699515,28cyclictest0-21swapper/106:08:001
3221699515,28cyclictest0-21swapper/106:08:001
3221699514,29cyclictest0-21swapper/106:22:321
3221699514,29cyclictest0-21swapper/104:37:591
3221699514,28cyclictest0-21swapper/108:27:051
3221699514,28cyclictest0-21swapper/105:33:011
3221699514,28cyclictest0-21swapper/104:53:001
223099510,5rtkit-daemon2229-21rtkit-daemon09:08:282
198522512,9sleep20-21swapper/206:23:392
91832502,9sleep20-21swapper/209:25:532
69142502,42sleep30-21swapper/309:22:233
3221899504,28cyclictest0-21swapper/306:58:053
32218995033,10cyclictest7626-21cpuspeed_turbos04:52:493
32218995031,12cyclictest9264-21gltestperf08:17:483
32218995031,12cyclictest9264-21gltestperf08:17:483
3221799504,27cyclictest0-21swapper/209:47:392
32217995035,10cyclictest0-21swapper/205:42:482
3221699504,28cyclictest0-21swapper/110:07:231
3221699504,28cyclictest0-21swapper/109:02:551
3221699504,28cyclictest0-21swapper/109:02:551
3221699504,27cyclictest0-21swapper/107:12:491
32216995035,9cyclictest5789-21sed04:47:561
32216995027,13cyclictest0-21swapper/106:32:321
32216995019,13cyclictest0-21swapper/107:07:411
295382503,11sleep10-21swapper/109:01:051
223102503,41sleep10-21swapper/108:45:091
139252503,10sleep10-21swapper/109:35:111
116892503,41sleep10-21swapper/105:00:311
32218994928,13cyclictest9990-21cpuspeed_turbos07:12:463
3221899492,21cyclictest0-21swapper/306:30:433
3221799493,39cyclictest0-21swapper/204:52:482
32217994927,14cyclictest0-21swapper/208:27:382
3221699495,35cyclictest3267-21gltestperf06:57:541
32216994932,10cyclictest6917-21crond09:22:241
31102492,9sleep20-21swapper/205:48:122
223099490,5rtkit-daemon2229-21rtkit-daemon08:57:583
223099490,5rtkit-daemon2229-21rtkit-daemon04:40:233
223099490,5rtkit-daemon2229-21rtkit-daemon04:39:082
114732492,40sleep20-21swapper/207:13:372
3221899485,34cyclictest0-21swapper/307:57:433
3221899484,34cyclictest0-21swapper/307:07:533
32218994829,12cyclictest19677-21unixbench_multi08:37:573
32218994817,13cyclictest0-21swapper/308:42:533
32217994832,11cyclictest0-21swapper/205:38:082
3221799483,38cyclictest0-21swapper/209:57:432
3221799483,28cyclictest0-21swapper/209:42:512
3221799483,28cyclictest0-21swapper/209:42:512
3221699484,27cyclictest0-21swapper/107:57:421
3221699484,27cyclictest0-21swapper/105:46:391
3221699484,14cyclictest0-21swapper/105:06:211
223099480,5rtkit-daemon2229-21rtkit-daemon06:47:483
3221899474,36cyclictest17931-21strings09:42:503
3221899474,36cyclictest17931-21strings09:42:503
3221899474,26cyclictest0-21swapper/307:26:553
32218994727,12cyclictest19398-21aten_r1power_vo05:17:473
32218994717,23cyclictest361ktimersoftd/304:58:063
3221799474,34cyclictest0-21swapper/208:02:402
32217994732,10cyclictest0-21swapper/210:02:362
32217994732,10cyclictest0-21swapper/204:47:532
32217994730,11cyclictest12483-21unixbench_multi08:23:002
3221799473,37cyclictest0-21swapper/208:42:582
3221799473,27cyclictest0-21swapper/207:02:512
3221799473,27cyclictest0-21swapper/205:33:022
32217994728,12cyclictest0-21swapper/205:10:162
32217994727,13cyclictest0-21swapper/206:52:422
3221699474,9cyclictest15892-21sendmail_mailqu05:08:041
3221699474,26cyclictest0-21swapper/105:31:371
3221699474,26cyclictest0-21swapper/105:31:371
3221699473,27cyclictest0-21swapper/106:36:471
32216994718,12cyclictest0-21swapper/108:32:431
200672473,9sleep20-21swapper/208:42:002
181832473,37sleep10-21swapper/107:27:591
44402463,9sleep20-21swapper/204:44:282
44402463,9sleep20-21swapper/204:44:282
32218994615,13cyclictest0-21swapper/308:32:573
32218994615,13cyclictest0-21swapper/305:12:513
32217994627,13cyclictest0-21swapper/207:27:292
32217994626,13cyclictest23129-21tr06:32:452
32217994625,9cyclictest0-21swapper/205:53:042
32217994617,13cyclictest0-21swapper/206:42:592
318372463,10sleep20-21swapper/206:48:582
223099460,4rtkit-daemon2229-21rtkit-daemon09:47:343
120422462,38sleep20-21swapper/207:17:402
3221899454,13cyclictest0-21swapper/306:52:533
3221899453,35cyclictest0-21swapper/305:47:323
3221799454,25cyclictest0-21swapper/208:52:092
3221799454,24cyclictest0-21swapper/205:17:502
32217994530,9cyclictest21753-21aten2.4-expect05:22:502
32217994530,10cyclictest0-21swapper/209:37:382
3221799453,37cyclictest1265-21runrttasks08:54:132
3221799453,25cyclictest0-21swapper/207:43:042
32217994529,10cyclictest25532-21grep06:37:462
3221699454,22cyclictest0-21swapper/109:52:381
318722453,35sleep30-21swapper/304:33:373
318382452,9sleep20-21swapper/207:57:502
283582453,10sleep20-21swapper/207:50:492
103392452,38sleep20-21swapper/208:19:552
103392452,38sleep20-21swapper/208:19:552
3221799444,34cyclictest0-21swapper/207:52:572
3221799444,33cyclictest0-21swapper/207:39:052
3221799444,33cyclictest0-21swapper/207:39:052
3221799444,33cyclictest0-21swapper/206:57:452
3221799444,33cyclictest0-21swapper/205:02:492
3221799444,13cyclictest0-21swapper/208:57:442
3221799443,36cyclictest0-21swapper/209:28:032
3221799443,35cyclictest0-21swapper/209:17:552
3221799443,35cyclictest0-21swapper/208:12:542
3221799443,24cyclictest0-21swapper/207:27:582
32217994429,10cyclictest0-21swapper/206:07:452
32217994429,10cyclictest0-21swapper/206:07:452
32216994427,10cyclictest1510-21cpuspeed_turbos05:47:481
223099440,5rtkit-daemon2229-21rtkit-daemon09:27:573
3221799434,33cyclictest0-21swapper/205:13:382
3221799434,12cyclictest0-21swapper/208:08:022
32217994227,10cyclictest0-21swapper/208:32:592
32217994226,10cyclictest19807-21grep07:32:482
32217994226,10cyclictest19807-21grep07:32:482
3221699406,23cyclictest0-21swapper/104:37:361
223099360,4rtkit-daemon2229-21rtkit-daemon04:37:092
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional