You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 21:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Wed Nov 19, 2025 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24612999637,10cyclictest26848-21cut08:11:561
24612999637,10cyclictest26848-21cut08:11:561
2461799954,14cyclictest0-21swapper/207:32:102
2461799925,14cyclictest0-21swapper/209:17:032
2461799924,11cyclictest30801-21aten2.4-expect10:36:422
2461799904,62cyclictest0-21swapper/210:42:072
2461799904,14cyclictest0-21swapper/210:31:582
24617999018,49cyclictest16298-21aten2.4-expect12:21:392
2461799894,13cyclictest15723-21grep07:47:102
24617998718,49cyclictest0-21swapper/208:00:272
2461799865,58cyclictest0-21swapper/207:41:492
2461799864,67cyclictest0-21swapper/210:01:442
24617998619,46cyclictest0-21swapper/211:36:472
24617998619,46cyclictest0-21swapper/209:36:502
2461799854,65cyclictest0-21swapper/208:40:332
2461799854,57cyclictest0-21swapper/210:26:462
2461799854,57cyclictest0-21swapper/210:26:462
2461799853,66cyclictest0-21swapper/211:51:402
2461799845,13cyclictest0-21swapper/210:55:552
2461799834,15cyclictest0-21swapper/207:02:072
24617998320,55cyclictest0-21swapper/209:01:352
24617998319,41cyclictest0-21swapper/207:17:112
24617998317,45cyclictest0-21swapper/212:11:082
24617998310,9cyclictest114550irq/46-eth009:46:532
2461799825,67cyclictest0-21swapper/208:29:362
2461799825,62cyclictest0-21swapper/207:25:212
2461799824,69cyclictest22799-21df_abs09:11:502
2461799824,59cyclictest0-21swapper/211:26:402
2461799824,18cyclictest0-21swapper/211:16:432
2461799824,13cyclictest4539-21aten_r1power_en07:26:512
2461799824,13cyclictest4539-21aten_r1power_en07:26:512
24617998220,54cyclictest18781-21tail12:26:402
24617998220,54cyclictest18781-21tail12:26:402
2461799815,68cyclictest0-21swapper/208:11:522
2461799815,68cyclictest0-21swapper/208:11:522
2461799805,67cyclictest0-21swapper/211:01:492
2461799805,67cyclictest0-21swapper/209:06:562
2461799805,14cyclictest0-21swapper/211:14:522
24617998024,49cyclictest23928-21tr08:06:522
2461799795,64cyclictest1369-21runrttasks10:13:462
2461799794,67cyclictest0-21swapper/209:56:442
24617997920,50cyclictest14476-21grep11:07:042
2461799785,64cyclictest0-21swapper/211:23:462
2461799785,16cyclictest0-21swapper/209:21:462
2461799784,54cyclictest0-21swapper/208:16:472
2461799784,23cyclictest0-21swapper/211:42:022
2461799784,13cyclictest0-21swapper/208:22:082
2461799783,16cyclictest0-21swapper/210:58:352
24617997826,45cyclictest1369-21runrttasks10:46:552
24617997824,47cyclictest2449-21metacity12:21:242
24617997819,51cyclictest27568-21grep07:06:542
24617997816,11cyclictest6163-21unixbench_singl09:42:072
24617997816,11cyclictest6163-21unixbench_singl09:42:072
2461799778,12cyclictest0-21swapper/211:56:392
2461799775,15cyclictest0-21swapper/208:56:562
2461799775,15cyclictest0-21swapper/208:56:562
2461799775,14cyclictest0-21swapper/212:11:522
2461799774,66cyclictest1243-21snmpd09:55:442
2461799774,65cyclictest0-21swapper/210:21:432
2461799774,65cyclictest0-21swapper/208:32:012
2461799774,58cyclictest0-21swapper/208:51:502
2461799774,16cyclictest0-21swapper/211:47:152
2461799774,16cyclictest0-21swapper/211:47:152
2461799774,14cyclictest0-21swapper/207:01:382
24617997723,46cyclictest2449-21metacity09:33:302
24617997719,9cyclictest2196-21nscd10:21:302
24617997715,42cyclictest0-21swapper/211:31:392
2461799765,11cyclictest1369-21runrttasks09:27:312
2461799764,64cyclictest0-21swapper/207:36:492
2461799764,22cyclictest0-21swapper/208:02:002
24617997624,44cyclictest11893-21switchtime08:47:082
2461799755,53cyclictest0-21swapper/207:11:522
2461799755,15cyclictest0-21swapper/207:51:512
24617997524,44cyclictest16398-21aten_r1power_en10:06:432
2461799744,63cyclictest0-21swapper/212:01:472
2461799725,15cyclictest0-21swapper/208:41:512
78062643,54sleep00-21swapper/009:46:550
24619996215,40cyclictest22436-21taskset11:26:413
24619996213,41cyclictest115061munin-node12:11:563
24619995926,26cyclictest1369-21runrttasks10:57:473
2461299593,45cyclictest0-21swapper/107:11:521
248122584,14sleep00-21swapper/011:31:410
24619995816,34cyclictest1243-21snmpd09:30:413
24619995813,29cyclictest11240-21hddtemp_smartct11:01:513
2460899585,36cyclictest0-21swapper/008:11:350
24619995715,35cyclictest30471-21iostat_ios11:41:523
24619995710,31cyclictest0-21swapper/309:31:473
2460899574,25cyclictest0-21swapper/009:36:470
243799570,6rtkit-daemon2436-21rtkit-daemon09:02:030
243799570,5rtkit-daemon2436-21rtkit-daemon08:36:490
2461999565,44cyclictest0-21swapper/307:47:003
2461999564,44cyclictest0-21swapper/309:55:403
24619995617,32cyclictest1369-21runrttasks07:29:543
24619995617,32cyclictest1369-21runrttasks07:29:543
24619995614,25cyclictest18485-21smartctl09:01:553
24619995611,39cyclictest0-21swapper/309:19:543
24612995639,10cyclictest27369-21tr11:36:441
243799560,5rtkit-daemon2436-21rtkit-daemon11:41:570
243799560,5rtkit-daemon2436-21rtkit-daemon08:41:560
278912554,10sleep00-21swapper/008:13:340
278912554,10sleep00-21swapper/008:13:340
2461999559,41cyclictest0-21swapper/309:37:573
2461999559,39cyclictest6788-21munin-node09:46:533
2461999558,33cyclictest0-21swapper/312:01:573
2461999557,41cyclictest2932-21latency_hist10:46:293
2461999557,31cyclictest0-21swapper/308:56:523
2461999557,31cyclictest0-21swapper/308:56:523
24619995537,11cyclictest26817-21head08:11:553
24619995537,11cyclictest26817-21head08:11:553
24619995512,34cyclictest1369-21runrttasks11:53:243
24619995510,36cyclictest22489-21stty09:11:473
2461299557,40cyclictest0-21swapper/110:57:021
2460899554,30cyclictest0-21swapper/010:36:300
24608995514,24cyclictest22065-21idleruntime10:16:550
243799550,5rtkit-daemon2436-21rtkit-daemon11:55:340
243799550,5rtkit-daemon2436-21rtkit-daemon11:46:570
243799550,5rtkit-daemon2436-21rtkit-daemon11:46:570
243799550,5rtkit-daemon2436-21rtkit-daemon07:41:510
243799550,5rtkit-daemon2436-21rtkit-daemon07:26:520
243799550,5rtkit-daemon2436-21rtkit-daemon07:26:520
326792543,10sleep00-21swapper/010:39:390
2461999549,37cyclictest11303-21latency_hist12:11:263
2461999548,39cyclictest0-21swapper/312:16:403
2461999545,41cyclictest5383-21if_eth009:41:543
2461999545,41cyclictest5383-21if_eth009:41:543
24619995437,10cyclictest22032-21tail08:01:553
24619995417,31cyclictest26012-21cat07:02:023
24619995415,31cyclictest1243-21snmpd11:23:023
24619995413,34cyclictest16353-21munin-node10:06:533
24612995438,9cyclictest31282-21sed09:27:071
2461299543,45cyclictest0-21swapper/109:56:001
24612995416,25cyclictest29174-21sshd07:09:131
24608995424,9cyclictest20485-21unixbench_singl10:12:050
24608995423,13cyclictest0-21swapper/008:52:030
24608995423,11cyclictest29607-21chrt07:11:500
243799540,6rtkit-daemon2436-21rtkit-daemon12:12:031
243799540,5rtkit-daemon2436-21rtkit-daemon12:31:250
243799540,5rtkit-daemon2436-21rtkit-daemon12:31:250
243799540,5rtkit-daemon2436-21rtkit-daemon12:01:491
243799540,5rtkit-daemon2436-21rtkit-daemon08:17:050
243799540,5rtkit-daemon2436-21rtkit-daemon07:17:070
232452543,44sleep00-21swapper/011:26:500
206512544,10sleep00-21swapper/007:57:370
146882544,10sleep00-21swapper/011:09:010
269372532,44sleep10-21swapper/107:06:071
2461999538,27cyclictest20359-21unixbench_multi12:27:003
2461999538,27cyclictest20359-21unixbench_multi12:27:003
24619995336,11cyclictest13083-21cut11:06:453
24619995313,33cyclictest20545-21df_inode09:06:503
24619995313,33cyclictest1369-21runrttasks08:34:193
24619995312,34cyclictest20090-21processes10:12:003
24619995311,35cyclictest27173-21tail09:21:453
24619995311,35cyclictest11296-21smartctl08:46:573
24619995310,36cyclictest23898-21munin-node10:21:463
24619995310,25cyclictest0-21swapper/307:21:513
2460899536,29cyclictest0-21swapper/009:06:510
2460899535,29cyclictest0-21swapper/011:13:020
2460899535,29cyclictest0-21swapper/007:52:120
2460899535,28cyclictest0-21swapper/010:56:300
2460899535,28cyclictest0-21swapper/009:55:440
2460899534,29cyclictest0-21swapper/010:46:280
2460899534,29cyclictest0-21swapper/010:21:580
2460899534,26cyclictest0-21swapper/012:16:390
24608995336,10cyclictest12120-21df09:56:480
24608995320,15cyclictest0-21swapper/008:57:070
24608995320,15cyclictest0-21swapper/008:57:070
243799530,5rtkit-daemon2436-21rtkit-daemon11:21:430
243799530,5rtkit-daemon2436-21rtkit-daemon10:17:001
243799530,5rtkit-daemon2436-21rtkit-daemon08:26:570
113642533,10sleep00-21swapper/007:41:130
73532523,43sleep10-21swapper/108:37:081
28792524,9sleep10-21swapper/108:30:031
2461999529,36cyclictest16652-21cpuspeed07:51:523
2461999523,35cyclictest1243-21snmpd08:40:333
24619995214,31cyclictest23352-21date10:21:303
24619995212,32cyclictest12139-21df_abs09:56:483
2460899526,27cyclictest0-21swapper/011:05:570
2460899525,29cyclictest0-21swapper/011:16:560
2460899525,28cyclictest0-21swapper/012:00:030
2460899525,27cyclictest1369-21runrttasks12:25:020
2460899524,30cyclictest0-21swapper/009:11:590
2460899524,29cyclictest0-21swapper/008:46:520
2460899524,28cyclictest0-21swapper/007:06:510
24608995221,14cyclictest0-21swapper/009:22:050
167662522,42sleep10-21swapper/108:57:071
167662522,42sleep10-21swapper/108:57:071
133812525,12sleep00-21swapper/012:13:000
130082522,43sleep10-21swapper/108:51:511
51332512,41sleep10-21swapper/110:49:111
33752513,41sleep10-21swapper/107:22:081
2461999515,38cyclictest0-21swapper/307:32:003
24619995130,15cyclictest0-21swapper/310:36:563
24619995117,26cyclictest1369-21runrttasks08:23:323
24619995113,31cyclictest1369-21runrttasks07:44:213
24612995134,11cyclictest13065-21cut11:06:451
2460899516,27cyclictest0-21swapper/010:26:560
2460899516,27cyclictest0-21swapper/010:26:560
2460899516,13cyclictest0-21swapper/010:50:540
2460899515,28cyclictest0-21swapper/012:01:540
2460899515,27cyclictest0-21swapper/009:32:060
2460899514,29cyclictest0-21swapper/011:36:470
2460899514,29cyclictest0-21swapper/010:06:570
2460899514,28cyclictest0-21swapper/007:26:380
2460899514,26cyclictest0-21swapper/009:17:020
243799510,5rtkit-daemon2436-21rtkit-daemon12:01:191
243799510,5rtkit-daemon2436-21rtkit-daemon08:22:120
243799510,5rtkit-daemon2436-21rtkit-daemon07:32:070
243799510,4rtkit-daemon2436-21rtkit-daemon10:21:521
243799510,4rtkit-daemon2436-21rtkit-daemon10:11:451
213362513,41sleep121400-21switchtime11:21:591
161532513,10sleep00-21swapper/007:51:290
110102513,10sleep00-21swapper/012:08:130
50802504,10sleep00-21swapper/008:32:360
39192502,41sleep10-21swapper/109:38:221
2461999506,37cyclictest24001-21cstates08:06:523
2461999504,11cyclictest0-21swapper/308:27:013
24619995018,25cyclictest357-21tail07:16:583
24619995015,27cyclictest1369-21runrttasks08:16:383
24619995014,25cyclictest4306-21sort10:46:553
24619995014,21cyclictest4467-21runrttasks11:56:393
2460899505,27cyclictest0-21swapper/011:00:560
2460899503,29cyclictest0-21swapper/008:06:350
24608995019,13cyclictest0-21swapper/009:26:510
243799500,4rtkit-daemon2436-21rtkit-daemon11:06:281
243799500,4rtkit-daemon2436-21rtkit-daemon09:42:240
243799500,4rtkit-daemon2436-21rtkit-daemon09:42:240
234272504,39sleep00-21swapper/006:57:010
193942506,10sleep10-21swapper/109:02:111
2461999498,35cyclictest14545-21diskstats10:01:483
2461999496,22cyclictest20945-21runrttasks08:01:273
24619994917,25cyclictest6401-21gltestperf10:51:513
24619994915,10cyclictest30732-21memory07:12:033
24619994914,28cyclictest16804-21munin-node12:21:433
2461299495,27cyclictest0-21swapper/108:20:301
2461299494,27cyclictest0-21swapper/107:47:131
2460899495,27cyclictest0-21swapper/007:01:530
24608994918,24cyclictest14094-21tail10:01:440
243799490,5rtkit-daemon2436-21rtkit-daemon09:20:021
220342492,40sleep10-21swapper/109:10:291
135562492,40sleep10-21swapper/109:59:181
2461999489,33cyclictest401-21sed11:46:523
2461999489,33cyclictest401-21sed11:46:523
2461999484,38cyclictest0-21swapper/308:42:083
2461999484,27cyclictest0-21swapper/310:27:023
2461999484,27cyclictest0-21swapper/310:27:023
24619994827,15cyclictest0-21swapper/311:35:213
24619994810,32cyclictest12909-21aten_r1power_po08:51:503
2461299484,27cyclictest0-21swapper/111:56:271
2461299483,27cyclictest0-21swapper/108:01:361
24612994816,25cyclictest1618-21nscd11:16:281
243799480,5rtkit-daemon2436-21rtkit-daemon10:06:381
243799480,5rtkit-daemon2436-21rtkit-daemon07:18:521
135882486,10sleep10-21swapper/107:44:081
301592472,39sleep10-21swapper/110:32:511
2461999475,36cyclictest0-21swapper/311:36:423
2461999474,26cyclictest0-21swapper/311:17:033
24612994732,9cyclictest26277-21cut10:26:481
24612994732,9cyclictest26277-21cut10:26:481
24612994731,10cyclictest24103-21unixbench_singl09:12:081
243799470,5rtkit-daemon2436-21rtkit-daemon12:06:411
62822462,36sleep10-21swapper/109:43:051
62822462,36sleep10-21swapper/109:43:051
52822463,37sleep10-21swapper/108:35:151
264962465,10sleep10-21swapper/111:32:151
2461999464,25cyclictest0-21swapper/310:35:523
24619994618,12cyclictest27220-21munin-node07:07:063
24619994612,24cyclictest16150-21irqstats11:11:543
2461299465,35cyclictest0-21swapper/108:01:481
2461299464,34cyclictest0-21swapper/107:31:381
2461299464,34cyclictest0-21swapper/107:31:381
2461299463,36cyclictest0-21swapper/107:36:371
24612994624,9cyclictest8013-21munin-node08:41:571
24612994617,13cyclictest0-21swapper/112:21:391
325992452,36sleep10-21swapper/108:22:461
2461299454,35cyclictest0-21swapper/111:28:021
2461299454,24cyclictest0-21swapper/107:36:491
24612994524,10cyclictest0-21swapper/112:16:481
24612994516,10cyclictest10494-21telnet08:46:481
13142452,37sleep10-21swapper/111:48:191
13142452,37sleep10-21swapper/111:48:191
326652444,10sleep10-21swapper/110:39:281
2461299444,33cyclictest0-21swapper/107:01:391
2461299443,26cyclictest0-21swapper/111:16:581
2461299443,25cyclictest0-21swapper/109:21:581
2461299443,23cyclictest18-21rcuc/110:52:011
24612994421,10cyclictest0-21swapper/110:02:051
24612994420,8cyclictest20817-21kworker/u8:212:31:251
24612994420,8cyclictest20817-21kworker/u8:212:31:251
14502442,36sleep10-21swapper/109:34:031
67872433,9sleep10-21swapper/109:46:441
2461999434,32cyclictest0-21swapper/307:01:393
24619994310,21cyclictest0-21swapper/307:36:493
2461299434,33cyclictest0-21swapper/110:42:021
24612994321,9cyclictest16409-21munin-node07:52:041
24612994222,13cyclictest0-21swapper/111:42:011
2461299413,34cyclictest0-21swapper/108:06:501
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional