You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 22:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Tue May 26, 2026 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
402121020,3sleep118-21rcuc/107:10:521
402121020,3sleep118-21rcuc/107:10:521
18411991004,28cyclictest0-21swapper/007:20:510
248599990,4rtkit-daemon2484-21rtkit-daemon05:46:072
248599990,4rtkit-daemon2484-21rtkit-daemon05:46:072
1841199994,86cyclictest0-21swapper/009:50:370
1841199994,84cyclictest20287-21yum-updatesd-he08:22:080
18411999820,13cyclictest0-21swapper/009:30:480
18411999820,13cyclictest0-21swapper/007:59:290
1841799975,25cyclictest0-21swapper/309:47:573
1841199974,84cyclictest0-21swapper/006:45:360
1841199974,78cyclictest0-21swapper/007:07:590
1841199974,29cyclictest0-21swapper/009:05:410
1841799965,29cyclictest0-21swapper/307:57:193
1841699963,85cyclictest14559-21crond06:40:352
1841199965,83cyclictest0-21swapper/009:15:460
1841199965,83cyclictest0-21swapper/009:15:460
1841199965,83cyclictest0-21swapper/009:10:420
1841199965,27cyclictest0-21swapper/009:43:220
1841199964,27cyclictest0-21swapper/004:41:290
1841199964,26cyclictest32633-21taskset07:53:030
1841199964,26cyclictest32633-21taskset07:53:030
18411999635,47cyclictest0-21swapper/008:45:160
18411999618,15cyclictest0-21swapper/006:46:050
1841699953,84cyclictest0-21swapper/209:55:542
1841199955,28cyclictest0-21swapper/008:25:530
1841199954,80cyclictest0-21swapper/009:50:280
1841199954,29cyclictest0-21swapper/009:01:390
1841199954,27cyclictest0-21swapper/008:50:490
1841199954,27cyclictest0-21swapper/007:20:110
1841799945,24cyclictest21175-21munin-node10:00:443
1841799945,24cyclictest21175-21munin-node10:00:443
1841699943,84cyclictest0-21swapper/209:00:562
1841199946,11cyclictest1419-21runrttasks09:56:000
1841199944,83cyclictest0-21swapper/009:22:090
1841199944,29cyclictest0-21swapper/006:30:050
1841199944,29cyclictest0-21swapper/005:32:050
1841199944,29cyclictest0-21swapper/004:36:000
1841199944,26cyclictest0-21swapper/006:55:480
18411999411,26cyclictest0-21swapper/007:00:510
1841799936,23cyclictest20207-21seq08:21:343
1841799935,28cyclictest0-21swapper/308:57:123
1841799934,27cyclictest0-21swapper/309:24:583
1841799934,26cyclictest0-21swapper/308:19:303
1841799934,26cyclictest0-21swapper/306:37:313
18417999317,14cyclictest0-21swapper/306:05:553
1841699937,78cyclictest0-21swapper/207:40:442
1841199935,27cyclictest0-21swapper/008:30:420
1841199935,27cyclictest0-21swapper/008:30:420
1841199934,82cyclictest0-21swapper/008:11:330
1841199934,80cyclictest0-21swapper/005:15:490
1841199934,27cyclictest0-21swapper/009:29:110
1841199934,27cyclictest0-21swapper/007:30:380
1841799925,25cyclictest7241-21irqstats08:50:523
1841799924,28cyclictest0-21swapper/308:30:473
1841799924,28cyclictest0-21swapper/308:30:473
1841799924,28cyclictest0-21swapper/306:16:103
1841799924,27cyclictest0-21swapper/309:30:423
1841799924,27cyclictest0-21swapper/307:03:423
1841799924,25cyclictest7626-21iostat_ios07:15:563
1841699923,43cyclictest0-21swapper/210:01:502
1841699923,43cyclictest0-21swapper/210:01:502
1841199925,30cyclictest0-21swapper/009:36:360
1841199924,79cyclictest0-21swapper/007:35:430
1841199924,28cyclictest0-21swapper/006:20:550
1841199924,27cyclictest0-21swapper/008:03:320
1841199924,27cyclictest0-21swapper/007:46:360
18411999211,25cyclictest0-21swapper/008:07:060
1841799915,27cyclictest0-21swapper/308:47:503
1841799915,27cyclictest0-21swapper/305:14:153
1841799914,28cyclictest0-21swapper/308:41:053
1841799914,27cyclictest11147-21diskmemload09:55:433
1841699913,81cyclictest4930-21grep07:11:062
1841699913,81cyclictest4930-21grep07:11:062
1841699913,81cyclictest0-21swapper/209:21:352
1841199914,79cyclictest0-21swapper/010:00:450
1841199914,79cyclictest0-21swapper/010:00:450
1841199914,79cyclictest0-21swapper/006:05:470
1841199914,78cyclictest0-21swapper/005:28:060
1841199914,32cyclictest0-21swapper/008:45:540
1841199914,28cyclictest0-21swapper/008:15:530
1841199914,27cyclictest0-21swapper/006:51:420
1841199914,27cyclictest0-21swapper/004:55:250
1841199914,26cyclictest0-21swapper/008:39:300
1841199914,26cyclictest0-21swapper/008:39:300
1841199914,26cyclictest0-21swapper/007:26:290
1841199913,25cyclictest0-21swapper/005:35:530
1841799905,26cyclictest0-21swapper/309:09:553
1841799905,24cyclictest0-21swapper/308:37:523
1841799905,24cyclictest0-21swapper/308:37:523
1841799904,26cyclictest0-21swapper/309:37:163
18417999029,51cyclictest448-21needreboot06:11:013
1841699904,80cyclictest0-21swapper/207:30:512
1841699904,65cyclictest28773-21ssh08:35:462
1841699904,65cyclictest28773-21ssh08:35:462
1841699903,80cyclictest23082-21python08:26:032
1841199904,29cyclictest0-21swapper/007:10:520
1841199904,29cyclictest0-21swapper/007:10:520
1841199904,28cyclictest0-21swapper/008:57:180
1841199904,26cyclictest0-21swapper/006:37:540
220372890,3sleep3361ktimersoftd/309:12:443
1841799896,72cyclictest0-21swapper/304:55:543
1841799895,26cyclictest0-21swapper/307:23:153
1841799894,78cyclictest19140-21fschecks_time06:45:533
1841699893,51cyclictest0-21swapper/209:17:042
1841699893,51cyclictest0-21swapper/209:17:042
1841199894,77cyclictest0-21swapper/007:40:470
1841799885,28cyclictest0-21swapper/305:41:013
1841799885,28cyclictest0-21swapper/305:41:013
1841799885,27cyclictest0-21swapper/309:00:303
1841799884,29cyclictest0-21swapper/305:25:573
1841799884,25cyclictest11147-21diskmemload07:11:003
1841799884,25cyclictest11147-21diskmemload07:11:003
18417998817,14cyclictest0-21swapper/307:44:003
1841699884,76cyclictest9567-21munin-node08:55:592
1841699883,77cyclictest0-21swapper/208:05:552
1841799875,75cyclictest0-21swapper/308:03:353
1841799875,75cyclictest0-21swapper/307:06:383
1841799875,29cyclictest1419-21runrttasks09:19:293
1841799875,29cyclictest1419-21runrttasks09:19:293
1841799875,28cyclictest0-21swapper/305:15:573
1841799875,27cyclictest0-21swapper/308:11:283
1841799874,26cyclictest0-21swapper/307:25:373
1841699874,75cyclictest1419-21runrttasks06:54:592
1841699874,64cyclictest0-21swapper/208:10:532
1841799869,9cyclictest121850irq/46-eth007:30:593
1841799866,10cyclictest7673-21munin-node09:41:013
1841799865,26cyclictest0-21swapper/309:52:313
1841799864,76cyclictest0-21swapper/304:40:563
1841799864,74cyclictest0-21swapper/306:55:493
1841799864,27cyclictest0-21swapper/305:54:223
1841799864,18cyclictest17305-21ssh06:43:033
1841699864,75cyclictest0-21swapper/207:23:482
1841699863,76cyclictest0-21swapper/209:45:142
1841699863,51cyclictest0-21swapper/207:00:462
1841199864,74cyclictest0-21swapper/005:10:070
1841799855,74cyclictest0-21swapper/306:54:173
1841799854,26cyclictest0-21swapper/307:52:503
1841799854,26cyclictest0-21swapper/307:52:503
18417998515,12cyclictest0-21swapper/307:38:013
1841699855,72cyclictest0-21swapper/207:50:592
1841699855,72cyclictest0-21swapper/207:50:592
1841699855,72cyclictest0-21swapper/207:15:542
1841699854,74cyclictest0-21swapper/204:39:532
1841699853,75cyclictest0-21swapper/207:45:462
1841699853,75cyclictest0-21swapper/206:49:192
1841199854,29cyclictest0-21swapper/006:31:070
1841199854,29cyclictest0-21swapper/006:31:070
1841199854,13cyclictest0-21swapper/006:17:360
18411998513,13cyclictest0-21swapper/005:50:370
18411998513,13cyclictest0-21swapper/005:50:370
1841799845,28cyclictest0-21swapper/307:50:163
1841799845,24cyclictest0-21swapper/308:26:463
1841799844,74cyclictest0-21swapper/309:27:303
18417998432,45cyclictest831-21cpuspeed_turbos05:05:523
1841699846,60cyclictest0-21swapper/208:16:592
1841699843,75cyclictest0-21swapper/207:30:222
1841699843,74cyclictest0-21swapper/208:40:442
1841699843,73cyclictest25794-21munin-node04:50:552
1841699843,54cyclictest0-21swapper/207:08:352
1841199844,29cyclictest0-21swapper/005:21:020
140312840,4sleep121-21ksoftirqd/109:49:591
1841799834,73cyclictest0-21swapper/308:06:073
1841699834,47cyclictest25155-21fschecks_count05:55:532
1841699833,74cyclictest0-21swapper/206:58:452
1841699833,73cyclictest6728-21ssh08:01:442
1841699833,73cyclictest19612-21cat09:10:302
1841699833,73cyclictest1419-21runrttasks06:08:192
1841699833,73cyclictest0-21swapper/208:31:042
1841699833,73cyclictest0-21swapper/208:31:042
1841699833,73cyclictest0-21swapper/208:23:092
1841199834,29cyclictest0-21swapper/006:00:490
18411998332,43cyclictest22856-21cut05:50:550
1841799825,27cyclictest0-21swapper/306:30:463
1841799825,27cyclictest0-21swapper/306:30:463
1841699824,69cyclictest4026-21crond09:35:282
1841699823,72cyclictest11147-21diskmemload09:48:582
1841699823,72cyclictest0-21swapper/205:45:372
1841699823,72cyclictest0-21swapper/205:45:372
1841699823,53cyclictest16236-21ssh06:41:042
1841199824,25cyclictest1419-21runrttasks05:40:520
1841199824,25cyclictest1419-21runrttasks05:40:520
1841199824,24cyclictest1247-21lldpd06:10:580
1841199824,13cyclictest0-21swapper/005:04:140
1841199824,13cyclictest0-21swapper/005:04:140
1841799816,19cyclictest0-21swapper/306:00:383
1841799815,26cyclictest0-21swapper/305:23:163
1841799814,22cyclictest0-21swapper/304:45:533
1841699815,68cyclictest0-21swapper/205:15:542
1841699813,71cyclictest20970-21gltestperf07:35:522
1841699813,70cyclictest0-21swapper/209:35:582
1841699813,69cyclictest29828-21munin-node09:25:522
1841699813,69cyclictest0-21swapper/205:36:012
1841699813,59cyclictest15011-21diskstats09:50:422
1841199814,15cyclictest0-21swapper/004:46:140
18411998121,14cyclictest0-21swapper/005:10:500
18411998120,13cyclictest0-21swapper/004:55:570
1841799804,69cyclictest0-21swapper/304:51:123
1841799804,11cyclictest5227-21kernelversion06:20:583
18417998019,12cyclictest31159-21fschecks_count05:00:563
18417998019,12cyclictest31159-21fschecks_count05:00:563
1841699804,68cyclictest7344-21ssh08:50:532
1841699803,70cyclictest1334-21telnet07:55:442
1841699803,70cyclictest0-21swapper/205:50:482
1841699803,68cyclictest0-21swapper/206:20:452
1841199804,30cyclictest26709-21latency_hist06:00:380
1841799794,24cyclictest18996-21tr04:35:533
1841699793,7cyclictest21808-21ssh09:11:552
1841699793,61cyclictest0-21swapper/208:47:412
1841699774,54cyclictest0-21swapper/204:55:502
1841699773,50cyclictest0-21swapper/205:00:542
1841699773,50cyclictest0-21swapper/205:00:542
1841799764,13cyclictest0-21swapper/305:55:473
18417997624,10cyclictest1419-21runrttasks05:37:393
18417997610,16cyclictest0-21swapper/306:26:013
1841699764,65cyclictest7338-21fschecks_time06:25:542
18417997510,9cyclictest121850irq/46-eth005:45:463
18417997510,9cyclictest121850irq/46-eth005:45:463
1841699752,67cyclictest0-21swapper/205:28:162
1841799743,23cyclictest132102sleep305:30:593
1841699744,63cyclictest0-21swapper/206:01:052
1841699743,44cyclictest0-21swapper/206:31:072
1841699743,44cyclictest0-21swapper/206:31:072
1841199744,18cyclictest0-21swapper/004:35:410
1841699733,64cyclictest1419-21runrttasks05:06:082
1841699733,64cyclictest0-21swapper/204:46:002
1841699733,63cyclictest0-21swapper/204:41:112
1841699713,61cyclictest9978-21latency_hist05:25:382
1841699703,59cyclictest0-21swapper/205:31:122
1841799695,16cyclictest0-21swapper/304:35:413
1841699693,60cyclictest0-21swapper/206:11:022
18415996916,23cyclictest1419-21runrttasks08:19:261
1841699673,57cyclictest0-21swapper/205:11:022
1841699673,44cyclictest1787-21telnet06:15:482
18415996619,40cyclictest25135-21timerandwakeup07:41:041
18415996615,43cyclictest0-21swapper/107:53:511
18415996615,43cyclictest0-21swapper/107:53:511
18415996611,34cyclictest0-21swapper/109:19:231
18415996611,34cyclictest0-21swapper/109:19:231
18415996517,38cyclictest1419-21runrttasks08:10:341
1841599644,52cyclictest0-21swapper/107:05:501
18415996413,32cyclictest0-21swapper/109:01:021
1841699633,54cyclictest0-21swapper/204:35:402
1841599637,48cyclictest0-21swapper/108:25:501
1841599633,33cyclictest0-21swapper/108:45:531
18415996325,21cyclictest9930-21munin-node07:20:561
18415996217,28cyclictest17281-21hwlatdetect09:05:501
18415996214,33cyclictest1419-21runrttasks08:44:041
18415996211,34cyclictest0-21swapper/109:22:031
18415996211,34cyclictest0-21swapper/108:51:051
18415996121,33cyclictest8595-21munin-node09:40:481
18415996119,27cyclictest1419-21runrttasks09:35:571
18415996116,38cyclictest18510-21cpuspeed_turbos08:20:431
18415996116,37cyclictest14484-21runrttasks09:50:361
18415996116,28cyclictest1419-21runrttasks08:05:461
18415996115,26cyclictest11984-21taskset08:57:591
18415996113,29cyclictest26107-21cpuspeed_turbos04:50:531
18415996112,34cyclictest0-21swapper/107:25:381
1841599607,46cyclictest0-21swapper/107:35:521
18415996019,33cyclictest28648-21ls08:35:451
18415996019,33cyclictest28648-21ls08:35:451
18415996018,35cyclictest0-21swapper/105:15:211
18415996017,37cyclictest20901-21sh10:00:281
18415996016,37cyclictest15920-21iostat_ios06:40:581
18415996016,29cyclictest25129-21telnet08:30:441
18415996016,29cyclictest25129-21telnet08:30:441
18415996011,29cyclictest0-21swapper/109:11:541
18415996010,21cyclictest0-21swapper/106:53:341
1841599598,44cyclictest0-21swapper/107:31:021
18415995915,35cyclictest31339-21python09:25:591
18415995911,33cyclictest0-21swapper/109:32:031
18415995911,27cyclictest21611-21df10:00:411
18415995911,27cyclictest21611-21df10:00:411
18415995910,29cyclictest0-21swapper/108:05:331
1841599589,30cyclictest0-21swapper/107:15:481
1841599588,34cyclictest0-21swapper/106:48:281
18415995814,25cyclictest23185-21awk04:45:391
18415995812,31cyclictest0-21swapper/106:57:501
1841599575,36cyclictest0-21swapper/105:49:211
1841599575,36cyclictest0-21swapper/105:49:211
18415995730,19cyclictest28629-21unixbench_multi07:46:051
18415995716,25cyclictest29307-21diskstats07:00:511
18415995716,20cyclictest13815-21runrttasks06:38:101
1841599569,40cyclictest0-21swapper/104:36:101
1841599569,31cyclictest0-21swapper/106:21:051
18415995615,26cyclictest1380-21ssh07:55:451
1841599558,40cyclictest0-21swapper/104:56:101
1841599543,43cyclictest7191-21fschecks_count06:25:531
18415995414,34cyclictest10271-21tail05:25:491
18415995411,37cyclictest17981-21munin-node05:40:541
18415995411,37cyclictest17981-21munin-node05:40:541
18415995311,34cyclictest9019-21telnet06:30:461
18415995311,34cyclictest9019-21telnet06:30:461
1841599524,41cyclictest0-21swapper/106:01:121
18415995223,23cyclictest13476-21sed05:31:001
1841599519,36cyclictest0-21swapper/105:05:391
1841599519,36cyclictest0-21swapper/105:05:391
1841599519,27cyclictest0-21swapper/105:51:411
18415995111,24cyclictest2827-21awk05:10:391
1841599508,18cyclictest0-21swapper/105:59:231
18415995013,31cyclictest0-21swapper/106:12:201
18415995012,32cyclictest7532-21sshd05:19:591
1841599501,42cyclictest0-21swapper/106:15:551
1841599496,36cyclictest0-21swapper/104:35:411
1841599492,28cyclictest0-21swapper/104:49:111
1841599488,34cyclictest0-21swapper/106:09:251
1841599485,35cyclictest0-21swapper/105:20:491
18415994831,11cyclictest16452-21sensors05:36:091
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional