You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 19:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot6s.osadl.org (updated Tue Apr 21, 2026 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31642991005,29cyclictest0-21swapper/110:22:581
3164299984,29cyclictest2583-21munin-node08:58:481
3164299956,32cyclictest0-21swapper/108:08:591
3164299956,32cyclictest0-21swapper/108:08:591
3164299953,25cyclictest19304-21unixbench_multi10:08:541
3164299953,25cyclictest19304-21unixbench_multi10:08:541
3163999954,72cyclictest0-21swapper/009:21:450
3163999944,12cyclictest0-21swapper/008:23:390
31639999410,75cyclictest0-21swapper/008:28:480
3163999933,68cyclictest0-21swapper/008:43:570
31639999317,54cyclictest0-21swapper/006:23:430
3163999924,12cyclictest0-21swapper/007:38:500
3163999914,9cyclictest28159-21munin-node08:48:520
3163999913,65cyclictest0-21swapper/009:28:590
3163999913,65cyclictest0-21swapper/009:28:590
31639999118,52cyclictest121850irq/46-eth009:34:560
3163999905,69cyclictest0-21swapper/007:12:180
3163999904,71cyclictest0-21swapper/008:38:530
3163999904,12cyclictest0-21swapper/007:59:010
3163999903,63cyclictest21070-21ssh09:23:550
31639999022,60cyclictest8648-21tune2fs08:18:430
3163999894,13cyclictest0-21swapper/007:28:320
3163999894,12cyclictest0-21swapper/006:53:330
3163999893,63cyclictest0-21swapper/008:06:300
3163999893,63cyclictest0-21swapper/008:06:300
3163999886,65cyclictest24401-21diskmemload10:13:370
3163999884,13cyclictest0-21swapper/007:14:020
3163999884,12cyclictest0-21swapper/010:08:460
3163999884,12cyclictest0-21swapper/010:08:460
3163999884,11cyclictest0-21swapper/007:23:450
3163999884,11cyclictest0-21swapper/007:23:450
3163999874,66cyclictest0-21swapper/008:36:480
3163999874,13cyclictest0-21swapper/008:53:370
3163999874,12cyclictest0-21swapper/008:08:430
3163999874,12cyclictest0-21swapper/008:08:430
3163999873,69cyclictest0-21swapper/007:21:460
3163999873,66cyclictest0-21swapper/007:45:100
3163999873,63cyclictest0-21swapper/010:18:420
31639998720,47cyclictest3704-21ssh08:58:470
31639998715,50cyclictest0-21swapper/008:13:480
31639998713,54cyclictest0-21swapper/009:58:450
31639998712,60cyclictest12960-21cpuspeed_turbos06:43:420
31639998712,60cyclictest12960-21cpuspeed_turbos06:43:420
3163999865,66cyclictest0-21swapper/009:52:420
3163999864,65cyclictest0-21swapper/009:43:340
3163999864,65cyclictest0-21swapper/009:05:210
3163999864,13cyclictest0-21swapper/009:15:140
3163999864,13cyclictest0-21swapper/009:09:020
3163999864,12cyclictest0-21swapper/009:42:130
3163999864,12cyclictest0-21swapper/006:49:000
3163999863,12cyclictest0-21swapper/006:34:010
248599860,4rtkit-daemon2484-21rtkit-daemon09:19:582
3163999854,12cyclictest0-21swapper/005:47:510
3163999854,11cyclictest0-21swapper/010:33:560
3163999853,12cyclictest0-21swapper/007:53:460
31639998514,54cyclictest0-21swapper/007:33:500
31639998513,10cyclictest0-21swapper/010:23:450
3164599843,75cyclictest0-21swapper/309:57:373
3163999844,12cyclictest0-21swapper/007:50:290
3163999843,62cyclictest0-21swapper/010:06:100
3163999843,12cyclictest0-21swapper/005:59:030
3163999843,12cyclictest0-21swapper/005:59:030
31639998418,49cyclictest31882-21gltestperf10:28:400
31639998418,42cyclictest0-21swapper/006:18:510
3163999834,59cyclictest0-21swapper/005:18:430
3163999834,12cyclictest0-21swapper/005:39:070
3163999834,12cyclictest0-21swapper/005:16:070
3163999834,12cyclictest0-21swapper/005:16:070
3163999833,12cyclictest0-21swapper/009:53:390
31639998319,47cyclictest0-21swapper/006:03:530
3163999824,62cyclictest0-21swapper/006:13:450
3163999824,12cyclictest0-21swapper/005:13:330
3163999823,71cyclictest9813-21cut05:28:490
3163999823,63cyclictest0-21swapper/007:03:280
3164599815,68cyclictest1419-21runrttasks10:23:413
3164599804,70cyclictest0-21swapper/308:49:123
3164599804,69cyclictest0-21swapper/307:14:013
3163999804,13cyclictest0-21swapper/005:48:540
3163999804,13cyclictest0-21swapper/005:48:540
3163999804,12cyclictest0-21swapper/007:08:280
3163999804,12cyclictest0-21swapper/006:38:470
31639998031,42cyclictest7142-21cat06:29:030
31639998022,41cyclictest30406-21awk06:13:300
3164599799,63cyclictest0-21swapper/308:58:573
3164599793,40cyclictest0-21swapper/310:02:273
3163999794,12cyclictest0-21swapper/005:53:580
3163999784,66cyclictest12673-21cat05:33:560
3163999784,12cyclictest0-21swapper/005:23:590
3164599775,64cyclictest0-21swapper/310:35:143
3164599775,63cyclictest19732-21ssh07:48:263
3164599775,52cyclictest0-21swapper/307:53:453
3164599774,55cyclictest0-21swapper/308:13:263
3164599774,55cyclictest0-21swapper/308:13:263
3164599773,65cyclictest29550-21munin-node09:38:443
3164599764,50cyclictest0-21swapper/308:40:423
3164599763,66cyclictest0-21swapper/307:24:403
3164599763,66cyclictest0-21swapper/307:24:403
3164599763,65cyclictest18430-21expr08:33:413
3164599755,52cyclictest0-21swapper/305:33:483
3164599754,64cyclictest12465-21rm07:36:333
3164599754,64cyclictest0-21swapper/308:08:263
3164599754,64cyclictest0-21swapper/308:08:263
3164599754,64cyclictest0-21swapper/307:48:483
3164599754,63cyclictest12616-21ntp_kernel_pll_08:23:523
3164599754,56cyclictest4617-21aten2.4-expect08:13:363
3164599754,51cyclictest0-21swapper/309:34:553
3164599753,65cyclictest1419-21runrttasks10:21:183
3164599753,64cyclictest1419-21runrttasks05:55:153
3164599753,63cyclictest55061sshd06:28:343
3164599753,61cyclictest6729-21perl09:03:423
3164599744,65cyclictest0-21swapper/309:28:343
3164599744,65cyclictest0-21swapper/309:28:343
3164599744,64cyclictest0-21swapper/308:53:553
3164599744,54cyclictest0-21swapper/308:46:343
31645997421,36cyclictest20432-21taskset10:12:443
31645997421,36cyclictest20432-21taskset10:12:443
3163999744,10cyclictest31637-21awk05:08:320
3164599735,50cyclictest0-21swapper/310:28:223
3164599734,61cyclictest9946-21diskstats09:08:393
3164599734,61cyclictest14316-21iostat_ios07:38:483
3164599734,60cyclictest16636-21df_abs09:18:383
3164599734,53cyclictest0-21swapper/307:29:453
3164599733,64cyclictest0-21swapper/309:13:393
3164599733,63cyclictest22060-21runrttasks09:26:343
3164599733,51cyclictest14743-21munin-node08:28:373
3164599732,64cyclictest10390-21ssh08:21:073
3164599724,52cyclictest0-21swapper/309:44:423
3164599724,50cyclictest0-21swapper/307:19:353
3164599723,61cyclictest24401-21diskmemload07:10:103
3164599723,60cyclictest26718-21gltestperf06:03:503
31645997213,33cyclictest0-21swapper/310:03:343
3164599714,61cyclictest0-21swapper/306:23:293
3164599714,60cyclictest0-21swapper/305:48:433
3164599714,60cyclictest0-21swapper/305:48:433
3164599714,50cyclictest0-21swapper/305:39:083
3164599714,49cyclictest0-21swapper/306:50:333
3164599713,61cyclictest0-21swapper/310:17:033
3164599712,52cyclictest0-21swapper/309:51:443
3164599712,48cyclictest0-21swapper/306:38:433
3164599704,48cyclictest0-21swapper/306:58:403
3164599702,62cyclictest0-21swapper/305:44:003
3164599693,59cyclictest32764-21chrt06:18:173
3164599693,57cyclictest0-21swapper/308:03:053
3164599693,47cyclictest8112-21cpu06:33:433
3164599684,57cyclictest19275-21runrttasks06:54:153
3164599684,56cyclictest0-21swapper/306:43:413
3164599684,56cyclictest0-21swapper/306:43:413
31645996728,32cyclictest1419-21runrttasks05:32:153
3164599672,47cyclictest0-21swapper/306:12:293
3164599664,54cyclictest0-21swapper/305:23:503
3164599663,56cyclictest3307-21cpu06:23:443
3164599663,55cyclictest23842-21timerwakeupswit07:04:003
31645996624,35cyclictest951-21timerandwakeup05:09:043
3164599652,57cyclictest0-21swapper/305:13:463
3164599652,57cyclictest0-21swapper/305:13:463
3164499643,54cyclictest30969-21munin-node10:28:392
3164299645,27cyclictest0-21swapper/108:23:571
31642996421,21cyclictest1128-21crond10:24:201
3164599632,56cyclictest0-21swapper/305:58:333
3164599632,56cyclictest0-21swapper/305:58:333
3164599632,55cyclictest0-21swapper/305:18:433
3164499615,27cyclictest0-21swapper/207:33:482
3164599603,50cyclictest31758-21cat05:08:333
3164499604,46cyclictest0-21swapper/209:48:572
31642996013,26cyclictest0-21swapper/109:33:361
3164499594,47cyclictest0-21swapper/209:23:352
326372584,46sleep20-21swapper/207:18:502
31642995821,15cyclictest14837-21kworker/u8:106:48:291
31642995821,15cyclictest14837-21kworker/u8:106:48:291
248599580,6rtkit-daemon0-21swapper/309:17:512
215112583,49sleep10-21swapper/109:24:331
104002584,13sleep20-21swapper/205:29:002
3164499574,45cyclictest0-21swapper/208:18:452
3164499574,45cyclictest0-21swapper/205:43:522
3164499573,45cyclictest0-21swapper/208:43:362
31644995719,8cyclictest3299-21ssh08:09:512
31644995719,8cyclictest3299-21ssh08:09:512
3164299574,37cyclictest0-21swapper/109:11:311
31642995712,26cyclictest0-21swapper/108:15:321
6662563,45sleep20-21swapper/208:53:592
3164499564,13cyclictest0-21swapper/206:48:432
31644995636,13cyclictest0-21swapper/208:23:432
3164299565,31cyclictest0-21swapper/106:43:291
3164299564,31cyclictest0-21swapper/107:48:491
31642995623,12cyclictest31425-21ntp_states08:03:521
31642995623,12cyclictest31425-21ntp_states08:03:521
31642995622,14cyclictest0-21swapper/110:14:151
31642995622,14cyclictest0-21swapper/107:13:421
31642995621,13cyclictest0-21swapper/109:19:181
31642995621,13cyclictest0-21swapper/108:52:531
31642995620,13cyclictest0-21swapper/105:49:051
31642995620,13cyclictest0-21swapper/105:49:051
31642995617,21cyclictest2376-21cut08:58:251
248599560,6rtkit-daemon0-21swapper/310:15:262
3164499554,42cyclictest19234-21rm08:33:522
3164499554,11cyclictest0-21swapper/208:08:092
3164499554,11cyclictest0-21swapper/208:08:092
3164299556,25cyclictest0-21swapper/109:03:481
3164299555,28cyclictest0-21swapper/106:53:301
3164299555,27cyclictest0-21swapper/109:41:311
3164299555,27cyclictest0-21swapper/107:18:481
3164299554,29cyclictest0-21swapper/108:23:271
3164299554,27cyclictest0-21swapper/109:54:591
3164299554,27cyclictest0-21swapper/108:28:441
3164299554,27cyclictest0-21swapper/107:58:261
3164299554,27cyclictest0-21swapper/105:39:021
3164299554,27cyclictest0-21swapper/105:18:331
3164299554,27cyclictest0-21swapper/105:18:331
31642995520,14cyclictest0-21swapper/110:31:081
31642995520,13cyclictest0-21swapper/107:38:461
31642995520,10cyclictest14542-21ssh09:13:571
31642995519,13cyclictest0-21swapper/107:43:501
31642995519,13cyclictest0-21swapper/106:58:571
248599550,5rtkit-daemon2484-21rtkit-daemon05:08:462
195552553,45sleep119563-21ssh08:33:591
3164499544,42cyclictest0-21swapper/207:24:022
3164499544,42cyclictest0-21swapper/207:24:022
3164299545,27cyclictest0-21swapper/106:32:591
3164299545,27cyclictest0-21swapper/106:23:511
3164299545,27cyclictest0-21swapper/105:19:011
3164299545,26cyclictest0-21swapper/106:22:561
3164299544,30cyclictest0-21swapper/109:58:441
3164299544,28cyclictest0-21swapper/109:43:351
3164299544,28cyclictest0-21swapper/109:28:341
3164299544,28cyclictest0-21swapper/109:28:341
3164299544,28cyclictest0-21swapper/106:33:471
3164299544,27cyclictest0-21swapper/110:03:341
3164299544,27cyclictest0-21swapper/109:49:031
3164299544,27cyclictest0-21swapper/108:44:201
3164299544,27cyclictest0-21swapper/107:09:301
3164299544,27cyclictest0-21swapper/105:48:311
3164299544,26cyclictest9832-21fschecks_time05:28:491
3164299544,26cyclictest0-21swapper/107:33:511
3164299544,26cyclictest0-21swapper/107:24:141
3164299544,26cyclictest0-21swapper/107:24:141
3164299544,15cyclictest0-21swapper/108:38:471
31642995419,13cyclictest0-21swapper/110:33:581
31642995419,12cyclictest0-21swapper/105:53:501
31642995417,13cyclictest0-21swapper/107:58:451
198342543,9sleep20-21swapper/210:10:342
198342543,9sleep20-21swapper/210:10:342
134232543,43sleep10-21swapper/105:35:311
3164499534,41cyclictest0-21swapper/207:38:492
3164499534,12cyclictest0-21swapper/209:43:372
31644995333,14cyclictest0-21swapper/209:53:522
3164299535,26cyclictest0-21swapper/106:58:031
3164299534,26cyclictest0-21swapper/107:28:491
31644995222,24cyclictest0-21swapper/205:53:312
31644995222,24cyclictest0-21swapper/205:53:312
31642995232,13cyclictest26600-21fschecks_time06:03:481
31642995222,12cyclictest0-21swapper/105:59:051
31642995222,12cyclictest0-21swapper/105:59:051
248962522,10sleep20-21swapper/210:18:402
3164499517,37cyclictest0-21swapper/207:58:592
31644995135,10cyclictest7605-21grep05:23:512
31644995133,12cyclictest0-21swapper/207:44:422
31644995132,13cyclictest0-21swapper/209:02:352
3164499513,44cyclictest0-21swapper/209:41:172
3164499513,10cyclictest2700-21tune2fs10:33:392
3164499512,43cyclictest0-21swapper/209:03:352
3164499512,42cyclictest281ktimersoftd/207:53:512
3164499512,35cyclictest0-21swapper/206:43:392
3164499512,35cyclictest0-21swapper/206:43:392
3164299514,27cyclictest0-21swapper/107:03:511
3164299514,24cyclictest0-21swapper/105:08:441
31642995125,10cyclictest0-21swapper/105:23:451
218932512,9sleep20-21swapper/207:02:012
3164499504,39cyclictest0-21swapper/210:23:352
3164499504,14cyclictest0-21swapper/207:48:562
31644995035,9cyclictest2832-21grep05:13:522
31644995035,9cyclictest2832-21grep05:13:522
31644995030,14cyclictest0-21swapper/209:13:032
3164499503,42cyclictest1419-21runrttasks09:35:162
31644995011,34cyclictest0-21swapper/209:28:562
31644995011,34cyclictest0-21swapper/209:28:562
31644995010,35cyclictest26-21rcuc/207:28:522
3164299504,27cyclictest0-21swapper/106:08:541
3164299502,43cyclictest0-21swapper/106:13:581
272192502,10sleep20-21swapper/207:11:372
242262502,9sleep20-21swapper/208:42:222
230962503,9sleep20-21swapper/205:57:012
74972492,9sleep20-21swapper/206:31:262
3164499494,32cyclictest0-21swapper/208:48:332
3164499494,28cyclictest0-21swapper/206:18:292
3164499493,40cyclictest0-21swapper/208:14:372
3164499493,39cyclictest13850-21crond10:03:212
3164499493,38cyclictest0-21swapper/208:30:032
3164499484,13cyclictest0-21swapper/210:08:212
3164499482,40cyclictest0-21swapper/205:39:372
254392482,9sleep20-21swapper/206:01:122
254392482,9sleep20-21swapper/206:01:122
3164499474,26cyclictest0-21swapper/207:13:502
3164499474,26cyclictest0-21swapper/206:04:042
3164499473,37cyclictest0-21swapper/206:53:412
3164499473,27cyclictest0-21swapper/206:33:462
103972472,9sleep20-21swapper/206:38:442
3164499464,24cyclictest0-21swapper/206:24:002
133272462,9sleep20-21swapper/205:34:152
62742452,36sleep20-21swapper/205:20:372
3164499433,24cyclictest0-21swapper/207:03:402
3164499423,32cyclictest0-21swapper/206:18:432
295272423,8sleep229583-21ps06:08:582
31642994111,12cyclictest0-21swapper/105:08:341
3164499383,27cyclictest31732-21latency_hist05:08:332
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional