You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 18:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Tue Apr 14, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8343283079,8266sleep21740-21kworker/2:211:46:292
232022265526,2554sleep220096-21kworker/2:007:25:422
464924937,474sleep24694-21ntpq09:51:492
464924937,474sleep24694-21ntpq09:51:492
158442286111,129sleep215981-21unixbench_multi10:26:542
7134226617,198sleep27186-21latency_hist10:01:222
18112226624,222sleep018153-21uniq08:51:410
1905225348,190sleep21926-21sort11:26:222
140752252118,19sleep20-21swapper/210:21:492
12810224842,185sleep212831-21idleruntime-cro08:36:202
595922468,226sleep26074-21cat11:36:562
238102239106,21sleep20-21swapper/207:26:472
25298223116,200sleep225366-21switchtime10:56:502
18246222742,143sleep218320-21awk12:16:502
21155221715,41sleep20-21swapper/209:01:372
21155221715,41sleep20-21swapper/209:01:372
7584220713,181sleep17635-21cat10:01:411
17785220413,151sleep217918-21latency_hist08:51:232
5167219945,91sleep20-21swapper/208:11:412
26391219591,28sleep20-21swapper/211:01:392
2401521957,13sleep00-21swapper/007:26:530
2786219112,165sleep22843-21fschecks_count09:46:402
13777218733,117sleep213821-21ntpq08:37:092
12544218418,126sleep20-21swapper/210:16:482
10744218332,32sleep210782-21latency10:11:432
16514217247,108sleep216119-21/usr/sbin/munin12:11:452
28448216914,141sleep328482-21irqstats07:41:423
27833216751,94sleep20-21swapper/211:06:232
31788216618,83sleep20-21swapper/211:16:452
12756216614,110sleep212766-21modprobe12:01:192
20864216342,106sleep220867-21cron09:01:202
7295216226,117sleep20-21swapper/211:41:492
31073215816,19sleep30-21swapper/307:51:233
23249215735,98sleep20-21swapper/209:06:502
23249215735,98sleep20-21swapper/209:06:502
18238215715,126sleep118190-21latency_hist07:11:241
14805215611,14sleep20-21swapper/212:06:402
1928821549,52sleep20-21swapper/212:21:342
25691215242,74sleep225770-21xargs07:31:522
23779215113,123sleep023852-21threads10:51:520
21092215142,75sleep20-21swapper/212:26:432
1622121518,102sleep20-21swapper/208:46:222
737621498,97sleep20-21swapper/210:01:342
143042141104,19sleep20-21swapper/212:06:202
13369214113,116sleep013487-21latency_hist10:21:230
23812214014,111sleep023926-21cmp12:36:220
1987214013,15sleep20-21swapper/208:01:362
286502139101,22sleep2357-21ntpd09:26:202
23542213995,22sleep20-21swapper/210:51:452
19362213815,108sleep219499-21latency_hist08:56:242
799921378,93sleep28142-21ls08:21:222
799921378,93sleep28142-21ls08:21:222
4601213518,96sleep20-21swapper/211:31:552
32431213227,18sleep20-21swapper/209:36:462
8298212911,105sleep28312-21grep08:21:392
8298212911,105sleep28312-21grep08:21:392
23261212811,90sleep123315-21grep10:51:401
11173211826,72sleep311205-21awk11:56:203
32680211638,14sleep20-21swapper/207:56:232
17406211619,77sleep017561-21grep10:31:530
332221155,11sleep00-21swapper/011:30:120
28583211514,15sleep20-21swapper/207:41:452
20116211510,69sleep220068-21apt-get10:42:042
17611211513,64sleep217763-21grep07:06:532
700210926,71sleep2747-21cat07:56:432
6607210911,83sleep06763-21sed11:41:220
27850210812,79sleep327851-21sshd07:40:253
5547210717,70sleep35612-21awk08:11:513
30760210750,40sleep20-21swapper/209:31:452
3818210647,40sleep20-21swapper/208:06:452
169821058,83sleep21132-21/usr/sbin/munin09:41:512
29999210411,46sleep20-21swapper/207:46:422
26669210448,38sleep20-21swapper/207:36:362
26482210418,67sleep326515-21/usr/sbin/munin11:01:403
2246221047,13sleep20-21swapper/212:31:402
5833210313,32sleep35896-21df09:56:383
24640210320,20sleep20-21swapper/209:11:462
1503210314,69sleep3580-21/usr/sbin/munin11:21:553
10561210316,71sleep310630-21fschecks_time10:11:403
12615210216,71sleep312672-21switchtime10:16:493
935821006,12sleep20-21swapper/210:06:472
13516210018,63sleep013556-21ps12:01:500
2621829920,54sleep10-21swapper/109:16:461
1718829910,70sleep017315-21unixbench_multi08:46:530
1152429922,61sleep311037-21apt-get08:31:333
2817929816,59sleep028067-21/usr/sbin/munin11:07:340
2059629815,67sleep120730-21latency_hist12:26:221
2081229711,58sleep00-21swapper/012:26:330
1029929614,22sleep20-21swapper/208:26:522
300802957,54sleep20-21swapper/209:29:522
2683729519,11sleep20-21swapper/209:17:562
1832429514,65sleep318416-21threads12:16:513
2947429416,61sleep129493-21munin-run07:46:211
994429319,60sleep19980-21idleruntime08:26:431
1782529217,54sleep017856-21fschecks_count12:16:400
25862917,46sleep20-21swapper/211:26:472
2049429117,55sleep120050-21/usr/sbin/munin10:42:161
400129010,66sleep04027-21cat09:51:220
3273929020,55sleep1404-21unixbench_multi09:36:531
2371729014,61sleep323809-21systemd-run12:36:213
2103429020,48sleep121086-21if_eth012:26:421
1683429012,56sleep116963-21/usr/sbin/munin12:11:521
1533429019,46sleep115476-21grep12:06:521
1021829012,58sleep310225-21/usr/sbin/munin08:26:493
109612886,74sleep00-21swapper/011:53:340
2288228715,57sleep2281-21dbus-daemon10:51:212
2236428517,30sleep10-21swapper/107:21:451
837428417,18sleep00-21swapper/011:46:350
368128412,25sleep10-21swapper/111:31:341
208272849,17sleep220871-21ntpq07:16:462
1718628417,48sleep017227-21latency07:06:420
1543328415,47sleep115469-21cat10:26:451
174828311,58sleep11846-21latency_hist08:01:231
1698628319,43sleep317015-21kernelversion10:31:433
101612830,7sleep10-21swapper/111:51:401
309222828,62sleep013441-21snmpd11:15:540
1893428213,12sleep20-21swapper/207:11:372
1569728217,29sleep00-21swapper/010:26:520
1154228211,56sleep311568-21sh11:56:363
483228118,46sleep34852-21latency_hist08:11:223
2969228126,14sleep212-21rcu_sched11:11:362
1724728122,15sleep20-21swapper/210:31:512
1270928112,56sleep013441-21snmpd08:35:050
201052809,55sleep320185-21tr08:56:493
188212808,54sleep118927-21users08:51:541
2483827918,18sleep00-21swapper/010:56:400
2480027822,34sleep324814-21idleruntime-cro07:31:203
2021927823,31sleep00-21swapper/007:16:230
1883027817,44sleep318900-21/usr/sbin/munin10:36:523
187052788,13sleep20-21swapper/210:36:472
1541827832,23sleep20-21swapper/207:01:402
1541427824,29sleep10-21swapper/107:01:391
3024727719,44sleep130309-21/usr/sbin/munin07:46:491
263172765,12sleep10-21swapper/107:35:551
200412755,15sleep30-21swapper/310:41:293
114002756,20sleep00-21swapper/010:12:060
847427415,40sleep18486-21/usr/sbin/munin08:21:421
847427415,40sleep18486-21/usr/sbin/munin08:21:421
3148527416,38sleep130974-21apt-get11:16:321
3024627422,32sleep330295-21latency_hist09:31:223
2520427415,43sleep325210-21ntp_states10:56:473
1963527414,24sleep10-21swapper/107:11:521
1646827414,30sleep30-21swapper/310:31:233
1108427416,36sleep011199-21latency_hist08:31:220
99692736,13sleep00-21swapper/010:08:340
2848527312,47sleep028529-21latency07:41:430
2170727314,31sleep10-21swapper/107:21:211
1708527316,18sleep30-21swapper/308:46:523
109512737,11sleep20-21swapper/211:53:282
840027225,26sleep10-21swapper/111:46:361
1387227222,21sleep00-21swapper/010:21:430
1099727213,43sleep111076-21mailstats10:11:501
485427118,26sleep00-21swapper/008:11:230
722027015,36sleep37288-21switchtime08:16:513
722027015,36sleep37288-21switchtime08:16:513
39782709,37sleep00-21swapper/011:31:450
2338327019,21sleep30-21swapper/310:51:423
1501327019,26sleep10-21swapper/108:41:411
3015926913,39sleep329667-21/usr/sbin/munin11:11:493
2807226915,36sleep027370-21/usr/sbin/munin09:21:540
246126917,27sleep10-21swapper/109:46:221
2330726915,41sleep323310-21latency_hist07:26:243
2216426922,23sleep10-21swapper/110:46:521
2139026921,25sleep00-21swapper/009:01:430
2139026921,25sleep00-21swapper/009:01:430
1928126923,25sleep30-21swapper/308:56:213
1912426913,41sleep319137-21kernelversion07:11:423
1339326919,28sleep00-21swapper/008:36:550
2797826817,31sleep328070-21/usr/sbin/munin09:21:533
2423826815,37sleep024240-21/usr/sbin/munin09:11:320
2962026714,22sleep00-21swapper/009:26:510
1960026717,28sleep00-21swapper/010:41:220
1658926724,26sleep316621-21munin-run07:06:213
735126616,33sleep37352-21munin-node10:01:313
69326617,34sleep1739-21fschecks_count11:21:411
433626621,26sleep00-21swapper/009:51:420
433626621,26sleep00-21swapper/009:51:420
3047226611,40sleep330484-21apt-get09:31:343
2837526617,25sleep30-21swapper/311:07:423
2657426614,36sleep126628-21sed11:01:421
2011926617,31sleep020185-21cpuspeed10:42:060
124226615,19sleep00-21swapper/009:41:410
35526512,36sleep3503-21cat11:21:223
3188826517,22sleep00-21swapper/011:16:470
2392726516,30sleep10-21swapper/112:36:231
1960026517,30sleep019594-21apt-get08:56:340
1492026518,28sleep30-21swapper/310:26:223
1486926517,27sleep114887-21modprobe10:26:201
402926415,25sleep10-21swapper/109:51:231
3100026411,37sleep131090-21mailstats09:31:531
2307026413,25sleep00-21swapper/009:06:440
2307026413,25sleep00-21swapper/009:06:440
1989426418,25sleep30-21swapper/312:21:523
1875726412,37sleep118770-21open_files10:36:501
1369726419,22sleep30-21swapper/312:01:533
1190226414,22sleep00-21swapper/011:56:450
634826314,29sleep16391-21systemctl08:16:211
2509626320,19sleep00-21swapper/007:31:380
246226317,31sleep32524-21wc08:01:493
1556726323,25sleep015601-21/usr/sbin/munin07:01:420
691226215,21sleep10-21swapper/111:41:401
2332226215,29sleep323429-21grep09:06:513
2332226215,29sleep323429-21grep09:06:513
1176626217,21sleep10-21swapper/111:56:421
942226114,21sleep10-21swapper/110:06:501
550326116,19sleep10-21swapper/109:56:201
550326116,19sleep10-21swapper/109:56:201
1333026114,28sleep112828-21apt-get08:36:281
2996126016,21sleep10-21swapper/111:11:431
2208226019,25sleep00-21swapper/007:21:400
1749726017,27sleep317640-21latency_hist12:16:233
796825924,22sleep10-21swapper/108:21:211
796825924,22sleep10-21swapper/108:21:211
395725913,21sleep30-21swapper/309:51:213
3096525921,24sleep00-21swapper/009:31:520
609325814,24sleep10-21swapper/109:56:441
2849125817,20sleep10-21swapper/111:07:441
2729425813,18sleep10-21swapper/107:36:521
2645425821,15sleep30-21swapper/309:16:523
2517825818,20sleep30-21swapper/307:31:413
2480625812,25sleep30-21swapper/309:11:513
2100225814,25sleep321073-21ls07:16:513
1805125822,18sleep10-21swapper/110:36:221
1183825821,19sleep10-21swapper/108:31:551
871825711,23sleep00-21swapper/008:21:490
871825711,23sleep00-21swapper/008:21:490
2593525710,29sleep026016-21grep09:16:410
2288525715,25sleep10-21swapper/109:06:411
2288525715,25sleep10-21swapper/109:06:411
808625614,25sleep08225-21/usr/sbin/munin10:01:520
657425612,18sleep30-21swapper/311:41:213
329625617,19sleep00-21swapper/008:06:220
2776525611,26sleep10-21swapper/109:21:471
2248525613,24sleep022532-21users10:46:550
900725512,24sleep39112-21users08:21:523
900725512,24sleep39112-21users08:21:523
45882556,11sleep10-21swapper/108:08:461
2906225513,24sleep129148-21fw_packets09:26:401
2683825515,17sleep00-21swapper/011:01:490
215725513,25sleep32150-21apt-get11:26:353
3141725412,23sleep031517-21perl07:51:410
2480225413,23sleep124824-21df10:56:371
2092325416,23sleep30-21swapper/312:26:403
1968625415,21sleep10-21swapper/108:56:391
1691425414,24sleep316910-21apt-get07:06:333
967925315,16sleep30-21swapper/311:51:233
959525315,23sleep39714-21timerwakeupswit10:06:523
3170625316,22sleep131743-21ntpq07:51:461
2293225314,23sleep322990-21latency_hist10:51:223
212025319,19sleep10-21swapper/111:26:251
1684725314,22sleep10-21swapper/108:46:461
1252025315,22sleep10-21swapper/110:16:471
68082526,35sleep00-21swapper/009:57:440
425925216,20sleep30-21swapper/311:31:523
3213325214,22sleep332140-21df_inode09:36:393
2797725215,19sleep00-21swapper/007:41:230
2246525213,24sleep322482-21latency_hist09:06:223
2246525213,24sleep322482-21latency_hist09:06:223
1854425213,22sleep118642-21users12:16:531
169725213,24sleep31720-21munin-run08:01:213
86225112,18sleep00-21swapper/009:41:210
433125111,16sleep30-21swapper/309:51:413
433125111,16sleep30-21swapper/309:51:413
2974025112,24sleep029759-21apt-get07:46:320
2498825114,21sleep125126-21awk09:11:531
1910625116,16sleep00-21swapper/012:21:240
1590225111,18sleep30-21swapper/312:11:223
1534725110,15sleep30-21swapper/308:41:473
1519125115,17sleep00-21swapper/008:41:440
186325013,17sleep00-21swapper/011:26:200
1614625012,22sleep016138-21apt-get12:11:350
160825012,22sleep31614-21ntp_states09:41:483
141162506,36sleep10-21swapper/112:03:531
689824913,21sleep36905-21df_inode11:41:383
2485224915,18sleep10-21swapper/107:31:221
173324911,22sleep01747-21munin-run08:01:220
3139524814,18sleep30-21swapper/307:51:403
1487124811,17sleep00-21swapper/012:06:420
1337124811,22sleep313358-21apt-get08:36:383
682624710,19sleep00-21swapper/011:41:350
572224712,17sleep10-21swapper/111:36:511
729124613,14sleep00-21swapper/008:16:520
729124613,14sleep00-21swapper/008:16:520
2160024611,16sleep10-21swapper/109:01:481
2160024611,16sleep10-21swapper/109:01:481
2107624510,17sleep00-21swapper/007:16:520
529724412,16sleep00-21swapper/008:11:430
225824313,15sleep10-21swapper/108:01:431
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional