You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-03 - 01:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat May 02, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
118282409113,4067sleep110412-21kworker/1:010:51:101
17434265921,618sleep117483-21memory11:10:401
10727263165,26sleep00-21swapper/012:30:500
183832523138,359sleep018513-21unixbench_multi09:30:490
758124409,394sleep07644-21unin-run07:15:180
24668244025,18sleep00-21swapper/009:50:520
24668244025,18sleep00-21swapper/009:50:520
1578237014,206sleep01726-21latency_hist12:05:180
1578237014,206sleep01726-21latency_hist12:05:180
502323567,333sleep05173-21sed07:05:490
27674232011,294sleep127713-21ntpq10:00:421
17409227217,238sleep017442-21grep09:30:170
14193227072,26sleep00-21swapper/007:35:370
14193227072,26sleep00-21swapper/007:35:370
7471223418,72sleep00-21swapper/008:55:490
10038222612,200sleep010149-21unixbench_multi10:45:480
6852220842,39sleep06958-21timerandwakeup10:35:460
32194220528,102sleep032213-21apt-get10:15:270
16956220410,55sleep016995-21latency_hist07:45:180
16956220410,55sleep016995-21latency_hist07:45:180
3433220217,166sleep03472-21fschecks_count12:10:350
4942220112,138sleep05008-21ls12:15:320
18466220015,166sleep018599-21latency_hist11:15:200
844421865,168sleep00-21swapper/012:26:100
7927218522,138sleep28134-21latency_hist12:25:192
687021768,156sleep06913-21ntpq12:20:450
27647217616,22sleep00-21swapper/011:44:110
27647217616,22sleep00-21swapper/011:44:110
2673217622,47sleep00-21swapper/008:40:460
23430217624,22sleep00-21swapper/008:05:330
19961217223,134sleep019981-21awk11:20:160
32705217112,135sleep032736-21cron08:35:170
26444216710,145sleep026525-21cat09:56:500
27882216510,24sleep00-21swapper/010:00:500
18507216512,37sleep018650-21latency_hist07:50:190
18507216512,37sleep018650-21latency_hist07:50:190
551321647,13sleep00-21swapper/010:30:500
551321647,13sleep00-21swapper/010:30:500
2579216486,24sleep00-21swapper/010:25:160
2998721639,83sleep030016-21fschecks_count08:25:360
25343215315,113sleep325493-21cat09:55:193
25343215315,113sleep325493-21cat09:55:193
14873215319,119sleep00-21swapper/009:20:400
23110214921,101sleep20-21swapper/211:30:202
20325214738,34sleep00-21swapper/007:55:320
632321466,12sleep00-21swapper/008:53:140
2305021468,123sleep023178-21unixbench_multi09:45:470
22138214610,118sleep022223-21latency_hist09:45:190
1355221459,124sleep00-21swapper/007:32:180
1355221459,124sleep00-21swapper/007:32:180
835921438,119sleep00-21swapper/009:00:330
31829214217,98sleep331972-21grep11:55:483
10341213811,114sleep09405-21/usr/sbin/munin07:20:510
3588213715,106sleep33646-21sendmail_mailqu10:25:483
3588213715,106sleep33646-21sendmail_mailqu10:25:483
1334921340,23sleep213354-21sed07:30:502
1334921340,23sleep213354-21sed07:30:502
2928221328,113sleep029310-21munin-run11:50:160
12076213110,106sleep012187-21ls09:10:470
10043213011,39sleep00-21swapper/009:05:380
16828212819,86sleep216855-21awk11:10:172
2611212022,83sleep12674-21apt-key10:25:171
8265211820,80sleep38268-21apt-get12:25:283
3664211835,13sleep00-21swapper/008:45:310
3664211835,13sleep00-21swapper/008:45:310
2118021175,100sleep00-21swapper/011:21:050
2663621167,39sleep00-21swapper/008:15:350
1532121166,36sleep00-21swapper/011:05:180
25046211454,45sleep00-21swapper/008:10:340
2200921136,35sleep00-21swapper/011:25:370
11089211222,76sleep311167-21unin-run12:35:173
983121118,17sleep20-21swapper/212:30:322
929211119,52sleep21062-21unixbench_multi12:00:482
929211119,52sleep21062-21unixbench_multi12:00:482
28430211035,36sleep00-21swapper/008:20:350
28430211035,36sleep00-21swapper/008:20:350
32578210917,69sleep315670-21sshd10:15:553
26208210914,78sleep225712-21/usr/sbin/munin09:56:442
15315210826,59sleep115464-21unixbench_singl09:20:491
5665210721,67sleep35833-21sort07:10:203
11561210711,40sleep00-21swapper/007:25:480
11561210711,40sleep00-21swapper/007:25:480
25185210420,70sleep125250-21http09:55:171
25185210420,70sleep125250-21http09:55:171
11500210314,71sleep211618-21timerandwakeup10:50:472
17444210215,74sleep117450-21/usr/sbin/munin09:30:181
4663210129,49sleep34665-21cpu10:30:323
4663210129,49sleep34665-21cpu10:30:323
22067210123,29sleep00-21swapper/008:00:410
2037021016,84sleep00-21swapper/009:38:080
10526210110,79sleep210603-21/usr/sbin/munin09:05:482
21631210017,26sleep20-21swapper/208:00:182
788529944,43sleep07913-21grep07:15:340
3197729911,35sleep00-21swapper/011:55:500
2504429914,70sleep325110-21fschecks_time11:35:383
2504429914,70sleep325110-21fschecks_time11:35:383
1583829925,62sleep015907-21sort11:05:390
849629819,57sleep38593-21timerandwakeup07:15:503
3182529812,23sleep10-21swapper/108:30:421
244572985,81sleep00-21swapper/011:33:210
1624129820,60sleep216307-21fschecks_time09:25:382
1562529819,49sleep315644-21apt-get07:40:293
1562529819,49sleep315644-21apt-get07:40:293
10342989,74sleep00-21swapper/008:35:460
2999729719,27sleep00-21swapper/010:08:490
1408629721,30sleep00-21swapper/009:17:210
1408629721,30sleep00-21swapper/009:17:210
3050329618,31sleep00-21swapper/011:51:050
140829623,54sleep21493-21fschecks_time10:20:382
2607729518,30sleep00-21swapper/011:38:540
2607729518,30sleep00-21swapper/011:38:540
1983729519,28sleep00-21swapper/007:53:100
1872129511,70sleep118740-21apt-get07:50:281
1201529518,29sleep00-21swapper/010:53:350
883429418,29sleep00-21swapper/010:42:290
1354929420,29sleep00-21swapper/010:57:550
3144129318,30sleep00-21swapper/010:12:200
31092936,32sleep00-21swapper/010:25:330
31092936,32sleep00-21swapper/010:25:330
285629319,28sleep00-21swapper/012:07:410
285629319,28sleep00-21swapper/012:07:410
1428929317,64sleep214316-21kernelversion11:00:402
989429214,66sleep39901-21basename09:05:293
413329223,54sleep14141-21ntp_states08:45:441
413329223,54sleep14141-21ntp_states08:45:441
50429113,66sleep1522-21kernelversion12:00:391
50429113,66sleep1522-21kernelversion12:00:391
2208529121,50sleep222109-21cron09:45:172
177229114,63sleep11785-21basename12:05:271
177229114,63sleep11785-21basename12:05:271
1086229128,31sleep30-21swapper/310:50:273
2839729034,43sleep325884-21systemd-journal10:03:113
951628926,28sleep20-21swapper/207:20:372
248928914,20sleep30-21swapper/308:40:413
1728728920,48sleep217378-21fschecks_time07:45:362
1728728920,48sleep217378-21fschecks_time07:45:362
2656128820,44sleep126655-21fschecks_time11:40:361
2656128820,44sleep126655-21fschecks_time11:40:361
2573728815,56sleep125204-21apt-get09:55:301
1350828815,52sleep113029-21/usr/sbin/munin09:15:441
1350828815,52sleep113029-21/usr/sbin/munin09:15:441
2884828725,40sleep228845-21apt-get10:05:282
815928619,47sleep38192-21/usr/sbin/munin10:40:423
679828515,40sleep20-21swapper/208:55:302
303682857,18sleep20-21swapper/208:25:462
2704628517,50sleep327128-21mailstats08:15:463
658128319,49sleep36670-21grep12:20:403
1447528314,52sleep10-21swapper/107:35:421
1447528314,52sleep10-21swapper/107:35:421
409628124,27sleep30-21swapper/307:05:203
375328117,31sleep20-21swapper/212:10:412
960728014,52sleep39660-21latency10:45:393
3016228027,26sleep10-21swapper/110:10:191
284128029,29sleep00-21swapper/007:00:370
3235927915,48sleep231420-21/usr/sbin/munin08:30:512
1728327916,45sleep117175-21/usr/sbin/munin07:45:351
1728327916,45sleep117175-21/usr/sbin/munin07:45:351
330127833,23sleep20-21swapper/207:00:472
787727720,39sleep27916-21cut10:40:372
32627713,30sleep10-21swapper/110:16:011
3122627711,46sleep20-21swapper/211:55:352
959327621,26sleep10-21swapper/112:30:191
914027628,30sleep20-21swapper/207:20:172
861627621,22sleep20-21swapper/212:26:242
1977427613,25sleep30-21swapper/309:35:453
1437527617,29sleep30-21swapper/307:35:403
1437527617,29sleep30-21swapper/307:35:403
3141927514,46sleep331421-21/usr/sbin/munin08:30:283
2633527521,27sleep20-21swapper/208:15:202
242192750,7sleep30-21swapper/311:30:483
3021327413,46sleep230332-21cat11:50:472
192527422,30sleep20-21swapper/208:40:202
774027326,22sleep10-21swapper/112:25:101
522327317,42sleep15247-21grep08:50:331
1591327312,16sleep30-21swapper/309:25:193
872627224,28sleep10-21swapper/109:00:421
66692726,14sleep30-21swapper/310:35:413
35732727,12sleep20-21swapper/210:25:472
35732727,12sleep20-21swapper/210:25:472
285182727,17sleep10-21swapper/111:45:451
2460727218,38sleep224611-21cron11:35:162
1298627216,19sleep10-21swapper/107:30:461
1298627216,19sleep10-21swapper/107:30:461
401627127,24sleep10-21swapper/107:05:181
1838527125,30sleep218462-21systemd-run11:15:172
1473227116,38sleep214813-21fw_packets09:20:382
2287327022,26sleep20-21swapper/209:45:452
2044927017,27sleep10-21swapper/109:39:101
1757627016,26sleep30-21swapper/307:45:423
1757627016,26sleep30-21swapper/307:45:423
2112326921,26sleep10-21swapper/111:20:501
64426813,39sleep1650-21/usr/sbin/munin08:35:341
2262526815,37sleep322772-21awk08:00:523
2137226816,12sleep10-21swapper/109:40:481
695926722,21sleep10-21swapper/110:35:471
3083126720,27sleep330845-21modprobe11:55:163
3058126718,29sleep10-21swapper/108:25:491
2632226713,26sleep10-21swapper/108:15:191
261872676,15sleep20-21swapper/211:40:172
261872676,15sleep20-21swapper/211:40:172
2033326714,21sleep30-21swapper/311:20:363
1423126718,24sleep10-21swapper/111:00:391
132426721,19sleep30-21swapper/310:20:353
1165426712,39sleep111782-21unixbench_multi07:25:491
1165426712,39sleep111782-21unixbench_multi07:25:491
1032026727,23sleep125884-21systemd-journal09:05:431
517426613,21sleep10-21swapper/107:05:501
487326617,16sleep20-21swapper/207:05:472
1063926616,31sleep210665-21latency_hist10:50:182
601826523,24sleep20-21swapper/207:10:242
2664826521,27sleep20-21swapper/208:15:372
74362648,47sleep10-21swapper/107:13:421
461226414,34sleep34613-21sshd08:46:103
461226414,34sleep34613-21sshd08:46:103
113462645,12sleep20-21swapper/207:25:422
113462645,12sleep20-21swapper/207:25:422
970726322,19sleep10-21swapper/110:45:411
552226320,23sleep20-21swapper/212:15:462
3154826316,29sleep131590-21/usr/sbin/munin11:55:421
3077026319,26sleep330815-21awk10:10:423
2091426327,22sleep30-21swapper/309:40:383
2072526314,35sleep320772-21/usr/sbin/munin07:55:433
842826214,33sleep28435-21df_inode09:00:352
642426219,19sleep20-21swapper/210:35:362
2803426218,20sleep10-21swapper/108:20:201
2525126219,25sleep225342-21gpgv09:55:182
2525126219,25sleep225342-21gpgv09:55:182
2008526216,19sleep30-21swapper/307:55:193
1916926217,22sleep10-21swapper/111:15:451
1910426216,28sleep219149-21/usr/sbin/munin11:15:422
1777326113,33sleep217780-21df_inode09:30:352
655126015,21sleep20-21swapper/208:55:182
361526014,30sleep33655-21latency12:10:383
3251526018,28sleep232575-21latency10:15:542
3107725923,20sleep20-21swapper/210:10:492
288402598,35sleep128910-21mailstats08:20:491
288402598,35sleep128910-21mailstats08:20:491
229952596,10sleep10-21swapper/108:03:281
220625916,28sleep22284-21fschecks_time08:40:362
187852595,16sleep10-21swapper/109:33:041
1103025919,21sleep30-21swapper/307:25:353
1103025919,21sleep30-21swapper/307:25:353
738025817,23sleep30-21swapper/308:55:473
658025821,18sleep30-21swapper/307:10:413
657225818,17sleep30-21swapper/308:55:193
310825822,16sleep30-21swapper/312:10:183
310825822,16sleep30-21swapper/312:10:183
3040725813,19sleep30-21swapper/308:25:473
2389625811,13sleep10-21swapper/108:05:461
1259625817,25sleep312605-21df_inode07:30:353
1259625817,25sleep312605-21df_inode07:30:353
2928925718,20sleep30-21swapper/308:21:113
2928925718,20sleep30-21swapper/308:21:113
2891225717,18sleep20-21swapper/208:20:502
2891225717,18sleep20-21swapper/208:20:502
1683125715,25sleep316952-21sed09:25:523
1602425715,23sleep116030-21ntp_states11:05:441
1376725720,19sleep20-21swapper/209:15:492
1376725720,19sleep20-21swapper/209:15:492
544225616,19sleep10-21swapper/112:15:451
502525620,15sleep30-21swapper/312:15:353
2778325617,21sleep20-21swapper/210:00:482
2206825614,25sleep10-21swapper/111:25:381
197642565,11sleep30-21swapper/311:17:453
1837425620,18sleep30-21swapper/311:15:103
164922566,39sleep30-21swapper/311:05:583
1470325617,15sleep30-21swapper/311:00:493
1305725618,17sleep30-21swapper/309:15:343
1305725618,17sleep30-21swapper/309:15:343
1253625619,19sleep10-21swapper/110:55:341
1246225614,25sleep212456-21apt-get10:55:312
919325513,18sleep10-21swapper/107:20:201
2647225515,24sleep326468-21apt-get11:40:303
2647225515,24sleep326468-21apt-get11:40:303
259022554,16sleep10-21swapper/111:36:371
259022554,16sleep10-21swapper/111:36:371
2571125515,23sleep325729-21apt09:55:273
236232557,12sleep30-21swapper/309:50:103
2342825512,26sleep323423-21apt-get08:05:323
155125516,23sleep31575-21latency_hist12:05:173
155125516,23sleep31575-21latency_hist12:05:173
126462555,12sleep10-21swapper/109:14:111
1233725513,23sleep212370-21users09:10:492
1074225515,18sleep10-21swapper/107:25:201
980625412,18sleep30-21swapper/307:20:423
473725412,19sleep10-21swapper/110:30:351
473725412,19sleep10-21swapper/110:30:351
338225419,15sleep10-21swapper/110:25:421
338225419,15sleep10-21swapper/110:25:421
2928525415,22sleep329304-21ntp_states10:05:413
2778525420,15sleep20-21swapper/211:45:182
2778525420,15sleep20-21swapper/211:45:182
2396025415,21sleep224031-21sensors_temp08:05:472
2318825418,22sleep20-21swapper/208:05:192
1410425415,23sleep214101-21apt-get07:35:312
1410425415,23sleep214101-21apt-get07:35:312
1275825414,22sleep312830-21memory10:55:403
836825316,18sleep10-21swapper/110:40:481
677625312,23sleep16795-21apt-get08:55:291
2160325316,22sleep221756-21cut11:25:192
211725312,24sleep12111-21apt-get08:40:311
178725314,23sleep21792-21apt-get12:05:282
178725314,23sleep21792-21apt-get12:05:282
2933425213,21sleep329464-21latency_hist11:50:193
2496825211,25sleep324966-21apt-get08:10:293
2249725212,24sleep122583-21fw_packets09:45:361
202952527,31sleep10-21swapper/107:55:251
125225215,19sleep30-21swapper/308:35:493
376925110,15sleep20-21swapper/208:45:372
376925110,15sleep20-21swapper/208:45:372
2999525113,23sleep130024-21ntp_states11:50:421
2343125116,16sleep10-21swapper/111:30:351
200712519,16sleep20-21swapper/209:35:482
1699625115,20sleep116932-21latency_hist07:45:191
124002515,11sleep30-21swapper/309:11:063
45372505,11sleep20-21swapper/210:30:192
45372505,11sleep20-21swapper/210:30:192
3048225013,21sleep130486-21diskstats10:10:361
2802225012,21sleep228018-21apt-get11:45:302
167752507,31sleep20-21swapper/207:43:352
167752507,31sleep20-21swapper/207:43:352
1572625016,16sleep10-21swapper/107:40:351
983024912,21sleep19825-21apt-get12:30:311
334824912,21sleep13416-21df12:10:321
2281524913,21sleep322816-21sshd11:27:103
1911524819,15sleep20-21swapper/207:50:412
835724411,16sleep30-21swapper/309:00:323
1814824411,18sleep30-21swapper/309:30:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional