You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 15:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Dec 22, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
178302101523,980sleep017837-21ntpq08:29:550
22804293695,803sleep022802-21ntp_states12:14:550
22804293695,803sleep022802-21ntp_states12:14:550
6444246717,435sleep36448-21ntp_states11:19:543
8703242911,399sleep08049-21/usr/sbin/munin07:59:590
21603238861,199sleep021669-21wc10:24:580
1934423347,314sleep019416-21latency_hist10:19:290
907323258,39sleep00-21swapper/011:29:440
2419023257,304sleep024278-21cat12:19:480
15824229630,250sleep315825-21users10:05:023
20046228317,251sleep00-21swapper/010:19:570
11902226613,190sleep012038-21unixbench_singl08:10:000
32262226340,205sleep00-21swapper/009:14:590
11010226216,231sleep111051-21ntpq11:34:541
27640226111,128sleep027793-21unixbench_multi09:00:170
1213221260,99sleep01251-21processes09:19:560
6957220719,44sleep00-21swapper/007:54:540
22107220529,158sleep122199-21fschecks_time08:44:471
9708219715,122sleep09746-21fschecks_count08:04:470
9708219715,122sleep09746-21fschecks_count08:04:470
22290219455,119sleep00-21swapper/008:44:500
1419021946,176sleep00-21swapper/011:44:590
1419021946,176sleep00-21swapper/011:44:590
2482621939,138sleep024868-21true08:54:270
1797321879,163sleep318047-21find08:29:583
7209218245,23sleep00-21swapper/009:39:510
25119217731,82sleep00-21swapper/008:54:430
25119217731,82sleep00-21swapper/008:54:430
635421768,32sleep00-21swapper/011:19:500
429821769,127sleep04363-21processes09:29:570
11803217517,140sleep211797-21apt-get09:54:432
25859216979,51sleep0314-21rs:main0
27092216540,71sleep027208-21latency_hist10:44:280
2634216425,102sleep02786-21date11:09:280
2609721579,136sleep026168-21timerandwakeup07:14:310
8817215519,121sleep28823-21df_inode09:45:062
19101215523,90sleep00-21swapper/011:59:570
1832215517,115sleep10-21swapper/111:04:571
13987215335,80sleep014103-21timerandwakeup09:59:580
13987215335,80sleep014103-21timerandwakeup09:59:580
25982215217,117sleep30-21swapper/307:14:303
27517214916,77sleep00-21swapper/010:44:490
24279214928,98sleep124328-21modprobe07:09:271
30489214116,16sleep00-21swapper/009:09:580
333021326,29sleep00-21swapper/009:26:420
15499213118,59sleep00-21swapper/010:04:590
3239213011,73sleep00-21swapper/011:09:530
14050213014,79sleep014174-21cat08:19:290
6209212913,100sleep16245-21idleruntime-cro07:54:261
13856212711,86sleep20-21swapper/209:59:562
13856212711,86sleep20-21swapper/209:59:562
16767212619,12sleep00-21swapper/008:25:190
7308212415,89sleep27353-21/usr/sbin/munin09:39:532
4511212420,80sleep34658-21unixbench_multi09:30:003
29232212340,16sleep00-21swapper/012:34:540
29232212340,16sleep00-21swapper/012:34:540
23847212315,12sleep00-21swapper/010:33:300
31747212162,16sleep00-21swapper/007:29:580
31747212162,16sleep00-21swapper/007:29:580
25182212114,88sleep225234-21true12:24:262
30777211716,20sleep20-21swapper/210:54:482
25870211717,80sleep10-21swapper/107:14:281
441321168,71sleep04403-21apt-get11:14:460
441321168,71sleep04403-21apt-get11:14:460
31037211630,68sleep00-21swapper/007:29:300
25875211524,74sleep325909-21cut10:39:473
15506211461,16sleep00-21swapper/011:49:430
21233211358,16sleep00-21swapper/012:09:540
2686211216,80sleep12732-21grep09:24:531
13007211215,11sleep00-21swapper/009:58:490
22872211022,70sleep222946-21sensors_temp12:14:572
22872211022,70sleep222946-21sensors_temp12:14:572
606921095,75sleep00-21swapper/007:52:360
5090210616,69sleep30-21swapper/307:49:483
9595210516,69sleep39603-21/usr/sbin/munin08:04:383
9595210516,69sleep39603-21/usr/sbin/munin08:04:383
7399210514,22sleep10-21swapper/109:39:561
30141210514,77sleep330157-21open_inodes07:24:563
10419210514,70sleep310479-21/usr/sbin/munin09:49:473
8472210414,75sleep38554-21apt-key09:44:303
11073210425,22sleep011107-21ps11:34:570
4063210341,41sleep00-21swapper/007:45:000
3023210216,74sleep00-21swapper/007:43:020
21128210218,64sleep30-21swapper/310:24:453
17350210213,70sleep317366-21/usr/sbin/munin10:09:593
1682210011,75sleep21831-21latency_hist07:39:292
3041529812,71sleep230507-21latency_hist10:54:282
134302989,54sleep013433-21cron11:44:260
1670029717,40sleep00-21swapper/010:09:480
1502129713,66sleep215112-21fschecks_time10:04:492
2432929519,57sleep224348-21munin-run07:09:282
3168329415,63sleep231684-21ntp_states07:29:562
3168329415,63sleep231684-21ntp_states07:29:562
2178229430,22sleep021877-21sed08:44:290
1682529446,21sleep20-21swapper/210:09:502
1871529316,58sleep217979-21/usr/sbin/munin10:14:582
2568129220,48sleep00-21swapper/012:24:360
2568129220,48sleep00-21swapper/012:24:360
2489629219,56sleep225034-21date08:54:292
1862029216,60sleep318667-21grep11:59:443
1788429214,62sleep217972-21mailstats08:29:572
3169029116,59sleep331743-21processes07:29:573
3169029116,59sleep331743-21processes07:29:573
3198829024,43sleep132026-21ntp_kernel_err09:14:521
1861429014,59sleep118712-21timerandwakeup10:14:571
1280429019,23sleep30-21swapper/308:14:483
325942888,16sleep00-21swapper/010:59:510
3092828823,42sleep21-21systemd07:29:272
319092878,20sleep031981-21systemd-tmpfile10:59:270
1334928710,16sleep10-21swapper/109:59:391
1334928710,16sleep10-21swapper/109:59:391
3012928628,30sleep10-21swapper/112:39:281
3012928628,30sleep10-21swapper/112:39:281
2887428618,41sleep128924-21ls10:49:301
100182867,18sleep010022-21munin-run09:49:280
887228518,51sleep18958-21fschecks_time09:45:131
63472855,69sleep00-21swapper/009:35:510
2271228415,41sleep122756-21ntpq12:14:521
2271228415,41sleep122756-21ntpq12:14:521
1931228327,41sleep019347-21ntpq08:34:560
1393528317,52sleep113985-21if_eth011:44:511
1393528317,52sleep113985-21if_eth011:44:511
84828230,41sleep032733-21/usr/sbin/munin07:34:550
84828230,41sleep032733-21/usr/sbin/munin07:34:550
2972728223,36sleep10-21swapper/109:09:261
2704028222,33sleep227057-21munin-run10:44:262
2569028226,26sleep30-21swapper/308:54:593
2569028226,26sleep30-21swapper/308:54:593
1156328227,19sleep011649-21grep09:54:290
2934928127,37sleep00-21swapper/010:50:100
2725828122,42sleep00-21swapper/012:29:410
2725828122,42sleep00-21swapper/012:29:410
2644528126,37sleep226516-21fschecks_count07:14:372
2640328115,51sleep125980-21munin-node07:14:361
2523528123,40sleep325326-21latency_hist12:24:273
2126128118,45sleep321380-21/usr/sbin/munin08:40:003
10232819,23sleep21162-21/usr/sbin/munin07:34:572
324328015,45sleep23244-21sshd09:25:352
291692808,35sleep00-21swapper/009:05:010
2870828016,50sleep228780-21sed07:19:582
1928228019,42sleep318890-21/usr/sbin/munin08:34:553
1675528029,33sleep316785-21munin-run11:54:273
281422789,35sleep00-21swapper/007:19:440
2238927815,47sleep222433-21ntpq08:44:522
917027724,30sleep20-21swapper/211:29:502
661927718,43sleep16692-21sh09:39:271
2048127715,25sleep10-21swapper/110:20:021
2028327718,38sleep120181-21latency_hist08:39:301
201352778,12sleep00-21swapper/012:04:330
1241627723,32sleep2228-21systemd-journal08:14:262
171762766,12sleep00-21swapper/011:54:480
141027621,25sleep30-21swapper/311:04:473
86572755,58sleep00-21swapper/011:27:500
2661227523,32sleep226386-21apt-get08:59:302
2661227523,32sleep226386-21apt-get08:59:302
2579927510,48sleep125809-21/usr/sbin/munin12:24:461
2579927510,48sleep125809-21/usr/sbin/munin12:24:461
2373527518,28sleep30-21swapper/308:49:503
2089927520,23sleep20-21swapper/212:09:482
2892627418,28sleep228601-21apt-get10:49:312
2818427420,24sleep20-21swapper/209:04:272
246762746,32sleep00-21swapper/010:34:550
1466727428,28sleep30-21swapper/308:19:533
3218727315,15sleep30-21swapper/309:14:583
31827312,46sleep2394-21expr10:59:572
147182738,51sleep00-21swapper/008:19:540
881827229,20sleep10-21swapper/108:00:001
472327219,38sleep14863-21latency_hist07:49:301
2919627221,34sleep329203-21/usr/sbin/munin10:49:493
1640327216,41sleep216496-21cat08:24:572
93132716,14sleep20-21swapper/208:04:132
2401927110,14sleep10-21swapper/108:49:561
2313027119,28sleep10-21swapper/110:29:551
2287527126,26sleep20-21swapper/210:29:482
159227115,41sleep21608-21/usr/sbin/munin11:04:502
1465427118,24sleep20-21swapper/208:19:522
1777827017,28sleep10-21swapper/110:14:291
1069627010,28sleep30-21swapper/311:34:483
53372698,48sleep227124-21ntpd09:34:362
369126925,29sleep30-21swapper/307:44:513
2747126923,22sleep10-21swapper/112:29:501
2747126923,22sleep10-21swapper/112:29:501
1731526910,17sleep20-21swapper/211:54:502
491826822,18sleep30-21swapper/311:15:013
491826822,18sleep30-21swapper/311:15:013
3105526820,31sleep331097-21awk10:54:533
2255526812,15sleep30-21swapper/308:44:573
1053026822,21sleep10-21swapper/109:49:491
3261426715,24sleep10-21swapper/110:59:521
281882678,12sleep20-21swapper/212:30:162
281882678,12sleep20-21swapper/212:30:162
2055426715,37sleep220585-21sed12:09:282
798726619,23sleep30-21swapper/311:24:543
71726625,22sleep30-21swapper/309:19:383
50226618,22sleep20-21swapper/209:19:292
2890026621,21sleep20-21swapper/212:34:482
2890026621,21sleep20-21swapper/212:34:482
1774026621,21sleep10-21swapper/108:29:521
536726527,21sleep35368-21cpu09:34:413
2755326513,38sleep327632-21wc09:00:163
2689426514,32sleep126386-21apt-get08:59:371
2591026512,37sleep225980-21nin-node07:14:292
2466626519,21sleep10-21swapper/112:19:571
2020126518,30sleep320327-21/usr/sbin/munin10:20:003
1249926527,21sleep10-21swapper/108:14:301
862726414,33sleep38700-21sort07:59:583
310226410,43sleep20-21swapper/207:44:052
3005026423,21sleep30-21swapper/312:39:263
3005026423,21sleep30-21swapper/312:39:263
2769826419,25sleep30-21swapper/310:44:523
1744926413,21sleep10-21swapper/111:54:531
1713226420,22sleep30-21swapper/308:29:303
1225226416,28sleep312261-21/usr/sbin/munin11:39:453
340326311,34sleep13472-21sensors_temp11:09:581
2734726316,23sleep30-21swapper/312:29:483
2734726316,23sleep30-21swapper/312:29:483
2013126314,30sleep220200-21/usr/sbin/munin10:19:592
649126215,24sleep10-21swapper/111:19:561
564426224,17sleep20-21swapper/207:49:592
466926214,21sleep20-21swapper/207:49:262
3276026224,19sleep30-21swapper/307:34:413
3276026224,19sleep30-21swapper/307:34:413
1456126214,30sleep114265-21/usr/sbin/munin08:19:511
1080726217,31sleep110808-21sshd08:08:361
1080726217,31sleep110808-21sshd08:08:361
2695526116,25sleep30-21swapper/307:14:553
241412615,11sleep00-21swapper/007:07:370
2279626114,30sleep30-21swapper/312:14:543
2279626114,30sleep30-21swapper/312:14:543
1230926118,25sleep10-21swapper/109:54:581
1188926123,17sleep30-21swapper/309:54:483
318026015,20sleep10-21swapper/107:44:291
2700826018,28sleep227032-21sed12:29:272
2700826018,28sleep227032-21sed12:29:272
2512026018,15sleep10-21swapper/108:54:441
2512026018,15sleep10-21swapper/108:54:441
2421026011,35sleep324205-21apt-get10:34:423
1957326018,26sleep219700-21cat08:35:022
1885826010,35sleep118889-21irqstats11:59:501
1396926022,23sleep213995-21munin-run08:19:272
1098626016,26sleep20-21swapper/209:49:592
945025911,33sleep39498-21ntp_kernel_pll_11:29:553
91125915,22sleep10-21swapper/107:34:561
91125915,22sleep10-21swapper/107:34:561
3111125919,24sleep131117-21ntp_states10:54:561
3047725917,20sleep30-21swapper/309:09:573
625125818,25sleep26272-21sed11:19:482
347425821,20sleep20-21swapper/211:09:592
2869025820,23sleep228746-21/usr/sbin/munin09:04:512
2627725823,19sleep10-21swapper/110:39:571
2348325816,25sleep323489-21ntp_states07:04:533
132672588,12sleep10-21swapper/111:42:171
85592575,12sleep10-21swapper/111:26:341
668425722,16sleep20-21swapper/207:54:482
647625712,18sleep10-21swapper/107:54:381
416725714,15sleep30-21swapper/311:14:283
2912125715,24sleep129188-21df10:49:471
2120125717,20sleep10-21swapper/110:24:471
2114325714,25sleep221259-21ls08:39:592
160372579,30sleep10-21swapper/108:24:471
1558625718,18sleep10-21swapper/110:05:001
1519225716,16sleep30-21swapper/311:49:303
1519225716,16sleep30-21swapper/311:49:303
1378925714,24sleep313825-21ntp_states09:59:533
1378925714,24sleep313825-21ntp_states09:59:533
119325716,26sleep21204-21ntpq09:19:542
452725615,24sleep24619-21perl11:14:522
452725615,24sleep24619-21perl11:14:522
2848025614,24sleep328471-21apt-get09:04:443
2112725615,23sleep221121-21apt-get10:24:442
1987725617,22sleep219927-21munin-run12:04:272
1510725613,27sleep215184-21munin-run11:49:292
1510725613,27sleep215184-21munin-run11:49:292
1326125616,26sleep113333-21sort08:14:581
79712557,36sleep30-21swapper/309:41:353
2442725514,24sleep224472-21if_eth010:34:492
1919825517,24sleep319199-21sshd10:18:353
1615325515,28sleep216217-21sed11:50:342
883125420,19sleep232320-1kworker/2:1H11:29:302
705725416,18sleep30-21swapper/307:54:573
469525411,21sleep10-21swapper/111:14:551
469525411,21sleep10-21swapper/111:14:551
39525418,16sleep30-21swapper/310:59:583
3009625414,23sleep130133-21ntp_states07:24:541
3003425416,21sleep10-21swapper/109:09:471
1178225415,18sleep30-21swapper/308:09:593
291922534,11sleep30-21swapper/307:21:533
2599225313,24sleep326009-21irqstats12:25:083
2599225313,24sleep326009-21irqstats12:25:083
1547325316,23sleep115474-21sshd11:49:361
391525213,24sleep23916-21diskstats09:29:472
3250225213,25sleep132518-21munin-run07:34:271
3250225213,25sleep132518-21munin-run07:34:271
2943825213,24sleep129530-21latency_hist07:24:301
1264825211,18sleep20-21swapper/211:39:562
390725116,17sleep10-21swapper/109:29:461
225425113,24sleep32256-21sshd09:24:353
2125825113,22sleep121264-21ntp_states12:09:551
2105625113,25sleep121142-21sed08:39:581
204725121,17sleep10-21swapper/109:24:291
19362516,11sleep30-21swapper/307:39:463
2266725012,23sleep322662-21apt-get10:29:413
969224919,16sleep20-21swapper/208:04:452
969224919,16sleep20-21swapper/208:04:452
311292498,32sleep20-21swapper/209:11:502
2421324919,13sleep10-21swapper/110:34:441
2128924815,16sleep30-21swapper/312:09:573
1105324711,21sleep211064-21ntp_states11:34:552
2058624611,15sleep30-21swapper/312:09:293
1713324614,17sleep30-21swapper/311:54:473
906824312,15sleep10-21swapper/111:29:411
2051124312,17sleep120552-21systemd-run12:09:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional