You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 20:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue May 19, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
178122408315,4055sleep19854-21kworker/1:110:13:191
2892129917,972sleep328957-21ntpq12:29:343
10385253914,511sleep110423-21ntpq08:05:461
25861249626,454sleep125904-21/usr/sbin/munin12:19:391
24349241766,303sleep324502-21sed12:14:403
3132237612,344sleep33153-21munin-plugin-st07:44:083
3132237612,344sleep33153-21munin-plugin-st07:44:083
101622350129,100sleep30-21swapper/309:49:113
14786223415,203sleep014825-21true10:04:100
14786223415,203sleep014825-21true10:04:100
14262222644,119sleep30-21swapper/308:19:083
20057222520,187sleep323758-21sshd12:02:013
32175222023,149sleep332215-21modprobe10:59:073
32175222023,149sleep332215-21modprobe10:59:073
27614221238,29sleep30-21swapper/312:24:373
9862221013,178sleep39881-21apt-get08:04:193
30531220614,175sleep330745-21latency_hist10:54:113
30531220614,175sleep330745-21latency_hist10:54:113
27491220351,113sleep327525-21ps08:59:383
26159218842,46sleep326135-21fschecks_count10:39:293
1714121868,164sleep317183-21date11:54:093
25604218511,158sleep325621-21idleruntime-cro07:14:083
13599218319,151sleep113605-21cat09:59:271
13599218319,151sleep113605-21cat09:59:271
2251121788,157sleep322575-21cat08:44:283
2251121788,157sleep322575-21cat08:44:283
30839217413,141sleep330987-21unixbench_singl12:34:433
31154216714,95sleep30-21swapper/309:11:003
3075221656,15sleep00-21swapper/010:54:120
3075221656,15sleep00-21swapper/010:54:120
14907216516,98sleep30-21swapper/310:04:133
14907216516,98sleep30-21swapper/310:04:133
9511216112,100sleep30-21swapper/308:04:103
9511216112,100sleep30-21swapper/308:04:103
4452215810,133sleep34519-21latency_hist11:14:093
4452215810,133sleep34519-21latency_hist11:14:093
11469215714,92sleep30-21swapper/311:34:363
22972215310,131sleep322995-21tune2fs10:29:283
22972215310,131sleep322995-21tune2fs10:29:283
123321466,10sleep30-21swapper/311:03:153
26788214134,20sleep30-21swapper/308:59:093
26788214134,20sleep30-21swapper/308:59:093
315821390,7sleep00-21swapper/007:44:080
315821390,7sleep00-21swapper/007:44:080
26289213616,106sleep026348-21cpuspeed07:14:250
14662213623,96sleep214687-21fschecks_count08:19:282
19878213516,99sleep219961-21tune2fs10:19:302
23656213413,105sleep123625-21latency_hist08:49:101
23656213413,105sleep123625-21latency_hist08:49:101
13516213317,98sleep213570-21cpuspeed09:59:242
13516213317,98sleep213570-21cpuspeed09:59:242
3878213231,20sleep30-21swapper/307:44:363
3878213231,20sleep30-21swapper/307:44:363
12581213217,93sleep212600-21fschecks_count11:39:272
14503212914,16sleep20-21swapper/211:44:362
12498212614,19sleep30-21swapper/311:39:243
18133212512,96sleep018139-21ntpq08:29:350
28264212318,84sleep128420-21grep10:44:401
22041212365,45sleep322051-21find12:09:193
22041212365,45sleep322051-21find12:09:193
16582212257,49sleep30-21swapper/308:24:383
27957211920,21sleep30-21swapper/307:19:293
16723211923,81sleep316805-21fschecks_time10:09:383
32001211812,80sleep30-21swapper/309:14:403
6939211572,33sleep30-21swapper/309:38:463
19684211515,28sleep10-21swapper/108:34:371
26371211440,62sleep326385-21grep07:14:283
25098211316,20sleep10-21swapper/112:19:101
21427211314,80sleep121451-21fschecks_count10:24:301
1860621128,93sleep3363-21snmpd11:58:423
9460211122,66sleep09521-21fschecks_time11:29:280
30388211119,78sleep130444-21apt-config10:54:091
30388211119,78sleep130444-21apt-config10:54:091
1183211127,21sleep30-21swapper/309:19:283
1183211127,21sleep30-21swapper/309:19:283
2864021105,12sleep30-21swapper/310:46:363
2649221105,18sleep30-21swapper/312:21:183
6922210812,83sleep36969-21ntpq07:54:343
6922210812,83sleep36969-21ntpq07:54:343
290821089,17sleep30-21swapper/309:24:413
290821089,17sleep30-21swapper/309:24:413
9347210715,58sleep30-21swapper/311:29:193
1747921070,6sleep00-21swapper/008:29:100
6038210624,68sleep36148-21cat11:19:103
24993210510,73sleep30-21swapper/310:34:453
24993210510,73sleep30-21swapper/310:34:453
2174921056,52sleep30-21swapper/308:40:123
2174921056,52sleep30-21swapper/308:40:123
1861210524,61sleep21342-21apt-get07:39:182
1861210524,61sleep21342-21apt-get07:39:182
15439210512,76sleep015467-21nfs4_client10:04:340
15439210512,76sleep015467-21nfs4_client10:04:340
20429210420,57sleep30-21swapper/308:38:303
1177321046,83sleep00-21swapper/009:54:130
1441121038,82sleep314455-21ntpq11:44:333
23151210116,64sleep023193-21ntp_kernel_pll_07:04:330
4254210014,63sleep00-21swapper/007:44:410
4254210014,63sleep00-21swapper/007:44:410
3194329918,62sleep031922-21apt-get09:14:330
778029821,55sleep17803-21latency_hist07:59:081
778029821,55sleep17803-21latency_hist07:59:081
2068929820,10sleep30-21swapper/310:20:003
243692977,76sleep323870-21/usr/sbin/munin08:49:373
243692977,76sleep323870-21/usr/sbin/munin08:49:373
4822966,15sleep30-21swapper/307:34:353
3137129616,64sleep231407-21/usr/sbin/munin10:55:032
3137129616,64sleep231407-21/usr/sbin/munin10:55:032
698729513,67sleep27051-21ps07:54:372
698729513,67sleep27051-21ps07:54:372
2991429535,18sleep30-21swapper/309:09:113
150672959,54sleep30-21swapper/310:04:213
150672959,54sleep30-21swapper/310:04:213
182129412,13sleep20-21swapper/209:19:442
182129412,13sleep20-21swapper/209:19:442
181702947,15sleep30-21swapper/310:14:223
1129329423,53sleep211337-21ntpq11:34:312
2917929316,62sleep029270-21latency_hist07:24:100
188352937,75sleep3363-21snmpd08:32:573
3213529220,10sleep30-21swapper/307:33:013
3213529220,10sleep30-21swapper/307:33:013
2755329221,52sleep227555-21cron07:19:082
115232926,14sleep30-21swapper/308:09:273
1083929219,10sleep30-21swapper/309:49:403
731929120,39sleep30-21swapper/311:21:243
605029121,10sleep30-21swapper/307:51:513
605029121,10sleep30-21swapper/307:51:513
3248729115,62sleep032506-21apt-get07:34:200
1358629016,55sleep10-21swapper/111:42:001
1190729024,51sleep111939-21proc_pri08:09:381
941928922,50sleep21-21systemd08:04:092
941928922,50sleep21-21systemd08:04:092
3143728916,57sleep231524-21latency_hist12:39:102
3140328819,51sleep031440-21http09:14:080
551028617,29sleep10-21swapper/107:49:391
551028617,29sleep10-21swapper/107:49:391
54282868,64sleep25435-21ntp_states07:49:362
54282868,64sleep25435-21ntp_states07:49:362
46322866,42sleep30-21swapper/309:29:423
297852868,64sleep30-21swapper/307:24:343
297852868,64sleep30-21swapper/307:24:343
2545528618,36sleep00-21swapper/008:54:260
2545528618,36sleep00-21swapper/008:54:260
2409528611,53sleep024160-21/usr/sbin/munin12:14:370
216612867,12sleep30-21swapper/312:07:423
1864128615,33sleep00-21swapper/010:14:400
1617528317,50sleep016259-21sed11:50:270
2048128221,42sleep020503-21munin-plugin-st08:39:080
76752816,41sleep30-21swapper/309:39:353
84322805,10sleep30-21swapper/311:24:393
28142795,11sleep30-21swapper/311:08:403
131632797,59sleep30-21swapper/309:58:443
131632797,59sleep30-21swapper/309:58:443
1115527920,32sleep00-21swapper/008:09:080
290427824,24sleep00-21swapper/011:09:100
222822785,39sleep30-21swapper/310:25:333
156692785,10sleep30-21swapper/308:22:033
114727821,27sleep00-21swapper/009:19:250
114727821,27sleep00-21swapper/009:19:250
2933227714,30sleep10-21swapper/109:04:411
2339127713,48sleep223538-21latency_hist12:14:112
2339127713,48sleep223538-21latency_hist12:14:112
2242827716,46sleep222508-21tune2fs08:44:272
2242827716,46sleep222508-21tune2fs08:44:272
3031227610,16sleep00-21swapper/012:34:310
2553927620,34sleep225546-21/usr/sbin/munin12:19:322
2461327617,25sleep20-21swapper/207:09:312
207592769,14sleep10-21swapper/112:04:291
2921727514,27sleep00-21swapper/010:49:270
2921727514,27sleep00-21swapper/010:49:270
2630127518,31sleep20-21swapper/210:39:322
2116927513,49sleep221216-21ntp_kernel_pll_08:39:342
2116927513,49sleep221216-21ntp_kernel_pll_08:39:342
57127417,36sleep2647-21swap10:59:392
2679127416,35sleep126875-21mailstats07:14:391
146427411,49sleep11504-21ntp_kernel_pll_09:19:351
146427411,49sleep11504-21ntp_kernel_pll_09:19:351
1153127426,25sleep00-21swapper/008:09:290
611327312,17sleep00-21swapper/009:34:350
506127323,19sleep00-21swapper/009:29:470
386627311,37sleep24017-21unixbench_singl11:09:402
386627311,37sleep24017-21unixbench_singl11:09:402
2863927321,24sleep20-21swapper/212:29:292
2551527329,25sleep10-21swapper/107:13:011
318327220,38sleep13272-21latency_hist07:44:101
318327220,38sleep13272-21latency_hist07:44:101
2551127216,42sleep225525-21apt-config10:36:542
2551127216,42sleep225525-21apt-config10:36:542
221592728,49sleep00-21swapper/012:09:280
221592728,49sleep00-21swapper/012:09:280
2157727214,45sleep021279-21/usr/sbin/munin10:24:330
177692728,52sleep00-21swapper/010:12:460
1588627223,28sleep00-21swapper/008:24:100
76427127,20sleep00-21swapper/010:59:410
2989627112,44sleep229913-21latency_hist09:09:102
2920627117,34sleep029330-21grep09:04:400
1942727122,28sleep20-21swapper/211:59:372
1788227128,25sleep20-21swapper/210:14:092
1376427118,35sleep013893-21/usr/sbin/munin08:14:410
496227024,25sleep10-21swapper/111:14:321
2848327015,25sleep20-21swapper/207:19:402
1120627016,36sleep211301-21sed08:09:102
2443126918,23sleep00-21swapper/008:49:380
2443126918,23sleep00-21swapper/008:49:380
236452699,45sleep00-21swapper/008:49:090
236452699,45sleep00-21swapper/008:49:090
1502026918,22sleep10-21swapper/108:19:351
1434126917,28sleep10-21swapper/108:19:101
3151826823,18sleep20-21swapper/209:14:102
2101826815,34sleep121028-21cron10:24:091
1971626820,23sleep20-21swapper/208:34:382
1896226821,20sleep10-21swapper/111:59:241
1788026816,24sleep20-21swapper/211:54:412
1024726814,29sleep00-21swapper/009:49:120
648126710,29sleep10-21swapper/111:19:281
44326710,44sleep2481-21/usr/sbin/munin07:34:342
3064626716,38sleep130742-21cut09:09:371
1431226724,24sleep10-21swapper/111:44:311
1021126718,23sleep10-21swapper/111:29:411
522626614,37sleep05297-21sensors_temp11:14:390
377926612,40sleep03780-21sshd09:28:010
377926612,40sleep03780-21sshd09:28:010
258302667,13sleep00-21swapper/012:19:380
1543326621,21sleep00-21swapper/011:49:100
712526518,27sleep00-21swapper/007:54:390
712526518,27sleep00-21swapper/007:54:390
305472655,15sleep00-21swapper/007:27:330
305472655,15sleep00-21swapper/007:27:330
2717326518,32sleep127220-21idleruntime12:24:271
2507426516,29sleep025196-21grep10:34:460
2507426516,29sleep025196-21grep10:34:460
2282326520,30sleep022827-21find10:29:170
2282326520,30sleep022827-21find10:29:170
1639726518,21sleep20-21swapper/208:24:332
1627626523,24sleep20-21swapper/211:50:292
1587026524,25sleep10-21swapper/111:49:511
131112656,11sleep00-21swapper/009:58:040
131112656,11sleep00-21swapper/009:58:040
1231226517,34sleep212341-21nfs4_client09:54:392
1231226517,34sleep212341-21nfs4_client09:54:392
1047126524,26sleep210481-21/usr/sbin/munin09:49:262
820126410,31sleep18248-21ntp_kernel_pll_11:24:331
1380326417,31sleep013851-21latency_hist11:44:090
782326319,28sleep27819-21apt-get11:24:212
776526315,26sleep00-21swapper/009:39:440
2963626314,32sleep229731-21cut10:49:382
2963626314,32sleep229731-21cut10:49:382
285126319,29sleep22870-21munin-run11:09:082
2594726318,20sleep10-21swapper/108:54:391
2594726318,20sleep10-21swapper/108:54:391
2080126324,19sleep10-21swapper/108:39:251
2080126324,19sleep10-21swapper/108:39:251
1804326318,20sleep00-21swapper/011:54:430
1767626317,26sleep117746-21df08:29:241
1619926313,35sleep116101-21/usr/sbin/munin08:24:271
1118826321,26sleep00-21swapper/011:34:290
313252626,12sleep10-21swapper/110:55:021
313252626,12sleep10-21swapper/110:55:021
2805026223,18sleep20-21swapper/210:44:372
2772226222,16sleep00-21swapper/012:24:380
2700426218,22sleep20-21swapper/208:59:212
2553026217,29sleep225428-21/usr/sbin/munin08:54:282
2553026217,29sleep225428-21/usr/sbin/munin08:54:282
2360926218,24sleep323656-21users07:04:403
1557326216,26sleep10-21swapper/110:04:391
1557326216,26sleep10-21swapper/110:04:391
512226125,19sleep00-21swapper/007:49:300
512226125,19sleep00-21swapper/007:49:300
368326116,30sleep23697-21grep07:44:312
368326116,30sleep23697-21grep07:44:312
3143926118,24sleep10-21swapper/107:29:361
3143926118,24sleep10-21swapper/107:29:361
289126113,34sleep22906-21uname09:24:402
289126113,34sleep22906-21uname09:24:402
2738026118,19sleep00-21swapper/010:44:110
2450626123,20sleep10-21swapper/108:49:391
2450626123,20sleep10-21swapper/108:49:391
3018126012,34sleep130186-21diskstats12:34:281
542125918,25sleep15438-21munin-run09:34:081
51225918,22sleep10-21swapper/110:59:381
275292598,11sleep00-21swapper/008:59:390
2264825922,19sleep20-21swapper/210:29:112
2080025917,25sleep020793-21apt-get08:39:240
2080025917,25sleep020793-21apt-get08:39:240
782625814,21sleep00-21swapper/011:24:240
72132585,11sleep10-21swapper/109:39:121
653225816,19sleep20-21swapper/211:19:292
51425815,27sleep10-21swapper/107:34:371
2700025816,25sleep226996-21apt-get12:24:202
233102587,38sleep10-21swapper/112:13:571
233102587,38sleep10-21swapper/112:13:571
2302725819,20sleep20-21swapper/207:04:312
3220025715,25sleep10-21swapper/109:15:141
3094125712,30sleep030959-21apt07:29:190
3094125712,30sleep030959-21apt07:29:190
3009725717,16sleep20-21swapper/207:24:412
3009725717,16sleep20-21swapper/207:24:412
262225714,27sleep10-21swapper/109:24:311
262225714,27sleep10-21swapper/109:24:311
2423525720,18sleep20-21swapper/212:14:392
238212577,15sleep20-21swapper/210:30:242
238212577,15sleep20-21swapper/210:30:242
2335925722,19sleep10-21swapper/107:04:381
208942579,23sleep00-21swapper/012:04:320
2025025715,17sleep00-21swapper/010:19:440
159325715,25sleep11590-21apt-get11:04:221
1923025613,25sleep019289-21latency11:59:310
1113325620,19sleep10-21swapper/109:49:501
2306325515,21sleep10-21swapper/110:29:301
2306325515,21sleep10-21swapper/110:29:301
1013825517,16sleep20-21swapper/209:49:102
942225416,16sleep10-21swapper/109:44:491
478225411,18sleep10-21swapper/107:49:111
2654525414,17sleep10-21swapper/110:39:401
1279325413,21sleep20-21swapper/208:14:112
375325313,18sleep10-21swapper/111:09:391
375325313,18sleep10-21swapper/111:09:391
296252536,37sleep00-21swapper/012:31:490
2760525314,16sleep10-21swapper/108:59:401
251982530,14sleep10-21swapper/110:34:461
251982530,14sleep10-21swapper/110:34:461
708425211,22sleep07176-21users11:19:390
651325212,22sleep26617-21users09:34:452
515325213,20sleep20-21swapper/211:14:372
936625111,16sleep10-21swapper/108:04:081
936625111,16sleep10-21swapper/108:04:081
2516125113,22sleep225185-21munin-run08:54:082
2516125113,22sleep225185-21munin-run08:54:082
2115625114,16sleep20-21swapper/212:04:382
1816225113,22sleep118165-21apt-get10:14:201
944725012,15sleep20-21swapper/211:29:262
290362506,33sleep10-21swapper/107:22:581
2882225017,15sleep10-21swapper/112:29:321
1864425017,16sleep10-21swapper/111:59:081
1862725014,16sleep20-21swapper/210:14:382
130802507,30sleep10-21swapper/109:57:411
130802507,30sleep10-21swapper/109:57:411
728724912,21sleep27281-21apt-get09:39:212
243282498,17sleep00-21swapper/007:09:210
1774724913,21sleep217753-21df_inode08:29:252
1100324911,23sleep10-21swapper/111:34:251
356224814,18sleep00-21swapper/011:09:340
356224814,18sleep00-21swapper/011:09:340
2803124811,16sleep00-21swapper/010:44:350
478324715,18sleep24880-21timerandwakeup09:29:452
1532924714,15sleep20-21swapper/210:04:322
1532924714,15sleep20-21swapper/210:04:322
2968624617,16sleep10-21swapper/107:24:321
2968624617,16sleep10-21swapper/107:24:321
90724512,20sleep2959-21latency_hist09:19:112
696924511,14sleep10-21swapper/109:39:081
349524511,16sleep10-21swapper/107:44:261
828424310,13sleep00-21swapper/007:59:310
828424310,13sleep00-21swapper/007:59:310
3138024111,13sleep20-21swapper/207:29:332
3138024111,13sleep20-21swapper/207:29:332
1324024112,15sleep20-21swapper/208:14:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional