You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 16:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Thu Jun 04, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48092300411,2975sleep14791-21apt-get08:53:301
7714282114,792sleep17726-21/usr/sbin/munin12:28:321
1361526686,648sleep112086-21kworker/1:009:19:381
1361526686,648sleep112086-21kworker/1:009:19:381
4211240053,289sleep00-21swapper/010:33:380
4338238225,345sleep04364-21grep12:18:290
28861236534,210sleep029059-21gpgv08:28:140
28861236534,210sleep029059-21gpgv08:28:140
30490229617,187sleep0225-21systemd-journal08:33:100
32454228416,247sleep232557-21fschecks_time08:38:292
10819227215,237sleep210784-21latency_hist09:13:132
18240226317,226sleep318277-21ps11:18:383
4354226213,206sleep04463-21apt-config08:53:130
4354226213,206sleep04463-21apt-config08:53:130
10109225072,89sleep010262-21latency_hist07:28:140
26260223910,216sleep026265-21bash09:59:100
26260223910,216sleep026265-21bash09:59:100
7963221732,140sleep08066-21timerandwakeup12:28:390
277912213177,22sleep327794-21sshd10:03:533
277912213177,22sleep327794-21sshd10:03:533
39822129,188sleep0467-21latency_hist10:23:120
12441221047,123sleep012539-21switchtime07:33:390
12441221047,123sleep012539-21switchtime07:33:390
9172220711,55sleep09233-21latency12:33:350
32042220517,39sleep032056-21modprobe08:38:110
172321978,176sleep01793-21cat08:43:320
32505219316,96sleep00-21swapper/012:03:400
19088219116,160sleep219208-21latency_hist11:23:132
17277218913,157sleep017252-21/usr/sbin/munin09:33:260
17277218913,157sleep017252-21/usr/sbin/munin09:33:260
2158521878,85sleep021621-21ntpq08:03:350
2158521878,85sleep021621-21ntpq08:03:350
12002218720,23sleep00-21swapper/010:58:360
25633217811,112sleep025695-21fschecks_time11:43:290
4763217714,65sleep04818-21latency_hist07:13:120
22628217418,138sleep00-21swapper/009:48:360
26957217372,86sleep027049-21perl10:03:300
26957217372,86sleep027049-21perl10:03:300
2522321738,100sleep0363-21snmpd11:42:410
1239221689,102sleep00-21swapper/009:18:120
1239221689,102sleep00-21swapper/009:18:120
2785321659,143sleep027916-21mailstats11:49:030
15497216439,25sleep00-21swapper/007:43:450
2982621599,138sleep1363-21snmpd08:29:241
1301215536,24sleep01361-21if_eth012:08:320
17984215413,39sleep00-21swapper/011:18:310
30269215315,123sleep230412-21cat11:58:132
808321490,42sleep28087-21fschecks_time09:03:312
808321490,42sleep28087-21fschecks_time09:03:312
5792214647,22sleep00-21swapper/007:13:370
5792214647,22sleep00-21swapper/007:13:370
14752214399,22sleep00-21swapper/007:43:120
14752214399,22sleep00-21swapper/007:43:120
6145214123,95sleep36296-21head07:13:443
6145214123,95sleep36296-21head07:13:443
22394214115,98sleep00-21swapper/011:33:200
3066821388,112sleep00-21swapper/011:58:310
5976213519,25sleep00-21swapper/008:57:040
10181213423,89sleep310325-21unixbench_multi09:08:423
20141213110,105sleep020174-21unin-run09:43:110
1548921309,16sleep00-21swapper/009:28:130
1548921309,16sleep00-21swapper/009:28:130
274021289,106sleep02760-21grep12:13:290
274021289,106sleep02760-21grep12:13:290
2275212690,21sleep00-21swapper/010:28:290
6746212517,92sleep36770-21netstat08:58:333
6746212517,92sleep36770-21netstat08:58:333
4134212418,79sleep30-21swapper/307:08:433
15205212310,98sleep00-21swapper/011:08:420
23732211967,17sleep00-21swapper/009:53:250
23732211967,17sleep00-21swapper/009:53:250
26628211727,24sleep30-21swapper/310:03:133
26628211727,24sleep30-21swapper/310:03:133
343211614,86sleep3396-21unin-run10:23:113
2439121167,14sleep00-21swapper/008:13:280
1641821149,59sleep00-21swapper/009:28:420
1641821149,59sleep00-21swapper/009:28:420
919421138,16sleep00-21swapper/009:08:110
919421138,16sleep00-21swapper/009:08:110
22649211112,81sleep222690-21sshd09:48:382
14920211123,63sleep315057-21head09:23:443
14920211123,63sleep315057-21head09:23:443
10323211110,18sleep00-21swapper/007:28:260
2124321098,64sleep00-21swapper/011:28:360
1821321098,20sleep018297-21perl07:53:340
1821321098,20sleep018297-21perl07:53:340
25995210812,77sleep226103-21perl08:18:312
1263121088,15sleep00-21swapper/009:18:240
1263121088,15sleep00-21swapper/009:18:240
570321077,19sleep05743-21ntpq10:38:340
2720921068,62sleep027307-21date08:23:130
5240210520,62sleep34773-21/usr/sbin/munin08:54:253
2969021057,86sleep0284-21dbus-daemon10:13:110
24714210511,77sleep224854-21unixbench_singl11:38:432
2314221047,14sleep00-21swapper/008:08:360
985621039,76sleep29947-21apt-config07:28:122
2064721037,14sleep00-21swapper/009:43:310
848921027,13sleep00-21swapper/010:48:270
2986421026,10sleep00-21swapper/011:54:360
1974121027,14sleep00-21swapper/007:58:300
1619021027,13sleep00-21swapper/011:13:220
1619021027,13sleep00-21swapper/011:13:220
2646121019,14sleep30-21swapper/311:43:453
20605210117,67sleep220120-21apt-get09:43:212
13953210017,57sleep10-21swapper/107:38:411
13953210017,57sleep10-21swapper/107:38:411
76772999,49sleep00-21swapper/010:43:430
312529941,32sleep33155-21munin-plugin-st07:08:103
2063529931,50sleep00-21swapper/011:28:110
169922997,13sleep00-21swapper/007:48:370
1118729945,19sleep00-21swapper/010:57:400
1118729945,19sleep00-21swapper/010:57:400
1029029919,61sleep39808-21apt-get07:28:213
1876729814,16sleep30-21swapper/307:53:453
1876729814,16sleep30-21swapper/307:53:453
57992978,50sleep00-21swapper/012:23:220
2249329714,24sleep222501-21diskstats11:33:262
205129634,30sleep00-21swapper/007:03:320
144312968,52sleep00-21swapper/011:08:130
300342956,13sleep00-21swapper/010:13:270
2350029419,57sleep323639-21unixbench_singl08:08:433
185429422,52sleep32006-21sed12:08:433
994829227,32sleep30-21swapper/307:28:133
63782927,33sleep00-21swapper/008:58:220
63782927,33sleep00-21swapper/008:58:220
105262926,11sleep00-21swapper/009:10:130
80882915,12sleep00-21swapper/007:21:120
80882915,12sleep00-21swapper/007:21:120
1054329112,59sleep110615-21wc10:53:411
1054329112,59sleep110615-21wc10:53:411
2768129010,22sleep30-21swapper/308:23:323
2768129010,22sleep30-21swapper/308:23:323
1703529012,64sleep217094-21latency_hist09:33:122
1703529012,64sleep217094-21latency_hist09:33:122
1102229020,51sleep10-21swapper/107:28:561
8162896,30sleep00-21swapper/008:38:430
2407428928,29sleep10-21swapper/108:13:131
157082898,13sleep30-21swapper/311:10:383
15622896,31sleep00-21swapper/010:23:490
2302528814,60sleep323125-21timerandwakeup11:33:393
2072628813,57sleep120733-21diskstats09:43:371
119628818,46sleep21212-21fschecks_count12:08:302
435228711,20sleep24426-21find10:33:402
829528621,41sleep38262-21latency_hist07:23:123
829528621,41sleep38262-21latency_hist07:23:123
3022928626,40sleep130293-21cat10:13:321
666728520,38sleep26693-21awk07:18:112
666728520,38sleep26693-21awk07:18:112
2445728419,47sleep224519-21/usr/sbin/munin08:13:302
1619428413,54sleep116261-21awk09:28:391
1619428413,54sleep116261-21awk09:28:391
97582827,10sleep10-21swapper/107:27:471
3087928216,45sleep230985-21tune2fs08:33:332
2016628125,33sleep30-21swapper/307:58:423
606727924,33sleep36218-21head10:38:413
2032827923,27sleep20-21swapper/209:43:132
1765327823,33sleep10-21swapper/109:33:351
1765327823,33sleep10-21swapper/109:33:351
162772789,55sleep216334-21grep11:13:282
162772789,55sleep216334-21grep11:13:282
839927712,46sleep18413-21/usr/sbin/munin09:03:391
839927712,46sleep18413-21/usr/sbin/munin09:03:391
2060727719,34sleep320626-21apt-get09:43:223
1633227719,19sleep30-21swapper/309:28:413
1633227719,19sleep30-21swapper/309:28:413
462127618,39sleep14227-21/usr/sbin/munin12:18:341
347627614,46sleep33513-21ps08:48:403
347627614,46sleep33513-21ps08:48:403
2411027616,42sleep324152-21sed09:53:363
2411027616,42sleep324152-21sed09:53:363
1893527622,34sleep318966-21fschecks_count09:38:303
1440227627,30sleep30-21swapper/311:08:123
1195727623,27sleep20-21swapper/207:33:292
1195727623,27sleep20-21swapper/207:33:292
739727516,43sleep27424-21df12:28:252
312282758,16sleep30-21swapper/310:18:113
311792757,46sleep30-21swapper/308:33:383
1281327520,30sleep20-21swapper/209:18:302
1281327520,30sleep20-21swapper/209:18:302
1122327535,27sleep211017-21/usr/sbin/munin09:13:302
1122327535,27sleep211017-21/usr/sbin/munin09:13:302
990127411,12sleep10-21swapper/109:08:361
3004627420,28sleep20-21swapper/210:13:292
2640027415,24sleep30-21swapper/308:18:403
2545327413,28sleep10-21swapper/109:58:301
2545327413,28sleep10-21swapper/109:58:301
663327218,23sleep30-21swapper/312:23:413
2967227215,41sleep329686-21idleruntime-cro10:13:103
232127221,29sleep20-21swapper/207:03:382
1204927220,23sleep30-21swapper/307:33:303
1204927220,23sleep30-21swapper/307:33:303
465027117,33sleep20-21swapper/212:18:352
316527118,37sleep33260-21grep12:13:403
316527118,37sleep33260-21grep12:13:403
205082717,53sleep10-21swapper/111:27:121
63727018,26sleep20-21swapper/210:23:262
427027024,29sleep24279-21modprobe08:53:112
427027024,29sleep24279-21modprobe08:53:112
3172827018,22sleep30-21swapper/312:03:103
293662708,12sleep10-21swapper/110:09:121
842126918,35sleep38486-21cpuspeed10:48:263
442926916,31sleep30-21swapper/310:33:413
3096526915,39sleep131038-21sendmail_mailtr11:58:401
265626913,43sleep12722-21ps10:28:381
153926924,27sleep10-21swapper/108:43:291
1528526924,22sleep10-21swapper/111:08:431
3226926820,29sleep132405-21grep10:18:411
3144426820,23sleep10-21swapper/108:33:441
222352686,9sleep20-21swapper/208:05:062
222352686,9sleep20-21swapper/208:05:062
126012685,11sleep20-21swapper/211:00:282
890426710,12sleep20-21swapper/212:33:292
54232676,51sleep30-21swapper/312:22:003
2569626717,29sleep125848-21latency_hist08:18:131
1342126716,34sleep313423-21/usr/sbin/munin07:38:233
1342126716,34sleep313423-21/usr/sbin/munin07:38:233
482026627,23sleep10-21swapper/107:13:131
327426620,27sleep10-21swapper/108:48:331
327426620,27sleep10-21swapper/108:48:331
302832666,10sleep20-21swapper/208:30:302
272426611,38sleep22793-21sensors_temp10:28:392
2569926618,17sleep10-21swapper/111:43:301
25492669,48sleep30-21swapper/308:46:053
247226622,20sleep10-21swapper/107:03:411
2383026611,42sleep223934-21grep09:53:312
2383026611,42sleep223934-21grep09:53:312
2130026610,15sleep30-21swapper/308:03:303
2130026610,15sleep30-21swapper/308:03:303
1162826619,21sleep20-21swapper/207:33:132
818426513,24sleep20-21swapper/210:48:132
732726518,27sleep30-21swapper/310:43:383
426026516,34sleep14351-21mailstats10:33:391
283982659,42sleep20-21swapper/211:50:382
2817726519,30sleep228302-21grep10:08:122
2817726519,30sleep228302-21grep10:08:122
2598226516,23sleep10-21swapper/108:18:301
1859426517,19sleep10-21swapper/107:53:431
1859426517,19sleep10-21swapper/107:53:431
1175126521,18sleep10-21swapper/109:13:411
1175126521,18sleep10-21swapper/109:13:411
64642647,12sleep20-21swapper/208:58:282
64642647,12sleep20-21swapper/208:58:282
3091326416,22sleep20-21swapper/211:58:372
2461726320,21sleep10-21swapper/111:38:421
245826323,22sleep30-21swapper/310:28:323
2352926319,17sleep10-21swapper/109:53:131
1511226318,20sleep20-21swapper/211:08:402
1444726314,19sleep10-21swapper/109:23:341
1444726314,19sleep10-21swapper/109:23:341
1349426321,24sleep213532-21ntpq11:03:352
824126220,27sleep18268-21munin-run07:23:101
824126220,27sleep18268-21munin-run07:23:101
648226220,17sleep20-21swapper/212:23:402
559626217,20sleep20-21swapper/210:38:322
2870526221,22sleep20-21swapper/208:28:122
2870526221,22sleep20-21swapper/208:28:122
250382626,47sleep30-21swapper/311:40:183
2097826220,19sleep20-21swapper/208:03:132
574526113,32sleep15777-21ntp_states10:38:351
334626116,31sleep13479-21sed12:13:421
334626116,31sleep13479-21sed12:13:421
1972226116,28sleep30-21swapper/311:23:343
1816726123,20sleep20-21swapper/207:53:322
1816726123,20sleep20-21swapper/207:53:322
162222615,48sleep30-21swapper/307:47:513
665026011,36sleep1363-21snmpd07:17:581
665026011,36sleep1363-21snmpd07:17:581
2175126016,29sleep121822-21sensors_temp08:03:401
2175126016,29sleep121822-21sensors_temp08:03:401
174562607,45sleep30-21swapper/307:48:483
1709626024,18sleep30-21swapper/309:33:133
1709626024,18sleep30-21swapper/309:33:133
1417926017,28sleep214171-21apt-get09:23:272
1417926017,28sleep214171-21apt-get09:23:272
580025914,28sleep15794-21apt-get12:23:231
2763725911,32sleep127668-21latency11:48:551
2418725914,30sleep124193-21ntp_states09:53:381
2418725914,30sleep124193-21ntp_states09:53:381
1963625919,23sleep10-21swapper/107:58:231
100152595,12sleep30-21swapper/312:36:213
637925815,25sleep16374-21apt-get08:58:231
637925815,25sleep16374-21apt-get08:58:231
2303625814,26sleep223128-21load08:08:342
3247825715,24sleep332503-21/usr/sbin/munin12:03:393
30532577,11sleep20-21swapper/212:13:352
30532577,11sleep20-21swapper/212:13:352
2344425719,21sleep30-21swapper/309:53:113
1822125714,24sleep118232-21ntp_states11:18:361
1371325719,18sleep20-21swapper/207:38:342
1371325719,18sleep20-21swapper/207:38:342
1153725716,25sleep111534-21apt-get10:58:211
1042225713,18sleep20-21swapper/210:53:382
1042225713,18sleep20-21swapper/210:53:382
738025620,16sleep10-21swapper/110:43:401
3209525620,19sleep10-21swapper/112:03:291
1779225612,26sleep117819-21munin-run07:53:101
1567225616,24sleep215793-21cat07:43:472
125825615,16sleep10-21swapper/110:23:421
11272568,12sleep30-21swapper/310:23:403
834525512,25sleep38387-21ntp_states09:03:373
834525512,25sleep38387-21ntp_states09:03:373
1743925518,18sleep30-21swapper/311:17:533
1743925518,18sleep30-21swapper/311:17:533
145525515,20sleep20-21swapper/208:43:262
3265825415,20sleep20-21swapper/212:03:422
2742125416,19sleep10-21swapper/108:23:221
2742125416,19sleep10-21swapper/108:23:221
1507725414,24sleep115081-21diskstats07:43:341
533225317,22sleep25339-21awk08:54:282
2730525315,23sleep326820-21apt-get11:48:213
253032535,38sleep30-21swapper/308:14:443
1861425314,16sleep20-21swapper/209:38:122
1861425314,16sleep20-21swapper/209:38:122
111125312,24sleep11105-21apt-get12:08:251
2920125213,24sleep328721-21apt-get08:28:213
2686325214,21sleep20-21swapper/211:48:122
1932525213,23sleep219392-21df11:23:252
1305025215,22sleep313082-21ntp_states09:18:353
1305025215,22sleep313082-21ntp_states09:18:353
1023825216,19sleep10-21swapper/112:38:121
3235825111,24sleep132352-21apt-get08:38:241
3022125119,19sleep130162-21latency_hist11:58:121
212032518,21sleep30-21swapper/311:28:353
1183925112,23sleep111858-21apt-get07:33:201
1183925112,23sleep111858-21apt-get07:33:201
90282508,25sleep29120-21timerandwakeup10:48:412
2915325018,19sleep30-21swapper/311:53:343
2129225011,23sleep221328-21ntp_states11:28:382
1679925010,23sleep116890-21sed11:13:391
1679925010,23sleep116890-21sed11:13:391
126325015,14sleep30-21swapper/308:43:143
1214725014,16sleep30-21swapper/310:58:403
973124914,17sleep10-21swapper/110:53:131
372124911,24sleep23755-21wc07:08:342
3154224911,22sleep231539-21apt-get10:18:222
2899024916,18sleep20-21swapper/210:08:412
1127824911,21sleep311308-21irqstats09:13:313
1127824911,21sleep311308-21irqstats09:13:313
2705024814,21sleep127099-21idleruntime10:03:311
2705024814,21sleep127099-21idleruntime10:03:311
2173424810,21sleep121781-21users11:28:451
1924724819,15sleep10-21swapper/109:38:361
99692459,18sleep310036-21df10:53:273
99692459,18sleep310036-21df10:53:273
819924512,18sleep20-21swapper/207:22:392
819924512,18sleep20-21swapper/207:22:392
2552124311,16sleep20-21swapper/209:58:312
2552124311,16sleep20-21swapper/209:58:312
2260224313,18sleep122395-21/usr/sbin/munin11:33:291
74252429,18sleep37432-21diskstats12:28:263
1693424215,14sleep20-21swapper/207:48:352
130732426,13sleep10-21swapper/111:03:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional