You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 04:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Aug 25, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23132275491,640sleep223159-21fschecks_count19:19:342
1371026948,672sleep213751-21ntpq22:14:372
1371026948,672sleep213751-21ntpq22:14:372
24233246912,441sleep024234-21cron22:49:110
24233246912,441sleep024234-21cron22:49:110
24209245515,425sleep324217-21ntp_states21:04:393
21827244735,398sleep221865-21ntp_kernel_err19:14:372
1866523586,341sleep2367-21snmpd00:21:292
1866523586,341sleep2367-21snmpd00:21:292
18776231516,187sleep218802-21idleruntime-cro20:49:112
856522888,264sleep08601-21ntpq20:14:380
25488225488,59sleep20-21swapper/219:24:462
258352252101,135sleep225808-21latency_hist22:54:132
19348223213,204sleep119440-21load20:49:351
17298223222,143sleep20-21swapper/220:44:152
18995222821,184sleep219158-21unixbench_multi22:29:552
3554222784,65sleep23697-21apt-config21:44:132
17784221115,181sleep117800-21kernelversion20:44:341
15746221142,109sleep215898-21latency_hist20:39:142
20357220610,180sleep220446-21mailstats22:34:402
29705220515,172sleep229729-21modprobe21:24:112
29705220515,172sleep229729-21modprobe21:24:112
21933218613,158sleep222009-21awk22:39:402
21933218613,158sleep222009-21awk22:39:402
2109821789,154sleep221158-21find20:54:412
15634217716,139sleep015783-21grep22:19:440
15634217716,139sleep015783-21grep22:19:440
175021709,137sleep20-21swapper/223:19:282
175021709,137sleep20-21swapper/223:19:282
1648721578,134sleep316506-21apt-get22:24:233
21351215512,120sleep121491-21grep20:54:441
1968214918,115sleep20-21swapper/221:39:132
10218214864,69sleep210265-21processes20:19:412
8127214451,77sleep20-21swapper/223:39:352
30462214216,108sleep230465-21cron23:09:112
25922214215,23sleep20-21swapper/221:09:422
2675213620,98sleep32793-21grep19:55:443
14592213561,26sleep214730-21unixbench_singl00:04:462
5916213240,12sleep20-21swapper/221:49:342
30129213138,15sleep20-21swapper/219:39:442
2458121317,14sleep00-21swapper/021:04:440
1634721315,17sleep20-21swapper/200:10:112
32361212918,94sleep032501-21sed19:49:150
32361212918,94sleep032501-21sed19:49:150
10516212971,29sleep00-21swapper/023:54:140
21334212734,22sleep220941-21/usr/sbin/munin00:29:372
27412212642,65sleep20-21swapper/222:59:142
2297421267,45sleep20-21swapper/222:44:252
2297421267,45sleep20-21swapper/222:44:252
1975821250,8sleep30-21swapper/300:25:003
1678212524,78sleep00-21swapper/023:19:260
1678212524,78sleep00-21swapper/023:19:260
23896212424,78sleep224050-21latency_hist00:39:142
19069212435,76sleep20-21swapper/220:49:282
26348212211,27sleep20-21swapper/219:29:232
10851212229,16sleep20-21swapper/222:04:442
1926212110,30sleep21929-21apt-get19:54:262
31887212016,21sleep20-21swapper/223:09:482
30260212027,16sleep20-21swapper/221:24:322
30260212027,16sleep20-21swapper/221:24:322
22725212022,76sleep122780-21sendmail-msp19:19:121
7986211511,89sleep18006-21users21:54:461
22688211524,75sleep122695-21modprobe22:44:111
22688211524,75sleep122695-21modprobe22:44:111
9837211418,73sleep19850-21fschecks_count20:19:311
19719211243,25sleep20-21swapper/200:25:002
1095211212,24sleep01101-21ntp_states21:34:390
418021117,91sleep24238-21sort21:44:422
2669821118,17sleep30-21swapper/319:30:123
19385211016,76sleep118891-21apt-get00:24:231
1864521096,91sleep00-21swapper/020:47:280
14157210811,19sleep20-21swapper/220:34:142
1801521070,16sleep112-21rcu_sched19:04:311
31511210626,61sleep231551-21proc_pri19:44:412
31511210626,61sleep231551-21proc_pri19:44:412
20892210510,79sleep00-21swapper/020:54:340
9506210415,72sleep39542-21cut20:19:133
54421031,90sleep20-21swapper/219:49:372
20940210324,28sleep10-21swapper/100:29:231
12290210322,47sleep312287-21apt-get23:59:223
843821027,22sleep20-21swapper/220:14:362
1332321016,20sleep20-21swapper/200:01:062
893521009,26sleep0367-21snmpd23:41:310
27473210011,72sleep20-21swapper/221:14:472
27473210011,72sleep20-21swapper/221:14:472
1596621007,80sleep215981-21apt-config22:21:322
1596621007,80sleep215981-21apt-config22:21:322
1985429913,67sleep00-21swapper/022:34:250
64252988,19sleep26525-21fschecks_time23:34:422
2781529817,61sleep127874-21idleruntime22:59:331
2076629811,17sleep21-21systemd19:14:132
1961629818,58sleep119741-21latency_hist22:34:141
1108729810,14sleep20-21swapper/223:56:182
2423729616,67sleep124284-21unin-run22:49:121
2423729616,67sleep124284-21unin-run22:49:121
2301929610,71sleep023036-21apt19:19:240
175822965,29sleep20-21swapper/222:26:482
515929511,67sleep05219-21fschecks_time20:04:330
515929511,67sleep05219-21fschecks_time20:04:330
382429513,63sleep20-21swapper/223:24:412
382429513,63sleep20-21swapper/223:24:412
708029419,59sleep37118-21grep20:09:413
708029419,59sleep37118-21grep20:09:413
66202949,69sleep20-21swapper/220:09:272
66202949,69sleep20-21swapper/220:09:272
3129529414,66sleep331326-21latency_hist21:29:133
3129529414,66sleep331326-21latency_hist21:29:133
271692949,67sleep027312-21unixbench_singl19:30:220
248522946,10sleep20-21swapper/221:07:082
3268329211,67sleep032695-21fschecks_count19:49:310
222002927,13sleep20-21swapper/220:59:292
180282927,11sleep30-21swapper/300:20:373
180282927,11sleep30-21swapper/300:20:373
117812927,12sleep20-21swapper/222:09:242
280692917,49sleep20-21swapper/222:59:382
12312915,52sleep20-21swapper/223:17:312
12312915,52sleep20-21swapper/223:17:312
117742917,15sleep30-21swapper/323:56:433
324732907,49sleep20-21swapper/221:30:012
1764029011,65sleep317725-21fschecks_time20:44:323
75332898,65sleep20-21swapper/221:54:392
73728914,59sleep1837-21grep21:34:321
504328959,14sleep20-21swapper/223:29:342
2271528913,61sleep022765-21if_eth000:34:310
602428822,50sleep36031-21modprobe23:34:113
295722889,61sleep029579-21df_inode19:39:300
843728713,57sleep0231-21systemd-journal21:59:130
50712878,41sleep20-21swapper/220:04:272
50712878,41sleep20-21swapper/220:04:272
3227128720,50sleep032417-21grep21:29:460
1790128714,34sleep10-21swapper/122:29:141
1540228713,60sleep00-21swapper/000:09:280
2975628617,56sleep029839-21latency_hist21:24:130
2975628617,56sleep029839-21latency_hist21:24:130
2228428614,56sleep10-21swapper/120:59:321
1177428612,17sleep20-21swapper/220:24:412
1177428612,17sleep20-21swapper/220:24:412
98502855,11sleep20-21swapper/222:02:582
96012857,10sleep20-21swapper/223:46:492
96012857,10sleep20-21swapper/223:46:492
46642856,12sleep20-21swapper/220:02:412
46642856,12sleep20-21swapper/220:02:412
322228520,28sleep10-21swapper/123:24:241
322228520,28sleep10-21swapper/123:24:241
289252845,12sleep20-21swapper/219:35:532
170892845,11sleep20-21swapper/200:16:292
585028214,43sleep15972-21/usr/sbin/munin20:04:461
585028214,43sleep15972-21/usr/sbin/munin20:04:461
100352826,10sleep20-21swapper/223:49:162
2930128113,18sleep10-21swapper/119:39:151
236028120,38sleep12498-21grep23:19:441
236028120,38sleep12498-21grep23:19:441
1513528118,41sleep115282-21unixbench_singl20:34:451
1137228116,50sleep311379-21df_inode20:24:303
1137228116,50sleep311379-21df_inode20:24:303
837028016,44sleep17866-21/usr/sbin/munin23:39:421
1342028019,44sleep113515-21perl22:14:321
1342028019,44sleep113515-21perl22:14:321
1304028031,28sleep10-21swapper/122:14:121
1224327923,25sleep00-21swapper/022:09:390
868127825,29sleep30-21swapper/321:59:283
2280527812,52sleep322913-21sed19:19:143
47682770,7sleep10-21swapper/121:44:521
2948927712,15sleep30-21swapper/323:04:363
1846227729,25sleep20-21swapper/219:04:412
1447027715,48sleep114591-21cat00:04:451
564127611,18sleep30-21swapper/320:04:443
564127611,18sleep30-21swapper/320:04:443
982427513,48sleep39834-21systemctl23:49:123
982427513,48sleep39834-21systemctl23:49:123
757227524,31sleep37597-21munin-run23:39:113
3260027519,42sleep132650-21cpuspeed19:49:281
218527523,20sleep30-21swapper/323:19:423
218527523,20sleep30-21swapper/323:19:423
1878127521,24sleep30-21swapper/322:29:523
135927520,36sleep11378-21idleruntime-cro23:19:111
135927520,36sleep11378-21idleruntime-cro23:19:111
1198027512,48sleep112131-21unixbench_multi20:24:441
1198027512,48sleep112131-21unixbench_multi20:24:441
771327418,39sleep37822-21ls21:54:443
1669027423,32sleep30-21swapper/320:39:443
347627321,33sleep00-21swapper/021:44:110
330827314,26sleep30-21swapper/323:24:303
330827314,26sleep30-21swapper/323:24:303
3113527323,29sleep131151-21sort19:44:311
3113527323,29sleep131151-21sort19:44:311
174822736,16sleep00-21swapper/022:25:290
3049027219,32sleep130569-21awk21:24:391
3049027219,32sleep130569-21awk21:24:391
682727115,29sleep00-21swapper/020:09:340
682727115,29sleep00-21swapper/020:09:340
2593627122,32sleep126014-21munin-run19:29:131
969627010,48sleep00-21swapper/023:48:000
969627010,48sleep00-21swapper/023:48:000
2584827015,37sleep325850-21cron19:29:113
1345527020,25sleep00-21swapper/020:29:440
1152727019,31sleep011672-21sed23:56:280
3154626915,33sleep331547-21cpu21:29:293
3144826925,20sleep30-21swapper/319:44:383
3144826925,20sleep30-21swapper/319:44:383
3132826914,23sleep00-21swapper/021:29:140
3132826914,23sleep00-21swapper/021:29:140
1895726917,32sleep019093-21apt-config00:24:140
1895726917,32sleep019093-21apt-config00:24:140
3126826814,29sleep00-21swapper/023:09:380
1837826824,24sleep00-21swapper/019:04:390
1078226815,22sleep10-21swapper/122:04:431
702026719,27sleep10-21swapper/121:54:181
51932675,49sleep00-21swapper/021:48:190
2088726719,28sleep020934-21munin-node19:14:150
2069426715,36sleep320703-21/usr/sbin/munin20:54:293
1985526615,33sleep119905-21cpuspeed22:34:261
886326520,23sleep00-21swapper/021:59:330
693426526,24sleep10-21swapper/123:34:541
3242026514,20sleep10-21swapper/121:29:471
3111826519,28sleep10-21swapper/123:09:341
2246726523,21sleep00-21swapper/020:59:350
2011326519,27sleep30-21swapper/322:34:323
1330826514,21sleep10-21swapper/120:29:411
1305826510,36sleep30-21swapper/320:29:353
3263226424,19sleep00-21swapper/023:14:340
3263226424,19sleep00-21swapper/023:14:340
2585326419,29sleep025935-21cmp19:29:120
2059326412,42sleep00-21swapper/000:28:090
1019826415,26sleep00-21swapper/020:19:390
2679826316,21sleep10-21swapper/119:30:141
1855526319,26sleep318687-21/usr/sbin/munin19:04:423
1156826313,34sleep011627-21/usr/sbin/munin20:24:350
1156826313,34sleep011627-21/usr/sbin/munin20:24:350
451226211,39sleep0367-21snmpd20:00:460
451226211,39sleep0367-21snmpd20:00:460
2774126218,20sleep00-21swapper/022:59:320
2587926215,23sleep30-21swapper/322:54:143
2038326218,21sleep00-21swapper/020:54:140
1665426222,24sleep316679-21munin-run00:14:113
1627126219,21sleep10-21swapper/122:24:131
1627126219,21sleep10-21swapper/122:24:131
1121526219,24sleep10-21swapper/123:56:211
1113226217,29sleep011146-21vmstat22:04:460
397426117,24sleep30-21swapper/321:44:233
3154126117,26sleep331609-21/usr/sbin/munin23:09:453
220226119,21sleep10-21swapper/119:55:341
2499426014,30sleep025058-21ps22:49:390
2351626012,23sleep30-21swapper/321:04:143
2150926016,27sleep321515-21diskstats22:39:293
2150926016,27sleep321515-21diskstats22:39:293
351125914,28sleep13543-21munin-run21:44:121
298692597,11sleep30-21swapper/319:39:373
2842625912,29sleep328444-21/usr/sbin/munin21:19:243
2842625912,29sleep328444-21/usr/sbin/munin21:19:243
2792325922,16sleep30-21swapper/322:59:353
2385425916,20sleep10-21swapper/100:39:131
238325915,25sleep00-21swapper/019:55:370
2140625910,36sleep021411-21find22:39:220
2140625910,36sleep021411-21find22:39:220
1205325919,18sleep30-21swapper/322:09:343
2435225810,33sleep324489-21latency_hist19:24:153
212352588,12sleep00-21swapper/000:29:350
1670625817,17sleep10-21swapper/100:14:131
1105825815,17sleep30-21swapper/320:24:143
73225715,17sleep30-21swapper/323:14:453
73225715,17sleep30-21swapper/323:14:453
71725716,25sleep3723-21diskstats21:34:293
581525715,21sleep00-21swapper/021:49:320
3061325718,20sleep10-21swapper/123:09:141
2771725714,18sleep00-21swapper/021:14:490
2771725714,18sleep00-21swapper/021:14:490
2634425717,17sleep30-21swapper/322:54:353
2328925714,25sleep323407-21users00:34:423
3041525610,34sleep129478-21/usr/sbin/munin19:39:471
2869525618,18sleep30-21swapper/319:34:453
1552325614,27sleep315628-21wc22:19:433
1552325614,27sleep315628-21wc22:19:433
1439825614,26sleep014392-21apt-get20:34:280
1360525618,21sleep00-21swapper/022:14:350
1360525618,21sleep00-21swapper/022:14:350
988125513,26sleep110034-21latency_hist23:49:151
561825512,28sleep35625-21df_inode21:49:273
525725516,21sleep00-21swapper/023:29:400
2531425514,25sleep125309-21apt-get21:09:251
2382325513,26sleep123830-21df_inode21:04:281
161525512,20sleep30-21swapper/319:54:143
112192554,12sleep30-21swapper/322:05:423
81322545,15sleep10-21swapper/120:14:191
477925412,16sleep30-21swapper/320:04:123
477925412,16sleep30-21swapper/320:04:123
391025413,26sleep13947-21sed19:59:381
391025413,26sleep13947-21sed19:59:381
2695425421,16sleep30-21swapper/321:14:363
2695425421,16sleep30-21swapper/321:14:363
2616225415,18sleep00-21swapper/022:54:320
2303725411,18sleep10-21swapper/119:19:251
2026425413,25sleep320265-21sshd20:53:203
122672549,23sleep10-21swapper/123:59:201
94972538,33sleep10-21swapper/123:45:311
334125313,25sleep03431-21perl23:24:310
334125313,25sleep03431-21perl23:24:310
2852225312,25sleep128531-21df_inode21:19:301
2852225312,25sleep128531-21df_inode21:19:301
2540125320,16sleep30-21swapper/321:09:313
2076225318,19sleep120763-21cron19:14:121
2644425214,24sleep126489-21awk22:54:371
2081325213,22sleep320882-21latency_hist19:14:143
1769625213,23sleep117700-21diskstats00:19:261
1769625213,23sleep117700-21diskstats00:19:261
1597725212,24sleep016046-21df20:39:290
1393225218,18sleep00-21swapper/000:04:320
891725114,21sleep18950-21irqstats21:59:341
674525111,22sleep06782-21ntp_states23:34:480
34382515,13sleep30-21swapper/321:43:423
192542516,35sleep00-21swapper/022:30:180
1341025114,21sleep30-21swapper/322:14:303
1341025114,21sleep30-21swapper/322:14:303
101372517,32sleep10-21swapper/123:50:321
2792425012,22sleep027920-21apt-get19:34:250
2096325011,19sleep30-21swapper/300:29:253
1047425016,17sleep30-21swapper/323:54:133
532324913,18sleep10-21swapper/123:29:411
1236624912,23sleep012373-21df_inode23:59:270
816324815,17sleep30-21swapper/323:39:363
3052924816,17sleep00-21swapper/023:09:130
669724712,20sleep16703-21diskstats20:09:301
669724712,20sleep16703-21diskstats20:09:301
230522477,14sleep30-21swapper/322:44:303
230522477,14sleep30-21swapper/322:44:303
1385224710,20sleep313849-21apt-get00:04:273
2657524611,20sleep026600-21munin-run21:14:120
1996024611,20sleep019966-21ntp_states19:09:390
1738524513,16sleep10-21swapper/100:19:141
171542455,31sleep10-21swapper/120:43:131
2820324212,16sleep10-21swapper/121:19:141
2820324212,16sleep10-21swapper/121:19:141
245242418,16sleep10-21swapper/122:49:241
1737324113,16sleep00-21swapper/000:19:130
173032407,15sleep30-21swapper/300:19:123
146382408,15sleep30-21swapper/320:34:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional