You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 04:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jun 30, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2211929558,886sleep022127-21ntpq23:55:030
17532295323,893sleep017576-21ntpq20:15:000
17532295323,893sleep017576-21ntpq20:15:000
1426829347,892sleep014304-21ntpq23:29:590
14306282712,797sleep114351-21/usr/sbin/munin23:30:001
14416248127,442sleep014457-21ntpq20:05:030
3010124096,371sleep0364-21snmpd22:37:410
1604224059,382sleep116050-21ntpq20:10:041
294912340160,30sleep029573-21latency_hist00:19:390
9432305138,146sleep01098-21unixbench_multi21:05:080
216762305127,159sleep021815-21tail22:10:270
2789722538,207sleep027988-21sort00:14:380
2789722538,207sleep027988-21sort00:14:380
5734225226,209sleep05822-21rm19:39:380
32558224163,28sleep00-21swapper/022:44:560
22602223518,199sleep022635-21fschecks_count22:14:550
3927223316,197sleep23994-21wc21:15:082
24624223014,194sleep00-21swapper/022:20:040
28759222929,118sleep028849-21fschecks_time19:09:580
31550221013,181sleep131643-21cut00:25:011
12570220935,22sleep00-21swapper/019:59:580
12570220935,22sleep00-21swapper/019:59:580
9901220049,16sleep00-21swapper/019:50:080
2695521938,162sleep00-21swapper/020:45:050
1023821900,69sleep30-21swapper/319:50:103
659321899,125sleep06191-21/usr/sbin/munin19:40:070
30845218919,152sleep230847-21cron20:59:372
325721888,163sleep01587-21kworker/0:200:39:380
2851321849,159sleep028530-21munin-run22:34:380
14680218214,148sleep214800-21timerwakeupswit20:05:092
30065217211,142sleep030099-21ps20:55:190
2285821715,33sleep00-21swapper/020:31:560
8091217019,83sleep00-21swapper/021:29:400
1998021709,149sleep020025-21systemd-run20:24:370
989821659,144sleep010026-21unixbench_multi23:15:090
989821659,144sleep010026-21unixbench_multi23:15:090
5891215916,93sleep00-21swapper/023:04:390
17524215816,72sleep00-21swapper/023:40:050
3197521558,129sleep00-21swapper/019:19:390
15535215511,76sleep015615-21fschecks_time23:34:560
3080521547,15sleep00-21swapper/022:39:510
26058214912,75sleep026098-21http00:09:370
24698214910,69sleep00-21swapper/020:39:400
24698214910,69sleep00-21swapper/020:39:400
15521214629,106sleep00-21swapper/021:51:290
573721448,100sleep05798-21ps21:20:590
9927214311,69sleep010007-21tune2fs21:34:570
23921214398,24sleep00-21swapper/000:00:080
1869921426,99sleep00-21swapper/022:02:070
24097213013,102sleep124092-21apt-get22:19:491
23456213018,88sleep20-21swapper/223:59:582
16630213034,81sleep016718-21cat21:55:080
30627212811,105sleep030639-21vmstat00:20:110
3316212726,14sleep00-21swapper/019:30:050
2898421235,35sleep00-21swapper/020:50:380
2892621230,7sleep30-21swapper/322:34:403
135621237,14sleep00-21swapper/019:24:560
135621237,14sleep00-21swapper/019:24:560
27348212218,78sleep227496-21tail00:10:562
27348212218,78sleep227496-21tail00:10:562
32642212113,88sleep10-21swapper/119:20:041
82721205,10sleep00-21swapper/019:22:090
2093212012,92sleep32151-21latency21:10:003
6397211926,57sleep00-21swapper/023:04:590
8149211817,80sleep17652-21/usr/sbin/munin23:10:051
2118921188,73sleep00-21swapper/023:52:160
2118921188,73sleep00-21swapper/023:52:160
1132211710,89sleep11243-21unixbench_singl00:30:101
1132211710,89sleep11243-21unixbench_singl00:30:101
1256321169,36sleep00-21swapper/023:24:580
19812211420,83sleep00-21swapper/023:49:260
2040721138,71sleep0364-21snmpd22:09:090
21136211235,56sleep321142-21fschecks_time22:10:143
11792211110,65sleep011840-21tr21:40:020
10996211110,55sleep00-21swapper/019:54:570
1659211032,35sleep00-21swapper/022:50:010
12179210825,16sleep30-21swapper/321:40:093
3156021077,89sleep031604-21ntpq21:00:030
1392221076,74sleep00-21swapper/021:45:470
1392221076,74sleep00-21swapper/021:45:470
1205721075,71sleep00-21swapper/023:24:300
418021066,72sleep00-21swapper/022:58:370
2051021067,33sleep00-21swapper/020:25:020
5823210526,64sleep16009-21latency_hist19:39:391
20200210417,66sleep120089-21/usr/sbin/munin23:49:551
20200210417,66sleep120089-21/usr/sbin/munin23:49:551
1065210421,68sleep31112-21latency_hist19:24:393
26221210317,71sleep326296-21switchtime22:25:083
144121035,87sleep00-21swapper/000:32:150
144121035,87sleep00-21swapper/000:32:150
28959210219,63sleep128471-21apt-get22:34:471
1997210112,69sleep21895-21/usr/sbin/munin00:34:552
14168210122,66sleep214223-21latency23:29:572
24386210018,57sleep224073-21/usr/sbin/munin22:19:582
3260529924,19sleep30-21swapper/300:29:393
2722729817,60sleep327298-21sort19:04:563
2611029812,71sleep126134-21proc_pri22:25:061
2461629721,56sleep124626-21modprobe20:39:371
2461629721,56sleep124626-21modprobe20:39:371
1469829617,65sleep314760-21if_eth021:49:583
653029537,46sleep36571-21ntp_offset23:05:023
217162956,11sleep20-21swapper/220:29:402
41102937,16sleep00-21swapper/021:15:100
2782929221,19sleep20-21swapper/220:49:392
2089629221,52sleep10-21swapper/120:25:111
1350829218,54sleep112938-21/usr/sbin/munin21:45:071
1350829218,54sleep112938-21/usr/sbin/munin21:45:071
3187829115,62sleep331880-21sshd22:42:163
153729117,56sleep21425-21/usr/sbin/munin22:49:592
271929017,55sleep12733-21vmstat21:10:101
1101629050,26sleep111064-21if_eth023:19:581
3124628922,19sleep20-21swapper/220:59:572
312522888,41sleep00-21swapper/000:24:480
2610328819,12sleep00-21swapper/022:25:050
30852878,65sleep03117-21munin-run21:14:370
2493428718,53sleep00-21swapper/020:39:520
1323428717,31sleep20-21swapper/221:45:002
1323428717,31sleep20-21swapper/221:45:002
1048728726,36sleep310517-21munin-run23:19:363
1048728726,36sleep310517-21munin-run23:19:363
474728625,40sleep14901-21latency_hist21:19:391
786328518,42sleep17955-21fschecks_time19:44:561
786328518,42sleep17955-21fschecks_time19:44:561
510328515,18sleep30-21swapper/323:00:053
2261928510,57sleep30-21swapper/320:30:103
1848028519,47sleep218582-21sed20:19:382
1848028519,47sleep218582-21sed20:19:382
1618728515,12sleep00-21swapper/020:10:070
927528418,29sleep00-21swapper/021:31:050
2624228422,31sleep20-21swapper/220:44:382
1920128419,47sleep318697-21/usr/sbin/munin20:20:083
884728326,45sleep08866-21apt-get19:47:180
884728326,45sleep08866-21apt-get19:47:180
690428311,58sleep36958-21fw_packets21:24:583
3248528319,28sleep20-21swapper/219:20:012
2810728318,44sleep328108-21/usr/sbin/munin00:14:453
1678328324,28sleep30-21swapper/323:39:393
76512828,38sleep00-21swapper/023:09:480
2659628220,42sleep126581-21apt-get00:09:581
2659628220,42sleep126581-21apt-get00:09:581
912528124,35sleep39258-21latency_hist19:49:393
912528124,35sleep39258-21latency_hist19:49:393
2998328126,39sleep130021-21munin-node19:14:391
1592428112,54sleep115982-21sendmail_mailqu23:35:051
2916428031,28sleep30-21swapper/322:35:013
2300528018,44sleep323107-21sed23:59:383
200528021,31sleep30-21swapper/300:34:563
1927028031,30sleep30-21swapper/322:04:563
538627925,26sleep30-21swapper/321:20:503
124427922,31sleep20-21swapper/200:30:112
124427922,31sleep20-21swapper/200:30:112
1713327828,30sleep30-21swapper/320:14:513
1713327828,30sleep30-21swapper/320:14:513
1615427817,38sleep216259-21tune2fs21:54:582
1224527822,29sleep20-21swapper/219:59:392
982527714,47sleep29841-21/usr/sbin/munin19:50:062
3150327719,33sleep20-21swapper/222:40:222
2917927718,43sleep229256-21readlink19:10:082
2911427718,44sleep229161-21fschecks_time22:35:002
586927616,44sleep25946-21grep21:21:012
2823327614,23sleep20-21swapper/220:50:002
2375227619,41sleep20-21swapper/220:35:012
2375227619,41sleep20-21swapper/220:35:012
2802927526,27sleep10-21swapper/120:49:541
2661527512,37sleep30-21swapper/300:10:133
2661527512,37sleep30-21swapper/300:10:133
1880527514,27sleep10-21swapper/123:44:581
1460327519,40sleep214696-21sort21:49:572
1206427518,39sleep212076-21cron23:24:362
1203727516,46sleep312038-21sshd23:24:173
756727426,25sleep10-21swapper/119:44:401
2361427414,31sleep10-21swapper/100:00:011
2351427417,28sleep30-21swapper/323:59:593
823327316,39sleep28274-21/usr/sbin/munin19:45:052
823327316,39sleep28274-21/usr/sbin/munin19:45:052
72827316,27sleep10-21swapper/121:05:051
3243527316,29sleep20-21swapper/222:44:542
3164427319,40sleep231659-21netstat00:25:022
275727326,26sleep10-21swapper/122:54:381
2717127329,23sleep20-21swapper/219:04:552
3019327215,34sleep329707-21/usr/sbin/munin00:20:063
2604727217,16sleep20-21swapper/222:25:032
3189527117,35sleep231920-21munin-run19:19:372
2446527113,39sleep324496-21/usr/sbin/munin22:19:593
206462710,8sleep30-21swapper/320:25:043
203942715,11sleep20-21swapper/222:08:592
1771427116,27sleep30-21swapper/321:59:563
1930927025,25sleep30-21swapper/323:45:093
1880727017,35sleep218895-21fschecks_time20:19:592
1361227017,29sleep20-21swapper/223:29:372
1285427016,30sleep20-21swapper/220:00:042
1285427016,30sleep20-21swapper/220:00:042
963426918,31sleep19698-21cat19:50:011
2139626914,39sleep121441-21ntp_offset22:10:201
2061126919,28sleep10-21swapper/122:09:391
1781426923,24sleep10-21swapper/121:59:581
167626818,33sleep31703-21awk19:25:033
167626818,33sleep31703-21awk19:25:033
1010926820,35sleep310176-21latency21:35:003
2618926719,25sleep20-21swapper/200:09:392
2291826723,19sleep10-21swapper/123:59:361
2011526716,33sleep220166-21/usr/sbin/munin23:49:522
2011526716,33sleep220166-21/usr/sbin/munin23:49:522
1642126714,35sleep216487-21/usr/sbin/munin20:10:102
1380226724,19sleep10-21swapper/120:04:391
1380226724,19sleep10-21swapper/120:04:391
1342326712,34sleep30-21swapper/321:45:053
1342326712,34sleep30-21swapper/321:45:053
1292926716,20sleep10-21swapper/123:25:071
433526617,31sleep14451-21latency_hist22:59:391
345826616,20sleep30-21swapper/321:14:563
293626616,36sleep32938-21diskstats19:29:563
222026620,25sleep30-21swapper/322:50:123
2126026626,23sleep20-21swapper/222:10:172
2048526625,23sleep30-21swapper/323:50:013
2048526625,23sleep30-21swapper/323:50:013
188226624,24sleep10-21swapper/119:25:091
188226624,24sleep10-21swapper/119:25:091
1757826623,23sleep10-21swapper/120:15:011
1757826623,23sleep10-21swapper/120:15:011
1598426611,24sleep20-21swapper/223:35:062
326826521,22sleep20-21swapper/222:55:002
3242726526,20sleep132438-21modprobe21:04:371
3127226515,23sleep10-21swapper/122:40:161
2539426514,18sleep10-21swapper/120:40:051
195826512,23sleep20-21swapper/219:25:102
195826512,23sleep20-21swapper/219:25:102
1833126522,23sleep20-21swapper/223:44:392
1579226512,19sleep30-21swapper/320:09:583
874526413,34sleep38752-21/usr/sbin/munin21:30:033
659526418,30sleep26655-21/usr/sbin/munin23:05:052
2784626410,35sleep227962-21timerwakeupswit22:30:082
2063326412,36sleep220645-21/usr/sbin/munin20:25:042
1512426415,22sleep10-21swapper/121:50:081
900726324,19sleep30-21swapper/323:14:393
46726314,34sleep2524-21irqstats21:04:582
1272726319,29sleep212743-21sed21:44:402
622626212,35sleep36296-21df19:39:583
2735526226,21sleep10-21swapper/119:04:591
62426115,26sleep30-21swapper/300:30:003
62426115,26sleep30-21swapper/300:30:003
367926123,17sleep30-21swapper/322:55:093
3247826117,26sleep332548-21latency_hist21:04:393
1856826118,26sleep218562-21apt-get23:44:512
1460726120,25sleep114679-21sed20:05:081
619026016,31sleep26204-21acpi19:39:482
2968526017,20sleep30-21swapper/320:55:093
2859826015,22sleep10-21swapper/122:34:391
2834526015,25sleep028375-21munin-run19:09:370
273952605,11sleep10-21swapper/120:45:141
242326021,19sleep10-21swapper/100:35:061
2189526015,28sleep121905-21/usr/sbin/munin20:29:551
1375126013,27sleep30-21swapper/320:04:373
1375126013,27sleep30-21swapper/320:04:373
1127926014,20sleep10-21swapper/119:55:021
897725913,22sleep20-21swapper/223:14:382
338425912,24sleep20-21swapper/219:30:072
2958625912,19sleep30-21swapper/319:10:133
1671925918,21sleep10-21swapper/121:55:091
972125816,26sleep20-21swapper/223:15:072
972125816,26sleep20-21swapper/223:15:072
31625817,24sleep1335-21ntp_states22:45:011
237025822,20sleep20-21swapper/221:10:072
1001025819,23sleep10-21swapper/121:34:581
674825714,25sleep16742-21apt-get21:24:521
366025714,17sleep10-21swapper/119:30:101
3016425714,25sleep330291-21df_inode19:14:413
2258625717,26sleep222593-21df_inode22:14:532
1816425718,18sleep20-21swapper/222:00:072
1035825721,19sleep29817-21/usr/sbin/munin21:35:072
866225624,18sleep10-21swapper/121:30:011
550325620,15sleep10-21swapper/121:20:521
465925617,22sleep14745-21fschecks_time22:59:551
3109425611,29sleep131100-21open_files19:15:081
3092425617,23sleep10-21swapper/120:59:401
2386325614,20sleep30-21swapper/322:19:393
2263725617,23sleep122723-21perl22:14:561
1760525613,22sleep117693-21/usr/sbin/munin23:40:061
1151325617,22sleep30-21swapper/319:55:083
738625518,20sleep10-21swapper/123:09:361
3015325519,21sleep130185-21ntp_states00:20:041
244142555,10sleep30-21swapper/320:37:043
244142555,10sleep30-21swapper/320:37:043
2173825510,28sleep221742-21diskstats23:54:532
1899125511,18sleep10-21swapper/120:20:021
162025514,17sleep30-21swapper/321:09:393
1605425516,23sleep316050-21apt-get21:54:513
1398525521,15sleep30-21swapper/323:29:543
1314325513,18sleep30-21swapper/323:25:093
829225413,24sleep28288-21apt-get21:29:502
486525416,22sleep34871-21ntp_states19:35:093
2535925417,16sleep30-21swapper/300:05:123
922525315,21sleep19292-21df23:14:541
922525315,21sleep19292-21df23:14:541
3000325310,18sleep20-21swapper/200:20:012
720325213,23sleep27209-21ntp_states21:25:052
2820325212,25sleep128212-21df_inode00:14:511
2478125215,19sleep10-21swapper/100:04:531
1700225214,22sleep317013-21apt-get23:39:473
1184125212,24sleep111867-21ntp_states21:40:031
1480325114,15sleep30-21swapper/320:05:103
542025010,15sleep20-21swapper/223:00:082
497525014,18sleep20-21swapper/219:35:122
2821324915,19sleep228216-21diskstats00:14:522
3246124813,20sleep332556-21perl22:44:553
2493224811,21sleep324928-21apt-get20:39:513
2802624512,16sleep30-21swapper/320:49:523
2680724511,17sleep30-21swapper/320:45:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional