You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 04:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Dec 09, 2025 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20475249696,4951sleep019149-21kworker/0:120:29:060
78102311420,3081sleep15076-21kworker/1:219:49:421
18336210426,1004sleep118372-21ntpq20:23:061
18336210426,1004sleep118372-21ntpq20:23:061
29245210368,1017sleep322356-1kworker/3:0H00:23:383
29245210368,1017sleep322356-1kworker/3:0H00:23:383
14946271919,681sleep215037-21/usr/sbin/munin20:13:052
19208245910,435sleep019257-21unin-run20:27:410
19208245910,435sleep019257-21unin-run20:27:410
4960242012,336sleep14962-21systemd23:07:421
298692279104,56sleep10-21swapper/122:42:591
21134226761,111sleep121205-21switchtime22:13:341
21134226761,111sleep121205-21switchtime22:13:341
19710225841,21sleep10-21swapper/123:53:101
24896223955,147sleep124915-21idleruntime-cro22:27:401
24869223670,125sleep125012-21unixbench_multi20:43:131
26884222698,65sleep126928-21fschecks_time22:33:011
26884222698,65sleep126928-21fschecks_time22:33:011
7903221177,61sleep17917-21fschecks_count21:33:001
4996221119,123sleep10-21swapper/119:42:431
9730221030,135sleep19733-21latency_hist23:22:451
21159220818,172sleep221197-21fschecks_count20:33:002
21159220818,172sleep221197-21fschecks_count20:33:002
11287220733,127sleep111334-21ntp_kernel_pll_21:43:061
11287220733,127sleep111334-21ntp_kernel_pll_21:43:061
11713220115,168sleep011714-21cpu20:02:570
12774219627,100sleep112898-21cat23:32:421
28723219010,62sleep128928-21gpgv00:22:431
479021858,167sleep024038-21kworker/u8:119:40:400
28697217447,51sleep10-21swapper/122:38:181
18794217116,24sleep118852-21fschecks_time22:08:011
18794217116,24sleep118852-21fschecks_time22:08:011
22717217021,129sleep322783-21fschecks_time20:38:023
24448216849,48sleep10-21swapper/100:08:131
1380821589,134sleep113919-21latency_hist21:52:431
25371215615,105sleep10-21swapper/122:28:041
1968121569,133sleep119742-21if_err_eth020:28:021
6467215519,44sleep26503-21sendmail23:12:402
16860215237,101sleep116863-21cron22:02:401
3033521458,100sleep130366-21awk00:27:391
3033521458,100sleep130366-21awk00:27:391
11388214119,104sleep211194-21apt-get20:02:432
2254321409,119sleep122577-21ps22:18:111
17421214028,70sleep10-21swapper/122:03:041
17421214028,70sleep10-21swapper/122:03:041
32724213910,115sleep132727-21cron19:27:411
6044213810,112sleep16193-21latency_hist21:27:431
16108213810,22sleep10-21swapper/120:17:401
31883213515,102sleep031887-21cron00:32:390
9370212911,39sleep10-21swapper/121:37:571
6761212915,100sleep26770-21acpi23:12:522
2766221299,101sleep10-21swapper/119:12:421
25929212954,38sleep125477-21apt-get20:47:451
13023212830,85sleep113100-21latency_hist20:07:431
16188212627,65sleep116086-21/usr/sbin/munin23:43:001
14521212611,102sleep25666-21snmpd20:12:322
662021259,80sleep10-21swapper/121:28:071
346721240,8sleep20-21swapper/219:37:422
3071221249,61sleep10-21swapper/100:28:001
1363521249,102sleep013682-21ntpq20:08:060
2951621237,19sleep129704-21gpgv19:17:431
27018212315,92sleep327216-21sed00:17:443
24604212112,66sleep10-21swapper/119:03:091
24604212112,66sleep10-21swapper/119:03:091
214021209,21sleep10-21swapper/119:32:581
25866211844,49sleep10-21swapper/100:13:051
2168021179,56sleep10-21swapper/120:33:111
2168021179,56sleep10-21swapper/120:33:111
181221157,55sleep10-21swapper/122:57:411
181221157,55sleep10-21swapper/122:57:411
412921145,33sleep10-21swapper/121:19:111
17661211431,61sleep017655-21apt-get23:47:560
16718211214,83sleep216794-21/usr/sbin/munin23:43:132
788721105,34sleep10-21swapper/123:15:381
6001210920,73sleep00-21swapper/021:27:420
5849210710,60sleep10-21swapper/119:43:111
31864210611,80sleep031907-21ntp_kernel_pll_19:23:070
20524210624,50sleep320517-21apt-get22:12:563
20524210624,50sleep320517-21apt-get22:12:563
15366210628,63sleep115496-21cat21:57:431
1087521065,90sleep10-21swapper/123:24:141
13655210517,67sleep30-21swapper/323:33:133
13655210517,67sleep30-21swapper/323:33:133
10540210519,61sleep10-21swapper/119:58:141
30675210417,62sleep030802-21grep19:18:510
31691210315,72sleep231707-21/usr/sbin/munin22:48:092
31691210315,72sleep231707-21/usr/sbin/munin22:48:092
1324421030,6sleep30-21swapper/320:07:523
27556210120,16sleep10-21swapper/120:52:581
30373210016,62sleep229884-21/usr/sbin/munin19:18:452
2204021005,67sleep10-21swapper/100:02:231
93562995,82sleep10-21swapper/119:54:101
91142990,9sleep20-21swapper/221:37:402
2986529922,23sleep30-21swapper/320:58:213
2986529922,23sleep30-21swapper/320:58:213
2810429915,68sleep028171-21/usr/sbin/munin20:53:130
186612995,67sleep10-21swapper/123:49:221
5362986,12sleep00-21swapper/019:27:520
1436029721,15sleep10-21swapper/123:37:431
1436029721,15sleep10-21swapper/123:37:431
3074929610,73sleep230834-21fw_packets00:28:012
2595029615,62sleep225477-21apt-get20:47:542
207029621,14sleep10-21swapper/122:57:491
1633729611,69sleep216483-21awk21:58:132
2725829422,56sleep326773-21apt-get00:17:523
2651929415,63sleep226556-21cat22:32:422
1368429421,12sleep10-21swapper/120:08:071
1091829414,65sleep010925-21cpu21:42:560
1091829414,65sleep010925-21cpu21:42:560
321722938,28sleep132175-21cron21:07:401
321722938,28sleep132175-21cron21:07:401
228332939,20sleep10-21swapper/100:03:121
2425329217,59sleep324372-21sed22:23:143
186982927,16sleep30-21swapper/322:07:553
186982927,16sleep30-21swapper/322:07:553
2049429113,58sleep219986-21apt-get22:12:512
2049429113,58sleep219986-21apt-get22:12:512
16732919,17sleep10-21swapper/121:13:011
2829529020,48sleep228442-21grep20:53:152
210029027,35sleep00-21swapper/022:57:560
182228910,18sleep11893-21sh19:32:411
3154228822,41sleep331565-21tune2fs22:48:053
3154228822,41sleep331565-21tune2fs22:48:053
58022867,16sleep10-21swapper/123:08:151
157112865,69sleep10-21swapper/123:41:151
2809128417,47sleep227609-21apt-get19:12:512
2618828425,32sleep026336-21grep00:13:120
2675628316,29sleep00-21swapper/020:48:290
834128212,53sleep20-21swapper/219:52:542
271332827,11sleep20-21swapper/222:33:062
271332827,11sleep20-21swapper/222:33:062
226292827,13sleep10-21swapper/120:37:561
1427028221,40sleep214299-21munin-plugin-st23:37:402
1427028221,40sleep214299-21munin-plugin-st23:37:402
2861728122,30sleep00-21swapper/000:22:420
2374028121,37sleep023817-21uniq22:23:020
1151928127,28sleep30-21swapper/323:28:003
3267428010,28sleep132711-21awk22:52:401
3267428010,28sleep132711-21awk22:52:401
2931828018,15sleep10-21swapper/120:57:511
2931828018,15sleep10-21swapper/120:57:511
1765628025,20sleep117706-21unin-run20:22:411
1765628025,20sleep117706-21unin-run20:22:411
570927914,49sleep05799-21mailstats23:08:140
3160127919,38sleep231751-21unixbench_multi21:03:172
3160127919,38sleep231751-21unixbench_multi21:03:172
2970727921,39sleep20-21swapper/200:23:532
2970727921,39sleep20-21swapper/200:23:532
2525727919,37sleep225197-21latency_hist00:12:422
1951527923,34sleep319553-21latency23:53:043
968327819,27sleep30-21swapper/319:57:433
495627816,47sleep24971-21cut21:23:042
687727724,32sleep30-21swapper/319:48:023
3112727715,48sleep331171-21/usr/sbin/munin22:47:413
2095927715,40sleep021001-21ntpq22:13:290
2095927715,40sleep021001-21ntpq22:13:290
222527613,31sleep00-21swapper/019:33:010
1812827618,43sleep218165-21ps23:48:102
1451427623,30sleep00-21swapper/021:53:100
496327521,27sleep20-21swapper/223:07:432
41582757,56sleep030406-21kworker/0:221:19:340
3183327521,30sleep30-21swapper/319:23:063
3061227522,32sleep230613-21cron21:02:402
3061227522,32sleep230613-21cron21:02:402
2809227520,33sleep328111-21apt-get19:12:523
2557927418,32sleep30-21swapper/320:47:433
2323527428,27sleep30-21swapper/318:58:113
2012527425,31sleep320250-21timerwakeupswit20:28:123
1297127420,37sleep312984-21apt20:07:413
299472736,29sleep10-21swapper/119:18:201
233792737,55sleep00-21swapper/000:04:440
219232738,13sleep30-21swapper/320:33:133
219232738,13sleep30-21swapper/320:33:133
1686227326,24sleep30-21swapper/320:18:113
1686227326,24sleep30-21swapper/320:18:113
813727221,28sleep00-21swapper/023:17:430
62127219,34sleep232438-21/usr/sbin/munin21:08:132
2482927213,12sleep30-21swapper/319:03:153
2482927213,12sleep30-21swapper/319:03:153
2332127230,20sleep10-21swapper/118:58:131
327152717,33sleep10-21swapper/121:08:041
2292127123,22sleep00-21swapper/018:58:040
2263327113,30sleep00-21swapper/020:37:590
16482719,50sleep01672-21sh21:13:000
1082027118,37sleep310832-21vmstat19:58:163
839527011,17sleep00-21swapper/021:33:110
2856027014,43sleep228577-21cat22:38:152
264627021,30sleep30-21swapper/322:58:153
2517727025,30sleep025191-21modprobe00:12:400
111627015,38sleep01192-21wc22:53:110
111627015,38sleep01192-21wc22:53:110
3722697,12sleep30-21swapper/321:08:063
1841126911,38sleep018416-21modprobe22:07:400
1841126911,38sleep018416-21modprobe22:07:400
3033126815,33sleep030333-21/usr/sbin/munin22:43:220
2860426813,20sleep00-21swapper/022:38:170
1122626818,25sleep20-21swapper/223:27:432
95302675,50sleep30-21swapper/323:21:503
726526717,28sleep00-21swapper/019:48:110
2576526721,27sleep025844-21irqstats19:07:440
2576526721,27sleep025844-21irqstats19:07:440
1846226720,26sleep30-21swapper/322:07:433
1846226720,26sleep30-21swapper/322:07:433
1323926724,22sleep00-21swapper/021:48:130
802526617,21sleep30-21swapper/321:33:023
477526622,26sleep30-21swapper/321:23:003
440026621,25sleep34434-21modprobe21:22:403
309892665,36sleep30-21swapper/319:20:163
3083526622,27sleep330891-21/usr/sbin/munin00:28:023
243352666,44sleep20-21swapper/220:43:022
190026614,27sleep30-21swapper/319:32:433
520926516,25sleep20-21swapper/219:42:522
3125226515,27sleep20-21swapper/222:47:442
2938526517,28sleep329409-21munin-run19:17:413
2873426520,23sleep20-21swapper/219:13:322
2771026516,26sleep20-21swapper/200:18:232
1789426524,19sleep20-21swapper/222:03:142
1789426524,19sleep20-21swapper/222:03:142
2579826412,32sleep325899-21/usr/sbin/munin22:28:143
2503126419,23sleep20-21swapper/219:03:232
2503126419,23sleep20-21swapper/219:03:232
1273626421,20sleep20-21swapper/221:48:032
1016326412,27sleep00-21swapper/019:58:050
993626324,25sleep010002-21cat21:38:120
2338926324,22sleep00-21swapper/022:22:420
232026319,28sleep32454-21unixbench_singl21:13:153
2206126314,33sleep222096-21munin-run00:02:402
624426218,31sleep06278-21cpu21:27:580
1453626216,18sleep20-21swapper/223:37:522
118012624,34sleep00-21swapper/023:28:050
377326112,21sleep30-21swapper/319:38:003
2882126119,27sleep328847-21munin-run20:57:403
2005826114,23sleep20-21swapper/220:28:112
1792826114,35sleep317942-21kernelversion23:48:033
1222926117,31sleep212326-21latency_hist21:47:432
1222926117,31sleep212326-21latency_hist21:47:432
528026022,22sleep30-21swapper/319:42:573
395626013,31sleep24014-21sed19:38:042
365226017,20sleep00-21swapper/023:02:570
2216226016,24sleep30-21swapper/322:18:013
15602606,11sleep20-21swapper/221:12:462
997125917,21sleep00-21swapper/023:23:000
77032596,11sleep20-21swapper/219:48:172
739625918,19sleep30-21swapper/323:13:143
2013125918,26sleep220274-21cut22:12:422
2013125918,26sleep220274-21cut22:12:422
1996825916,26sleep020042-21apt-config22:12:400
1996825916,26sleep020042-21apt-config22:12:400
1607925912,31sleep316146-21/usr/sbin/munin21:58:103
1311525918,17sleep00-21swapper/023:33:020
1311525918,17sleep00-21swapper/023:33:020
669625816,26sleep36704-21ntp_states21:28:093
2957525811,31sleep329610-21awk22:42:403
228725810,24sleep20-21swapper/222:58:062
2259125813,25sleep222625-21/usr/sbin/munin00:03:052
1802025814,26sleep218029-21diskstats20:22:582
1802025814,26sleep218029-21diskstats20:22:582
1492925816,26sleep314965-21ntpq23:38:063
1436625822,23sleep314412-21memory21:53:053
955425712,27sleep39616-21if_eth021:38:033
3155125712,26sleep20-21swapper/219:23:012
303782576,12sleep00-21swapper/020:59:430
303782576,12sleep00-21swapper/020:59:430
2676525717,24sleep026759-21apt-get22:32:560
2676525717,24sleep026759-21apt-get22:32:560
1560125715,27sleep015596-21apt-get21:57:550
1187525714,20sleep20-21swapper/220:03:162
2829925614,26sleep028361-21fw_packets19:13:220
2276825616,23sleep222764-21apt-get18:57:542
2143925615,19sleep30-21swapper/323:58:133
1794525612,25sleep317940-21apt-get20:22:553
1794525612,25sleep317940-21apt-get20:22:553
1743125610,31sleep317571-21cut23:47:433
1660725615,17sleep00-21swapper/020:18:040
1660725615,17sleep00-21swapper/020:18:040
1456025611,27sleep014555-21apt-get23:37:540
392025512,21sleep30-21swapper/323:03:043
2390525513,22sleep30-21swapper/300:07:583
2080625511,30sleep020874-21df23:57:580
1615925512,20sleep30-21swapper/320:17:423
641725410,31sleep05666-21snmpd19:46:380
624125413,24sleep26237-21apt-get21:27:562
2684325415,24sleep326848-21diskstats22:32:593
2684325415,24sleep326848-21diskstats22:32:593
235025419,17sleep30-21swapper/319:33:043
1255825417,17sleep30-21swapper/321:48:003
877125313,23sleep28794-21ntp_states23:18:092
601225314,22sleep36104-21users23:08:173
354125321,18sleep20-21swapper/221:18:072
319932535,39sleep00-21swapper/021:05:210
319932535,39sleep00-21swapper/021:05:210
267632539,16sleep00-21swapper/019:08:130
2520425328,16sleep00-21swapper/020:44:370
2362825313,23sleep223624-21apt-get22:22:562
935225213,22sleep29364-21apt-get21:37:542
61925214,22sleep3623-21diskstats22:52:583
61925214,22sleep3623-21diskstats22:52:583
3091525211,23sleep330911-21apt-get21:02:573
3091525211,23sleep330911-21apt-get21:02:573
3040025214,23sleep030529-21latency_hist00:27:430
3040025214,23sleep030529-21latency_hist00:27:430
1931925213,23sleep219325-21df_inode23:52:592
1152925219,17sleep30-21swapper/321:43:123
1152925219,17sleep30-21swapper/321:43:123
2232825111,22sleep30-21swapper/300:02:533
1991725110,21sleep00-21swapper/023:53:130
1648425112,18sleep20-21swapper/220:18:022
1648425112,18sleep20-21swapper/220:18:022
1528625112,19sleep00-21swapper/020:13:110
880325014,24sleep08833-21proc_pri23:18:110
86925013,22sleep0916-21users21:08:160
264852504,36sleep20-21swapper/200:14:112
2333725011,23sleep223366-21munin-run22:22:402
2263024913,21sleep222624-21apt-get20:37:572
1170224912,23sleep211704-21irqstats23:28:032
3156924811,15sleep00-21swapper/022:48:060
3156924811,15sleep00-21swapper/022:48:060
2160924813,17sleep00-21swapper/020:33:100
2160924813,17sleep00-21swapper/020:33:100
161812477,16sleep00-21swapper/023:42:590
876924613,19sleep00-21swapper/019:53:080
66024616,16sleep20-21swapper/222:53:012
66024616,16sleep20-21swapper/222:53:012
103152468,19sleep30-21swapper/323:23:073
2663324513,17sleep30-21swapper/320:48:283
96224312,12sleep20-21swapper/219:28:062
251792437,20sleep00-21swapper/022:27:550
1431424214,14sleep20-21swapper/221:53:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional