You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu Sep 17, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
231232392918,3899sleep312143-21kworker/3:012:05:203
231232392918,3899sleep312143-21kworker/3:012:05:203
190902275017,2719sleep31572-21kworker/3:210:09:203
24089294821,909sleep324091-21sendmail-mta08:44:203
28600282340,733sleep128634-21unin-run07:17:481
28600282340,733sleep128634-21unin-run07:17:481
16580262118,586sleep316611-21/usr/sbin/munin10:03:103
1465242611,400sleep01504-21ntpq07:33:140
2236238117,345sleep02355-21/usr/sbin/munin09:18:210
5180237115,340sleep05183-21sshd07:44:200
2325236614,303sleep1231-21systemd-journal07:37:471
2904823428,296sleep129197-21latency_hist10:42:501
15732232811,301sleep215768-21ntpq08:18:142
15732232811,301sleep215768-21ntpq08:18:142
1586231818,247sleep11651-21ls09:18:061
32062229492,100sleep132100-21fschecks_count07:28:101
347422629,215sleep13513-21ntpq11:03:131
4568225861,184sleep14619-21ntp_kernel_pll_07:43:121
25967225143,192sleep126014-21latency_hist08:52:501
24047223968,93sleep124067-21munin-run10:27:481
10554223640,186sleep1367-21snmpd09:47:051
10554223640,186sleep1367-21snmpd09:47:051
8947223319,195sleep09039-21fschecks_time07:58:050
12533223139,122sleep112602-21fschecks_time09:53:071
1567022158,169sleep115702-21tune2fs11:43:051
5823220814,153sleep15876-21fschecks_count07:48:051
5823220814,153sleep15876-21fschecks_count07:48:051
2673222059,180sleep126824-21timerandwakeup10:33:211
20258219816,98sleep10-21swapper/111:57:591
19912218617,90sleep10-21swapper/110:13:101
19912218617,90sleep10-21swapper/110:13:101
9110218531,97sleep19165-21latency_hist09:42:491
3255021806,16sleep00-21swapper/007:28:210
25663218048,50sleep10-21swapper/112:13:171
29800217847,25sleep10-21swapper/109:03:161
6724217718,103sleep10-21swapper/109:33:161
5178217711,99sleep10-21swapper/109:28:161
3021321686,150sleep130249-21ntpq12:28:111
26958216112,134sleep327015-21awk12:18:083
12574216012,133sleep212614-21grep08:08:132
14128215666,73sleep10-21swapper/108:13:131
23360215412,62sleep10-21swapper/108:43:091
20591214618,54sleep10-21swapper/108:33:201
17681214511,119sleep117754-21cat11:48:181
10679214115,109sleep310783-21latency_hist11:27:503
2812321358,59sleep10-21swapper/108:58:111
7582213427,57sleep10-21swapper/111:17:501
107421348,113sleep11077-21modprobe10:56:061
4400212918,30sleep04446-21unin-run11:07:480
15158212814,21sleep20-21swapper/209:58:132
2719721255,108sleep10-21swapper/108:54:201
1304621258,43sleep10-21swapper/111:33:161
5993212325,80sleep110103-1kworker/1:0H11:12:501
31758212142,28sleep10-21swapper/112:33:111
18193212163,40sleep10-21swapper/108:27:501
22539211816,86sleep322654-21sed10:22:493
22539211816,86sleep322654-21sed10:22:493
12816211819,83sleep212422-21/usr/sbin/munin09:53:122
1165421165,12sleep10-21swapper/108:04:241
2123211514,78sleep22198-21switchtime10:58:172
557921145,16sleep00-21swapper/011:08:210
3289211111,18sleep10-21swapper/107:38:201
3110421108,52sleep131156-21cat09:08:091
2155621108,62sleep121593-21grep08:38:041
17624211043,25sleep117588-21/usr/sbin/munin10:07:491
29931210915,75sleep030006-21fschecks_time12:28:060
2317921095,11sleep10-21swapper/112:06:031
2317921095,11sleep10-21swapper/112:06:031
3493210523,65sleep10-21swapper/109:23:111
1329521056,12sleep10-21swapper/108:10:381
2218821045,12sleep10-21swapper/110:18:551
2218821045,12sleep10-21swapper/110:18:551
1087021048,47sleep10-21swapper/109:47:581
1087021048,47sleep10-21swapper/109:47:581
106721048,12sleep30-21swapper/307:33:053
9237210128,21sleep19278-21ntpq07:58:101
27758210010,75sleep027777-21apt-get08:57:580
301852988,76sleep130251-21cat07:22:491
2737629811,73sleep327397-21fschecks_count07:13:063
2737629811,73sleep327397-21fschecks_count07:13:063
2889829742,16sleep10-21swapper/112:23:181
157982978,44sleep10-21swapper/108:18:181
157982978,44sleep10-21swapper/108:18:181
1065829719,58sleep310644-21apt-get08:03:053
2507929637,36sleep30-21swapper/307:07:493
1968929614,46sleep119692-21cron08:32:471
1968929614,46sleep119692-21cron08:32:471
2434229316,60sleep324483-21awk12:08:193
2277629217,56sleep30-21swapper/310:23:013
1475729111,58sleep314897-21unixbench_singl11:38:193
133929016,54sleep01389-21wc09:17:490
133929016,54sleep01389-21wc09:17:490
121122905,51sleep10-21swapper/111:31:591
81932878,36sleep18240-21ntpq11:18:151
266022868,14sleep10-21swapper/107:12:481
266022868,14sleep10-21swapper/107:12:481
26152869,19sleep20-21swapper/207:37:562
2317028511,18sleep00-21swapper/008:43:040
988128414,19sleep00-21swapper/009:43:170
988128414,19sleep00-21swapper/009:43:170
735928422,46sleep10-21swapper/107:53:061
238332849,55sleep10-21swapper/112:08:081
2281028422,42sleep022824-21apt08:42:480
194352849,17sleep10-21swapper/110:12:501
1372428415,50sleep013789-21lockfile-remove11:37:480
2910028312,58sleep329171-21latency_hist09:02:503
1994128318,47sleep10-21swapper/111:57:471
1940328313,55sleep019434-21date10:12:490
984728221,46sleep10-21swapper/111:23:151
2869828215,50sleep028744-21ntp_offset12:23:130
1226028122,32sleep00-21swapper/009:52:500
220028018,44sleep32325-21timerwakeupswit10:58:183
1560428010,21sleep10-21swapper/109:58:201
152362806,63sleep00-21swapper/011:41:590
54627916,39sleep2617-21ls09:13:182
54627916,39sleep2617-21ls09:13:182
3180227919,12sleep10-21swapper/110:48:521
303002799,60sleep1367-21snmpd10:45:081
270492798,15sleep10-21swapper/112:18:101
668427821,37sleep06722-21grep09:33:150
3265127818,44sleep032660-21modprobe12:37:470
2474427814,46sleep024836-21fschecks_time08:48:090
2413527825,32sleep224180-21munin-run10:27:492
226572780,7sleep00-21swapper/010:22:490
226572780,7sleep00-21swapper/010:22:490
159192780,7sleep30-21swapper/308:18:193
159192780,7sleep30-21swapper/308:18:193
1215727815,50sleep212206-21systemd-run09:52:482
1215727815,50sleep212206-21systemd-run09:52:482
913127615,36sleep38835-21/usr/sbin/munin07:58:083
912727617,38sleep39091-21latency_hist11:22:493
495327615,48sleep25023-21latency11:08:102
3248027619,40sleep332547-21mailstats07:28:203
2829727620,31sleep20-21swapper/212:23:032
2665227619,39sleep326677-21sort07:12:503
2665227619,39sleep326677-21sort07:12:503
87832757,10sleep00-21swapper/009:39:130
439527520,32sleep30-21swapper/311:07:473
3013327526,32sleep330146-21apt07:22:473
2501927522,33sleep225078-21true07:07:482
2471027515,11sleep10-21swapper/110:28:241
246927518,11sleep10-21swapper/110:58:241
4732747,17sleep1289-21dbus-daemon09:13:171
4732747,17sleep1289-21dbus-daemon09:13:171
3088427417,40sleep230398-21/usr/sbin/munin07:23:152
2089927420,27sleep00-21swapper/011:58:180
1702027415,42sleep017148-21unixbench_multi10:03:200
1505127416,11sleep10-21swapper/111:39:341
818327319,32sleep00-21swapper/011:18:140
2666927313,43sleep226705-21wc08:53:162
1895027317,32sleep20-21swapper/211:53:082
1602127317,29sleep00-21swapper/010:02:470
2287127218,24sleep00-21swapper/010:23:060
854327117,31sleep28575-21munin-run07:57:472
390127123,28sleep00-21swapper/007:42:470
3038327112,42sleep230450-21swap12:28:162
2896527113,41sleep029047-21cut07:18:080
1299627119,37sleep313070-21sort09:53:173
3272927021,22sleep20-21swapper/212:37:492
1904327013,28sleep30-21swapper/311:53:093
1535027015,16sleep20-21swapper/211:42:502
2790726917,37sleep327965-21/usr/sbin/munin10:38:073
261592696,10sleep30-21swapper/308:52:523
2516426919,19sleep00-21swapper/012:13:040
2091326921,24sleep20-21swapper/210:17:472
2091326921,24sleep20-21swapper/210:17:472
1932726920,30sleep219333-21sshd08:28:202
1932726920,30sleep219333-21sshd08:28:202
107442699,48sleep010748-21sh08:03:180
659826820,24sleep30-21swapper/311:13:123
634226815,29sleep00-21swapper/007:48:160
634226815,29sleep00-21swapper/007:48:160
362526819,29sleep23701-21/usr/sbin/munin09:23:162
3093826818,25sleep20-21swapper/210:48:082
2233526826,27sleep20-21swapper/212:03:172
2233526826,27sleep20-21swapper/212:03:172
2043826815,41sleep220459-21awk08:33:172
1007226821,25sleep00-21swapper/011:23:180
2775826713,38sleep027796-21processes07:13:150
2775826713,38sleep027796-21processes07:13:150
2612226710,42sleep30-21swapper/310:33:033
215626722,28sleep32231-21grep09:18:203
132292679,13sleep30-21swapper/311:33:193
807126616,14sleep30-21swapper/309:38:113
290952660,7sleep20-21swapper/207:18:092
290052669,43sleep029046-21cat10:42:490
2629126615,28sleep00-21swapper/008:53:050
173102669,42sleep017333-21ntp_states08:23:130
631626514,28sleep20-21swapper/211:13:072
593126519,27sleep05961-21awk09:32:470
3069426538,17sleep130781-21sed07:23:101
3045126513,37sleep330566-21sed12:28:173
2668026520,20sleep00-21swapper/007:12:510
2668026520,20sleep00-21swapper/007:12:510
2475126517,26sleep20-21swapper/210:28:382
2443826511,27sleep20-21swapper/208:47:502
395226418,24sleep30-21swapper/307:42:503
2992226418,23sleep30-21swapper/310:43:203
717326320,19sleep30-21swapper/309:33:513
550226319,28sleep20-21swapper/207:47:502
257022638,11sleep00-21swapper/010:31:060
123426318,28sleep01271-21sh10:57:470
3252826211,20sleep30-21swapper/309:13:073
3252826211,20sleep30-21swapper/309:13:073
295182624,11sleep30-21swapper/312:27:203
1843026218,26sleep018423-21apt-get08:28:020
1843026218,26sleep018423-21apt-get08:28:020
42522617,14sleep00-21swapper/009:26:040
3215526120,20sleep00-21swapper/009:12:480
2687126121,21sleep20-21swapper/212:18:072
2601726117,18sleep20-21swapper/208:52:512
2099526123,19sleep00-21swapper/010:17:490
2099526123,19sleep00-21swapper/010:17:490
2011226116,30sleep319621-21/usr/sbin/munin10:13:163
2011226116,30sleep319621-21/usr/sbin/munin10:13:163
1771026118,25sleep20-21swapper/210:07:502
1246126115,30sleep212455-21apt-get11:33:012
83662605,16sleep30-21swapper/307:55:303
610526018,29sleep26142-21ntp_kernel_err07:48:102
610526018,29sleep26142-21ntp_kernel_err07:48:102
3239526012,29sleep332390-21apt-get10:53:013
2543526016,25sleep20-21swapper/207:08:082
2543526016,25sleep20-21swapper/207:08:082
2512326012,25sleep00-21swapper/007:07:500
2456526017,26sleep324560-21apt-get10:28:023
2378226017,30sleep023832-21cat12:08:070
221842609,11sleep20-21swapper/210:18:522
221842609,11sleep20-21swapper/210:18:522
214662609,40sleep20-21swapper/212:02:002
183772609,24sleep20-21swapper/210:08:182
730325917,25sleep07358-21df07:53:050
1898025914,19sleep30-21swapper/308:28:173
1898025914,19sleep30-21swapper/308:28:173
1138225917,24sleep311468-21mailstats09:48:163
1138225917,24sleep311468-21mailstats09:48:163
934725816,24sleep29432-21mailstats07:58:162
73072585,10sleep20-21swapper/209:35:312
3106325810,32sleep031120-21irqstats10:48:100
30922588,37sleep3201-21jbd2/mmcblk2p2-09:22:523
304132586,17sleep30-21swapper/309:05:243
2542225815,20sleep20-21swapper/212:13:102
1598125819,19sleep30-21swapper/311:43:123
1531225813,29sleep015330-21apt08:18:000
1531225813,29sleep015330-21apt08:18:000
1375025816,25sleep213746-21apt-get08:13:022
1147025815,23sleep00-21swapper/009:48:170
1147025815,23sleep00-21swapper/009:48:170
1100225814,29sleep011062-21/usr/sbin/munin11:28:070
805825716,21sleep20-21swapper/211:18:122
3234325719,20sleep20-21swapper/207:28:152
3220625715,17sleep20-21swapper/209:12:502
2933725714,27sleep229334-21apt-get09:03:012
268272579,32sleep026725-21/usr/sbin/munin12:18:050
1383425711,30sleep313837-21/usr/sbin/munin08:13:073
2905325616,24sleep229098-21systemd-run09:02:482
2473425617,18sleep30-21swapper/308:48:083
2443525618,16sleep10-21swapper/107:03:191
2123425616,26sleep021258-21latency_hist08:37:470
934925512,17sleep30-21swapper/309:43:033
934925512,17sleep30-21swapper/309:43:033
780125513,25sleep27794-21apt-get09:38:022
3131725513,25sleep30-21swapper/307:23:203
3098825515,25sleep331061-21fw_packets10:48:093
208142559,28sleep320874-21switchtime11:58:173
1886225513,28sleep018920-21/usr/sbin/munin11:53:060
1775725516,18sleep20-21swapper/211:48:192
1411325514,20sleep00-21swapper/008:13:120
1220625512,26sleep012200-21apt-get08:08:030
3139025413,24sleep331386-21apt-get12:33:013
310425414,24sleep23100-21apt-get11:03:022
3084025415,23sleep030876-21ntp_states07:23:130
2561525415,23sleep325621-21ntp_states12:13:153
2360325413,24sleep223597-21apt-get12:08:012
2162825416,22sleep321637-21diskstats08:38:063
1848825415,17sleep00-21swapper/011:52:490
1242425415,23sleep312473-21irqstats08:08:103
1222125317,16sleep20-21swapper/211:32:492
1118425320,20sleep30-21swapper/311:28:103
644925212,24sleep06498-21irqstats11:13:090
3116925213,22sleep331262-21memory09:08:113
2004725213,23sleep320056-21df_inode08:33:053
1536125213,19sleep30-21swapper/309:58:183
1192625217,21sleep211966-21unin-run08:07:492
285522519,18sleep20-21swapper/210:38:192
2214125111,16sleep00-21swapper/012:03:070
2214125111,16sleep00-21swapper/012:03:070
2155225113,22sleep221549-21apt-get08:38:012
934825013,19sleep20-21swapper/209:43:022
934825013,19sleep20-21swapper/209:43:022
493425020,16sleep20-21swapper/209:28:082
418925013,18sleep30-21swapper/307:43:023
2969425013,21sleep229701-21ntp_states10:43:152
233325015,16sleep00-21swapper/010:58:190
1766425011,22sleep217779-21users08:23:192
1566325011,14sleep00-21swapper/011:43:040
107825013,18sleep20-21swapper/210:56:072
514424914,19sleep35164-21/usr/sbin/munin11:08:153
512724913,20sleep30-21swapper/309:28:143
2612424911,20sleep026119-21apt-get10:33:040
1595224911,21sleep20-21swapper/211:43:112
152324914,21sleep21529-21ntp_states07:33:162
594724811,20sleep36004-21irqstats07:48:073
594724811,20sleep36004-21irqstats07:48:073
2691224812,21sleep226973-21ntp_states07:12:532
2691224812,21sleep226973-21ntp_states07:12:532
3061324712,18sleep00-21swapper/010:47:500
2621224715,18sleep20-21swapper/210:33:102
1778024714,18sleep30-21swapper/308:23:203
1712624711,20sleep317129-21apt-get11:48:003
2942424513,17sleep00-21swapper/009:03:070
943924215,14sleep20-21swapper/211:23:042
16532409,13sleep20-21swapper/209:18:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional