You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 00:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jun 11, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
109622240111,2364sleep010532-21kworker/0:010:23:050
206462526491,22sleep220648-21sshd12:35:052
843725219,498sleep0194-21jbd2/mmcblk2p2-11:57:510
843725219,498sleep0194-21jbd2/mmcblk2p2-11:57:510
23299247928,433sleep323407-21fschecks_time07:38:063
2498523749,351sleep324471-21apt-get07:43:033
287092329136,175sleep328747-21processes11:18:133
287092329136,175sleep328747-21processes11:18:133
2969123286,282sleep329727-21ntpq09:38:093
3784230431,257sleep23816-21tune2fs08:18:152
4162230016,200sleep34198-21ntp_kernel_err11:43:083
1491822909,265sleep315010-21basename07:12:503
22365225189,118sleep322507-21unixbench_singl07:33:173
18917224162,165sleep30-21swapper/310:48:083
1148222279,201sleep311590-21timerandwakeup10:23:163
31813221951,25sleep30-21swapper/311:28:183
31813221951,25sleep30-21swapper/311:28:183
21275221738,21sleep30-21swapper/309:13:013
26195220927,166sleep326277-21unin-run11:12:473
28715220811,179sleep328826-21timerwakeupswit07:53:173
17365220831,40sleep317376-21kernelversion10:43:073
8752220125,26sleep30-21swapper/308:33:073
8752220125,26sleep30-21swapper/308:33:073
270921988,18sleep30-21swapper/311:38:123
15272219011,165sleep315336-21find08:53:143
1784218947,77sleep30-21swapper/309:53:073
1784218947,77sleep30-21swapper/309:53:073
2016621857,16sleep30-21swapper/310:52:563
14853218512,153sleep314687-21/usr/sbin/munin12:18:063
2571921839,153sleep325763-21latency_hist09:27:473
31361217428,62sleep330839-21/usr/sbin/munin09:43:143
27352216111,135sleep327539-21latency_hist09:32:483
19007215920,11sleep30-21swapper/312:28:523
2012521567,13sleep30-21swapper/309:08:293
17642215011,117sleep30-21swapper/307:18:153
27845214812,98sleep327920-21latency_hist07:52:483
15555214627,50sleep30-21swapper/310:38:023
655321455,47sleep30-21swapper/311:52:053
1443121438,11sleep30-21swapper/308:51:493
2888821395,12sleep30-21swapper/309:36:153
3062921386,10sleep30-21swapper/307:58:583
3080021345,11sleep30-21swapper/311:24:313
1341221345,10sleep30-21swapper/310:29:103
10094213410,66sleep30-21swapper/312:03:033
1052621336,11sleep30-21swapper/310:22:033
2074221315,10sleep30-21swapper/312:36:193
1458521315,10sleep30-21swapper/307:11:123
105521318,86sleep31102-21ntpq11:33:193
2400321305,47sleep30-21swapper/309:21:133
21854212928,42sleep321872-21sort10:58:053
582521285,50sleep30-21swapper/310:06:443
523321278,16sleep00-21swapper/008:23:020
2592821257,15sleep00-21swapper/009:27:520
647921246,85sleep30-21swapper/308:27:373
2431821205,83sleep30-21swapper/311:04:193
19149212010,17sleep30-21swapper/307:23:163
19149212010,17sleep30-21swapper/307:23:163
961921185,80sleep30-21swapper/312:01:143
961921185,80sleep30-21swapper/312:01:143
7049211621,61sleep312-21rcu_sched10:08:183
30869211511,90sleep230870-21sshd08:02:042
14749211116,82sleep114781-21sed07:12:481
778421099,18sleep37910-21timerwakeupswit08:29:273
31072210923,30sleep30-21swapper/308:02:503
246921099,19sleep30-21swapper/308:13:163
8153210710,14sleep30-21swapper/311:57:463
17229210725,15sleep30-21swapper/308:58:193
1673521077,14sleep20-21swapper/208:58:102
22108210614,73sleep022170-21sendmail_mailst07:33:140
1115210524,27sleep30-21swapper/309:51:223
1115210524,27sleep30-21swapper/309:51:223
24173210413,76sleep124254-21latency_hist09:22:481
7500210317,71sleep07536-21latency_hist10:12:470
2039321027,13sleep30-21swapper/307:28:093
421621005,62sleep30-21swapper/310:00:593
25155210015,68sleep225302-21awk09:23:182
1733321008,30sleep30-21swapper/312:23:433
1733321008,30sleep30-21swapper/312:23:433
1390221006,32sleep30-21swapper/312:13:173
2848029921,58sleep228609-21unixbench_singl09:33:232
271992999,48sleep30-21swapper/311:13:483
85682989,28sleep10-21swapper/108:33:011
85682989,28sleep10-21swapper/108:33:011
1832929764,15sleep30-21swapper/309:03:113
1397529721,38sleep00-21swapper/010:32:580
101402977,51sleep30-21swapper/308:37:593
511129623,51sleep20-21swapper/211:47:482
2295929615,61sleep122903-21latency_hist07:37:481
124902965,63sleep30-21swapper/312:08:243
2624529524,50sleep126264-21munin-run07:47:451
152372958,32sleep30-21swapper/310:37:463
12842955,58sleep30-21swapper/308:09:083
12842955,58sleep30-21swapper/308:09:083
88412945,10sleep30-21swapper/310:15:373
1510129416,62sleep215126-21ls07:12:532
3259729115,60sleep232619-21munin-run11:32:452
3259729115,60sleep232619-21munin-run11:32:452
116862916,28sleep30-21swapper/308:43:003
2845129026,51sleep028217-21/usr/sbin/munin11:18:050
2845129026,51sleep028217-21/usr/sbin/munin11:18:050
380128819,51sleep13931-21sed09:58:181
1011028825,42sleep110155-21sshd12:03:051
3241828626,24sleep20-21swapper/209:48:012
3241828626,24sleep20-21swapper/209:48:012
1630928611,14sleep20-21swapper/212:22:492
1630928611,14sleep20-21swapper/212:22:492
1603428618,22sleep00-21swapper/012:22:470
279972858,58sleep228013-21cat11:17:472
1978528523,40sleep119798-21sh07:27:461
1978528523,40sleep119798-21sh07:27:461
1312228529,34sleep013128-21modprobe07:07:460
1294928522,42sleep012970-21modprobe08:47:460
336328418,23sleep00-21swapper/009:58:090
3197228415,54sleep232079-21awk08:03:172
288952847,64sleep00-21swapper/009:36:200
2613128417,50sleep226175-21apt-key11:12:462
2068228416,25sleep20-21swapper/207:28:162
855728323,34sleep20-21swapper/211:58:062
855728323,34sleep20-21swapper/211:58:062
1051328313,47sleep210118-21/usr/sbin/munin08:38:112
895528219,42sleep20-21swapper/210:17:052
324632814,65sleep00-21swapper/008:07:390
151682810,6sleep20-21swapper/212:18:122
1222328123,31sleep30-21swapper/307:03:093
41602807,62sleep34194-21ps08:18:253
420127916,18sleep00-21swapper/011:43:090
3276127923,26sleep10-21swapper/108:08:011
3276127923,26sleep10-21swapper/108:08:011
3217927922,22sleep10-21swapper/109:47:481
2533827919,36sleep125402-21ps11:08:131
2527627913,52sleep025357-21latency07:43:240
1681127918,43sleep216849-21modprobe10:42:462
142727917,39sleep21568-21grep11:33:262
3015827815,17sleep20-21swapper/211:23:082
276872786,15sleep00-21swapper/011:14:460
1286627812,51sleep112879-21ntpq10:28:111
911927718,37sleep19266-21latency_hist10:17:491
365027723,30sleep00-21swapper/008:17:570
2103727616,41sleep221179-21latency_hist09:12:482
152122769,13sleep00-21swapper/008:53:110
1498327617,27sleep10-21swapper/112:18:081
2939527514,22sleep00-21swapper/009:38:020
284327513,46sleep222896-1kworker/2:0H09:57:472
284327513,46sleep222896-1kworker/2:0H09:57:472
2332627519,34sleep20-21swapper/209:18:132
1908027521,34sleep219147-21grep07:23:152
1908027521,34sleep219147-21grep07:23:152
1814627517,40sleep118253-21fschecks_time12:28:051
1292827525,28sleep00-21swapper/010:28:140
1175627513,44sleep111819-21/usr/sbin/munin12:08:061
980627413,46sleep09807-21sendmail-mta08:37:320
980627413,46sleep09807-21sendmail-mta08:37:320
230122749,52sleep00-21swapper/011:02:260
177372747,55sleep00-21swapper/012:27:280
177372747,55sleep00-21swapper/012:27:280
94827320,24sleep00-21swapper/008:08:160
94827320,24sleep00-21swapper/008:08:160
856927316,42sleep28592-21cpu08:33:022
856927316,42sleep28592-21cpu08:33:022
2059127328,33sleep120624-21ntp_kernel_pll_10:53:111
1996527316,41sleep120009-21ntpq12:33:101
1763627322,31sleep10-21swapper/109:02:451
1478227326,20sleep20-21swapper/207:12:492
736927214,42sleep17419-21processes11:53:121
3062227215,28sleep130651-21sed09:42:471
2702227213,41sleep226514-21/usr/sbin/munin07:48:312
2153127211,47sleep221674-21cat10:57:482
1829027216,41sleep018400-21latency_hist07:22:480
595627117,25sleep10-21swapper/110:07:471
319627119,36sleep23283-21gpgv08:17:472
3184427120,26sleep10-21swapper/108:03:161
1686327120,26sleep00-21swapper/010:42:480
1141527125,19sleep20-21swapper/210:23:152
2651027017,36sleep125931-21/usr/sbin/munin09:28:131
2452327011,41sleep024620-21cut07:42:470
1926227019,26sleep219352-21apt-config09:07:472
57332697,51sleep00-21swapper/010:05:320
34426917,34sleep1459-21gpgv11:32:481
34426917,34sleep1459-21gpgv11:32:481
2821626919,27sleep20-21swapper/211:17:552
2821626919,27sleep20-21swapper/211:17:552
18782699,47sleep01893-21netstat09:53:080
18782699,47sleep01893-21netstat09:53:080
933126820,31sleep19400-21df10:18:051
833126818,24sleep10-21swapper/108:32:481
773926812,42sleep00-21swapper/010:12:590
3110426819,17sleep10-21swapper/111:27:481
2779426724,20sleep10-21swapper/107:52:461
2604226715,38sleep226060-21fschecks_count09:28:022
1928326718,32sleep019423-21/usr/sbin/munin07:23:170
1928326718,32sleep019423-21/usr/sbin/munin07:23:170
125552676,13sleep00-21swapper/012:09:100
1132526723,24sleep20-21swapper/212:07:472
2846326616,35sleep128513-21awk07:53:101
1134926621,27sleep111413-21sendmail_mailqu10:23:141
346126514,34sleep23505-21ntpq09:58:112
3002826516,23sleep20-21swapper/209:38:162
277126523,27sleep11-21systemd09:57:461
277126523,27sleep11-21systemd09:57:461
258826518,18sleep00-21swapper/008:13:170
1995326523,27sleep20-21swapper/210:52:472
1873326519,23sleep00-21swapper/010:48:050
1775926532,18sleep117760-21cron12:27:451
1775926532,18sleep117760-21cron12:27:451
1231626520,27sleep212322-21ntp_states07:03:122
1075526515,21sleep00-21swapper/008:38:170
2805826419,26sleep028077-21apt-get07:52:570
2317326418,28sleep223195-21sshd07:37:572
223372645,14sleep10-21swapper/109:15:051
2031026416,19sleep00-21swapper/012:33:170
1680026417,25sleep10-21swapper/107:17:491
1518926419,26sleep015311-21munin-node07:12:550
1353826420,24sleep00-21swapper/007:08:060
275326220,22sleep00-21swapper/011:38:130
2670426221,25sleep126646-21apt-get11:13:341
126842627,13sleep20-21swapper/208:44:212
584326111,33sleep15917-21find08:23:191
56526114,22sleep10-21swapper/109:48:171
56526114,22sleep10-21swapper/109:48:171
312326115,26sleep13152-21munin-run08:17:461
240662617,44sleep00-21swapper/009:22:020
769026022,18sleep20-21swapper/208:29:262
742026017,22sleep20-21swapper/211:53:132
665126014,20sleep20-21swapper/208:27:492
2965826016,28sleep029727-21df07:58:020
244962606,13sleep10-21swapper/109:23:041
167642606,16sleep10-21swapper/110:42:081
1313426019,27sleep1284-21dbus-daemon07:07:471
1050126013,20sleep10-21swapper/108:38:101
80082599,38sleep00-21swapper/011:55:530
2851425916,24sleep228538-21ntp_states07:53:112
2836625917,22sleep10-21swapper/109:33:221
2048425917,25sleep10-21swapper/107:28:111
1447425923,17sleep10-21swapper/112:17:471
940225817,26sleep29409-21df_inode10:18:062
543125818,19sleep10-21swapper/111:48:031
1974225813,27sleep00-21swapper/009:08:090
182382589,38sleep10-21swapper/110:46:121
1002425815,26sleep010017-21apt-get12:03:000
319302576,11sleep00-21swapper/009:45:130
3077525710,34sleep0363-21snmpd11:24:110
2396925719,18sleep20-21swapper/211:03:172
180342575,13sleep20-21swapper/207:20:082
1718325717,19sleep00-21swapper/010:43:040
1164525713,18sleep20-21swapper/212:08:032
729825613,27sleep07351-21/usr/sbin/munin08:29:160
365225616,24sleep13163-21apt-get08:17:581
2824125615,24sleep128236-21apt-get11:17:581
2824125615,24sleep128236-21apt-get11:17:581
2002825615,24sleep120080-21sed09:08:271
1990525616,26sleep119933-21/usr/sbin/munin10:52:461
1348625610,15sleep20-21swapper/208:48:082
1333825613,25sleep013432-21perl08:48:060
1311625615,24sleep013111-21apt-get12:12:580
828625517,17sleep10-21swapper/111:57:481
828625517,17sleep10-21swapper/111:57:481
627825515,20sleep20-21swapper/210:08:052
547725513,18sleep20-21swapper/211:48:042
464525514,25sleep24639-21apt-get10:03:002
37025517,22sleep2373-21df08:08:022
37025517,22sleep2373-21df08:08:022
3185325518,17sleep00-21swapper/011:28:190
3185325518,17sleep00-21swapper/011:28:190
2332225517,23sleep023318-21apt-get11:02:590
2303425515,26sleep223060-21munin-run11:02:452
224525515,24sleep12240-21apt-get11:37:571
1768825517,17sleep00-21swapper/009:02:480
1168425515,23sleep111679-21apt-get08:42:581
2055125416,17sleep00-21swapper/007:28:140
1935525412,25sleep219428-21date12:32:472
1930525415,25sleep019315-21modprobe12:32:450
1488125411,26sleep214963-21fschecks_time08:53:052
1379925421,16sleep20-21swapper/210:32:482
127425417,21sleep00-21swapper/009:52:480
127425417,21sleep00-21swapper/009:52:480
110525413,24sleep01137-21ntp_states11:33:200
941925317,17sleep00-21swapper/010:18:080
664925314,24sleep16655-21ntp_states10:08:121
355225315,17sleep20-21swapper/211:42:472
2557325315,17sleep20-21swapper/207:43:342
2320025314,23sleep123192-21apt-get07:38:001
1837725316,18sleep00-21swapper/009:03:130
1539525311,24sleep115389-21apt-get07:13:001
1378725315,21sleep213898-21timerwakeupswit12:13:162
3123525211,22sleep20-21swapper/209:43:092
2965425214,22sleep129649-21apt-get07:57:591
2519225210,26sleep025239-21memory11:08:080
2135825215,22sleep221359-21diskstats09:13:042
2127225213,23sleep021268-21apt-get09:12:580
1838925215,21sleep118390-21ntp_states09:03:141
1487925215,22sleep114880-21diskstats08:53:041
67125110,24sleep1660-21apt-get11:33:011
2448625112,22sleep024495-21diskstats09:23:030
2653725013,22sleep026532-21apt-get07:48:010
2332325013,20sleep10-21swapper/111:03:001
2145225015,16sleep00-21swapper/010:57:460
152312504,11sleep20-21swapper/210:37:412
379224912,21sleep13787-21apt-get11:42:581
1531724814,14sleep10-21swapper/110:37:481
579424710,23sleep05802-21sh11:48:100
1455424612,14sleep10-21swapper/108:52:481
2184024410,13sleep10-21swapper/110:58:031
2170824310,15sleep20-21swapper/207:33:042
182752437,24sleep20-21swapper/210:46:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional