You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 19:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Tue Jul 28, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23439213566,1339sleep221998-21kworker/2:110:58:382
27707280715,770sleep127770-21/usr/sbin/munin07:48:151
315726016,583sleep23204-21ntpq11:38:192
9399259327,543sleep29496-21fschecks_time08:33:132
25975250324,463sleep326019-21/usr/sbin/munin07:43:083
27736242728,379sleep327832-21/usr/sbin/munin09:28:253
102662333127,184sleep2225-21systemd-journal12:02:532
47442315141,152sleep24758-21fschecks_count08:18:112
144752295125,23sleep20-21swapper/210:32:552
257372287128,31sleep225788-21latency_hist07:42:552
12349224828,200sleep012499-21unixbench_singl10:23:240
12426223357,27sleep20-21swapper/208:43:102
1978223116,201sleep01989-21sh08:08:200
5569220612,23sleep20-21swapper/210:03:162
10964220422,32sleep20-21swapper/208:38:142
10964220422,32sleep20-21swapper/208:38:142
31319219721,49sleep2225-21systemd-journal09:42:532
2171921939,167sleep221735-21apt-get09:12:542
29377218214,149sleep228462-21/usr/sbin/munin09:33:292
1918121819,155sleep219336-21latency_hist10:47:572
243412179134,24sleep20-21swapper/209:18:232
22676217810,27sleep222718-21ntp_kernel_err09:13:592
31329216413,134sleep031337-21sh07:58:190
8095216322,21sleep20-21swapper/211:53:252
1160216311,137sleep11168-21ntpq09:48:211
19061216111,16sleep20-21swapper/212:28:202
21420215914,49sleep20-21swapper/212:37:532
21420215913,19sleep20-21swapper/210:53:222
21420215913,19sleep20-21swapper/210:53:222
2701821558,134sleep227036-21fschecks_count09:28:122
16720215218,117sleep316777-21grep07:13:133
8025215024,28sleep20-21swapper/208:28:172
12346215012,121sleep212397-21sort12:08:102
12346215012,121sleep212397-21sort12:08:102
316621499,125sleep23207-21fschecks_count08:13:152
1654214814,28sleep20-21swapper/208:08:142
20038214498,22sleep20-21swapper/210:48:252
629021417,117sleep20-21swapper/208:23:122
30624214194,25sleep20-21swapper/211:23:132
18709213954,71sleep218850-21cat09:02:572
16065213994,29sleep216221-21latency_hist10:37:552
2950021389,114sleep229560-21cat07:53:212
30648213317,96sleep130767-21latency_hist07:57:561
1281321339,113sleep2364-21snmpd10:27:142
1496221329,101sleep20-21swapper/207:08:322
1496221329,101sleep20-21swapper/207:08:322
28599213112,104sleep228460-21latency_hist11:17:562
27260212917,88sleep327359-21fschecks_time11:13:133
24804212710,102sleep224958-21sed11:03:272
1748621266,12sleep20-21swapper/210:42:072
16955212555,56sleep217096-21cat08:57:562
2614221239,91sleep20-21swapper/211:08:232
338212110,21sleep20-21swapper/208:03:182
2891212110,18sleep20-21swapper/209:53:232
1977221218,18sleep20-21swapper/207:23:092
1977221218,18sleep20-21swapper/207:23:092
14678212123,79sleep314664-21apt-get07:08:123
14678212123,79sleep314664-21apt-get07:08:123
18901212036,53sleep318897-21apt-get09:03:103
5094211920,76sleep05117-21munin-plugin-st10:02:530
21852211917,81sleep022008-21apt-key09:12:560
1181321188,97sleep211863-21apt-config12:07:542
30679211618,82sleep130805-21timerwakeupswit09:38:241
1430521157,14sleep20-21swapper/212:13:192
692921147,19sleep26923-21apt-get10:08:082
25359211421,12sleep20-21swapper/209:23:032
25359211421,12sleep20-21swapper/209:23:032
32210211383,19sleep232294-21perl11:28:132
3215211318,27sleep03216-21ntp_states11:38:210
22275211322,71sleep022222-21latency_hist10:57:560
22275211322,71sleep022222-21latency_hist10:57:560
66321120,27sleep1668-21timerandwakeup08:03:241
4698211215,84sleep14735-21sed11:43:181
15623211217,77sleep015726-21fschecks_time08:53:140
648421117,13sleep20-21swapper/211:48:242
5273211120,12sleep20-21swapper/211:44:122
2777121117,15sleep20-21swapper/207:48:162
26960211023,54sleep20-21swapper/207:44:242
19455211020,56sleep20-21swapper/207:22:362
1706621106,15sleep20-21swapper/212:23:082
9539210921,54sleep20-21swapper/210:15:152
431121096,14sleep20-21swapper/209:58:232
941210819,68sleep31011-21latency_hist11:32:553
3533210814,77sleep13539-21ntp_states08:13:221
30521086,91sleep20-21swapper/209:46:402
30521086,91sleep20-21swapper/209:46:402
24002210821,54sleep20-21swapper/207:36:332
32067210721,52sleep20-21swapper/208:01:522
9786210620,72sleep39870-21mailstats08:33:223
1548621068,80sleep20-21swapper/212:18:072
18874210522,55sleep20-21swapper/209:03:052
1438821056,60sleep20-21swapper/208:48:192
14109210537,44sleep114154-21awk07:07:541
14109210537,44sleep114154-21awk07:07:541
1648921035,12sleep20-21swapper/207:13:042
1577210218,70sleep01614-21sed11:33:200
956821010,15sleep10-21swapper/111:58:231
11214210121,53sleep20-21swapper/210:21:322
334729917,62sleep33433-21threads11:38:243
2901329929,44sleep329014-21apt-get11:18:263
2360029816,68sleep323709-21sed09:17:553
222362986,11sleep20-21swapper/207:28:422
2411129720,62sleep124178-21modprobe07:37:531
19562975,12sleep20-21swapper/209:52:502
987329613,60sleep09943-21/usr/sbin/munin08:33:230
301142965,55sleep20-21swapper/211:21:112
1585629612,69sleep015895-21ntpq12:18:170
713429516,55sleep37157-21munin-plugin-st11:52:533
2010629515,65sleep1225-21systemd-journal09:07:151
1640829420,55sleep116083-21munin-node07:13:031
22472935,56sleep20-21swapper/211:35:282
171512938,15sleep30-21swapper/312:23:133
1261929321,50sleep312671-21if_eth008:43:153
1557229125,42sleep315635-21cut12:18:123
1782529013,61sleep117886-21processes08:58:361
603428816,15sleep00-21swapper/011:48:140
3185028821,49sleep031978-21latency_hist11:27:550
3097328814,17sleep10-21swapper/111:23:201
163752888,12sleep30-21swapper/310:38:133
515328614,53sleep05234-21/usr/sbin/munin08:18:220
2664428616,51sleep026650-21modprobe09:27:530
2664428616,51sleep026650-21modprobe09:27:530
1370428618,53sleep313773-21/usr/sbin/munin10:28:243
2588128518,50sleep325897-21kernelversion11:08:163
2318328423,45sleep022487-21/usr/sbin/munin10:58:240
3063528227,24sleep30-21swapper/311:23:143
1498628127,37sleep014692-21/usr/sbin/munin10:33:170
1250428116,48sleep112513-21/usr/sbin/munin10:23:251
1335428024,38sleep313449-21cut07:03:243
1335428024,38sleep313449-21cut07:03:243
2442827922,32sleep30-21swapper/309:18:243
150727918,44sleep11524-21apt08:08:051
194662788,19sleep00-21swapper/007:22:430
1713227820,41sleep117143-21/usr/sbin/munin12:23:111
1415627833,25sleep20-21swapper/207:07:552
1415627833,25sleep20-21swapper/207:07:552
2035027716,23sleep10-21swapper/107:23:241
2035027716,23sleep10-21swapper/107:23:241
1523527714,29sleep10-21swapper/112:17:541
1108227710,11sleep00-21swapper/010:19:490
818727614,24sleep30-21swapper/310:12:533
348327616,27sleep00-21swapper/008:13:210
283927619,35sleep32802-21latency_hist08:12:563
746427520,30sleep30-21swapper/308:27:533
1186427518,34sleep311881-21systemd-run12:07:553
502627413,38sleep35052-21netstat08:18:173
1324727423,21sleep10-21swapper/110:28:131
1781827312,18sleep00-21swapper/008:58:350
1602027318,38sleep116100-21sendmail_mailtr08:53:231
2761527239,18sleep00-21swapper/007:48:140
1977327120,32sleep319836-21/usr/sbin/munin07:23:103
1977327120,32sleep319836-21/usr/sbin/munin07:23:103
186132716,47sleep00-21swapper/012:28:060
1709727117,33sleep316806-21latency_hist08:57:573
1638727123,25sleep00-21swapper/010:38:140
1525027110,37sleep30-21swapper/310:33:243
438727014,26sleep30-21swapper/311:43:123
323472709,15sleep00-21swapper/011:28:160
1188027016,23sleep30-21swapper/310:23:153
115127025,27sleep00-21swapper/009:48:200
720126916,33sleep06905-21/usr/sbin/munin10:08:180
596026915,31sleep15978-21latency_hist08:22:551
2968426914,42sleep029814-21timerwakeupswit11:18:470
1604226916,33sleep10-21swapper/110:37:541
640326818,23sleep10-21swapper/111:48:231
322226815,22sleep10-21swapper/111:38:221
2824426819,24sleep10-21swapper/109:32:551
1164326823,22sleep30-21swapper/308:38:273
1164326823,22sleep30-21swapper/308:38:273
667326726,23sleep00-21swapper/008:23:220
2201026721,17sleep10-21swapper/109:12:571
2092326721,26sleep30-21swapper/309:08:233
201226717,26sleep30-21swapper/309:52:553
194622678,13sleep10-21swapper/110:48:121
1759826720,24sleep10-21swapper/110:42:551
429026614,38sleep14310-21/usr/sbin/munin09:58:221
2999126616,35sleep029284-21/usr/sbin/munin07:53:310
287292665,13sleep30-21swapper/309:33:173
2069826617,22sleep00-21swapper/010:52:550
940326530,20sleep00-21swapper/011:58:190
785026517,18sleep30-21swapper/308:28:143
2898526521,27sleep128978-21apt-get11:18:071
2774726519,20sleep10-21swapper/111:13:231
2575526513,23sleep10-21swapper/109:23:181
2575526513,23sleep10-21swapper/109:23:181
2564826515,26sleep30-21swapper/309:23:163
2564826515,26sleep30-21swapper/309:23:163
1405026514,36sleep314054-21diskstats08:48:123
1369926412,26sleep00-21swapper/012:12:560
1243026418,23sleep10-21swapper/112:08:131
1243026418,23sleep10-21swapper/112:08:131
54702638,13sleep00-21swapper/010:03:140
439826314,33sleep04486-21fschecks_time11:43:130
301926317,19sleep30-21swapper/309:53:243
2736526311,11sleep00-21swapper/011:13:140
2224926315,30sleep022243-21apt-get09:13:080
2025626323,21sleep30-21swapper/312:33:143
1975126312,22sleep30-21swapper/310:48:183
1444126314,34sleep114478-21ps08:48:221
1070326316,20sleep10-21swapper/110:18:241
1065926321,22sleep10-21swapper/108:37:571
3060726217,26sleep00-21swapper/009:38:230
2443026217,22sleep00-21swapper/007:38:090
2384226225,21sleep10-21swapper/109:18:101
213972627,11sleep10-21swapper/110:53:211
213972627,11sleep10-21swapper/110:53:211
1046226217,26sleep310493-21/usr/sbin/munin10:18:183
36626112,36sleep0368-21sshd09:47:230
36626112,36sleep0368-21sshd09:47:230
320942618,13sleep30-21swapper/309:43:233
320942618,13sleep30-21swapper/309:43:233
2906626118,23sleep328793-21apt-get07:52:573
2848926115,29sleep10-21swapper/109:33:111
2779426117,27sleep327883-21/usr/sbin/munin07:48:173
2537826133,15sleep30-21swapper/311:07:563
2392526114,19sleep00-21swapper/009:18:130
1441826114,29sleep014624-21apt-key07:07:570
1389426116,28sleep013926-21cpu12:13:080
11102616,17sleep00-21swapper/008:06:140
350826022,23sleep00-21swapper/009:57:530
3128126021,22sleep30-21swapper/307:58:183
2823426013,24sleep00-21swapper/009:32:540
198326010,33sleep32081-21users11:33:273
1977026018,25sleep019766-21apt-get07:23:070
1977026018,25sleep019766-21apt-get07:23:070
1953326021,19sleep10-21swapper/107:22:561
158972605,11sleep10-21swapper/112:18:181
141542606,44sleep00-21swapper/010:29:300
1239926020,22sleep10-21swapper/108:43:051
118126015,28sleep11176-21apt-get11:33:091
946725917,27sleep39496-21/usr/sbin/munin11:58:223
2308825916,19sleep30-21swapper/310:58:233
2306125917,24sleep30-21swapper/307:33:153
2297425923,19sleep10-21swapper/110:58:211
835225814,23sleep30-21swapper/311:53:383
394225811,24sleep33980-21users08:13:283
390525817,23sleep33911-21diskstats09:58:123
2910225812,22sleep00-21swapper/009:33:270
1593525815,28sleep115981-21systemd-run07:12:551
1593525815,28sleep115981-21systemd-run07:12:551
1451425823,18sleep30-21swapper/312:13:243
1108825818,17sleep00-21swapper/008:38:160
1108825818,17sleep00-21swapper/008:38:160
982625716,24sleep10-21swapper/110:17:561
3095925715,28sleep330971-21vmstat09:38:263
251792579,11sleep10-21swapper/111:06:001
185142578,13sleep10-21swapper/110:43:231
1309225717,20sleep00-21swapper/007:03:170
1055925715,25sleep110553-21apt-get12:03:081
775425614,26sleep07750-21apt-get08:28:070
2103725618,23sleep121134-21cat09:08:251
1365225617,20sleep30-21swapper/312:12:553
1365225617,20sleep30-21swapper/312:12:553
1195725616,25sleep012170-21cat12:07:560
866025513,26sleep08661-21sshd11:57:380
3222425515,24sleep132343-21sed09:43:251
3222425515,24sleep132343-21sed09:43:251
3133825513,29sleep131346-21ntpq07:58:201
2597325513,25sleep125968-21apt-get07:43:061
2117125516,21sleep30-21swapper/310:53:143
2117125516,21sleep30-21swapper/310:53:143
2016425515,25sleep120159-21apt-get12:33:081
1992725514,17sleep10-21swapper/112:32:551
1752825522,18sleep00-21swapper/012:23:220
61812546,36sleep10-21swapper/108:23:031
3208925413,25sleep132082-21apt-get11:28:081
2607525414,24sleep026076-21ntp_states11:08:210
2508925417,19sleep30-21swapper/307:38:243
2260625417,22sleep30-21swapper/307:32:553
1734825419,20sleep30-21swapper/308:58:133
133682545,37sleep00-21swapper/008:43:530
668225314,23sleep16691-21munin-run10:07:551
248182538,27sleep10-21swapper/107:38:181
1929625318,19sleep30-21swapper/312:28:253
1791625319,17sleep00-21swapper/010:43:100
1523525315,22sleep00-21swapper/008:52:530
1242225313,17sleep00-21swapper/012:08:120
739425218,18sleep10-21swapper/110:08:231
50525215,23sleep3579-21sensors_temp08:03:233
2221125216,19sleep10-21swapper/107:28:371
2150025217,18sleep00-21swapper/012:37:550
1783625213,23sleep317833-21apt-get10:43:053
1145325219,18sleep10-21swapper/108:38:251
1145325219,18sleep10-21swapper/108:38:251
96362516,16sleep00-21swapper/010:16:300
906625112,16sleep10-21swapper/108:32:561
2432125112,22sleep024343-21irqstats11:03:170
199952514,35sleep00-21swapper/009:05:490
1662725113,22sleep016630-21apt-get07:13:060
593125016,17sleep30-21swapper/310:03:253
266625011,21sleep12702-21ntp_states09:53:181
2598425017,17sleep10-21swapper/111:08:181
750824911,17sleep10-21swapper/111:53:111
1902124913,21sleep319119-21users07:18:263
974724810,24sleep39770-21munin-run10:17:543
2293224815,16sleep00-21swapper/007:33:120
2208424812,21sleep021206-21/usr/sbin/munin07:28:360
2602224712,17sleep00-21swapper/007:43:090
1469624512,16sleep00-21swapper/007:08:260
1469624512,16sleep00-21swapper/007:08:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional